ampere-kernel-decoders/phase4_plan_iter2.md

# Phase 4 — iter2 plan (concrete implementation sequence)

Drafted 2026-05-16 evening, post-Phase-3-baseline. This file is what Phase 6 executes verbatim, strict scope per `feedback_dev_process` §6 ("scope strictly to plan — resist feature creep").

## Files touched

| Status | Path | Approx. LOC | Origin |
|--------|------|-------------|--------|
| NEW | `src/h265_parser/h265parser.c` | ~5410 | vendored from `gstreamer/subprojects/gst-plugins-bad/gst-libs/gst/codecparsers/gsth265parser.c` (LGPL v2.1+, headers preserved verbatim) |
| NEW | `src/h265_parser/h265parser.h` | ~2440 | vendored, same source path |
| NEW | `src/h265_parser/bitreader.c` | ~300 | vendored from `gstreamer/subprojects/gstreamer/gst/parse/gstbitreader.c` (LGPL v2.1+) |
| NEW | `src/h265_parser/bitreader.h` | ~300 | vendored, same source path |
| NEW | `src/hevc-ctrls/v4l2-hevc-ext-controls.h` | ~50 | hand-written, mirrors kernel UAPI 7.0 verbatim with citation |
| MODIFIED | `src/h265.c` | +~80, -~5 | new include of v4l2-hevc-ext-controls.h + h265_parser/; extend `h265_set_controls` to add 2 new entries to the `controls[]` array when probed-supported; per-driver-kind gate |
| MODIFIED | `src/context.c` | +~30 | runtime probe via `VIDIOC_QUERYCTRL` for the two new CIDs at context init; store result on `driver_data` |
| MODIFIED | `src/request.h` (or `src/codec.h`, TBD) | +~5 | add `bool has_hevc_ext_sps_rps;` field on `struct request_data` |
| MODIFIED | `src/Makefile.am` | +~5 | add new source files to `libv4l2_request_drv_video_la_SOURCES` |
| MODIFIED | `README.md` | +~15 | new "Vendored third-party components" section listing the 4 LGPL files with upstream URLs + pinned-revision SHA |
| MODIFIED | `STUDY.md` (optional) | +~20 | brief note that the H.265 parser is vendored verbatim from GStreamer for upstream-bug-fix-sync simplicity |

Total: ~8650 LOC added (~7900 vendored unchanged, ~750 hand-written), modifications to 5 existing files.

## Phase 6 step-by-step sequence

Strict ordering — each step is a separate commit so Phase 7 can bisect if needed.

### Step 1 — vendor the parser unchanged

Fetch from GStreamer main mirror (or pin to GStreamer 1.28 tag — confirm in Phase 5 review). Copy verbatim into `src/h265_parser/`. Drop into git as-is. The four vendored files compile WILL not compile yet (GLib deps not satisfied). Build is broken at this commit — that's intentional: this step is "upstream code baseline, ready to be adapted."

Commit message clear: "vendor GStreamer H.265 parser unchanged @ <SHA> — does not build yet, see step 2."

### Step 2 — GLib-to-libc mechanical adaptation

Per the 6 replacements documented in Phase 2:

1. `GArray *` → `struct { void *data; size_t count; size_t element_size; }` plus 4-5 helper functions matching `g_array_*` API surface. Either inline-replace at every call site OR add a thin wrapper header `src/h265_parser/array_compat.h`. Phase 5 review decides which.
2. `g_malloc` / `g_free` / `g_slice_*` → libc `malloc` / `free`. Direct substitution; check for size-overflow patterns that `g_malloc` traps and libc doesn't (none expected in parser code, but verify).
3. `g_clear_pointer (&p, free)` → `do { free(p); p = NULL; } while (0)` inline.
4. `g_assert (cond)` → propagate to the caller as a parser-failure return code. CRITICAL: do NOT use `assert()` here per Phase 2 §"new failure modes" §5; the parser must return an error, not abort the process.
5. `gboolean` / `gint*` / `guint*` / `gsize` → `<stdbool.h>` + `<stdint.h>` C99 types (`bool`, `int8_t`, `int16_t`, `int32_t`, `uint8_t`, `uint16_t`, `uint32_t`, `size_t`).
6. `GST_DEBUG_CATEGORY` / `GST_DEBUG (...)` / `GST_WARNING (...)` / `GST_ERROR (...)` → backend's `request_log` / `error_log` (check `src/utils.h` for existing log macros — likely there).

Build now succeeds; parser is callable but unused (no callers yet).

Commit message: "GLib-to-libc mechanical adaptation of vendored parser; no logic changes." Reference the 6-row table verbatim.

### Step 3 — add UAPI header + runtime probe

Write `src/hevc-ctrls/v4l2-hevc-ext-controls.h` with verbatim copies of:
- `#define V4L2_CID_STATELESS_HEVC_EXT_SPS_ST_RPS (V4L2_CID_CODEC_STATELESS_BASE + 408)`
- `#define V4L2_CID_STATELESS_HEVC_EXT_SPS_LT_RPS (V4L2_CID_CODEC_STATELESS_BASE + 409)`
- `struct v4l2_ctrl_hevc_ext_sps_st_rps { … }` (exact kernel definition)
- `struct v4l2_ctrl_hevc_ext_sps_lt_rps { … }`
- `#define V4L2_HEVC_EXT_SPS_ST_RPS_FLAG_INTER_REF_PIC_SET_PRED 0x1`
- `#define V4L2_HEVC_EXT_SPS_LT_RPS_FLAG_USED_LT 0x1`

Header citation block: upstream kernel UAPI source URL + commit + dated as 7.0-rc3 reference; include guard.

Add `bool has_hevc_ext_sps_rps;` to `struct request_data` (in `src/request.h`).

Add runtime probe to context init (`src/context.c` or new helper in `src/codec.c`):

```c
static bool probe_hevc_ext_sps_controls(int video_fd) {
    struct v4l2_queryctrl q = { .id = V4L2_CID_STATELESS_HEVC_EXT_SPS_ST_RPS };
    if (ioctl(video_fd, VIDIOC_QUERYCTRL, &q) < 0) return false;
    q = (struct v4l2_queryctrl){ .id = V4L2_CID_STATELESS_HEVC_EXT_SPS_LT_RPS };
    return ioctl(video_fd, VIDIOC_QUERYCTRL, &q) >= 0;
}
/* on the rkvdec fd at init time: */
driver_data->has_hevc_ext_sps_rps = probe_hevc_ext_sps_controls(rkvdec_fd);
```

Per-driver-kind also (the gate is "kernel supports it AND driver-kind is vdpu381/383"):
- For RK3399 (`driver_data->driver_kind == 'r'` in iter38 terms): probe returns false on its rkvdec fd, so `has_hevc_ext_sps_rps` is naturally false. No explicit gate needed beyond the probe.
- For ampere RK3588 (also `'r'` per iter38 — same driver-kind tag): probe returns true on the rkvdec fd. The gate fires correctly.

Commit message: "add HEVC EXT_SPS_*_RPS UAPI header + runtime probe (no callers yet)."

### Step 4 — wire h265_set_controls

Extend `h265_set_controls` in `src/h265.c`. After the existing 5-control block (~line 660-690):

```c
if (driver_data->has_hevc_ext_sps_rps) {
    /* Parse the SPS NAL from the bitstream buffer to source RPS data.
     * VAAPI's VAPictureParameterBufferHEVC does NOT expose the RPS array
     * contents (only counts), per the upstream-survey finding documented in
     * ~/src/ampere-kernel-decoders/phase0_findings_iter2.md.
     */
    /* call into vendored gst_h265_parser_* to fill st_rps[] + lt_rps[] arrays
     * from the SPS NAL bytes the backend already has access to (the
     * VAEncSequenceParameterBuffer / source_data buffer) */
    /* ... allocate dynamic arrays sized to num_short_term_ref_pic_sets +
     *     num_long_term_ref_pics_sps; field-by-field map per the GStreamer
     *     reference mapping in fill_ext_sps_rps */

    controls[n++] = (struct v4l2_ext_control){
        .id = V4L2_CID_STATELESS_HEVC_EXT_SPS_ST_RPS,
        .ptr = st_rps_array,
        .size = sizeof(struct v4l2_ctrl_hevc_ext_sps_st_rps) * num_st_rps,
    };
    controls[n++] = (struct v4l2_ext_control){
        .id = V4L2_CID_STATELESS_HEVC_EXT_SPS_LT_RPS,
        .ptr = lt_rps_array,
        .size = sizeof(struct v4l2_ctrl_hevc_ext_sps_lt_rps) * num_lt_rps,
    };
}
```

The actual field-by-field mapping mirrors GStreamer's `gst_v4l2_codec_h265_dec_fill_ext_sps_rps` literally — per Phase 0 survey, the names match the H.265 spec on both sides, so it's a mechanical copy.

The SPS NAL must be located in the bitstream buffer. The backend has the buffer already (from VASliceData / source_data); a small helper walks it for `nal_unit_type == 33` (SPS) and feeds the NAL bytes to the parser.

Commit message: "h265: populate EXT_SPS_ST_RPS + LT_RPS controls when kernel supports them."

### Step 5 — build, install, test (Phase 7 territory but Step 5 is iter2's "smoke check")

```sh
cd ~/src/libva-v4l2-request-fourier
ninja -C build
sudo install -m644 build/src/v4l2_request_drv_video.so /usr/lib/dri/
sudo systemctl reboot  # clear any wedged m2m state from Phase 3
# (wait for ampere to come back)
# Smoke check (NOT full Phase 7):
LIBVA_DRIVER_NAME=v4l2_request timeout 8 ffmpeg \
    -hwaccel vaapi -hwaccel_output_format vaapi \
    -i ~/measurements/encoded/bbb_60s_720p.hevc.mp4 \
    -vf "hwdownload,format=nv12" -frames:v 5 -f null -
# Predicted: exit 0, no kernel OOPS in dmesg.
```

If smoke fails (still OOPSes): F1 fires → loop back to Phase 0 with re-opened `kernel-agent#11`.

Smoke success advances to full Phase 7 (C1-C8 sweep).

### Step 6 — README + STUDY.md updates

- `README.md`: add "Vendored third-party components" section listing the 4 vendored files with upstream URLs + SHA pin + GPL note.
- `STUDY.md` (optional, if file exists and is appropriate): brief paragraph that the H.265 parser is vendored verbatim from GStreamer to preserve future bug-fix sync capability.

Commit message: "README: document vendored GStreamer H.265 parser."

## Predicted Phase 7 outcomes (in C1-C8 terms)

| Criterion | Prediction | Loopback if violated |
|-----------|------------|----------------------|
| C1 (HEVC decode completes) | PASS — 1440 frames decoded, exit 0 | F1 → Phase 0 |
| C2 (HW path engaged, new ioctls fire) | PASS — `VIDIOC_S_EXT_CTRLS` count up from 4 to 6; `MEDIA_REQUEST_IOC_QUEUE` up from 1 to ≥1440 | F1 |
| C3 (frame 0 byte-identical vs SW) | PASS — HEVC I-frame at t=0, no inter-prediction, same sha as ampere-fourier iter1 H.264/VP8/MPEG-2 (`3214803d8be74416`) | F2 → Phase 4 (parser bisect) |
| C4 (frame 720 SSIM Y in H.264 territory ~0.65±0.05) | Likely PASS in that band, but RK3588 may differ; record actual value as new in-session-acquired data point | not a falsifier per Phase 1 — document and proceed |
| C5 (FPS N=3) | Unconstrained, expect 100-400 fps; matches H.264 / VP8 / MPEG-2 order of magnitude | not a falsifier |
| C6 (dmesg clean) | PASS — no `rkvdec_hevc_prepare_hw_st_rps` OOPS | F1 |
| C7 (firefox-fourier HEVC, opportunistic) | PASS if SDDM auto-login Wayland session still up | not iter2-blocking |
| C8 (iter1 3-codec baseline regression check) | PASS — patch is HEVC-only behind a gate; H.264 / VP8 / MPEG-2 paths unchanged | F3 → Phase 4 (per-driver-gate audit) |

## Risk register

| Risk | Probability | Mitigation |
|------|-------------|------------|
| Vendored parser doesn't compile cleanly without GLib (more deps than mapped in Phase 2) | medium | Phase 2 mapping is mechanical; expect 1-2 surprises (GTypeClass usage somewhere). Address inline in Step 2; don't extend scope further. |
| Parser produces RPS data slightly different from kernel's spec interpretation | medium | F2 falsifier — bisect against GStreamer gst-launch on same clip if it fires |
| `linux-api-headers` bump happens unexpectedly mid-iter2 | low | UAPI header is internal; works regardless of host headers version |
| ampere reboot during Step 5 takes longer than expected | low | SDDM auto-login restores Wayland session automatically |
| Step 4's SPS-NAL-location helper misidentifies the SPS in some bitstream layouts | low-medium | Phase 5 review item; test with multiple BBB encodes if needed |

## Risks NOT mitigated (accepted as is)

- The full Phase 6 sequence is substantial work (~8.7k LOC added, multiple build/test cycles, kernel reboots). Per operator directive, time/effort don't matter; the budget is "however long correctness takes."

## Phase 4 close

5 commits, 8.7k LOC, 5 file modifications. Phase 7 predictions concrete (C1-C8 mapped). 5 risks named, 4 mitigated. Ready for Phase 5 second-model review.

---

## Phase 5 amendments (applied 2026-05-16 post-review)

### Amendment §Step 3 — per-driver-kind probe storage

Replace single `bool has_hevc_ext_sps_rps;` field on `struct request_data` with a pair:

```c
bool has_hevc_ext_sps_rps_rkvdec;
bool has_hevc_ext_sps_rps_hantro;
```

Probe runs at the time each fd is opened (init for primary, alt-probe for secondary per iter38). Storage parallels `video_fd_rkvdec` / `video_fd_hantro`. h265_set_controls consults only the rkvdec entry (HEVC routes through rkvdec only).

§Step 4 control-submission guard becomes explicit:

```c
if (driver_data->driver_kind == 'r' && driver_data->has_hevc_ext_sps_rps_rkvdec) {
    /* … populate + submit EXT_SPS_*_RPS controls … */
}
```

The `driver_kind == 'r'` clause is logically redundant given the probe-result gate but provides reviewer-flagged human-readable intent.

### Amendment §Step 4 — explicit two-struct usage + SPS caching

The GStreamer reference `fill_ext_sps_rps` reads from BOTH `GstH265ShortTermRefPicSet` AND `GstH265ShortTermRefPicSetExt` (both structs are in the vendored gsth265parser.h — see Phase 5 Finding #1 rebut). Step 4 mapping:

| V4L2 field | Source in vendored parser |
|------------|---------------------------|
| `delta_idx_minus1`, `delta_rps_sign`, `abs_delta_rps_minus1`, `num_negative_pics`, `num_positive_pics`, `flags` | `GstH265ShortTermRefPicSet` (members `delta_idx_minus1`, `delta_rps_sign`, `abs_delta_rps_minus1`, `NumNegativePics`, `NumPositivePics`, `inter_ref_pic_set_prediction_flag`) |
| `use_delta_flag`, `used_by_curr_pic`, `delta_poc_s0_minus1[]`, `delta_poc_s1_minus1[]` | `GstH265ShortTermRefPicSetExt` (members `use_delta_flag[]`, `used_by_curr_pic_flag[]`, `delta_poc_s0_minus1[]`, `delta_poc_s1_minus1[]`) |

If GStreamer's `gst_h265_decoder_get_sps_ext()` helper is in the base-decoder class (not in gsth265parser.c), inline its body — it likely just reaches into `sps->short_term_ref_pic_set_ext[i]` on the parsed `GstH265SPS`.

**SPS parse-and-cache strategy** (Phase 5 Finding #3):

- Add a cache slot on `struct request_data`: `struct hevc_sps_cache_entry { uint8_t sps_id; bool valid; GstH265SPS parsed_sps; } hevc_sps_cache[16]` (sized to match the H.265 spec's max of 16 active SPS).
- At first `h265_set_controls` invocation OR whenever the SPS NAL is present in `source_data` (walk for `nal_unit_type == 33`): parse, store at `hevc_sps_cache[sps_id]`, mark valid.
- On subsequent frames: look up `hevc_sps_cache[pps->seq_parameter_set_id]`, reuse if valid; if invalid AND SPS NAL not in current source_data, log warning + skip the EXT_SPS_*_RPS controls (graceful fallback — better to fail soft than re-OOPS).
- Cache cleared in `RequestDestroyContext`.

This adds approximately +20 LOC to Step 4 (cache plumbing) on top of the original control-submission code.

### Amendment §"Phase 7 predictions" C3 row

Replace:
> C3 — PASS — HEVC I-frame at t=0, no inter-prediction, same sha as ampere-fourier iter1 H.264/VP8/MPEG-2 (`3214803d8be74416`)

With:
> C3 — PASS — HEVC frame 0 HW-decode SHA matches HEVC SW-decode SHA from the same encoded clip (anchored per-clip, not cross-codec). Cross-codec sha convergence observed in iter1 was empirical-not-guaranteed: encoder's quantization choices differ across codecs and the SHA match was a happy coincidence, not a contract.

### Updated risk register (added risk #6)

| 6 | SPS NAL absent from `source_data` on non-IDR frames; without caching, h265_set_controls submits zero-filled RPS for every non-IDR → re-OOPS | high (would have fired in practice) | parse-and-cache per Amendment §Step 4 above; first IDR populates cache, subsequent non-IDR frames reuse it; graceful skip + warning if cache miss AND no SPS NAL available |