fresnel-fourier/phase0_findings_iter4.md

# Iteration 4 — Phase 0 (substrate / motivation / inventory) → Phase 1 lock

Opens 2026-05-08 immediately after iter3 close ([`phase8_iteration3_close.md`](phase8_iteration3_close.md), commit `d5d4beb`). Per `feedback_dev_process.md` Phase 0, this document captures iter4's locked research question + substrate + scope, ending with the Phase 1 measurable success criterion.

## Locked research question (iteration 4)

> **"Make VP9 the fifth codec to pass boolean-correctness on fresnel via the libva-v4l2-request-fourier path — `mpv --hwdec=vaapi bbb_720p10s_vp9.webm` engages the backend cleanly on rkvdec, and HW pixel readback yields byte-identical output to a software-decoded reference for the same frames."**

Pass/fail (boolean):

1. **Profile enumeration**. `vainfo` with the rkvdec env binding (`LIBVA_V4L2_REQUEST_VIDEO_PATH=/dev/video<rkvdec>`, `LIBVA_V4L2_REQUEST_MEDIA_PATH=/dev/media<rkvdec>` — re-verify per-boot device numbering) lists `VAProfileVP9Profile0`. Currently NOT enumerated — `config.c` has no VP9 enumeration block (same starting condition as iter3 VP8).
2. **Config creation succeeds.** `vaCreateConfig(VAProfileVP9Profile0, VAEntrypointVLD)` returns `VA_STATUS_SUCCESS`.
3. **End-to-end ffmpeg-direct decode**. `ffmpeg -hwaccel vaapi -i ~/fourier-test/bbb_720p10s_vp9.webm -frames:v 5 -f null -` (with rkvdec env binding) shows the libva chain in stderr, no `EINVAL` from `VIDIOC_S_EXT_CTRLS` for the VP9_FRAME or VP9_COMPRESSED_HDR controls, exits 0.
4. **HW=SW byte-identical, with HW engagement verified per memory `feedback_hw_decode_engagement_check.md`**:
   - **Step 0**: confirm mpv `--hwdec=vaapi` engages for VP9 via `mpv -v` log (must NOT say `Using software decoding` or `Selected decoder: vp9 - <SW name>`). If mpv falls back like it did for VP8, fall through to transitive proof.
   - **Step 1 (preferred — direct)**: `mpv --hwdec=vaapi --vo=image --frames=2 --start=00:00:02 ~/fourier-test/bbb_720p10s_vp9.webm` produces JPEGs byte-identical to a `--hwdec=no` SW run for both frame 1 and frame 2; frames 1+2 hash-differ (real motion).
   - **Step 2 (fallback — transitive per `reference_dmabuf_resv_blocker.md`)**: if mpv falls back, OR if rkvdec hits the same dma_resv kernel issue as hantro on iter3, prove HW correctness via (A) libva backend `S_EXT_CTRLS` payload == kernel-direct ffmpeg-v4l2request payload AND (B) kernel-direct decode == SW reference.
5. **Four-codec regression**. iter1 MPEG-2 + iter2 HEVC + iter3 VP8 + T4 H.264 reference hashes all hold:
   - H.264 +30s: `f623d5f7...` and `7d7bc6f2...`
   - MPEG-2 +02s: `6e7873030dbf...` and `ccc7ce08810d...`
   - HEVC +02s: `47a5f3850df5...` and `a467b3bc9d7b...`
   - VP8 (criterion-4 reference, when re-verifiable): kernel-direct + SW match per iter3 Phase 7 transitive proof; OR JPEGs `e43757a4...` + `a86bf885...` once kernel patches land.

A clean iter4 close has all five checks green. Phase 7 → Phase 4 loopback per `feedback_dev_process.md` if any fail.

## Mechanism the question targets

Phase 0 cross-validator sweep ([`phase0_evidence/2026-05-07/cross_validator_traces.md`](phase0_evidence/2026-05-07/cross_validator_traces.md)) established that the kernel + rkvdec driver path works for VP9: rkvdec advertises VP9 capability per Phase 0 V4L2 inventory:

```
rkvdec /dev/video3 (boot-dependent path):
  OUTPUT_MPLANE codec_format VP9F (V4L2_PIX_FMT_VP9_FRAME, compressed)
  Stateless controls (per kernel UAPI <linux/v4l2-controls.h>:2696,2797):
    V4L2_CID_STATELESS_VP9_FRAME           = 0xa40b2c (CODEC_STATELESS_BASE + 300)
    V4L2_CID_STATELESS_VP9_COMPRESSED_HDR  = 0xa40b2d (CODEC_STATELESS_BASE + 301)
```

The broken link is the libva backend at sites analogous to iter3:

- **`src/config.c::RequestQueryConfigProfiles`** — no VP9 enumeration block. iter3 added VP8 block; iter4 adds VP9 block probing `V4L2_PIX_FMT_VP9_FRAME` and adding `VAProfileVP9Profile0` (and possibly `VAProfileVP9Profile2` if scope extends).
- **`src/config.c::RequestCreateConfig`** — no `case VAProfileVP9Profile0:`. Mirror iter3 VP8 case shape with `break;`.
- **`src/config.c::RequestQueryConfigEntrypoints`** — switch case missing; add to the VLD case list.
- **`src/vp9.c`** — file does NOT exist. NEW file.
- **`src/vp9.h`** — file does NOT exist. NEW file.
- **`src/meson.build`** — `'vp9.c'` + `'vp9.h'` aren't in the sources/headers lists.
- **`src/picture.c::codec_set_controls`** — no `VAProfileVP9Profile0` dispatch case.
- **`src/picture.c::codec_store_buffer`** — no VP9 cases for VAPictureParameterBufferType, VASliceParameterBufferType.
- **`src/surface.h`** — `params` union has no VP9 member.
- **`src/buffer.c`** — predicted Commit D fix-forward per memory `feedback_runtime_enumerates_allowlists.md`. Verify whether VP9's buffer types fall in the existing allow-list at `buffer.c:59-70`. VAAPI VP9 uses VAPictureParameterBufferType + VASliceParameterBufferType + VASliceDataBufferType — all already whitelisted (used by H.264). **Predicted no Commit D needed** for iter4 buffer.c, but plan for fix-forward if a runtime miss surfaces.

VP9 contract surface from kernel UAPI:

- **TWO controls per frame** (vs iter3 VP8's ONE):
  - `V4L2_CID_STATELESS_VP9_FRAME` — frame parameters, includes 8-entry DPB, segmentation, quantization, loop filter.
  - `V4L2_CID_STATELESS_VP9_COMPRESSED_HDR` — probability updates from compressed header (separate control because parsing it requires running the boolean decoder).
- 8 reference frames in DPB (`active_ref_frames[8]` in `v4l2_ctrl_vp9_frame` — vs VP8's 3: last/golden/altref).
- Tile-based decoding (VP9 has 1..N tiles per frame; rkvdec supports parallel tile decode).
- VP9 Profile 0 = 8-bit 4:2:0 only (out-of-scope: Profile 1 4:4:4, Profile 2 10-bit, Profile 3 10-bit 4:4:4).

Phase 4 plan must cite the contract before patching: read kernel `drivers/staging/media/rkvdec/rkvdec-vp9.c`, read FFmpeg `libavcodec/v4l2_request_vp9.c`, read kernel UAPI `<linux/v4l2-controls.h>` for both VP9 control structs, state the contract before any code lands.

## Predecessor carry-over (iter3 → iter4)

### State that carries forward (re-verified at iter4 open)

- **Hardware**: fresnel RK3399, kernel `linux-eos-arm 6.19.9-99-eos-arm`. Custom OC kernel.
- **rkvdec node**: device numbering shuffles per boot. iter1 had `/dev/video3+/dev/media1`; iter2 had `/dev/video3+/dev/media1`; iter3 last boot had `/dev/video3+/dev/media1` (consistent so far). iter4 binding cells re-verify via `v4l2-ctl --info` at session start.
- **Decoder formats** (rkvdec, from Phase 0 v4l2_inventory): OUTPUT_MPLANE = `H264`, `HEVC`, `VP9F`. CAPTURE_MPLANE = `NV12`.
- **Kernel UAPI**: `V4L2_CID_STATELESS_VP9_FRAME` + `V4L2_CID_STATELESS_VP9_COMPRESSED_HDR` available in `/usr/include/linux/v4l2-controls.h` (verified Phase 0 iter4 open).
- **Backend build state**: libva-v4l2-request-fourier post-iter3-close. Fork tip `e1aca9c`. SHA256 `0ab5b2ba22df19569be26228629968ee254c030cd3664ce7afd1bc0396c254ef` of `/usr/lib/dri/v4l2_request_drv_video.so` on fresnel.
- **Test fixture**: `~/fourier-test/bbb_720p10s_vp9.webm` on fresnel (3.4 MB, VP9 Profile 0, 1280×720@24fps yuv420p, 10s). Provenance in [`phase0_evidence/2026-05-07/test_fixtures.md`](phase0_evidence/2026-05-07/test_fixtures.md).
- **Reference fixtures for regression**: bbb_1080p30_h264.mp4, bbb_720p10s_mpeg2.ts, bbb_720p10s_hevc.mp4, bbb_720p10s_vp8.webm.
- **Reference hashes for criterion 5**:
  - H.264 (T4) at +30s: `f623d5f7a41697f67dd227275c6f1b21ffc257f65626d32fde8229357f8764c9` + `7d7bc6f2146dda8b2d223bba622c4b9fbe9674181ff1e02afe286b620342e0a8`
  - MPEG-2 (iter1) at +02s: `6e7873030dbf0403c67f35dd106ebef3c7909a0fd12433b82ad758e7fee9f092` + `ccc7ce08810d4a96e9ba7a19f4f95bbf6cc861bda9337604b5c668ad52bef7de`
  - HEVC (iter2) at +02s: `47a5f3850df5d8c732767a227830c2272ff78402a7b6adeea329e29838808be5` + `a467b3bc9d7b6374b6786ecfac46932d6c7bb932ab11d311edaa233d7863e656`
  - VP8 (iter3) — kernel-direct decoded YUV per-frame SHAs (Phase 3 baseline) + libva backend transitive-equivalent. SW reference JPEGs `e43757a40e5d71...` + `a86bf885e58825...` re-verifiable when `vb2_dma_resv` kernel patches land (memory `reference_dmabuf_resv_blocker.md`).
- **Cross-validator anchor**: ffmpeg-v4l2request VP9 trace from [`phase0_evidence/2026-05-07/cross_validator/vp9/`](phase0_evidence/2026-05-07/cross_validator/vp9/). Phase 3 will re-capture verbatim payloads.
- **Reference sources local**:
  - `/home/mfritsche/src/libva-multiplanar/references/ffmpeg-kwiboo/libavcodec/v4l2_request_vp9.c`
  - `/home/mfritsche/src/libva-multiplanar/references/ffmpeg-kwiboo/libavcodec/vaapi_vp9.c`
  - `/home/mfritsche/src/libva-multiplanar/references/linux-mainline/drivers/staging/media/rkvdec/rkvdec-vp9.c` (verify presence at Phase 2)

### Data that does NOT carry forward (re-acquire if needed)

- VP9-specific Phase 3 baseline payloads — Phase 3 captures fresh.
- Pre-iter4 VP9 trace data — none exists in this fork.
- ohm/RK3568 rkvdec VP9 behaviour — ohm rkvdec also supports VP9 but the libva-multiplanar campaign never tested VP9 end-to-end. Reference history only.

### Open questions inherited from iter3 close

- **iter3-flags-anomaly bit 0x40** — VP8-specific, doesn't apply to VP9.
- **iter3-criterion-4-readback (dmabuf-resv kernel issue)** — was hantro-specific. Open question for iter4: does rkvdec exhibit the same all-zero pages from libva readback? rkvdec is a different driver subsystem (drivers/staging/media/rkvdec/, not drivers/media/platform/verisilicon/hantro). Likely DOES NOT hit the same bug — iter1+iter2's mpv-DMA-BUF-GL paths worked for rkvdec. **Phase 3 baseline answers: re-test ffmpeg-vaapi-hwdownload on rkvdec and check if output is non-zero.**
- **iter3-mpv-vp8-fallback** — mpv 0.41.0 silently falls back to SW for VP8 hwdec=vaapi. Open question for iter4: does mpv engage HW for VP9? If yes (and rkvdec doesn't have dmabuf-resv issue), criterion 4 direct verification works. If no, transitive proof per `reference_dmabuf_resv_blocker.md`.
- **Phase 4 cross-cutting backlog** — same as iter3 close (B1, B3, B4, B5, B6, L3 inherited). Not iter4 scope.

## Tooling and measurement-instrument inventory (live verification)

Re-verified at iter4 open (carries forward from iter3; all proven working):

- `strace -ff -tt -y -v -e trace=ioctl,openat,close` — V4L2 + media-request ioctl tracing.
- `LIBVA_TRACE` environment variable — vaCreate/vaQuery/vaInitialize call traces.
- `mpv -v` for HW engagement check (per memory `feedback_hw_decode_engagement_check.md`).
- `mpv --hwdec=vaapi --vo=image` — cache-safe pixel verifier (when mpv engages).
- `ffmpeg -hwaccel v4l2request` — independent V4L2 client cross-validator (kernel-direct, no libva backend).
- `ffmpeg -hwaccel vaapi` — through-libva-backend test path.
- Backend build: `ninja -C ~/src/libva-v4l2-request-fourier/build && sudo ninja -C build install`.
- Phase 3 decoder (`/tmp/iter3_phase3/decode_vp8.py` reusable for VP9 with new field layout).
- gcc test-compile for VAAPI field-availability checks per `feedback_review_empirical_over_theoretical.md` Direction 2.

## In-scope (LOCKED 2026-05-08 for iteration 4)

- libva-v4l2-request-fourier backend VP9 path on rkvdec.
- `src/config.c` — ADD VP9 enumeration block in `RequestQueryConfigProfiles`; ADD `case VAProfileVP9Profile0:` with `break;` in `RequestCreateConfig`; ADD case in `RequestQueryConfigEntrypoints`. Same shape as iter3 VP8 changes.
- `src/picture.c::codec_set_controls` — ADD `VAProfileVP9Profile0` dispatch to `vp9_set_controls()`.
- `src/picture.c::codec_store_buffer` — ADD VP9 cases for VAPictureParameterBufferType + VASliceParameterBufferType. (VP9 doesn't use VAProbabilityBufferType — that was VP8-specific. VP9 doesn't use VAIQMatrixBufferType — quantization is in the picture-parameter struct.)
- `src/vp9.c` — NEW file. Implements `vp9_set_controls()` against `V4L2_CID_STATELESS_VP9_FRAME` + `V4L2_CID_STATELESS_VP9_COMPRESSED_HDR`. Predicted ~250-350 LOC (between iter3 VP8 and iter2 HEVC scope).
- `src/vp9.h` — NEW file. Declares `vp9_set_controls()`.
- `src/surface.h` — extend `params` union with `params.vp9` struct holding VAAPI VP9 buffer types.
- `src/meson.build` — add `'vp9.c'` + `'vp9.h'` to sources/headers lists.
- iter4 binding-cell test harness: re-run iter1's Phase 7 5-criterion shape with VP9 fixture substituted + 4-codec regression block.
- Pixel verify uses mpv-DMA-BUF-GL if mpv engages HW for VP9; transitive proof if not (per memory `reference_dmabuf_resv_blocker.md`).
- VP9 Profile 0 only (8-bit 4:2:0).

## Out-of-scope (LOCKED 2026-05-08 for iteration 4)

- VP9 Profile 1 (4:4:4 8-bit), Profile 2 (4:2:0 10/12-bit), Profile 3 (4:4:4 10/12-bit).
- VP9 super-frames (multi-frame packets — not present in BBB fixture).
- Multi-tile parallel decode performance optimization (kernel may decode tiles serially in single-threaded mode; not iter4 scope).
- VP9 SVC (scalable layers).
- Performance metrics.
- Long-duration VP9 stress (>10s).
- Phase 4 cross-cutting backlog items (B1 device-discovery, B3 BeginPicture profile-aware reset, B4 context.c log suppression, B5 vbv_buffer_size, B6 SPS fidelity, L3 vaDeriveImage cache-stale).
- Fixing the kernel-side dma_resv issue (sibling campaign `dmabuf-modifier-triage` iter1).
- Fixing mpv-VP8-fallback (consumer-side, not iter4 scope).
- chromium-fourier 149 install on fresnel.
- Upstream Linux engagement.

## Phase 1 success criterion (LOCKED 2026-05-08)

The five Pass/fail bullets at the top of this document are the iter4 success criterion. Phase 3 baseline measurements feed Phase 4 plan; Phase 7 verification re-runs all five against the patched backend.

If Phase 3 baseline reveals the chosen criterion is the wrong target (Phase 3 → Phase 1 loopback per `feedback_dev_process.md`), the criterion will be rewritten and re-locked. Plausible reasons that would trigger the loopback:

- VP9 fixture is malformed in a way that exposes a fixture-side bug rather than a backend-side bug. (Mitigation: ffmpeg-v4l2request decoded the fixture cleanly per Phase 0 cross-validator; unlikely.)
- VAAPI's VP9 buffer types include something not exposed by mpv-vaapi consumer chain (e.g. compressed-header buffer not wired through). Phase 3 baseline LIBVA_TRACE will surface.
- mpv `--hwdec=vaapi` falls back to SW for VP9 (analogous to iter3 VP8 mpv fallback). Mitigation: criterion 4 falls through to transitive proof per memory `reference_dmabuf_resv_blocker.md`.
- rkvdec exhibits the same dma_resv kernel issue as hantro (unlikely but possible — rkvdec is a different driver but might share the videobuf2 vb2_dma_resv gap). Mitigation: same transitive proof.

The other four Phase 1 criteria hold as locked.

## Phase 2 source-read targets

For the upcoming Phase 2 situation analysis:

- `src/config.c` — `RequestQueryConfigProfiles` (lines 121-165): pattern for VP9 enumeration block; `RequestCreateConfig` (lines 54-78): pattern for adding `case VAProfileVP9Profile0:`; `RequestQueryConfigEntrypoints` (lines 167-191): adding to entry-point case list.
- `src/picture.c::codec_set_controls` (lines 188-225): pattern for VP9 dispatch.
- `src/picture.c::codec_store_buffer` (lines 54-186): patterns for adding VP9 cases per VAAPI buffer type. VAAPI VP9 sends 2 distinct buffer types per frame (Picture + Slice). NOT 4 like iter3 VP8 (no Probability + IQMatrix).
- `src/surface.h` (lines 92-119): `params` union pattern for adding `vp9` member.
- VAAPI `<va/va_dec_vp9.h>` — VAAPI VP9 buffer struct definitions: `VADecPictureParameterBufferVP9`, `VASliceParameterBufferVP9`.
- Kernel UAPI `<linux/v4l2-controls.h>:2696-2870` — `V4L2_CID_STATELESS_VP9_FRAME` + `V4L2_CID_STATELESS_VP9_COMPRESSED_HDR` + sub-structs (segmentation, loop filter, quantization, MV probabilities).
- Linux mainline kernel `drivers/staging/media/rkvdec/rkvdec-vp9.c` — rkvdec VP9 driver source.
- FFmpeg downstream `libavcodec/v4l2_request_vp9.c` — independent V4L2 client implementation. Submission shape, per-frame field mapping.
- FFmpeg `libavcodec/vaapi_vp9.c` — VAAPI source-side reference (used in iter3 transitive proof Step A).

## What "iteration 4 close" looks like

A clean iter4 close per `feedback_dev_process.md` Phase 8:

- All 5 Phase 1 criteria green (criterion 4 either direct or transitive — criteria don't differentiate).
- `phase8_iteration4_close.md` summarizing the bug, contract, fix, binding-cell numbers.
- Fifth-codec passing on the campaign-level scoreboard: 4/5 → 5/5 (campaign complete).
- Memory entries distilled for any new lessons.
- Debug-instrumentation sweep at close.
- Phase 5 sonnet-architect review pass signed off.
- Commits all authored as `claude-noether` per memory `feedback_gitea_as_claude_noether.md`.
- `src/vp9.c` + `src/vp9.h` added to repo, enabled in meson.build.

Predicted iter4 difficulty vs iter1+iter2+iter3:

- **vs iter3 VP8** (single control, ~370 LOC): VP9 has 2 controls (FRAME + COMPRESSED_HDR), more DPB state (8 refs vs 3), more segmentation+lf complexity. Larger.
- **vs iter2 HEVC** (5 controls, ~470 LOC): VP9 has 2 controls + per-frame submission (no init-time menus expected). Smaller in control-count but field-fidelity comparable.
- **Predicted scope**: ~400-500 LOC, 3-4 commits + 0-1 fix-forwards (Commit D may not be needed; VP9 buffer types are already in buffer.c allow-list).

If Phase 7 misses a check, most likely culprits:

1. **Compressed header probability mapping**: `v4l2_ctrl_vp9_compressed_hdr` is a separate control with VP9 prob-update tables. VAAPI may not expose all bits the kernel needs; gap candidates similar to iter3's `first_part_header_bits`.
2. **8-frame DPB**: VP9 has 8 reference frame slots. Mapping VAAPI's reference indices to V4L2 timestamps for 8 entries is more complex than VP8's 3-entry mapping.
3. **Tile sizes**: VP9 frames may be split into tiles. The kernel's tile-size handling vs. VAAPI's slice-data-buffer concept may need careful mapping.
4. **mpv-vaapi VP9 fallback**: same risk as iter3. Verify at Phase 0+3 via `mpv -v`.
5. **Phase 5 review will catch most of these in advance** per iter1+iter2+iter3 precedent (4 Critical findings in iter3 alone, all empirically validated correct).