fresnel-fourier/phase0_findings_iter4.md

Iteration 4 — Phase 0 (substrate / motivation / inventory) → Phase 1 lock

Opens 2026-05-08 immediately after iter3 close (phase8_iteration3_close.md, commit d5d4beb). Per feedback_dev_process.md Phase 0, this document captures iter4's locked research question + substrate + scope, ending with the Phase 1 measurable success criterion.

Locked research question (iteration 4)

"Make VP9 the fifth codec to pass boolean-correctness on fresnel via the libva-v4l2-request-fourier path — mpv --hwdec=vaapi bbb_720p10s_vp9.webm engages the backend cleanly on rkvdec, and HW pixel readback yields byte-identical output to a software-decoded reference for the same frames."

Pass/fail (boolean):

1. Profile enumeration. vainfo with the rkvdec env binding (LIBVA_V4L2_REQUEST_VIDEO_PATH=/dev/video<rkvdec>, LIBVA_V4L2_REQUEST_MEDIA_PATH=/dev/media<rkvdec> — re-verify per-boot device numbering) lists VAProfileVP9Profile0. Currently NOT enumerated — config.c has no VP9 enumeration block (same starting condition as iter3 VP8).
2. Config creation succeeds. vaCreateConfig(VAProfileVP9Profile0, VAEntrypointVLD) returns VA_STATUS_SUCCESS.
3. End-to-end ffmpeg-direct decode. ffmpeg -hwaccel vaapi -i ~/fourier-test/bbb_720p10s_vp9.webm -frames:v 5 -f null - (with rkvdec env binding) shows the libva chain in stderr, no EINVAL from VIDIOC_S_EXT_CTRLS for the VP9_FRAME or VP9_COMPRESSED_HDR controls, exits 0.
4. HW=SW byte-identical, with HW engagement verified per memory feedback_hw_decode_engagement_check.md:
   • Step 0: confirm mpv --hwdec=vaapi engages for VP9 via mpv -v log (must NOT say Using software decoding or Selected decoder: vp9 - <SW name>). If mpv falls back like it did for VP8, fall through to transitive proof.
   • Step 1 (preferred — direct): mpv --hwdec=vaapi --vo=image --frames=2 --start=00:00:02 ~/fourier-test/bbb_720p10s_vp9.webm produces JPEGs byte-identical to a --hwdec=no SW run for both frame 1 and frame 2; frames 1+2 hash-differ (real motion).
   • Step 2 (fallback — transitive per reference_dmabuf_resv_blocker.md): if mpv falls back, OR if rkvdec hits the same dma_resv kernel issue as hantro on iter3, prove HW correctness via (A) libva backend S_EXT_CTRLS payload == kernel-direct ffmpeg-v4l2request payload AND (B) kernel-direct decode == SW reference.
5. Four-codec regression. iter1 MPEG-2 + iter2 HEVC + iter3 VP8 + T4 H.264 reference hashes all hold:
   • H.264 +30s: f623d5f7... and 7d7bc6f2...
   • MPEG-2 +02s: 6e7873030dbf... and ccc7ce08810d...
   • HEVC +02s: 47a5f3850df5... and a467b3bc9d7b...
   • VP8 (criterion-4 reference, when re-verifiable): kernel-direct + SW match per iter3 Phase 7 transitive proof; OR JPEGs e43757a4... + a86bf885... once kernel patches land.

A clean iter4 close has all five checks green. Phase 7 → Phase 4 loopback per feedback_dev_process.md if any fail.

Mechanism the question targets

Phase 0 cross-validator sweep (phase0_evidence/2026-05-07/cross_validator_traces.md) established that the kernel + rkvdec driver path works for VP9: rkvdec advertises VP9 capability per Phase 0 V4L2 inventory:

rkvdec /dev/video3 (boot-dependent path):
  OUTPUT_MPLANE codec_format VP9F (V4L2_PIX_FMT_VP9_FRAME, compressed)
  Stateless controls (per kernel UAPI <linux/v4l2-controls.h>:2696,2797):
    V4L2_CID_STATELESS_VP9_FRAME           = 0xa40b2c (CODEC_STATELESS_BASE + 300)
    V4L2_CID_STATELESS_VP9_COMPRESSED_HDR  = 0xa40b2d (CODEC_STATELESS_BASE + 301)

The broken link is the libva backend at sites analogous to iter3:

• src/config.c::RequestQueryConfigProfiles — no VP9 enumeration block. iter3 added VP8 block; iter4 adds VP9 block probing V4L2_PIX_FMT_VP9_FRAME and adding VAProfileVP9Profile0 (and possibly VAProfileVP9Profile2 if scope extends).
• src/config.c::RequestCreateConfig — no case VAProfileVP9Profile0:. Mirror iter3 VP8 case shape with break;.
• src/config.c::RequestQueryConfigEntrypoints — switch case missing; add to the VLD case list.
• src/vp9.c — file does NOT exist. NEW file.
• src/vp9.h — file does NOT exist. NEW file.
• src/meson.build — 'vp9.c' + 'vp9.h' aren't in the sources/headers lists.
• src/picture.c::codec_set_controls — no VAProfileVP9Profile0 dispatch case.
• src/picture.c::codec_store_buffer — no VP9 cases for VAPictureParameterBufferType, VASliceParameterBufferType.
• src/surface.h — params union has no VP9 member.
• src/buffer.c — predicted Commit D fix-forward per memory feedback_runtime_enumerates_allowlists.md. Verify whether VP9's buffer types fall in the existing allow-list at buffer.c:59-70. VAAPI VP9 uses VAPictureParameterBufferType + VASliceParameterBufferType + VASliceDataBufferType — all already whitelisted (used by H.264). Predicted no Commit D needed for iter4 buffer.c, but plan for fix-forward if a runtime miss surfaces.

VP9 contract surface from kernel UAPI:

• TWO controls per frame (vs iter3 VP8's ONE):
  • V4L2_CID_STATELESS_VP9_FRAME — frame parameters, includes 8-entry DPB, segmentation, quantization, loop filter.
  • V4L2_CID_STATELESS_VP9_COMPRESSED_HDR — probability updates from compressed header (separate control because parsing it requires running the boolean decoder).
• 8 reference frames in DPB (active_ref_frames[8] in v4l2_ctrl_vp9_frame — vs VP8's 3: last/golden/altref).
• Tile-based decoding (VP9 has 1..N tiles per frame; rkvdec supports parallel tile decode).
• VP9 Profile 0 = 8-bit 4:2:0 only (out-of-scope: Profile 1 4:4:4, Profile 2 10-bit, Profile 3 10-bit 4:4:4).

Phase 4 plan must cite the contract before patching: read kernel drivers/staging/media/rkvdec/rkvdec-vp9.c, read FFmpeg libavcodec/v4l2_request_vp9.c, read kernel UAPI <linux/v4l2-controls.h> for both VP9 control structs, state the contract before any code lands.

Predecessor carry-over (iter3 → iter4)

State that carries forward (re-verified at iter4 open)

• Hardware: fresnel RK3399, kernel linux-eos-arm 6.19.9-99-eos-arm. Custom OC kernel.
• rkvdec node: device numbering shuffles per boot. iter1 had /dev/video3+/dev/media1; iter2 had /dev/video3+/dev/media1; iter3 last boot had /dev/video3+/dev/media1 (consistent so far). iter4 binding cells re-verify via v4l2-ctl --info at session start.
• Decoder formats (rkvdec, from Phase 0 v4l2_inventory): OUTPUT_MPLANE = H264, HEVC, VP9F. CAPTURE_MPLANE = NV12.
• Kernel UAPI: V4L2_CID_STATELESS_VP9_FRAME + V4L2_CID_STATELESS_VP9_COMPRESSED_HDR available in /usr/include/linux/v4l2-controls.h (verified Phase 0 iter4 open).
• Backend build state: libva-v4l2-request-fourier post-iter3-close. Fork tip e1aca9c. SHA256 0ab5b2ba22df19569be26228629968ee254c030cd3664ce7afd1bc0396c254ef of /usr/lib/dri/v4l2_request_drv_video.so on fresnel.
• Test fixture: ~/fourier-test/bbb_720p10s_vp9.webm on fresnel (3.4 MB, VP9 Profile 0, 1280×720@24fps yuv420p, 10s). Provenance in phase0_evidence/2026-05-07/test_fixtures.md.
• Reference fixtures for regression: bbb_1080p30_h264.mp4, bbb_720p10s_mpeg2.ts, bbb_720p10s_hevc.mp4, bbb_720p10s_vp8.webm.
• Reference hashes for criterion 5:
  • H.264 (T4) at +30s: f623d5f7a41697f67dd227275c6f1b21ffc257f65626d32fde8229357f8764c9 + 7d7bc6f2146dda8b2d223bba622c4b9fbe9674181ff1e02afe286b620342e0a8
  • MPEG-2 (iter1) at +02s: 6e7873030dbf0403c67f35dd106ebef3c7909a0fd12433b82ad758e7fee9f092 + ccc7ce08810d4a96e9ba7a19f4f95bbf6cc861bda9337604b5c668ad52bef7de
  • HEVC (iter2) at +02s: 47a5f3850df5d8c732767a227830c2272ff78402a7b6adeea329e29838808be5 + a467b3bc9d7b6374b6786ecfac46932d6c7bb932ab11d311edaa233d7863e656
  • VP8 (iter3) — kernel-direct decoded YUV per-frame SHAs (Phase 3 baseline) + libva backend transitive-equivalent. SW reference JPEGs e43757a40e5d71... + a86bf885e58825... re-verifiable when vb2_dma_resv kernel patches land (memory reference_dmabuf_resv_blocker.md).
• Cross-validator anchor: ffmpeg-v4l2request VP9 trace from phase0_evidence/2026-05-07/cross_validator/vp9/. Phase 3 will re-capture verbatim payloads.
• Reference sources local:
  • /home/mfritsche/src/libva-multiplanar/references/ffmpeg-kwiboo/libavcodec/v4l2_request_vp9.c
  • /home/mfritsche/src/libva-multiplanar/references/ffmpeg-kwiboo/libavcodec/vaapi_vp9.c
  • /home/mfritsche/src/libva-multiplanar/references/linux-mainline/drivers/staging/media/rkvdec/rkvdec-vp9.c (verify presence at Phase 2)

Data that does NOT carry forward (re-acquire if needed)

• VP9-specific Phase 3 baseline payloads — Phase 3 captures fresh.
• Pre-iter4 VP9 trace data — none exists in this fork.
• ohm/RK3568 rkvdec VP9 behaviour — ohm rkvdec also supports VP9 but the libva-multiplanar campaign never tested VP9 end-to-end. Reference history only.

Open questions inherited from iter3 close

• iter3-flags-anomaly bit 0x40 — VP8-specific, doesn't apply to VP9.
• iter3-criterion-4-readback (dmabuf-resv kernel issue) — was hantro-specific. Open question for iter4: does rkvdec exhibit the same all-zero pages from libva readback? rkvdec is a different driver subsystem (drivers/staging/media/rkvdec/, not drivers/media/platform/verisilicon/hantro). Likely DOES NOT hit the same bug — iter1+iter2's mpv-DMA-BUF-GL paths worked for rkvdec. Phase 3 baseline answers: re-test ffmpeg-vaapi-hwdownload on rkvdec and check if output is non-zero.
• iter3-mpv-vp8-fallback — mpv 0.41.0 silently falls back to SW for VP8 hwdec=vaapi. Open question for iter4: does mpv engage HW for VP9? If yes (and rkvdec doesn't have dmabuf-resv issue), criterion 4 direct verification works. If no, transitive proof per reference_dmabuf_resv_blocker.md.
• Phase 4 cross-cutting backlog — same as iter3 close (B1, B3, B4, B5, B6, L3 inherited). Not iter4 scope.

Tooling and measurement-instrument inventory (live verification)

Re-verified at iter4 open (carries forward from iter3; all proven working):

• strace -ff -tt -y -v -e trace=ioctl,openat,close — V4L2 + media-request ioctl tracing.
• LIBVA_TRACE environment variable — vaCreate/vaQuery/vaInitialize call traces.
• mpv -v for HW engagement check (per memory feedback_hw_decode_engagement_check.md).
• mpv --hwdec=vaapi --vo=image — cache-safe pixel verifier (when mpv engages).
• ffmpeg -hwaccel v4l2request — independent V4L2 client cross-validator (kernel-direct, no libva backend).
• ffmpeg -hwaccel vaapi — through-libva-backend test path.
• Backend build: ninja -C ~/src/libva-v4l2-request-fourier/build && sudo ninja -C build install.
• Phase 3 decoder (/tmp/iter3_phase3/decode_vp8.py reusable for VP9 with new field layout).
• gcc test-compile for VAAPI field-availability checks per feedback_review_empirical_over_theoretical.md Direction 2.

In-scope (LOCKED 2026-05-08 for iteration 4)

• libva-v4l2-request-fourier backend VP9 path on rkvdec.
• src/config.c — ADD VP9 enumeration block in RequestQueryConfigProfiles; ADD case VAProfileVP9Profile0: with break; in RequestCreateConfig; ADD case in RequestQueryConfigEntrypoints. Same shape as iter3 VP8 changes.
• src/picture.c::codec_set_controls — ADD VAProfileVP9Profile0 dispatch to vp9_set_controls().
• src/picture.c::codec_store_buffer — ADD VP9 cases for VAPictureParameterBufferType + VASliceParameterBufferType. (VP9 doesn't use VAProbabilityBufferType — that was VP8-specific. VP9 doesn't use VAIQMatrixBufferType — quantization is in the picture-parameter struct.)
• src/vp9.c — NEW file. Implements vp9_set_controls() against V4L2_CID_STATELESS_VP9_FRAME + V4L2_CID_STATELESS_VP9_COMPRESSED_HDR. Predicted ~250-350 LOC (between iter3 VP8 and iter2 HEVC scope).
• src/vp9.h — NEW file. Declares vp9_set_controls().
• src/surface.h — extend params union with params.vp9 struct holding VAAPI VP9 buffer types.
• src/meson.build — add 'vp9.c' + 'vp9.h' to sources/headers lists.
• iter4 binding-cell test harness: re-run iter1's Phase 7 5-criterion shape with VP9 fixture substituted + 4-codec regression block.
• Pixel verify uses mpv-DMA-BUF-GL if mpv engages HW for VP9; transitive proof if not (per memory reference_dmabuf_resv_blocker.md).
• VP9 Profile 0 only (8-bit 4:2:0).

Out-of-scope (LOCKED 2026-05-08 for iteration 4)

• VP9 Profile 1 (4:4:4 8-bit), Profile 2 (4:2:0 10/12-bit), Profile 3 (4:4:4 10/12-bit).
• VP9 super-frames (multi-frame packets — not present in BBB fixture).
• Multi-tile parallel decode performance optimization (kernel may decode tiles serially in single-threaded mode; not iter4 scope).
• VP9 SVC (scalable layers).
• Performance metrics.
• Long-duration VP9 stress (>10s).
• Phase 4 cross-cutting backlog items (B1 device-discovery, B3 BeginPicture profile-aware reset, B4 context.c log suppression, B5 vbv_buffer_size, B6 SPS fidelity, L3 vaDeriveImage cache-stale).
• Fixing the kernel-side dma_resv issue (sibling campaign dmabuf-modifier-triage iter1).
• Fixing mpv-VP8-fallback (consumer-side, not iter4 scope).
• chromium-fourier 149 install on fresnel.
• Upstream Linux engagement.

Phase 1 success criterion (LOCKED 2026-05-08)

The five Pass/fail bullets at the top of this document are the iter4 success criterion. Phase 3 baseline measurements feed Phase 4 plan; Phase 7 verification re-runs all five against the patched backend.

If Phase 3 baseline reveals the chosen criterion is the wrong target (Phase 3 → Phase 1 loopback per feedback_dev_process.md), the criterion will be rewritten and re-locked. Plausible reasons that would trigger the loopback:

• VP9 fixture is malformed in a way that exposes a fixture-side bug rather than a backend-side bug. (Mitigation: ffmpeg-v4l2request decoded the fixture cleanly per Phase 0 cross-validator; unlikely.)
• VAAPI's VP9 buffer types include something not exposed by mpv-vaapi consumer chain (e.g. compressed-header buffer not wired through). Phase 3 baseline LIBVA_TRACE will surface.
• mpv --hwdec=vaapi falls back to SW for VP9 (analogous to iter3 VP8 mpv fallback). Mitigation: criterion 4 falls through to transitive proof per memory reference_dmabuf_resv_blocker.md.
• rkvdec exhibits the same dma_resv kernel issue as hantro (unlikely but possible — rkvdec is a different driver but might share the videobuf2 vb2_dma_resv gap). Mitigation: same transitive proof.

The other four Phase 1 criteria hold as locked.

Phase 2 source-read targets

For the upcoming Phase 2 situation analysis:

• src/config.c — RequestQueryConfigProfiles (lines 121-165): pattern for VP9 enumeration block; RequestCreateConfig (lines 54-78): pattern for adding case VAProfileVP9Profile0:; RequestQueryConfigEntrypoints (lines 167-191): adding to entry-point case list.
• src/picture.c::codec_set_controls (lines 188-225): pattern for VP9 dispatch.
• src/picture.c::codec_store_buffer (lines 54-186): patterns for adding VP9 cases per VAAPI buffer type. VAAPI VP9 sends 2 distinct buffer types per frame (Picture + Slice). NOT 4 like iter3 VP8 (no Probability + IQMatrix).
• src/surface.h (lines 92-119): params union pattern for adding vp9 member.
• VAAPI <va/va_dec_vp9.h> — VAAPI VP9 buffer struct definitions: VADecPictureParameterBufferVP9, VASliceParameterBufferVP9.
• Kernel UAPI <linux/v4l2-controls.h>:2696-2870 — V4L2_CID_STATELESS_VP9_FRAME + V4L2_CID_STATELESS_VP9_COMPRESSED_HDR + sub-structs (segmentation, loop filter, quantization, MV probabilities).
• Linux mainline kernel drivers/staging/media/rkvdec/rkvdec-vp9.c — rkvdec VP9 driver source.
• FFmpeg downstream libavcodec/v4l2_request_vp9.c — independent V4L2 client implementation. Submission shape, per-frame field mapping.
• FFmpeg libavcodec/vaapi_vp9.c — VAAPI source-side reference (used in iter3 transitive proof Step A).

What "iteration 4 close" looks like

A clean iter4 close per feedback_dev_process.md Phase 8:

• All 5 Phase 1 criteria green (criterion 4 either direct or transitive — criteria don't differentiate).
• phase8_iteration4_close.md summarizing the bug, contract, fix, binding-cell numbers.
• Fifth-codec passing on the campaign-level scoreboard: 4/5 → 5/5 (campaign complete).
• Memory entries distilled for any new lessons.
• Debug-instrumentation sweep at close.
• Phase 5 sonnet-architect review pass signed off.
• Commits all authored as claude-noether per memory feedback_gitea_as_claude_noether.md.
• src/vp9.c + src/vp9.h added to repo, enabled in meson.build.

Predicted iter4 difficulty vs iter1+iter2+iter3:

• vs iter3 VP8 (single control, ~370 LOC): VP9 has 2 controls (FRAME + COMPRESSED_HDR), more DPB state (8 refs vs 3), more segmentation+lf complexity. Larger.
• vs iter2 HEVC (5 controls, ~470 LOC): VP9 has 2 controls + per-frame submission (no init-time menus expected). Smaller in control-count but field-fidelity comparable.
• Predicted scope: ~400-500 LOC, 3-4 commits + 0-1 fix-forwards (Commit D may not be needed; VP9 buffer types are already in buffer.c allow-list).

If Phase 7 misses a check, most likely culprits:

1. Compressed header probability mapping: v4l2_ctrl_vp9_compressed_hdr is a separate control with VP9 prob-update tables. VAAPI may not expose all bits the kernel needs; gap candidates similar to iter3's first_part_header_bits.
2. 8-frame DPB: VP9 has 8 reference frame slots. Mapping VAAPI's reference indices to V4L2 timestamps for 8 entries is more complex than VP8's 3-entry mapping.
3. Tile sizes: VP9 frames may be split into tiles. The kernel's tile-size handling vs. VAAPI's slice-data-buffer concept may need careful mapping.
4. mpv-vaapi VP9 fallback: same risk as iter3. Verify at Phase 0+3 via mpv -v.
5. Phase 5 review will catch most of these in advance per iter1+iter2+iter3 precedent (4 Critical findings in iter3 alone, all empirically validated correct).