fresnel-fourier/phase0_findings_iter5.md

# Iteration 5 — Phase 0 (substrate / motivation / inventory) → Phase 1 lock

Opens 2026-05-10 immediately after iter4 close ([`iter4_phase7_close.md`](iter4_phase7_close.md), commit `9d2b7c1`). Per `feedback_dev_process.md` Phase 0, this document captures iter5's substrate state + open issue inventory and proposes candidate research questions for Phase 1 lock.

**This iteration breaks pattern.** iter1..iter4 all targeted "the next codec." With VP9 closed at iter4, the campaign-level scope ("5 codecs boolean-correct") is technically green (4/5 direct + 1/5 transitive). iter5 must therefore answer a *different* class of research question than "add codec N+1." Three substantive candidates surfaced from iter4 close are recorded below; Phase 1 lock picks one.

## Substrate state (verified 2026-05-10 21:50 CEST on fresnel)

| Property | Value | Notes |
|---|---|---|
| Kernel | `linux-fresnel-fourier 7.0-1` | kernel-agent product. v7.0 + 3 PBP DTS patches. **NOT** besser-direct. `vb2_dma_resv` + panfrost iommu-cache excluded pending RFC v2. |
| Boot device numbering (today) | `/dev/video0` rga · `/dev/video1` vpu-enc · `/dev/video2` vpu-dec (hantro MPEG-2/VP8) · `/dev/video3` rkvdec (H.264/HEVC/VP9) · `/dev/video4,5` USB cams | `/dev/media0=hantro` · `/dev/media1=rkvdec` · `/dev/media2=uvc`. **Different from iter4 Phase 6 boot** — auto-detect picks hantro encoder first (iter4-B1 not fixed). |
| Fork tip | `692eaa0` | iter4 Phase 7 fix-forward (`h264_start_code` profile-gating) |
| Backend installed | `/usr/lib/dri/v4l2_request_drv_video.so` SHA256 `6e90b7a9b2c33480dd3ffc2da8423ab0bcef14f23c68cf18dc2ae2ff66ac808c` | matches fork tip build |
| Codec scoreboard | 5/5 PASS (4 direct + 1 transitive) | iter1 MPEG-2 · iter2 HEVC · T4 H.264 · iter3 VP8 · iter4 VP9 |

## Open issue inventory from iter4 close

Carried forward into iter5 (any/all can be locked as the iter5 target):

### Substrate-class issues (block direct verification through libva backend)

**Bug 2 — cap_pool readback regression substrate-wide** on `linux-fresnel-fourier 7.0-1`. Symptom: libva-backend ffmpeg-vaapi-hwdownload returns the cap_pool init pattern (`RGB(0,0x4c,0)` constant) for all 4 already-shipping codecs at iter4 Phase 7 capture. H.264 partial exception (keyframe reads correctly, inter frames stuck). Hash `93dd9db5...` appears across VP9 + MPEG-2. Sibling to iter3 dma_resv issue but different signature: constant `0x4c` instead of all-zero. Kernel-direct ffmpeg-v4l2request works fine (iter4 Leg 3 PASS). So the broken path is libva → cap_pool → kernel binding, not the kernel.

**Bug 3 — hantro UAPI drift** on MPEG-2 + VP8: kernel `Unable to set control(s)` rejection at submission time. Phase 2 of iter1 (6.19.9-99) had `v4l2_ctrl_vp9_frame` size 144 / `_compressed_hdr` 1947; iter4 Phase 3 (7.0-1) had 168 / 2040. Probably struct-size or new-required-field drift on hantro controls (MPEG-2/VP8 sequence/picture/quant), analogous to the VP9 struct evolution.

### Backend-class issues (iter4 Phase 6 backlog)

- **iter4-B1** — Auto-detect picks first-enumerated `/dev/media*` driver-match without discriminating decoder vs encoder. Today's boot enumerates hantro-vpu **encoder** (`rk3399-vpu-enc`) at `/dev/media0`; auto-detect picks it for VP9 (wrong). Workaround: explicit `LIBVA_V4L2_REQUEST_NO_AUTODETECT=1 LIBVA_V4L2_REQUEST_VIDEO_PATH=... LIBVA_V4L2_REQUEST_MEDIA_PATH=...`. Fix: walk should require `MEDIA_ENT_F_PROC_VIDEO_DECODER` entity in the topology, not just driver-name match.
- **iter4-B2** — mpv-vaapi `Could not create device` for VP9. Consumer-side, not backend.
- **iter4-Q6** — per-segment `seg_param[s].luma_ac_quant_scale` → kernel `feature_data[s][ALT_Q]` is lossy for non-BBB segmentation-enabled streams. Dead code for BBB.
- **iter4-COLOR_RANGE** — VAAPI exposes no `color_range`; backend leaves `V4L2_VP9_FRAME_FLAG_COLOR_RANGE_FULL_SWING` clear. Wrong for full-range JPEG-encoded VP9.
- **B3** — picture.c BeginPicture profile-aware reset. iter1-4 added per-profile reset lines without refactoring.
- **B4** — context.c log suppression for `Unable to set control(s)` warnings from codecs the bound driver doesn't support.
- **B5** — mpeg2 vbv_buffer_size polish (iter1 S2 finding).
- **B6** — h265 SPS bitstream-parse fidelity gap.
- **L3** — vaDeriveImage cache-stale.

### Regression invariants (iter1-3 hashes invalidated by substrate change)

iter4 Phase 7 documented all 4 prior codecs FAIL on `linux-fresnel-fourier 7.0-1`:

| Codec | Driver | iter4 P7 verdict | Underlying cause |
|---|---|---|---|
| H.264 | rkvdec | PARTIAL (keyframe-only) | Bug 2 |
| HEVC | rkvdec | FAIL (constant fill) | Bug 2 |
| MPEG-2 | hantro | FAIL + `Unable to set control(s)` | Bug 2 + Bug 3 |
| VP8 | hantro | FAIL + `Unable to set control(s)` | Bug 2 + Bug 3 |
| VP9 | rkvdec | FAIL until fix-forward | iter4-specific (start-code prepend), closed |

The campaign-level "5/5 codecs passing" claim is true on iter4's substrate frame-of-reference (transitive proof discipline) but NOT byte-identical on the new substrate. Until iter5 work establishes Bug 2 + Bug 3 closure, iter5+ regression test means **transitive proof** + control-payload byte-compare, not direct pixel hashes.

## Candidate iter5 research questions

### Candidate A — Re-anchor 4-codec regression block on `linux-fresnel-fourier 7.0-1`

> **"Restore direct pixel verification for at least 3 of {H.264, HEVC, MPEG-2, VP8} on the new kernel substrate. Lock the regression-test reference hashes for iter5+ regression-block-of-5."**

Pass/fail: For ≥3 codecs of the four iter1-3 codecs, ffmpeg-vaapi-hwdownload + libva backend produces a SHA256 (or transitive-proof equivalent) that locks as iter5+ regression invariant. The 4-codec block in iter5 binding cells is GREEN.

**Scope**: Investigate Bug 2 (cap_pool readback) — is it a libva-backend cap_pool refactor needed, or a kernel patch? Investigate Bug 3 (hantro UAPI drift) — what changed in 7.0-1 struct shapes for MPEG-2/VP8 controls? Both can be backend-only OR cross-cutting into kernel-agent.

**Pro**: Establishes a re-grounded baseline for iter5+ campaign claims. Without it, every future iteration must say "re-verify against substrate" before declaring regression-block clean.

**Con**: Bug 2 might require kernel patches (RFC v2 vb2_dma_resv hasn't landed); fix-forward backend may not be enough. Scope risk: could expand into a kernel-iteration.

### Candidate B — Substrate stabilization: land vb2_dma_resv + hantro UAPI drift fixes in `kernel-agent`

> **"Remove the cap_pool readback workaround across the campaign by landing kernel patches in the `linux-fresnel-fourier` product. Direct verification becomes possible for all 5 codecs."**

Pass/fail: Post-iter5 kernel ships with patches; ffmpeg-vaapi-hwdownload returns correct pixels for all 5 codecs (no transitive proof needed); the iter4 transitive proof for VP9 can be replaced by direct pixel hash.

**Scope**: Cross-product work in `kernel-agent` (sibling memory `project_kernel_agent.md`). vb2_dma_resv RFC v2 is in design upstream — track + land. Hantro UAPI drift root cause may be in linux-media tree, not local patch.

**Pro**: Solves the *real* underlying issue. iter1-4 all become direct-verifiable; transitive proofs become belt-and-suspenders. iter6+ can lock pixel hashes confidently.

**Con**: Large scope. Kernel-iteration timing, possibly multi-week. May not be ready for a single iter5 close cycle. Memory `reference_dmabuf_resv_blocker.md` notes RFC v1 rejected, v2 in design — landing window uncertain.

### Candidate C — Auto-detect device-discovery harden (iter4-B1)

> **"Backend auto-detect picks the right decoder for each codec on every boot, without env-override workarounds. Validate against the iter4-style enumeration shuffle (today: hantro-vpu encoder enumerates first)."**

Pass/fail: With auto-detect (no env vars), `vainfo` enumerates all 5 codec profiles correctly. ffmpeg-vaapi VP9 + H.264 + HEVC route to rkvdec; MPEG-2 + VP8 route to hantro-vpu-dec. Tested on at least 2 reboots with different device-numbering orders.

**Scope**: backend-only. ~50-100 LOC change in `request.c::v4l2_request_init` to enrich the media topology walk: require `MEDIA_ENT_F_PROC_VIDEO_DECODER` on at least one entity, OR scan media-controller entities for decode-capable function, OR maintain a codec-to-card-name map.

**Pro**: Smallest scope. Pure-backend fix. Removes a campaign-wide fragility. Mechanical.

**Con**: Doesn't address Bug 2 + Bug 3 (substrate-class blockers). iter5 close would say "5/5 codecs auto-route correctly" but still need transitive proof to verify any of them actually decode.

### Candidate D — Performance / measurement iteration

> **"Quantify per-codec performance on fresnel — CPU utilization, frames-per-second, drop count over a 60s window — for H.264/HEVC/VP9 paths. Establish iter5 binding-cell numbers for future performance-regression-checks."**

Pass/fail: For each of {H.264 1080p30, HEVC 720p, VP9 720p}, mpv `--hwdec=vaapi --frames=10s` reports CPU% < some-threshold, no dropped frames, libva path engaged. The numbers themselves become the iter5+ binding-cell anchors.

**Scope**: pure measurement + harness. NO backend code changes. Maybe a `perf` / `top` / `mpv --status` harness script. Reproducible per-codec invocations.

**Pro**: Campaign README locked this as "follow-up iteration after correctness lands." Correctness has landed. Now is the formally-next iteration. NOT blocked by Bug 2/3 (perf is engagement-side, not pixel-side).

**Con**: Bug 2 + Bug 3 still block the iter1-3 codecs from reporting *correct* pixels — performance numbers without pixel correctness are vacuous (could measure 60fps of constant green). MPEG-2 + VP8 may not even submit-controls cleanly. Likely needs Candidate A or B first.

## Recommendation

If pressed to recommend a sequence: **A → C → B → D**.

A first because regression invariants underpin everything else. C next because the fragility hits the user every boot. B because it's the real fix but timing-uncertain. D last because it depends on A.

A and C could plausibly be one iteration if scope cooperates; B is a separate beast; D is genuinely "after correctness."

## Locked research question (iteration 5, 2026-05-10)

User pick: **Candidate B — both Bug 2 + Bug 3 in one close, kernel-agent local-carry acceptable.**

> **"Land vb2_dma_resv (Bug 2) AND fix hantro UAPI drift (Bug 3) in the `linux-fresnel-fourier` kernel product so that every shipping codec on RK3399 — H.264, HEVC, VP9 via rkvdec; MPEG-2, VP8 via hantro-vpu-dec — produces directly-readable pixels through the libva backend's cap_pool path, on the next kernel product shipped from kernel-agent. The transitive-proof discipline becomes optional belt-and-suspenders, not the only viable verifier."**

### Pass/fail (boolean, per `feedback_dev_process.md` Phase 1)

1. **Bug 2 closed — cap_pool readback returns kernel-decoded pixels on the new kernel.** For all 3 rkvdec codecs (H.264 1080p30, HEVC 720p, VP9 720p), `ffmpeg -hwaccel vaapi -hwaccel_output_format vaapi -vf hwdownload,format=nv12,format=yuv420p -f rawvideo` via the libva backend on the new kernel returns raw YUV byte-identical to the SW reference (or transitive equivalent that explicitly demonstrates cap_pool is delivering kernel-written pages, not init pattern). The `RGB(0,0x4c,0)` constant-fill signature is gone.

2. **Bug 3 closed — hantro MPEG-2 + VP8 controls accepted on the new kernel.** `ffmpeg -hwaccel vaapi ... bbb_720p10s_{mpeg2.ts,vp8.webm}` via the libva backend on the new kernel runs without kernel `Unable to set control(s)` log messages, and produces YUV byte-matching the SW reference (with or without transitive proof depending on Bug 2 status for the hantro path specifically).

3. **Substrate ships from `kernel-agent`.** The new kernel is built and promoted via the kernel-agent pipeline (per memory `project_kernel_agent.md` — `ka-promote` + `ka-close --status success` semantics) and tagged `linux-fresnel-fourier 7.0-2` (or whatever version the agent produces next). Local-carry patches in the kernel-agent product are acceptable for iter5 close — mainline-landing is bonus, not required. Patch scope-tagged per agent discipline (Hard rule #7).

4. **No regression on already-shipping codec contracts.** The Phase 3 anchor payloads (FRAME + COMPRESSED_HDR + slice for all 5 codecs) on the new kernel still byte-match the iter1-4 captures, modulo the documented S4 uv_mode VP9 carve-out. The libva backend tip `692eaa0` is unmodified; only the kernel substrate changes.

5. **Five-codec direct-verification block.** With the new kernel + unmodified backend tip `692eaa0`, all 5 codecs pass criterion-4 *directly* (not transitively). VP9 in particular: byte-identical YUV from libva-backend hwdownload to SW reference. The campaign scoreboard becomes "5/5 direct" (was "4 direct + 1 transitive").

A clean iter5 close has all five criteria green. Phase 7 → Phase 4 loopback per `feedback_dev_process.md` if any fail.

### Scope locks

**In scope:**

- vb2_dma_resv kernel patch (RFC v2 carry, or local equivalent). Source: linux-media list RFC v2 thread (resume from `reference_dmabuf_resv_blocker.md`).
- Hantro UAPI drift root-cause + fix. Probably struct-size / field-additions in `drivers/media/platform/verisilicon/hantro_*.c` between 6.19 → 7.0. Phase 3 baseline diffs the headers in both kernels.
- Kernel-agent product build + promote pipeline (per `project_kernel_agent.md`). Iter5 is the first user-driven exercise of `ka-promote` + `ka-close` for the fresnel-fourier campaign tag.
- Re-verification matrix on new kernel: all 5 codecs through libva, direct pixel hash.

**Out of scope:**

- iter4-B1 (auto-detect device discrimination) — backend-side fragility, separate iteration.
- iter4-B2 (mpv-vaapi create-device failure) — consumer-side.
- iter4-Q6, COLOR_RANGE, B3/B4/B5/B6/L3 backlog — backend, not kernel.
- panfrost IOMMU_CACHE — sibling issue not on Bug-2/Bug-3 critical path.
- Performance metrics — that's Candidate D, deferred again.
- Front-end libva (campaign-wide, unchanged).
- AV1 (no decoder block).
- Bootlin / Collabora upstreaming — local-carry acceptable per user pick. Stays default-deferred.
- Mainline merge — bonus, not gating.

### Open questions to resolve in Phase 0 inventory deepening / Phase 2 source-read

1. **vb2_dma_resv RFC v2 status** — what is the current patch revision on linux-media? Is it apply-able to v7.0 directly, or does it need rebase? Phase 2 task.
2. **Hantro 7.0 driver delta** — diff `drivers/media/platform/verisilicon/hantro_{mpeg2,vp8}.c` between v6.19.9 and v7.0 upstream. What controls' struct shapes changed? What new mandatory fields are there? Phase 3 baseline.
3. **kernel-agent operational status** — memory notes the agent is spec'd 2026-05-09, not yet operational; `ka-*` CLI verbs not implemented. Does iter5 trigger building the agent, or use the manual kernel-build path as fallback? Coordinate with the operator at Phase 1 close.
4. **Patch attribution** — vb2_dma_resv is somebody else's patch (linux-media author). For local-carry the kernel-agent's patch metadata needs the upstream author preserved + a "Backported-by: claude-noether <sentinel-email>" trailer. Phase 6 detail.
5. **Cross-substrate verification** — once new kernel ships, must we also rebuild libva backend against the new kernel UAPI headers? Probably no (backend reads UAPI at runtime, not build-time), but Phase 3 confirms.
6. **Test corpus** — same 5 fixtures (bbb_*) as iter1-4. New kernel could break the fixture-parse path if hantro's new UAPI requires different bitstream framing — unlikely but Phase 3 verifies.
7. **Reboot discipline** — installing a new kernel requires fresnel reboot. Schedule with the operator (Pinebook Pro is a working machine).

### Phase 2 source-read targets

- `drivers/media/v4l2-core/videobuf2-dma-contig.c` (or wherever vb2_dma_resv lives upstream — RFC v2 patch context).
- `drivers/media/platform/verisilicon/hantro_mpeg2.c` + `hantro_vp8.c` — v6.19 vs v7.0 delta.
- `drivers/media/v4l2-core/v4l2-ctrls-defs.c` — control-struct evolution for VP8 + MPEG-2.
- `/usr/include/linux/v4l2-controls.h` on fresnel (currently 7.0 headers).
- Memory `project_kernel_agent.md` — agent operating envelope.
- Memory `reference_fresnel_kernel_substrate.md` — current product version + excluded-patch list.
- linux-media archive — RFC v2 thread + reviewer feedback.

### Phase 3 baseline (predicted shape)

Phase 3 captures:

- Verbatim FRAME + COMPRESSED_HDR (VP9) + slice (H.264/HEVC/MPEG-2/VP8) control payloads on `linux-fresnel-fourier 7.0-1` (current kernel) for all 5 codecs. Strace + `extract.py` from iter4 close evidence applies directly.
- Raw YUV from ffmpeg-vaapi-hwdownload on the current kernel (will be all-`0x4c` constant-fill = init pattern for ≥3 codecs, confirms Bug 2 is reproduced).
- Raw YUV from ffmpeg-v4l2request kernel-direct on the current kernel for all 5 codecs (the reference, since kernel-direct bypasses cap_pool).
- Kernel `Unable to set control(s)` log capture for hantro MPEG-2 + VP8 (confirms Bug 3 reproduced).
- Header diff: `linux-fresnel-fourier 7.0-1` vs `linux-eos-arm 6.19.9-99` `v4l2-controls.h` + hantro driver source.

Once Phase 3 lands, Phase 4 designs patches against the diff.

### Predicted iter5 difficulty + shape

- **vs iter1-3 (codec adds, ~300-800 LOC libva backend)**: iter5 is kernel-side. Patch scope likely small (vb2_dma_resv is reportedly ~50-100 lines mainline; hantro UAPI drift may be smaller still, just struct-size guards). Build + promote pipeline is the time sink.
- **vs iter4 (single new codec)**: iter5 touches NO codec. It touches the kernel that codecs run on.
- **Predicted scope**: 1-3 kernel patches (~50-200 LOC), kernel build + install per kernel-agent flow, libva-backend regression matrix re-run. No fork patches expected unless Phase 3 surfaces a backend-side amplification of Bug 2/3.
- **Most likely Phase 7 failure mode**: vb2_dma_resv RFC v2 doesn't apply cleanly to v7.0; hantro 7.0 driver delta turns out to need a downstream patch the kernel maintainers haven't shipped yet; cap_pool readback fails for reasons beyond what vb2_dma_resv addresses.

### Predecessor close-out summary

Per the campaign 8(+1)-phase loop, iter4 closed at `9d2b7c1` (campaign repo) + fork tip `692eaa0`. iter4's transitive proof was the *workaround* for Bug 2; iter5 sets out to *remove* that workaround for the rkvdec block, and to *remove the regression-block falsification* introduced by Bug 3 on the hantro block.

Memory rules carry forward (no rule retirement expected at iter5 open):
- `feedback_dev_process` (campaign-wide)
- `feedback_gitea_as_claude_noether` (updated at iter4 close)
- `feedback_no_session_termination_attempts`
- `feedback_review_empirical_over_theoretical` (both directions)
- `feedback_unconditional_codec_state` (iter4 lesson, applies to any new backend flag work)
- `feedback_pixel_compare_in_yuv_not_png` (iter4 lesson, applies to criterion 4/5)
- `feedback_hw_decode_engagement_check` (iter3 lesson)
- `feedback_runtime_enumerates_allowlists` (iter3 lesson)
- `reference_dmabuf_resv_blocker` (iter5 STARTS HERE — this is the bug iter5 closes)
- `reference_fresnel_kernel_substrate` (iter5 produces the next entry's version bump)
- `project_kernel_agent` (iter5 is the first campaign-side exercise of the agent flow)

iter5 → iter6 handoff prediction:

- If Bug 2 + Bug 3 close cleanly: iter6 picks up Candidate A (regression block re-anchored on new substrate, formal iter5+ binding-cell numbers) or Candidate C (auto-detect harden).
- If Bug 2 closes but Bug 3 doesn't: iter6 = Bug 3 alone, or iter6 swaps the iter5 plan to iter5b mid-loop.
- If neither closes: Phase 7 → Phase 4 loopback within iter5, OR Phase 7 → Phase 0 reset.

## Memory state at iter5 open

Active rules (`MEMORY.md` index): 11 entries. Recently added at iter4 close:

- `feedback_unconditional_codec_state` (iter4 Phase 7 root-cause)
- `feedback_pixel_compare_in_yuv_not_png` (iter4 Phase 7 close)
- `feedback_gitea_as_claude_noether` (updated — SSH push now wired, user is `gitea` at port 2222)

No new memory needed at iter5 open until Phase 1 locks the target.