iter10 close + iter11 Phase 0: pivot to HEVC wire-byte diff for Bug 5

iter10 closed negative at Phase 0 (Bommarito unreachable on RK3399).
Saved kernel build + reboot cycle by source-tree reachability check.

iter11 opens with Bug 5 (HEVC libva all-zero) as research target.
Replay iter8/iter9 methodology: deep strace HEVC libva vs kdirect,
decode V4L2_CID_STATELESS_HEVC_* control payload bytes, find the
diff that causes rkvdec to produce all-zero output for libva while
kdirect's submission produces correct decode.

In scope: src/h265.c (libva HEVC), Phase 3 strace + byte-decode.
Out of scope: ext_sps_st/lt_rps (VDPU381/383-only, not RK3399),
kernel patches until empirical evidence of a kernel-side gap.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-14 01:37:45 +00:00
parent 917e9b2691
commit 5f94d7a9ae
2 changed files with 146 additions and 0 deletions
+33
View File
@@ -0,0 +1,33 @@
# Iteration 10 — Phase 8 (close)
Closes 2026-05-14. iter10 = "is Bommarito's May 13 rkvdec HEVC bounds-check the Bug 5 fix vehicle?" Answered NO at Phase 0 via reachability check. Iteration ran 1 phase, ~20 minutes wallclock.
## Outcome
Phase 0 reachability analysis showed `rkvdec_hevc_assemble_hw_rps()` is called only from `rkvdec-vdpu381-hevc.c` (RK3576) and `rkvdec-vdpu383-hevc.c` (RK3588). RK3399's `rk3399_variant_ops` routes HEVC through `rkvdec-hevc.c` which doesn't touch the patched function. Bommarito's patch is sound (KASAN-driven, narrow, low-risk) but inert on fresnel.
Iter10 saved a kernel build + reboot cycle by Phase-0 source check. No code changed.
## Lessons distilled
1. **Reachability checks before patch application** — saved a wasted cycle. Memorialized as [[feedback-rkvdec-patch-reachability]].
2. **The RK3399 HEVC handler is its own ecosystem**. `rkvdec-hevc.c` is the standalone RK3399 HEVC code; the VDPU381/383 split files are separate. Mainline patches against the VDPU381/383 codebase don't reach RK3399 by construction.
3. **Bommarito's methodology DOES transfer** — KUnit harness + KASAN under a small wrapper of the assembly helper, with bounds-check fix. The RK3399 path (`rkvdec-hevc.c::rkvdec_hevc_run_setup`) has structurally analogous patterns (`num_l0_refs` loop, `dpb[ref_idx_l0[i]]` indirect indexing) where a similar KUnit could find OOB gaps. **But libva submits spec-valid values**, so a defensive patch wouldn't fix Bug 5 — it would harden the kernel against malicious userspace, distinct from the libva→rkvdec correctness problem.
## Substrate state at iter10 close
- Fork tip `e0be4e6` (iter9 close). Unchanged.
- Backend SHA `a17e3c39…`. Unchanged.
- Kernel unchanged.
- Diagnostic instrumentation preserved.
## iter10 → iter11 handoff
Bug 5 (HEVC libva all-zero) is still open and is **NOT** a kernel-side OOB issue (which is what Bommarito's patch class addresses). It's a wire-protocol or control-payload issue similar in shape to Bug 4 but for HEVC.
iter11 candidate: replay the iter8/iter9 wire-byte methodology for HEVC. Specifically:
- Deep strace HEVC libva vs kdirect.
- Decode the `V4L2_CID_STATELESS_HEVC_*` control payloads (SPS, PPS, SLICE_PARAMS, SCALING_MATRIX, DECODE_PARAMS).
- Find the first byte-level diff and narrow.
That's iter11 Phase 0 lock target. Route any resulting kernel work through kernel-agent per user directive.