Files

T

marfrit 5f94d7a9ae iter10 close + iter11 Phase 0: pivot to HEVC wire-byte diff for Bug 5

iter10 closed negative at Phase 0 (Bommarito unreachable on RK3399).
Saved kernel build + reboot cycle by source-tree reachability check.

iter11 opens with Bug 5 (HEVC libva all-zero) as research target.
Replay iter8/iter9 methodology: deep strace HEVC libva vs kdirect,
decode V4L2_CID_STATELESS_HEVC_* control payload bytes, find the
diff that causes rkvdec to produce all-zero output for libva while
kdirect's submission produces correct decode.

In scope: src/h265.c (libva HEVC), Phase 3 strace + byte-decode.
Out of scope: ext_sps_st/lt_rps (VDPU381/383-only, not RK3399),
kernel patches until empirical evidence of a kernel-side gap.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

2026-05-14 01:37:45 +00:00

2.4 KiB

Raw Permalink Blame History

Iteration 10 — Phase 8 (close)

Closes 2026-05-14. iter10 = "is Bommarito's May 13 rkvdec HEVC bounds-check the Bug 5 fix vehicle?" Answered NO at Phase 0 via reachability check. Iteration ran 1 phase, ~20 minutes wallclock.

Outcome

Phase 0 reachability analysis showed rkvdec_hevc_assemble_hw_rps() is called only from rkvdec-vdpu381-hevc.c (RK3576) and rkvdec-vdpu383-hevc.c (RK3588). RK3399's rk3399_variant_ops routes HEVC through rkvdec-hevc.c which doesn't touch the patched function. Bommarito's patch is sound (KASAN-driven, narrow, low-risk) but inert on fresnel.

Iter10 saved a kernel build + reboot cycle by Phase-0 source check. No code changed.

Lessons distilled

Reachability checks before patch application — saved a wasted cycle. Memorialized as feedback-rkvdec-patch-reachability.
The RK3399 HEVC handler is its own ecosystem. rkvdec-hevc.c is the standalone RK3399 HEVC code; the VDPU381/383 split files are separate. Mainline patches against the VDPU381/383 codebase don't reach RK3399 by construction.
Bommarito's methodology DOES transfer — KUnit harness + KASAN under a small wrapper of the assembly helper, with bounds-check fix. The RK3399 path (rkvdec-hevc.c::rkvdec_hevc_run_setup) has structurally analogous patterns (num_l0_refs loop, dpb[ref_idx_l0[i]] indirect indexing) where a similar KUnit could find OOB gaps. But libva submits spec-valid values, so a defensive patch wouldn't fix Bug 5 — it would harden the kernel against malicious userspace, distinct from the libva→rkvdec correctness problem.

Substrate state at iter10 close

Fork tip e0be4e6 (iter9 close). Unchanged.
Backend SHA a17e3c39…. Unchanged.
Kernel unchanged.
Diagnostic instrumentation preserved.

iter10 → iter11 handoff

Bug 5 (HEVC libva all-zero) is still open and is NOT a kernel-side OOB issue (which is what Bommarito's patch class addresses). It's a wire-protocol or control-payload issue similar in shape to Bug 4 but for HEVC.

iter11 candidate: replay the iter8/iter9 wire-byte methodology for HEVC. Specifically:

Deep strace HEVC libva vs kdirect.
Decode the V4L2_CID_STATELESS_HEVC_* control payloads (SPS, PPS, SLICE_PARAMS, SCALING_MATRIX, DECODE_PARAMS).
Find the first byte-level diff and narrow.

That's iter11 Phase 0 lock target. Route any resulting kernel work through kernel-agent per user directive.