iter10 close + iter11 Phase 0: pivot to HEVC wire-byte diff for Bug 5
iter10 closed negative at Phase 0 (Bommarito unreachable on RK3399). Saved kernel build + reboot cycle by source-tree reachability check. iter11 opens with Bug 5 (HEVC libva all-zero) as research target. Replay iter8/iter9 methodology: deep strace HEVC libva vs kdirect, decode V4L2_CID_STATELESS_HEVC_* control payload bytes, find the diff that causes rkvdec to produce all-zero output for libva while kdirect's submission produces correct decode. In scope: src/h265.c (libva HEVC), Phase 3 strace + byte-decode. Out of scope: ext_sps_st/lt_rps (VDPU381/383-only, not RK3399), kernel patches until empirical evidence of a kernel-side gap. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,113 @@
|
||||
# Iteration 11 — Phase 0 (substrate / motivation / inventory) → Phase 1 lock
|
||||
|
||||
Opens 2026-05-14 after iter10 negative close ([`phase8_iteration10_close.md`](phase8_iteration10_close.md)). Bommarito's patch confirmed unreachable on RK3399. Bug 5 (HEVC libva all-zero) remains open; not kernel-OOB.
|
||||
|
||||
## Pivot: replay the iter8/iter9 methodology for HEVC
|
||||
|
||||
The same approach that surfaced Bug 4's structure (kernel decodes wrong/incomplete, libva wire-payload search-space exhausted with 6 eliminations, real cause still TBD) can be applied to Bug 5. Phase 3 in iter8 already showed libva HEVC produces all-zero output (`06b2c5a0…`) while kdirect HEVC works. Same kernel, same readback path. So the diff is in the libva submission.
|
||||
|
||||
## Locked research question (iter11, 2026-05-14)
|
||||
|
||||
> *"What specific byte(s) in libva's HEVC control submission cause rkvdec to produce all-zero CAPTURE output while kdirect's submission produces correct decode? After fix: `libva_hevc.yuv == kdirect_hevc.yuv` byte-identical for `bbb_720p10s_hevc.mp4` 3-frame test."*
|
||||
|
||||
### Pass/fail (boolean)
|
||||
|
||||
1. **HEVC libva == kdirect**: `cmp -s libva_hevc.yuv kdirect_hevc.yuv` returns 0.
|
||||
2. **H.264 unchanged**: `71ac099b…` (Bug 4 still open, no regression).
|
||||
3. **VP9 unchanged**: `4f1565e8…` PASS.
|
||||
4. **MPEG-2 unchanged**: `19eefbf4…` PASS.
|
||||
5. **VP8 unchanged**: `bcc57ed5…` (Bug 6 still open, no regression).
|
||||
6. **Control-payload anchors hold** for 4 non-HEVC codecs.
|
||||
|
||||
Clean iter11 close = all 6 PASS. Partial close acceptable if Bug 5 narrowed via wire-byte elimination.
|
||||
|
||||
## Substrate state at iter11 open
|
||||
|
||||
| Property | Value |
|
||||
|---|---|
|
||||
| Kernel | `linux-fresnel-fourier 7.0-1` (unchanged) |
|
||||
| Fork tip | `e0be4e6` (iter9 close) |
|
||||
| Backend SHA | `a17e3c39…` on fresnel |
|
||||
| Diagnostic | γ + IMP-1 + α-2 + α-7 instrumentation preserved (env-gated off by default) |
|
||||
| HEVC anchor | `06b2c5a0c01e515d009c0bfbe0e61fafb105a54da5ec621104915cd5949849e8` (libva, all-zero) |
|
||||
| HEVC kdirect | TBD (need to capture this iter; iter8 didn't do kdirect HEVC) |
|
||||
|
||||
## Mechanism the question targets
|
||||
|
||||
HEVC kernel UAPI controls (linux/v4l2-controls.h):
|
||||
- `V4L2_CID_STATELESS_HEVC_SPS` (0xa40a95)
|
||||
- `V4L2_CID_STATELESS_HEVC_PPS` (0xa40a96)
|
||||
- `V4L2_CID_STATELESS_HEVC_SLICE_PARAMS` (0xa40a97)
|
||||
- `V4L2_CID_STATELESS_HEVC_SCALING_MATRIX` (0xa40a98)
|
||||
- `V4L2_CID_STATELESS_HEVC_DECODE_PARAMS` (0xa40a99)
|
||||
- `V4L2_CID_STATELESS_HEVC_DECODE_MODE` / `V4L2_CID_STATELESS_HEVC_START_CODE` (0xa40a9a / 0xa40a9b)
|
||||
- VDPU381/383-specific: `V4L2_CID_STATELESS_HEVC_EXT_SPS_ST_RPS` / `_LT_RPS` (0xa40a... — newer, not on RK3399)
|
||||
|
||||
Bug 5 surfaces as: every libva HEVC DQBUF returns without FLAG_ERROR (kernel accepts), CAPTURE bytes all-zero. kdirect submits structurally same controls and gets correct decode.
|
||||
|
||||
## Scope locks
|
||||
|
||||
**In scope**:
|
||||
- `src/h265.c` (libva backend HEVC handler).
|
||||
- `src/picture.c` HEVC dispatch.
|
||||
- Phase 3 strace + byte-decode of HEVC controls.
|
||||
- Comparison against kdirect's strace.
|
||||
|
||||
**Out of scope**:
|
||||
- Bug 4 / Bug 6 / iter4-B1b / other backlog.
|
||||
- Kernel patches (until empirical evidence of a specific kernel-side gap, in which case route via kernel-agent).
|
||||
- ext_sps_st/lt_rps (VDPU381/383-only).
|
||||
|
||||
## Phase 2 source-read targets
|
||||
|
||||
- `src/h265.c` — full file. Map per-frame HEVC pipeline (BeginPicture → controls → QBUF).
|
||||
- `src/picture.c::codec_set_controls` — VAProfileHEVCMain dispatch.
|
||||
- `src/picture.c::codec_store_buffer` — HEVCMain buffer storage cases.
|
||||
- Kernel UAPI `struct v4l2_ctrl_hevc_sps/pps/decode_params/slice_params/scaling_matrix`.
|
||||
- Reference: `~/src/aur/ffmpeg-git/src/FFmpeg/libavcodec/v4l2_request_hevc.c` (kdirect HEVC fill_*).
|
||||
|
||||
## Phase 3 plan
|
||||
|
||||
1. Run libva HEVC with strace -s 8192 → capture full control payloads.
|
||||
2. Run kdirect HEVC with strace -s 8192 → same.
|
||||
3. Byte-decode SPS, PPS, DECODE_PARAMS, SCALING_MATRIX, SLICE_PARAMS structures (analogous to what iter9 did for H.264 DPB).
|
||||
4. Identify byte-level diffs.
|
||||
5. For each diff, check rkvdec source for whether the field is load-bearing.
|
||||
|
||||
## Phase 4 plan shape (predicted)
|
||||
|
||||
Contingent on Phase 3 findings. Three likely outcomes:
|
||||
|
||||
- **Outcome A**: Single field diff in HEVC SPS / PPS / DECODE_PARAMS. Mechanical libva-side fix (~5-15 LOC in h265.c).
|
||||
- **Outcome B**: Multi-field divergence. Field-by-field amendment of libva's HEVC control fill.
|
||||
- **Outcome C**: No meaningful diff — controls are identical, but bytes-in-OUTPUT-buffer differ. Pivot to OUTPUT bitstream byte-level inspection.
|
||||
|
||||
## Predicted iter11 cadence
|
||||
|
||||
- Phase 0: this doc.
|
||||
- Phase 2: source-read h265.c. ~30 min.
|
||||
- Phase 3: empirical strace + decode. ~60 min.
|
||||
- Phase 4: targeted plan. ~20 min.
|
||||
- Phase 5: sonnet-architect review. ~30 min.
|
||||
- Phase 6: implement. ~30 min.
|
||||
- Phase 7: verify. ~30 min.
|
||||
- Phase 8: close. ~10 min.
|
||||
|
||||
Total: ~3.5 hours wallclock contingent on fresnel uptime.
|
||||
|
||||
## What "iter11 PASS" looks like
|
||||
|
||||
If Bug 5 fixed:
|
||||
- iter11 PASS. HEVC row goes from DEGRADED (transitive-only) to PASS direct.
|
||||
- Codec scoreboard: 3 direct PASS (VP9, MPEG-2, HEVC) + 2 PARTIAL (H.264, VP8).
|
||||
|
||||
If iter11 PARTIAL:
|
||||
- Bug 5 narrowed (some hypotheses eliminated). iter12 candidates derived from narrowing.
|
||||
|
||||
## Memory-rule guidance for this iter
|
||||
|
||||
- `feedback_review_empirical_over_theoretical.md` — apply Phase 5 review.
|
||||
- `feedback_wire_vs_behavior.md` — Phase 7 must verify criterion-1 hash test, not just wire-byte match.
|
||||
- `feedback_unconditional_codec_state.md` — any libva backend change must be profile-gated (HEVC-only).
|
||||
- `feedback_per_driver_kludge_gating.md` — any driver-specific workaround should gate on driver_kind.
|
||||
- `feedback_rkvdec_patch_reachability.md` — if iter11 ends up proposing a kernel patch, verify reachability before submission.
|
||||
@@ -0,0 +1,33 @@
|
||||
# Iteration 10 — Phase 8 (close)
|
||||
|
||||
Closes 2026-05-14. iter10 = "is Bommarito's May 13 rkvdec HEVC bounds-check the Bug 5 fix vehicle?" Answered NO at Phase 0 via reachability check. Iteration ran 1 phase, ~20 minutes wallclock.
|
||||
|
||||
## Outcome
|
||||
|
||||
Phase 0 reachability analysis showed `rkvdec_hevc_assemble_hw_rps()` is called only from `rkvdec-vdpu381-hevc.c` (RK3576) and `rkvdec-vdpu383-hevc.c` (RK3588). RK3399's `rk3399_variant_ops` routes HEVC through `rkvdec-hevc.c` which doesn't touch the patched function. Bommarito's patch is sound (KASAN-driven, narrow, low-risk) but inert on fresnel.
|
||||
|
||||
Iter10 saved a kernel build + reboot cycle by Phase-0 source check. No code changed.
|
||||
|
||||
## Lessons distilled
|
||||
|
||||
1. **Reachability checks before patch application** — saved a wasted cycle. Memorialized as [[feedback-rkvdec-patch-reachability]].
|
||||
2. **The RK3399 HEVC handler is its own ecosystem**. `rkvdec-hevc.c` is the standalone RK3399 HEVC code; the VDPU381/383 split files are separate. Mainline patches against the VDPU381/383 codebase don't reach RK3399 by construction.
|
||||
3. **Bommarito's methodology DOES transfer** — KUnit harness + KASAN under a small wrapper of the assembly helper, with bounds-check fix. The RK3399 path (`rkvdec-hevc.c::rkvdec_hevc_run_setup`) has structurally analogous patterns (`num_l0_refs` loop, `dpb[ref_idx_l0[i]]` indirect indexing) where a similar KUnit could find OOB gaps. **But libva submits spec-valid values**, so a defensive patch wouldn't fix Bug 5 — it would harden the kernel against malicious userspace, distinct from the libva→rkvdec correctness problem.
|
||||
|
||||
## Substrate state at iter10 close
|
||||
|
||||
- Fork tip `e0be4e6` (iter9 close). Unchanged.
|
||||
- Backend SHA `a17e3c39…`. Unchanged.
|
||||
- Kernel unchanged.
|
||||
- Diagnostic instrumentation preserved.
|
||||
|
||||
## iter10 → iter11 handoff
|
||||
|
||||
Bug 5 (HEVC libva all-zero) is still open and is **NOT** a kernel-side OOB issue (which is what Bommarito's patch class addresses). It's a wire-protocol or control-payload issue similar in shape to Bug 4 but for HEVC.
|
||||
|
||||
iter11 candidate: replay the iter8/iter9 wire-byte methodology for HEVC. Specifically:
|
||||
- Deep strace HEVC libva vs kdirect.
|
||||
- Decode the `V4L2_CID_STATELESS_HEVC_*` control payloads (SPS, PPS, SLICE_PARAMS, SCALING_MATRIX, DECODE_PARAMS).
|
||||
- Find the first byte-level diff and narrow.
|
||||
|
||||
That's iter11 Phase 0 lock target. Route any resulting kernel work through kernel-agent per user directive.
|
||||
Reference in New Issue
Block a user