Files
fresnel-fourier/phase0_findings_iter11.md
T
marfrit 5f94d7a9ae iter10 close + iter11 Phase 0: pivot to HEVC wire-byte diff for Bug 5
iter10 closed negative at Phase 0 (Bommarito unreachable on RK3399).
Saved kernel build + reboot cycle by source-tree reachability check.

iter11 opens with Bug 5 (HEVC libva all-zero) as research target.
Replay iter8/iter9 methodology: deep strace HEVC libva vs kdirect,
decode V4L2_CID_STATELESS_HEVC_* control payload bytes, find the
diff that causes rkvdec to produce all-zero output for libva while
kdirect's submission produces correct decode.

In scope: src/h265.c (libva HEVC), Phase 3 strace + byte-decode.
Out of scope: ext_sps_st/lt_rps (VDPU381/383-only, not RK3399),
kernel patches until empirical evidence of a kernel-side gap.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-14 01:37:45 +00:00

5.6 KiB
Raw Blame History

Iteration 11 — Phase 0 (substrate / motivation / inventory) → Phase 1 lock

Opens 2026-05-14 after iter10 negative close (phase8_iteration10_close.md). Bommarito's patch confirmed unreachable on RK3399. Bug 5 (HEVC libva all-zero) remains open; not kernel-OOB.

Pivot: replay the iter8/iter9 methodology for HEVC

The same approach that surfaced Bug 4's structure (kernel decodes wrong/incomplete, libva wire-payload search-space exhausted with 6 eliminations, real cause still TBD) can be applied to Bug 5. Phase 3 in iter8 already showed libva HEVC produces all-zero output (06b2c5a0…) while kdirect HEVC works. Same kernel, same readback path. So the diff is in the libva submission.

Locked research question (iter11, 2026-05-14)

"What specific byte(s) in libva's HEVC control submission cause rkvdec to produce all-zero CAPTURE output while kdirect's submission produces correct decode? After fix: libva_hevc.yuv == kdirect_hevc.yuv byte-identical for bbb_720p10s_hevc.mp4 3-frame test."

Pass/fail (boolean)

  1. HEVC libva == kdirect: cmp -s libva_hevc.yuv kdirect_hevc.yuv returns 0.
  2. H.264 unchanged: 71ac099b… (Bug 4 still open, no regression).
  3. VP9 unchanged: 4f1565e8… PASS.
  4. MPEG-2 unchanged: 19eefbf4… PASS.
  5. VP8 unchanged: bcc57ed5… (Bug 6 still open, no regression).
  6. Control-payload anchors hold for 4 non-HEVC codecs.

Clean iter11 close = all 6 PASS. Partial close acceptable if Bug 5 narrowed via wire-byte elimination.

Substrate state at iter11 open

Property Value
Kernel linux-fresnel-fourier 7.0-1 (unchanged)
Fork tip e0be4e6 (iter9 close)
Backend SHA a17e3c39… on fresnel
Diagnostic γ + IMP-1 + α-2 + α-7 instrumentation preserved (env-gated off by default)
HEVC anchor 06b2c5a0c01e515d009c0bfbe0e61fafb105a54da5ec621104915cd5949849e8 (libva, all-zero)
HEVC kdirect TBD (need to capture this iter; iter8 didn't do kdirect HEVC)

Mechanism the question targets

HEVC kernel UAPI controls (linux/v4l2-controls.h):

  • V4L2_CID_STATELESS_HEVC_SPS (0xa40a95)
  • V4L2_CID_STATELESS_HEVC_PPS (0xa40a96)
  • V4L2_CID_STATELESS_HEVC_SLICE_PARAMS (0xa40a97)
  • V4L2_CID_STATELESS_HEVC_SCALING_MATRIX (0xa40a98)
  • V4L2_CID_STATELESS_HEVC_DECODE_PARAMS (0xa40a99)
  • V4L2_CID_STATELESS_HEVC_DECODE_MODE / V4L2_CID_STATELESS_HEVC_START_CODE (0xa40a9a / 0xa40a9b)
  • VDPU381/383-specific: V4L2_CID_STATELESS_HEVC_EXT_SPS_ST_RPS / _LT_RPS (0xa40a... — newer, not on RK3399)

Bug 5 surfaces as: every libva HEVC DQBUF returns without FLAG_ERROR (kernel accepts), CAPTURE bytes all-zero. kdirect submits structurally same controls and gets correct decode.

Scope locks

In scope:

  • src/h265.c (libva backend HEVC handler).
  • src/picture.c HEVC dispatch.
  • Phase 3 strace + byte-decode of HEVC controls.
  • Comparison against kdirect's strace.

Out of scope:

  • Bug 4 / Bug 6 / iter4-B1b / other backlog.
  • Kernel patches (until empirical evidence of a specific kernel-side gap, in which case route via kernel-agent).
  • ext_sps_st/lt_rps (VDPU381/383-only).

Phase 2 source-read targets

  • src/h265.c — full file. Map per-frame HEVC pipeline (BeginPicture → controls → QBUF).
  • src/picture.c::codec_set_controls — VAProfileHEVCMain dispatch.
  • src/picture.c::codec_store_buffer — HEVCMain buffer storage cases.
  • Kernel UAPI struct v4l2_ctrl_hevc_sps/pps/decode_params/slice_params/scaling_matrix.
  • Reference: ~/src/aur/ffmpeg-git/src/FFmpeg/libavcodec/v4l2_request_hevc.c (kdirect HEVC fill_*).

Phase 3 plan

  1. Run libva HEVC with strace -s 8192 → capture full control payloads.
  2. Run kdirect HEVC with strace -s 8192 → same.
  3. Byte-decode SPS, PPS, DECODE_PARAMS, SCALING_MATRIX, SLICE_PARAMS structures (analogous to what iter9 did for H.264 DPB).
  4. Identify byte-level diffs.
  5. For each diff, check rkvdec source for whether the field is load-bearing.

Phase 4 plan shape (predicted)

Contingent on Phase 3 findings. Three likely outcomes:

  • Outcome A: Single field diff in HEVC SPS / PPS / DECODE_PARAMS. Mechanical libva-side fix (~5-15 LOC in h265.c).
  • Outcome B: Multi-field divergence. Field-by-field amendment of libva's HEVC control fill.
  • Outcome C: No meaningful diff — controls are identical, but bytes-in-OUTPUT-buffer differ. Pivot to OUTPUT bitstream byte-level inspection.

Predicted iter11 cadence

  • Phase 0: this doc.
  • Phase 2: source-read h265.c. ~30 min.
  • Phase 3: empirical strace + decode. ~60 min.
  • Phase 4: targeted plan. ~20 min.
  • Phase 5: sonnet-architect review. ~30 min.
  • Phase 6: implement. ~30 min.
  • Phase 7: verify. ~30 min.
  • Phase 8: close. ~10 min.

Total: ~3.5 hours wallclock contingent on fresnel uptime.

What "iter11 PASS" looks like

If Bug 5 fixed:

  • iter11 PASS. HEVC row goes from DEGRADED (transitive-only) to PASS direct.
  • Codec scoreboard: 3 direct PASS (VP9, MPEG-2, HEVC) + 2 PARTIAL (H.264, VP8).

If iter11 PARTIAL:

  • Bug 5 narrowed (some hypotheses eliminated). iter12 candidates derived from narrowing.

Memory-rule guidance for this iter

  • feedback_review_empirical_over_theoretical.md — apply Phase 5 review.
  • feedback_wire_vs_behavior.md — Phase 7 must verify criterion-1 hash test, not just wire-byte match.
  • feedback_unconditional_codec_state.md — any libva backend change must be profile-gated (HEVC-only).
  • feedback_per_driver_kludge_gating.md — any driver-specific workaround should gate on driver_kind.
  • feedback_rkvdec_patch_reachability.md — if iter11 ends up proposing a kernel patch, verify reachability before submission.