Files
fresnel-fourier/phase8_iteration35_close.md
marfrit 7c06c519e7 iter35 close: MPEG-2 verified libva-correct; HW IDCT precision intrinsic
Investigation requested via /user 'mpeg2 next' after iter33 closed
MPEG-2 at libva==kdirect with libva!=SW gap. iter35 verifies:

1. libva==kdirect holds across:
   - BBB 720p MPEG-2 240F (sha a75015b10fe205ea)
   - Synthetic testsrc 720p MPEG-2 48F (sha 5557dae9fa99d01b)
   Rules out fixture coincidence.

2. v4l2 ctrl structs (SEQUENCE + PICTURE + QUANTISATION) semantically
   correct per V4L2 spec via iter35 env-gated DIAG dump. Intra Q
   matrix in zigzag scan order; flags = PROGRESSIVE+FRAME_PRED_DCT.

3. HW vs SW gap quantified:
   - mean byte-diff = 0.015, max = 3
   - SSIM = 0.999923 (visually identical)
   Within IEEE 1180 / ISO 13818-2 Annex A MPEG-2 IDCT precision
   tolerance. Not fixable at V4L2/libva layer.

ffmpeg's own SW IDCT options (-idct auto/int/simple/...) produce 4
distinct hashes for the same bitstream — libavcodec internally has
the same precision divergence between impls. Hantro HW IDCT is yet
another impl, not matching any SW.

Backend tip 48fd028 (added env-gated LIBVA_MPEG2_DUMP_FRAME).
Memory: feedback_mpeg2_hw_sw_idct_precision.md added.
2026-05-14 17:58:31 +00:00

3.7 KiB
Raw Permalink Blame History

Iteration 35 — Phase 8 (close): MPEG-2 investigation — libva CORRECT, HW IDCT precision intrinsic

Closes 2026-05-14, sixth campaign-day milestone after iter34 (kernel 7.0-14 clean ship). User asked "mpeg2 next" after 5/5 PASS; this iteration verifies libva MPEG-2 is genuinely correct (not a coincidence) and documents why the HW != SW gap is unfixable from libva.

Goal

After iter33 closed MPEG-2 at "libva==kdirect bit-exact, libva!=SW by ≤1 LSB/67 px (HW IDCT precision)", verify:

  1. The libva==kdirect bit-exact property holds across non-BBB content (rule out fixture coincidence).
  2. libva is sending correct v4l2 ctrl structs to the kernel.
  3. The HW != SW divergence stays bounded (within IEEE 1180 MPEG-2 IDCT tolerance).

Verification

Multi-fixture libva==kdirect:

Source Length libva sha-16 kdirect sha-16 Match
BBB 720p mpeg2 (existing fixture) 10F 95c5905890c937d4 95c5905890c937d4
BBB 720p mpeg2 (full) 240F a75015b10fe205ea a75015b10fe205ea
Synthetic testsrc 720p MPEG-2 (b=4M, B-frames=2, GOP=12) 48F 5557dae9fa99d01b 5557dae9fa99d01b

libva backend produces bit-exact output to kdirect (ffmpeg-v4l2request) across multiple distinct MPEG-2 streams. Not a fixture coincidence.

v4l2 ctrl struct verification (iter35 DIAG LIBVA_MPEG2_DUMP_FRAME=1):

mpeg2: SEQ size=12  00 05 d0 02 00 00 10 00 00 00 01 01      ← 1280x720, vbv=1M, chroma=1, flags=1 (progressive)
mpeg2: PIC size=32 [0..32]=… 82 00 00 00 0f 0f 0f 0f 01 03   ← flags=0x82 (PROGRESSIVE+FRAME_PRED_DCT), I-pic, FRAME
mpeg2: PIC frame2…  82 00 00 00 01 01 0f 0f 02 03 00 00      ← f_code 1/1, P-pic
mpeg2: QUANT size=256 [0..16]=08 10 10 13 10 13 16 16 …       ← MPEG-2 default intra Q in ZIGZAG order
mpeg2: forward_ref_ts: 0, 1000, 2000, 3000, …                ← α-7 counter scheme; CAPTURE-lookup works

All fields semantically correct per V4L2 spec (v4l2-controls.h:1990-2090). Intra Q matrix in zigzag-scan order as required.

HW-vs-SW divergence quantified:

Metric Value
Mean byte-diff (sampled, 240F = 331M bytes) 0.015
Max byte-diff 3
Fraction of bytes nonzero-diff (sampled) ~1.5%
SSIM_Y 0.999919
SSIM_U 0.999922
SSIM_V 0.999941
SSIM overall 0.999923

≤3 LSB / pixel and SSIM > 0.9999 — well within IEEE 1180 / ISO/IEC 13818-2 Annex A "implementation-defined IDCT precision" tolerance. Visually identical to SW reference.

ffmpeg's SW IDCT options also disagree with HW:

Sweep of -idct auto|int|simple|simplemmx|simplearm|faani|simpleneon: produces 4 distinct YUV hashes. None match HW. None match each other except aliases. This confirms IDCT precision differences are expected between any two MPEG-2 implementations.

Conclusion

MPEG-2 through libva is correct:

  • Bit-exact with the canonical kdirect HW reference path across 3 distinct fixtures (288 frames total).
  • All V4L2 ctrl struct fields semantically valid per spec.
  • HW vs SW residual gap is intrinsic to MPEG-2 IDCT precision tolerance; not fixable at the V4L2/libva layer.

5/5 codecs PASS for the libva-vs-kdirect correctness contract. MPEG-2 retains its "L==K, L!=SW(IDCT precision)" status from iter33; no further fix possible or warranted.

Substrate state at iter35 close

  • Backend fork tip 48fd028 (α-25 + α-29 + α-30 + iter29/30/33/35 env-gated DIAG probes).
  • Kernel linux-fresnel-fourier 7.0-14 clean.
  • iter35 added LIBVA_MPEG2_DUMP_FRAME=1 env-gated DIAG to mpeg2.c — env-gated, no behavior change without env.

Memory entry added

feedback_mpeg2_hw_sw_idct_precision.md — HW MPEG-2 IDCT differs from libavcodec SW MPEG-2 IDCT by ≤3 LSB. Per MPEG-2 spec IEEE 1180 tolerance. Not a libva-fixable bug.