fresnel-fourier/phase0_findings.md

Phase 0 — fresnel-fourier

This campaign's substrate, locked research question, and pre-Phase-1 inventory work for the fresnel/RK3399 leg of the libva-v4l2-request-fourier rollout. Peer to libva-multiplanar (ohm/RK3568); see README.md for topology.

Empirical corrections 2026-05-07 evening. This document was locked 2026-05-07 morning before the V4L2 surface was empirically enumerated and before iter6–iter8 work in libva-multiplanar was reflected in fresnel-fourier's substrate framing. Two facts have been corrected inline: (1) hantro-vpu-dec on RK3399 does not advertise H.264 — only MPEG-2 + VP8 (so Open Question #2 is null); (2) substrate is libva-v4l2-request-fourier master tip 65969da (iter8 Phase 4), not "iter5-end". See phase0_evidence/2026-05-07/v4l2_inventory_findings.md and phase0_evidence/2026-05-07/iter8_build_smoke.md for the empirical evidence.

Campaign-contained data discipline (governing rule)

Per feedback_dev_process.md Phase 0 + feedback_replicate_baseline_first.md:

This campaign acquires its own measurement data on fresnel in-session. libva-multiplanar's ohm-side findings (iter1..iter5 close documents, the patch-0011 cache-coherency lesson, the FFmpeg/GStreamer source-read references, the H.264 control-submission corrections) are documented for state carry-over — bug catalog, contract analysis, source pointers, build recipe shapes — but every claim that depends on hardware behaviour ("hantro accepts X", "the parser rejects Y", "DPB needs Z") is reference history until re-verified on RK3399's rkvdec and rk3399-vpu-dec blocks. Different silicon, possibly different firmware, definitely different driver path: don't assume.

Research question (LOCKED 2026-05-07)

"Make libva-v4l2-request-fourier accepted by VA-API consumers on fresnel (Rockchip RK3399), providing access to all decode-capable codecs the SoC ships — H.264 + HEVC + VP9 via the rkvdec block, MPEG-2 + VP8 via the hantro-vpu-dec block — end-to-end. Performance metrics are explicitly deferred to a follow-up iteration."

Pass/fail is boolean correctness per codec, not throughput:

• For each of {H.264, HEVC, VP9, MPEG-2, VP8}: does the consumer dlopen v4l2_request_drv_video.so, complete the VA-API surface lifecycle without falling back to SW, route to the correct V4L2 device node (rkvdec vs hantro-vpu-dec), and produce a CAPTURE buffer with non-zero, non-sentinel, semantically-correct decoded pixels (cache-coherency-safe verification, per the libva-multiplanar iter1 0011-sentinel lesson)?

If yes for a codec → that codec passes for the iteration. Frame-rate / CPU% / drops measurement is a separate iteration whose binding cells will be locked separately.

Mechanism the question targets

fresnel exposes two V4L2 stateless decode interfaces (verified 2026-05-07 via v4l2-ctl --info + --list-formats):

Node	Driver	DT compatible	Codecs (kernel CIDs)	Capture format
/dev/video3	rkvdec	rockchip,rk3399-vdec	H.264 (V4L2_PIX_FMT_H264_SLICE), HEVC (V4L2_PIX_FMT_HEVC_SLICE), VP9 (V4L2_PIX_FMT_VP9_FRAME)	NV12
/dev/video5	hantro-vpu	rockchip,rk3399-vpu	MPEG-2 (V4L2_PIX_FMT_MPEG2_SLICE), VP8 (V4L2_PIX_FMT_VP8_FRAME)	NV12

Both are V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE + V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE, both speak the request-API for control submission, both produce NV12 output. H.264 lands only on rkvdec — RK3399's hantro-vpu-dec does not advertise H.264 (corrected 2026-05-07 evening; the original lock cited H.264 on hantro, which was incorrect on running kernel 6.19.9-99-eos-arm). There is no two-block routing decision for H.264 on this SoC.

VA-API consumers (mpv, Firefox, chromium-fourier, vainfo) speak libva. The bridge they expect is libva-v4l2-request-fourier. As of libva-multiplanar iter5-close (2026-05-05), the fork is hardened against ohm's hantro-vpu (rk3568-vpu) — the test consumers run end-to-end on H.264 with all known iter1..iter4 bugs fixed. Whether that hardening transfers to:

1. fresnel's hantro-vpu-dec (rk3399-vpu-dec) — same driver family, possibly different SoC quirks
2. fresnel's rkvdec — a different driver entirely; new code paths in the libva backend may be needed
3. HEVC + VP9 + MPEG-2 + VP8 — codecs the fork has never decoded on real hardware to date

are open empirical questions that this campaign answers.

External pointers (carried from libva-multiplanar/phase0_findings.md):

• Mozilla bug 1833354 / 1965646 — Firefox HW decode on RK35xx via libva-v4l2-request.
• Bootlin upstream bootlin/libva-v4l2-request — dormant since 2021, sunxi-cedrus single-plane only.
• Linux kernel drivers/staging/media/rkvdec/ — the rkvdec driver source. H.264 + HEVC + VP9 control-submission contract reference for fresnel.
• Linux kernel drivers/media/platform/verisilicon/hantro_* — the hantro-vpu driver source. RK3399-vpu-dec MPEG-2/H.264/VP8 control protocol reference.
• Christian Hewitt patch series (LKML 2025/12/26/206) — RK3566 rkvdec2/vdpu346 (NOT applicable here; fresnel's rkvdec is mainline and works).

Predecessor close-out summary (state carry-over, not data)

From ~/src/libva-multiplanar/ — iter1..iter5 closed on ohm

The full chain of bugs found and fixed on ohm/RK3568 hantro is the substrate. Read in order:

• phase8_iteration1_close.md — iter1: contract trace lands; surface-export DMA-BUF lifecycle race; multi-resolution session corruption; Mesa WSI 64-pitch alignment; patch-0011 sentinel cache-coherency bug (the readback was lying — kernel was writing zeros).
• phase8_iteration2_close.md — iter2: Fix 1 (resolution-change format-cache invalidation), Fix 2 (DRM_FORMAT_MOD_INVALID conditional for non-64 pitch), Fix 3 (decoupled cap_pool with LRU recycling).
• phase8_iteration3_close.md — iter3: Firefox RDD sandbox fix (broker policy + seccomp + select() → poll() driver migration); frame-11 EINVAL reproduced deterministically.
• phase8_iteration4_close.md — iter4: frame-11 EINVAL fixed (DPB fields=FRAME_REF, fresh request_fd per frame, B-slice L1 reflist .fields copy-paste). mpv 2130 BeginPictures over 90s with 0 EINVAL.
• phase8_iteration5_close.md — iter5: ~339 lines of debug instrumentation removed; firefox-fourier 150.0.1-1.1 rebuilt non-PGO (21× smaller libxul, 2.7× faster decode); LAST_OUTPUT_* moved per-driver-data; mpv --vo=gpu 0 segfaults. iter6+ caveat: cap_pool resolution-change race latent under untested consumer probe patterns (Phase 5 sonnet C4).

The iter5-end fork is the substrate fresnel-fourier starts from. Corrected 2026-05-07 evening: the substrate is master tip 65969da (iter8 Phase 4) — libva-multiplanar continued past iter5 into iter6 (per-OUTPUT-slot REINIT request_fd binding), iter7 (slot-leak fix, cap_pool harness, msync verify, OUTPUT-pool teardown), and iter8 (perf binding cell harness) between the 2026-05-05 iter5 close and the 2026-05-07 fresnel-fourier scaffold. Building from master inherits all that hardening at zero cost. Codepaths not exercised on ohm — the rkvdec driver path entirely; HEVC + VP9 + MPEG-2 + VP8 control submission — are unknown territory and the most likely Phase 6 work.

From the libva-v4l2-request-fourier fork itself

Carry-over state (re-verify before treating as current):

• 12 commits ahead of bootlin a3c2476 plus 5 iterations of patches, all on master. Build harness: meson setup + ninja. Install: /usr/lib/dri/v4l2_request_drv_video.so. Activation: LIBVA_DRIVER_NAME=v4l2_request + LIBVA_V4L2_REQUEST_VIDEO_PATH=... + LIBVA_V4L2_REQUEST_MEDIA_PATH=....
• Build artefact size on ohm: ~265 KB .so (small — no distcc).

From the SDDM-greeter recovery plan (~/.claude/plans/dynamic-forging-piglet.md)

fresnel currently cannot graphical-login — /usr/bin/sddm-greeter-qt6 crash-loops with a qFatal() call from libqxcb-glx-integration.so after QOpenGLContext::create returned failure. Coredump bt confirmed; qt6-base-fourier is exonerated (downgrade to stock didn't fix it; bt path is GLX, our patches gate on ES 3+). The plan lists three step-1 invocations to capture the qFatal message, an iterative downgrade test against the recent pacman -Syyuu (mesa, qt6-base, qt6-declarative, libdrm-pinebookpro candidates), and two workaround options (Wayland greeter via SDDM Compositor, or TTY autologin with exec startplasma-wayland).

This recovery is fresnel-fourier Phase 0 task 1 (LOCKED 2026-05-07). Without graphical login there's no Plasma session to host VAAPI consumers under; headless Xvfb is documented (libva-multiplanar phase0) to give the wrong answer for Firefox.

Current fresnel state (verified 2026-05-07 over SSH)

• Kernel: linux-eos-arm 6.19.9-99 (custom OC kernel, ALARM/EndeavourOS-ARM). CONFIG_FTRACE=y, CONFIG_FUNCTION_TRACER=y, CONFIG_DYNAMIC_FTRACE=y, CONFIG_TRACING=y confirmed via zcat /proc/config.gz. /sys/kernel/tracing/ populated.
• V4L2 decode nodes: /dev/video3 (rkvdec), /dev/video5 (hantro-vpu-dec) — listed above.
• Encoder: /dev/video4 (hantro-vpu-enc, JPEG only).
• RGA: /dev/video0 (rockchip-rga, scaling/format-conversion accelerator — out of scope but worth knowing about).
• USB cameras: /dev/video{1,2} — irrelevant.
• Graphical session: DOWN — sddm-greeter-qt6 crash-loop. Phase 0 task 1.

In-scope (LOCKED 2026-05-07)

• libva-v4l2-request-fourier backend only.
• Hardware target: fresnel RK3399, both decode nodes (rkvdec + hantro-vpu-dec).
• Codecs: H.264 + HEVC + VP9 + MPEG-2 + VP8 — everything fresnel can hardware-decode. Each codec is a separate boolean cell.
• Test consumers (LOCKED 2026-05-07):
  • vainfo — smoke test, enumerates profiles + entrypoints per codec.
  • mpv --hwdec=vaapi (and --hwdec=vaapi-copy for the headless intermediate runs) — most directly testable end-to-end consumer for HW decode validation.
  • Firefox via media.ffmpeg.vaapi.enabled=true + LIBVA env — Mozilla bug 1965646 anchor.
  • chromium-fourier 149 — regression check inherited from libva-multiplanar.
  • Brave 1.89 — deferred (chromeos-pipeline gating is upstream of libva, same as libva-multiplanar's reasoning).
• Phase 1 success criterion: per-codec boolean correctness including cache-coherency-safe pixel-content verification (NOT the patch-0011 mmap pattern — that lesson is non-negotiable per libva-multiplanar iter1 close).
• Phase 0 task 1: recover fresnel SDDM (per dynamic-forging-piglet.md). Bookkept inside this campaign.

Out-of-scope (LOCKED 2026-05-07)

• Front-end libva.
• Other hardware (ohm in libva-multiplanar; ampere/boltzmann RK3588 are a separate not-yet-named campaign).
• AV1 (no decoder block on RK3399 supports it).
• Userspace bitstream parsing (kernel V4L2-stateless does this).
• Performance metrics (CPU%, fps, drops_60s, panfrost freq, GPU residency). Explicitly deferred. Do not lock Phase 1 binding cells around performance.
• KWin / panfrost / Mali-T860 work — orthogonal until proven otherwise. A kwin-fourier-fresnel campaign would be a separate decision.
• cros-codecs Rust replacement (per user_stance_rust.md).
• Bootlin / Collabora upstreaming — default-deferred (per feedback_no_upstream.md). Same discipline as libva-multiplanar.
• /dev/video0 rockchip-rga acceleration — not on the libva path.

Open questions before Phase 1 lock

1. SDDM recovery root cause — is the qFatal from a Mesa 26.0.5 panfrost regression on Mali-T860 (Midgard), Qt 6.11.0 xcb-glx-integration internals, or qt6-declarative? Phase 0 task 1 must determine this before Phase 1 locks, because it affects rig stability for measurement.
2. Two-block H.264 routing — RESOLVED 2026-05-07 evening (null). Empirical V4L2 enumeration showed hantro-vpu-dec on RK3399 does not advertise H.264. Only rkvdec accepts H.264 on this SoC. Single bind, no routing decision, single test cell per codec.
3. HEVC control-submission contract on rkvdec — the fork has had HEVC stripped per the libva-multiplanar build-cleanly stack. Re-introducing it for rkvdec is a code addition, not a regression-test. Phase 2 source-read of rkvdec H.265 path + FFmpeg's v4l2_request_hevc.c is gating.
4. VP9 control-submission contract on rkvdec — never been in the fork. Same Phase 2 source-read need: kernel rkvdec_vp9.c + FFmpeg v4l2_request_vp9.c. Plus VAAPI-VP9 consumer testing — does mpv exercise it? does Firefox?
5. MPEG-2 control-submission contract on hantro-vpu-dec — was iter1 backlog in libva-multiplanar, dropped at iter6 close because A55 CPU handles it fine. On RK3399 the A53 cluster is weaker; HW MPEG-2 decode might be more useful. Phase 2 source-read: kernel hantro_mpeg2.c + FFmpeg v4l2_request_mpeg2.c.
6. VP8 control-submission contract on hantro-vpu-dec — same shape: source-read kernel + FFmpeg. VP8 consumer matrix is thin; mostly Firefox / WebRTC paths.
7. rkvdec node device path env var — backend likely uses LIBVA_V4L2_REQUEST_VIDEO_PATH for one node. Two nodes may need a richer config: explicit per-codec routing or a probe loop. Phase 4 design decision.
8. firefox-fourier suitability for fresnel — the iter3 RDD sandbox patch is Mali-G52-rig-tested. Does it apply on Mali-T860 / panfrost on RK3399 unchanged, or are sandbox-side surprises lurking? Phase 0 inventory item: confirm the iter5 firefox-fourier package installs and runs on fresnel.
9. Test corpus per codec — bbb_1080p30_h264.mp4 (carryover). HEVC, VP9, MPEG-2, VP8 test clips need sourcing or generating. Doppler / /moviedata/fourier-test/ may already have some.
10. Brave 1.89 / chromium-fourier 149 binaries on aarch64 for fresnel — both already exist for ohm. Same aarch64 architecture; should run on fresnel without rebuild. Verify.

What Phase 0 will deliver

Independent of detail, Phase 0 produces:

1. fresnel SDDM recovered — either root-caused + downgraded to a working version, or workaround-A (Wayland greeter) installed. Findings logged in this directory under phase0_recovery_<date>.md. Loop back to phase 0 (not phase 1) if a root cause has campaign-relevant implications (e.g., a Mesa 26.0.5 panfrost regression that also affects video display paths).
2. fresnel V4L2 inventory written — v4l2-ctl enumeration of every codec controls + supported formats per node, captured to phase0_evidence/<date>/v4l2_inventory.txt. Already partially done in this document; expand to include --list-ctrls-menus, --all per node, and supported OUTPUT_MPLANE codec formats.
3. iter5-end fork built on fresnel and installed — meson setup && ninja directly on fresnel (no distcc per libva-multiplanar precedent). Install to /usr/lib/dri/v4l2_request_drv_video.so. Run vainfo, capture profile enumeration. This is the smoke test before any iteration even starts — does the existing fork even load on RK3399?
4. H.264 baseline trace — mpv --hwdec=vaapi-copy --frames=2 against bbb_1080p30_h264.mp4 on fresnel, ftrace-instrumented, with cache-coherency-safe pixel verification per the libva-multiplanar iter1 patch-0011 lesson. Compare control-submission ioctl sequence against ohm's iter5 trace. Identify divergences.
5. Per-codec test fixture inventory — what test clips exist for HEVC / VP9 / MPEG-2 / VP8, what we need to source or generate, and where they'll live.
6. Phase 0 baseline anchor — for the boolean-success criterion per codec, the anchor is again a contract trace, not a metric distribution. Capture the V4L2 request-API ioctl sequence on a known-working consumer per codec on fresnel — chromium-fourier 149 binary is the cross-validator (does it engage rkvdec for HEVC? VP9? for the codecs it natively uses) — for 1 frame's decode each, in-session, before any fork modifications.

Phase 0 is not the place to start coding HEVC/VP9 support in the fork. That work, if it lands, is Phase 4 onwards of an iteration whose Phase 1 has already locked the codec scope.

Source-read references (carried + extended for fresnel)

For Phase 2 source-read and Phase 6 implementation:

Carried from libva-multiplanar (H.264 path):

• FFmpeg — libavcodec/v4l2_request.c, v4l2_request_buffer.c, v4l2_request_h264.c. Active downstream: code.ffmpeg.org/Kwiboo/FFmpeg.git branch v4l2-request-n8.1.
• GStreamer v4l2codecs — gst-plugins-bad/sys/v4l2codecs/gstv4l2decoder.c + gstv4l2codecsh264dec.c.
• Chromium — media/gpu/v4l2/v4l2_video_decoder_backend_stateless.{h,cc} + v4l2_queue.cc.

Added for fresnel (HEVC/VP9/MPEG-2/VP8):

• FFmpeg v4l2_request_hevc.c — HEVC slice-mode control submission.
• FFmpeg v4l2_request_vp9.c — VP9 frame control submission.
• FFmpeg v4l2_request_mpeg2.c — MPEG-2 slice control submission.
• FFmpeg v4l2_request_vp8.c — VP8 frame control submission.
• GStreamer — gstv4l2codecsh265dec.c, gstv4l2codecsvp9dec.c, gstv4l2codecsmpeg2dec.c, gstv4l2codecsvp8dec.c.
• Linux kernel — drivers/staging/media/rkvdec/{rkvdec_h264.c, rkvdec_hevc.c, rkvdec_vp9.c} for the rkvdec block; drivers/media/platform/verisilicon/hantro_h264.c, hantro_mpeg2.c, hantro_vp8.c for the hantro block.

Test fixtures

• H.264: bbb_1080p30_h264.mp4 (carry-over from fourier_attribution / libva-multiplanar). Pull via hertz lxc file pull from data:/moviedata/fourier-test/ or copy from ohm's /home/mfritsche/fourier-test/.
• HEVC, VP9, MPEG-2, VP8: TBD per Phase 0 deliverable #5. Likely Big Buck Bunny in those codecs from the same operator-internal /moviedata/ if available; otherwise generate via ffmpeg -i bbb_1080p30_h264.mp4 -c:v <codec> -y bbb_1080p30_<codec>.<ext>.

Build + install on fresnel

• meson setup build && ninja -C build directly on fresnel. Same default as libva-multiplanar: small library, no distcc.
• Install path: /usr/lib/dri/v4l2_request_drv_video.so.
• Activation: LIBVA_DRIVER_NAME=v4l2_request + (per-node env vars; exact layout depends on Phase 4 device-routing decision).
• Once a working state is reached: package as marfrit/libva-v4l2-request-fourier-fresnel next to existing fourier packages — out of Phase 1 scope, post-Phase-7.

Cross-link to libva-multiplanar

This campaign is a peer, not a child. libva-multiplanar Phase 8 closes happen iteration-by-iteration on ohm independently of fresnel-fourier progress. Cross-references:

• libva-multiplanar/README.md:87 names fresnel + RK3588 as "future iterations after ohm path is solid" — that promise is now formally taken up here.
• Code-side work in the shared fork (../libva-multiplanar/libva-v4l2-request-fourier/) lands either as #ifdef/runtime-detected paths on master (preferred when behavior is RK3399-specific but doesn't break ohm) or on a feature branch (when scope diverges sharply). Phase 2 source-read of each iteration decides.
• libva-multiplanar's iter1 patch-0011 cache-coherency lesson is non-negotiable for fresnel-fourier: pixel-content verification must use cache-flushed reads (e.g. mpv --vo=image-sequence, or DMA-BUF GL import via mpv --vo=gpu, or a small C reproducer with msync(MS_SYNC|MS_INVALIDATE)), never the cached mmap readback pattern.