ampere-fourier/phase0_findings.md

Phase 0 — Substrate / Motivation / Inventory

Closed 2026-05-16. Iteration 1 of the ampere-fourier campaign. Locks the research question, captures the substrate, anchors the in-session baseline, surfaces open questions.

Research question

Can the libva-v4l2-request-fourier backend (iter38b, fresnel-certified) deliver HW decode on RK3588 for every codec the SoC actually exposes via mainline-rc3 V4L2 stateless — and where it can't, what's the minimal kernel-agent experiment / backend iteration that closes each remaining codec?

Operator-supplied mechanism (verbatim 2026-05-16):

ampere should not have fourier-specific kernel patches. If any are necessary, those should be added as an experiment with kernel-agent.

So the campaign is userspace-first: validate the codecs that the clean kernel supports, and for everything else, file kernel-agent experiments (separate fleet target, not the baseline).

Hardware substrate

Property	Value
Board	CoolPi CM5 GenBook
Compatible chain	coolpi,pi-cm5-genbook coolpi,pi-cm5 rockchip,rk3588
Booted kernel	7.0.0-rc3-devices+ from linux-ampere-fourier 7.0rc3.kafr1-1 (vanilla torvalds v7.0-rc3 + ampere DTS/board patches; no codec patches)
Boot stack	extlinux entry arch_mainline
kernel cmdline	… cma=256M cgroup_enable=cpuset cgroup_memory=1 cgroup_enable=memory coherent_pool=2M memtest=2
libva backend	libva-v4l2-request-fourier 1.0.0.r348.7ac934e-1 packaged BUT broken on HEVC (campaign sibling issue marfrit-packages#17). Working hand-build (md5 0c9a7efaab6a31b74536ac1abff18dfa, 484 800 B) reinstalled via sudo install -m644 over the package's file.
ffmpeg	ffmpeg-v4l2-request-fourier 2:8.1.r123329.b57fbbe-4
mpv	mpv-fourier 1:0.41.0-10 (iter2, ships --hwdec=v4l2request-copy)
libva runtime	libva 2.23.0-1, libva-utils 2.22.0-1, libdrm 2.4.133-1, linux-api-headers 6.19-1

/usr/include/linux/v4l2-controls.h sha256 f70d5224… — same as fresnel and fermi (verified during marfrit-packages#17 forensics). V4L2 control struct layout is byte-identical across the fleet.

V4L2 decoder topology (v4l2-ctl -d /dev/videoN --info)

Node	Media	Driver	Card type (DT compatible)	OUTPUT pixfmts
/dev/video0	—	rockchip-rga	rockchip-rga	BA24 / AR24 / XR24 / RGB3 / BGR3 / AR12 / AR15 / RGBP / XR15 / XB24 / AB24 / BX24 / RA24 / BA15 / RA15 / YU12 / YV12 / NV12 / NV21 / YUYV / UYVY / NV16 / NV61 / YM12 / YM21 / NM12 / NM21 / YM16 / YM61 / TL44 / RM12 — 2D blit, out of scope
/dev/video1	/dev/media0	rkvdec	(no DT card name in this kernel)	S265 (HEVC), S264 (H.264)
/dev/video2	/dev/media1	hantro-vpu	rockchip,rk3568-vpu-dec	S264 (H.264), MG2S (MPEG-2), VP8F (VP8)
/dev/video3	/dev/media2	hantro-vpu	rockchip,rk3588-vepu121-enc	YM12 / NM12 / YUYV / UYVY — encoder, out of scope
/dev/video4	/dev/media3	hantro-vpu	rockchip,rk3588-av1-vpu-dec	AV1F (AV1)
/dev/video5, /dev/video6	/dev/media4	uvcvideo	USB Camera	not decoders

Three independent decoder cores. No VP9 pixfmt is exposed by any node on this kernel.

dmesg at boot also reports four … missing multi-core support, ignoring this instance lines (fdc40100, fdba4000, fdba8000, fdbac000) — RK3588 has additional rkvdec+hantro cores that mainline doesn't bind because the multi-core glue isn't implemented. So the advertised throughput cap is single-core per block, not full-SoC.

libva backend behavior

$ LIBVA_DRIVER_NAME=v4l2_request vainfo
v4l2-request: auto-selected codec device: /dev/video1 + /dev/media0
v4l2-request: iter38: also opened hantro-vpu decoder at /dev/video2 + /dev/media1
vainfo: VA-API version: 1.23 (libva 2.22.0)
vainfo: Driver version: v4l2-request
vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            : VAEntrypointVLD
      VAProfileMPEG2Main              : VAEntrypointVLD
      VAProfileH264Main               : VAEntrypointVLD
      VAProfileH264High               : VAEntrypointVLD
      VAProfileH264ConstrainedBaseline: VAEntrypointVLD
      VAProfileH264MultiviewHigh      : VAEntrypointVLD
      VAProfileH264StereoHigh         : VAEntrypointVLD
      VAProfileHEVCMain               : VAEntrypointVLD
      VAProfileVP8Version0_3          : VAEntrypointVLD

9 profiles — same count as iter38 pre-bounds-fix on fresnel (no VP9 because the kernel doesn't expose it for rkvdec to pick up; no AV1 because the iter38 auto-probe is hard-capped at 2 fds: one rkvdec + one hantro-vpu instance). The third decoder (/dev/video4 AV1) is therefore invisible to the libva session even though the kernel exposes it.

In-session baseline anchor (Phase 0 N=1)

Test material: BBB 60 s 720p, re-encoded on ampere via ffmpeg-v4l2-request-fourier with same encoder presets as fresnel-fourier iter1 measurements (x264 medium CRF 23, x265 fast CRF 28, mpeg2video qscale 4, libvpx CRF 30, libvpx-vp9 CRF 30). 5 clips in ~/measurements/encoded/.

Probe (libva ffmpeg, ~/measurements/safe_probe.sh), single-shot post-reboot, full clip decode:

=== ampere baseline: libva HW decode probe (H.264 + VP8 + MPEG-2) ===
  h264    rkvdec /dev/video1         OK  elapsed=3.15s
  vp8     hantro /dev/video2         OK  elapsed=6.62s     v4l2-request: Unable to set control(s): Invalid argument
  mpeg2   hantro /dev/video2         OK  elapsed=7.23s     v4l2-request: Unable to set control(s): Invalid argument

3-of-6 codecs decode end-to-end. The "Unable to set control(s)" warnings on VP8 / MPEG-2 are from the cross-codec capability probe phase (rkvdec rejects VP8 / MPEG-2 controls and hantro rejects HEVC controls; iter38 auto-probe walks both fds at startup, captures these as warnings, ignores). Not a decode-time failure.

dmesg clean post-probe — only the usual boot init lines, no OOPSes.

This is N=1, not enough to lock as a load-bearing floor. Phase 3 will repeat at N=3+ with real instrumentation (lsof on the v4l2 fds, perf counters on rkvdec/hantro IRQs, libva trace, byte-comparison of decoded YUV vs SW reference — not stdout-grepped FPS).

Codecs blocked, and why

HEVC — kernel OOPS in rkvdec_hevc_prepare_hw_st_rps

First HEVC decode attempt produced this kernel oops (full dmesg captured):

Modules linked in: ...rockchip_vdec...
lr : rkvdec_hevc_prepare_hw_st_rps+0x38/0x300 [rockchip_vdec]
Call trace:
 __pi_memcmp+0x10/0x110 (P)
 rkvdec_hevc_assemble_hw_rps+0x1c/0xac [rockchip_vdec]
 rkvdec_hevc_run+0x12c/0x148 [rockchip_vdec]
 rkvdec_device_run+0x48/0x104 [rockchip_vdec]
 v4l2_m2m_try_run+0x80/0x13c [v4l2_mem2mem]
 v4l2_m2m_request_queue+0xe8/0x140 [v4l2_mem2mem]
 media_request_ioctl_queue+0x1bc/0x2f4 [mc]
 media_request_ioctl+0xc8/0x160 [mc]
 __arm64_sys_ioctl+0xa4/0x100

Cascading effect: after this oops the whole v4l2_mem2mem path wedges. Subsequent H.264 (rkvdec) AND VP8 / MPEG-2 (hantro) ffmpeg -hwaccel vaapi invocations block in futex wait inside libva, with the underlying ioctl thread blocked in the kernel m2m queue. The wedge persists until reboot.

This is a kernel bug in the RK3588 mainline rkvdec HEVC path — specifically the RPS (Reference Picture Set) preparation function. Not a libva backend bug (the working hand-built 0c9a7efa… backend is the one in use; the libva controls are what fresnel sends successfully to its own RK3399 rkvdec). The faulting instruction is __pi_memcmp — likely a NULL or unmapped pointer dereference during RPS comparison.

Per memory feedback_va_st_rps_bits_is_slice_field: fresnel iter31 had a related RPS issue, but that one was on the libva side (writing st_rps_bits into slice_params rather than decode_params). The libva-side fix is already in place at backend 7ac934e. The ampere oops is downstream of that — kernel-side.

Path to unblock: kernel-agent experiment with a candidate fix. Either:

• Find an existing upstream fix on linux-media or rockchip-linux trees (RK3588 rkvdec HEVC may have a fix in a later -rc).
• Bisect within mainline to localize the regression.
• Write a defensive guard around the memcmp call in rkvdec_hevc_prepare_hw_st_rps.

Filed as separate kernel-agent issue to follow.

VP9 — kernel doesn't expose V4L2_PIX_FMT_VP9_FRAME on rkvdec

v4l2-ctl -d /dev/video1 --list-formats-out returns only S264 + S265. The kernel module v4l2_vp9 is loaded (per lsmod: used by rockchip_vdec and hantro_vpu), but the RK3588 rkvdec binding does not register V4L2_PIX_FMT_VP9_FRAME as an output format. Likely needs DTS work + a driver patch to add the VP9 control set to the RK3588 rkvdec variant_ops, matching what RK3399 has.

Path to unblock: kernel-agent experiment with VP9 enablement patch for RK3588 rkvdec. Out of scope for the baseline per operator policy.

AV1 — libva backend iter38 only probes 2 fds, not 3

/dev/video4 (AV1) is exposed by the kernel — v4l2-ctl --list-formats-out shows AV1F. But the libva-v4l2-request-fourier auto-probe (iter38 multi-device-probe in src/request.c::request_data) tracks only video_fd_rkvdec + video_fd_hantro — there's no slot for a third decoder kind. The AV1 fd is never opened, so VAProfileAV1 never appears in vainfo.

Path to unblock: backend iter39. Extend struct request_data to a third fd (or to a table-driven dispatch over an arbitrary number of fds), update request_device_kind_for_profile() to return 'a' (av1) and route AV1 profiles to the new fd, add bounds and termination handling. Pure userspace work; no kernel involvement.

Predecessor data — what carries vs what doesn't

Per feedback_dev_process.md Phase 0 rules:

Carries (state):

• libva-v4l2-request-fourier source pin 7ac934e (iter38b, fresnel-certified)
• The marfrit-packages CI build bug for libva (marfrit-packages#17) — same ae611d80… packaged binary deployed to ampere, same workaround (hand-build over) applies
• The encode preset set (x264 medium CRF 23, x265 fast CRF 28, mpeg2video qscale 4, libvpx CRF 30, libvpx-vp9 CRF 30)
• Memory entries that govern decoder-output verification (feedback_pixel_compare_in_yuv_not_png, feedback_hw_decode_engagement_check, feedback_rockchip_pixel_verify_path)

Does NOT carry (data):

• Per-codec FPS numbers from fresnel-fourier iter1 measurements (RK3588 ≠ RK3399, different CPU + decoder block specs, no read-across)
• Per-codec bit-exact PASS criterion (fresnel's libva == kdirect held on RK3399; needs separate proof on RK3588)
• mpv --hwdec=v4l2request-copy 236 FPS for H.264 from fresnel — recompute on ampere

Ampere binding cells in this campaign anchor to in-session-acquired numbers, even when fresnel measured an identical condition.

Open questions tabled into Phase 1

1. What is the success metric? "All 6 codecs HW-decode" requires three external dependencies (kernel HEVC fix, kernel VP9 enablement, backend iter39). Phase 1 must pick whether the iteration locks against (a) the 3 currently-decodable codecs, (b) all 5 web-relevant codecs (excludes MPEG-2 from a Firefox-consumer view since video/mp2t is not supported by Gecko), (c) all 6.
2. What is bit-exactness anchored to on RK3588? fresnel used libva == kdirect (the kernel-direct V4L2 stateless test rig that bypasses libva). Does kdirect still build / run on RK3588? Probably needs adaptation. Alternative: anchor on libva == ffmpeg-SW-decode byte-for-byte, accepting that some codecs (MPEG-2 IDCT, H.264 inter-pred drift) won't be bit-exact and need an SSIM-floor instead.
3. Does the kernel HEVC OOPS cascade trip BEFORE actual decode, or only AFTER the first frame is queued? Phase 3 instrumentation (ftrace on rkvdec_hevc_run entry, perf on __pi_memcmp site) will tell us — that determines whether we can sandbox HEVC via try_decode_one_frame without wedging the m2m subsystem.
4. Are firefox-fourier vendor defaults sufficient on ampere? Not yet installed. Fresnel: yes for 3/4 codecs after firefox-fourier 150.0.1-5 shipped rockchip-fourier-defaults.js. Same package on RK3588 — probably yes, but unverified.
5. What's the AV1 source clip? None of BBB / Sintel are AV1-encoded in their standard distributions. Need to either (a) re-encode BBB to AV1 via libaom-av1 (slow but reproducible) or (b) source a short AV1 sample from elsewhere (e.g. AV1 sample bag) for the iter39 backend test once that lands.

Phase 0 close

Substrate, motivation, inventory captured. Baseline anchor at N=1 confirms the working 3-codec subset; HEVC kernel oops + VP9 kernel gap + AV1 backend gap documented as the three open work items. Ready for Phase 1 goal lock.

Iteration log:

• iter1: 2026-05-16 — this document.