marfrit/fresnel-fourier
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

iter4 Phase 3: VP9 baseline + 4-codec regression on 7.0 substrate

Browse Source 
Captured on linux-fresnel-fourier 7.0-1 (post 6.19 decommission).

VP9 baseline (kernel-direct via ffmpeg-v4l2request on rkvdec):
- 5-frame SW reference PNG SHA256 anchors (criterion-4)
- VIDIOC_S_EXT_CTRLS strace with full payload at -s 16384
- Empirical struct sizes 168 B (FRAME) / 2040 B (COMPRESSED_HDR)
  supersede Phase 2 estimates of 144 / 1947
- Probe pattern: count=1 (FRAME-only) then count=2 (FRAME + COMPRESSED_HDR)

Phase 2 doc fix: control IDs corrected 0xa40b2c/d -> 0xa40a2c/d.

4-codec regression (H.264, MPEG-2, HEVC, VP8): all fall back to SW on
default config because /dev/video0 is now rockchip-rga (RGB color
converter), not a codec device. Fork hardcodes /dev/video0 in
request.c:149. Env override LIBVA_V4L2_REQUEST_VIDEO_PATH /
_MEDIA_PATH restores per-driver profile enumeration; mitigation A/B/C
queued for user decision.

New contract clauses surfaced:
- Clause 11: uncompressed-header partial parse for lf_delta /
  base_q_idx (VAAPI doesn't expose these; keyframe ref_deltas non-zero
  for BBB so leave-at-zero is wrong)
- Clause 12: compile-time sizeof asserts on the two control structs
  so future UAPI shifts fail loudly

iter4_phase3.tgz: full Phase 3 artifact bundle (strace + PNG refs).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Markus Fritsche
2026-05-09 20:31:53 +00:00
parent 2651e4cfdf
commit 56abe3d6a2
3 changed files with 139 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
iter4_phase3.tgz
BIN
View File
Binary file not shown.
phase2_iter4_situation.md
+2 -2
View File
@@ -149,7 +149,7 @@ Per memory `feedback_runtime_enumerates_allowlists.md`: plan for fix-forward Com
```c ```c
#define V4L2_CID_STATELESS_VP9_FRAME (V4L2_CID_CODEC_STATELESS_BASE + 300) #define V4L2_CID_STATELESS_VP9_FRAME (V4L2_CID_CODEC_STATELESS_BASE + 300)
/* = 0xa40b2c */ /* = 0xa40a2c */
struct v4l2_ctrl_vp9_frame { struct v4l2_ctrl_vp9_frame {
struct v4l2_vp9_loop_filter lf; /* 16 bytes; ref_deltas[4] + mode_deltas[2] struct v4l2_vp9_loop_filter lf; /* 16 bytes; ref_deltas[4] + mode_deltas[2]
@@ -187,7 +187,7 @@ Total size: ~144 bytes (vs iter3 VP8's 1232 bytes — much smaller because VP9_F
```c ```c
#define V4L2_CID_STATELESS_VP9_COMPRESSED_HDR (V4L2_CID_CODEC_STATELESS_BASE + 301) #define V4L2_CID_STATELESS_VP9_COMPRESSED_HDR (V4L2_CID_CODEC_STATELESS_BASE + 301)
/* = 0xa40b2d */ /* = 0xa40a2d */
struct v4l2_ctrl_vp9_compressed_hdr { struct v4l2_ctrl_vp9_compressed_hdr {
__u8 tx_mode; /* V4L2_VP9_TX_MODE_* (0..4) */ __u8 tx_mode; /* V4L2_VP9_TX_MODE_* (0..4) */
phase3_iter4_baseline.md
+137
View File
@@ -0,0 +1,137 @@
# Iteration 4 — Phase 3 (baseline capture)
VP9 baseline anchor for iter4, plus 4-codec regression block under new substrate `linux-fresnel-fourier 7.0-1`. Captured 2026-05-09 22:05–22:13 CEST.
## Substrate fingerprint
```
Linux fresnel 7.0.0-fresnel-fourier #1 SMP PREEMPT Sat May 9 19:24:42 CEST 2026 aarch64
Driver: rkvdec @ /dev/video1 + /dev/media0 (advertises HEVC/H.264/VP9F)
Driver: hantro-vpu @ /dev/video2 (enc) + /dev/video3 (dec) + /dev/media1 (advertises MPEG-2/VP8)
Backend: /usr/lib/dri/v4l2_request_drv_video.so SHA256 0ab5b2ba22df19569be26228629968ee254c030cd3664ce7afd1bc0396c254ef
(built earlier 2026-05-09 01:04, before 7.0 install at 20:06)
Sample: ~/fourier-test/bbb_720p10s_vp9.webm — 3,433,189 bytes
```
Phase 2 doc had wrong control IDs `0xa40b2c`/`0xa40b2d` — corrected to **`0xa40a2c` (VP9_FRAME)** and **`0xa40a2d` (VP9_COMPRESSED_HDR)** in-doc and verified empirically.
## Anchor 1 — SW reference frames (criterion-4 anchor)
Generated via `mpv --hwdec=no --vo=image --vo-image-format=png --frames=5 ~/fourier-test/bbb_720p10s_vp9.webm`. Decoder: libavcodec.
```
sw_ref/00000001.png b055681b27f2b3cd4151ffdf632876cfcb18ff1e44696d9da4c3f164510b1047
sw_ref/00000002.png faa84e17aca0f908b9456eab1a1c04ac9832436fa2ff279096de24ae1b072c87
sw_ref/00000003.png f5d7ed402901052d8b8c0f6f421a164379e9d0da8c3be18429756ccb3998fadf
sw_ref/00000004.png 06ff497a2debe3f93b6038645976a093cbca5058fee0d121a438ec3f87863af2
sw_ref/00000005.png df3e23ae926ccbf9a8c1a3207f0c449365ba04bb9b26573bb18a7a5d5019e82a
```
These hashes are the `iter4` Phase 7 criterion-4 reference. Persisted at `fresnel:/tmp/iter4_phase3/sw_ref/`.
## Anchor 2 — Kernel-direct VP9 control payloads (`ffmpeg -hwaccel v4l2request`)
`strace -ff -tt -y -v -s 16384 -e trace=ioctl ffmpeg -hwaccel v4l2request -i bbb_720p10s_vp9.webm -frames:v 5 -f null -`
**Verbatim ioctl observation** (frame 1, keyframe submission, after S_FMT/REQBUFS):
```
ioctl(/dev/video1, VIDIOC_S_EXT_CTRLS,
ctrl_class=0xf010000, count=2,
controls=[
{ id=0xa40a2c (V4L2_CID_STATELESS_VP9_FRAME), size=168, ... },
{ id=0xa40a2d (V4L2_CID_STATELESS_VP9_COMPRESSED_HDR), size=2040, ... }
]) = 0
```
**Empirical struct sizes — Phase 2 doc estimates were off**:
| Control | Phase 2 estimate | Empirical 7.0 | Delta |
|---|---|---|---|
| `v4l2_ctrl_vp9_frame` | 144 B | **168 B** | +24 |
| `v4l2_ctrl_vp9_compressed_hdr` | 1947 B | **2040 B** | +93 |
The Phase 4 plan must allocate / memcpy / cast against the empirical sizes; Phase 5 review should re-cite from `<linux/v4l2-controls.h>` on a 7.0-installed host (fresnel) rather than a 6.19 working copy.
**Frame-1 keyframe FRAME payload (decoded prefix)**:
- `lf` (16 B): `ref_deltas={1,0,-1,-1}, mode_deltas={0,0}, level=3, sharpness=0, flags=3, reserved[7]=0`
- `quant` (8 B): `base_q_idx=0x2e=46, delta_q_*=0`
- `seg` (~80 B): all zeros (segmentation disabled for keyframe)
- (Full byte-by-byte decode deferred to Phase 4 mapping clauses)
**Strace + decode artifacts** persisted: `fresnel:/tmp/iter4_phase3/vp9_strace_full.*`. Tarball pulled to `noether:~/src/fresnel-fourier/iter4_phase3.tgz` (8.0 MB).
**Initial probe call observation**: ffmpeg first issues a `count=1` S_EXT_CTRLS with only `0xa40a2c` (no compressed-header) — that's the runtime probe to detect kernel CID support. rkvdec accepts (CID is registered). Subsequent submissions use `count=2`. iter4 backend should mirror this 1→2 pattern OR unconditionally send 2 (rkvdec mandatorily-requires COMPRESSED_HDR per `rkvdec-vp9.c:752`).
## Anchor 3 — VP9 mpv-vaapi engagement (negative baseline)
```
mpv -v --hwdec=vaapi --frames=2 ~/fourier-test/bbb_720p10s_vp9.webm
[vd] Opening decoder vp9
[vd] Looking at hwdec vp9-vaapi...
[vd] Using software decoding. ← expected: backend has no VP9 yet
[vd] Selected decoder: vp9 - Google VP9
```
**Expected**: SW fallback. iter4 backend not built. After Phase 6 install, this exact command should switch to "Selected decoder: vp9_vaapi" + non-SW selection — that is the engagement check per memory `feedback_hw_decode_engagement_check.md`.
## 4-codec regression block — REGRESSION CONFIRMED
Substrate change exposed a pre-existing fork bug. Pre-existing `iter1`/`iter2`/`iter3` PASS results captured on 6.19.9 do **not** reproduce on 7.0 with the env-defaulted device path:
| Codec | Driver/device pre-7.0 | Driver/device 7.0 | mpv-vaapi engages? (no env override) |
|---|---|---|---|
| H.264 | rkvdec (probably video0) | rkvdec at /dev/video1 | NO (SW fallback) |
| MPEG-2 | hantro-dec | hantro-dec at /dev/video3 | NO |
| HEVC | rkvdec | rkvdec at /dev/video1 | NO |
| VP8 | hantro-dec | hantro-dec at /dev/video3 | NO |
**Root cause** (`request.c:149`):
```c
video_path = getenv("LIBVA_V4L2_REQUEST_VIDEO_PATH");
if (!video_path)
video_path = "/dev/video0";
```
On 7.0, `/dev/video0` is now `rockchip-rga` (RGB color converter) — its OUTPUT formats are pure color, no compressed codec — so the backend enumerates 0 supported codec profiles. `vainfo` confirms: empty profile list under default env. With explicit env `LIBVA_V4L2_REQUEST_VIDEO_PATH=/dev/video1 LIBVA_V4L2_REQUEST_MEDIA_PATH=/dev/media0`, rkvdec advertises H.264×5 + HEVCMain. With `/dev/video3` + `/dev/media1`, hantro advertises MPEG-2×2 + VP8Version0_3.
**Why this didn't surface earlier**: 6.19.9's bind order put rkvdec or hantro at `video0` by default. 7.0's binding/probe order changed. The fork's `/dev/video0` default was always implicit-numbering-dependent; the kernel upgrade decommissioned the assumption.
**Mitigation options for iter4** (decide before Phase 4):
- **A** (cheapest, env-only): document that users must set `LIBVA_V4L2_REQUEST_VIDEO_PATH` + `LIBVA_V4L2_REQUEST_MEDIA_PATH` per codec class. Add to `~/.config/mpv/config` system-wide. No fork patch.
- **B** (in-iter4 fork patch): walk `/dev/video0..N`, query VIDIOC_QUERYCAP, pick first device whose driver name is in {`rkvdec`, `hantro-vpu`}. Adds ~30 LOC to `request.c`. Cleaner end-user experience, but expands iter4 scope beyond VP9.
- **C** (defer to iter5): document as known issue for now; ship iter4 with env-var workaround in regression tests.
**Note**: there is also an mpv-vaapi `Could not create device` failure mode visible at `[vd] Looking at hwdec ...` even with env vars set, suggesting a second issue (likely vaapi-DRM render-node path). This is potentially a separate fix from the device-path issue. Investigation deferred to Phase 6 — ffmpeg-vaapi vs mpv-vaapi may use different device discovery paths.
## Open questions resolved by this baseline
1. **Loop filter deltas**: keyframe `ref_deltas={1,0,-1,-1}, mode_deltas={0,0}` — non-zero. So for BBB the libva backend can't leave them at zero. Source: bitstream `loop_filter_level/lf_delta_enabled/lf_ref_delta` parsed by libavcodec (kernel-direct path uses VP9Context internal state). VAAPI `VADecPictureParameterBufferVP9` does NOT expose these — backend must parse the uncompressed header for them, OR fail BBB criterion-4. **Decision for Phase 4**: ADD an uncompressed-header partial parser for `lf_delta_enabled` + `lf_ref_delta[4]` + `lf_mode_delta[2]`.
2. **Quantization base_q_idx**: keyframe `base_q_idx=46`. VAAPI exposes `seg_param[0].luma_ac_quant_scale` only; need inverse mapping via VP9 spec quantization table at `[1][q]` — feasible but slow. **Decision**: also pull `base_q_idx` from uncompressed header parse (already needed for #1).
3. **Reference mode**: deferred — verbatim payload byte at offset for `reference_mode` field needs decode. Phase 4 plan will pick from explicit byte.
4. **Reset frame context, interpolation filter, segmentation feature mapping**: byte-decode + cross-validate against VAAPI fields — Phase 4 mapping clauses will cite the empirical bytes, not the FFmpeg-inferred mapping.
5. **mpv VP9 hwdec engagement**: SW fallback expected (no VP9 backend yet). After Phase 6 install, expect the engagement string to change.
6. **rkvdec readback non-zero**: deferred to Phase 6/7 — predicted yes (rkvdec passed for HEVC iter2, H.264 T4) but cannot test without engaging libva first, which the device-path issue blocks. Resolve in Phase 6 install with env override.
## New iter4 contract clauses surfaced from baseline
The Phase 4 plan must add:
- **Clause 11** (uncompressed-header partial parse): backend reads bytes 0..uncompressed_header_size from `surface_object->source_data`, runs a minimal-state VPX bool reader to extract `loop_filter_level`/`loop_filter_sharpness`/`lf_delta_enabled`/`lf_ref_delta[4]`/`lf_mode_delta[2]`/`base_q_idx`/`y_dc_delta_q`/`uv_dc_delta_q`/`uv_ac_delta_q`. ~40 LOC. Replaces the "leave loop-filter-deltas at zero" predicted approach.
- **Clause 12** (struct sizing): use empirical 168/2040 B sizes; assert `sizeof(struct v4l2_ctrl_vp9_frame) == 168 && sizeof(struct v4l2_ctrl_vp9_compressed_hdr) == 2040` at compile time so any future kernel UAPI shift fails loudly instead of silently corrupting.
## Decisions queued for the user
- **device-path mitigation**: A / B / C above. Affects iter4 LOC budget (B adds ~30 LOC; A/C none).
- **engagement test path**: `mpv-vaapi-copy` has TWO failure modes (device-path + Could-not-create-device); `ffmpeg-vaapi -hwaccel_output_format vaapi -vf hwdownload` has the device-create issue separately. Should iter4 verify HW=SW via libva at all, or use the kernel-direct-only transitive proof from iter3? (Predicted Phase 7 work.)
## Substrate state at Phase 3 close
- Phase 3 captures persisted on fresnel `/tmp/iter4_phase3/` and tar'd to `noether:~/src/fresnel-fourier/iter4_phase3.tgz` (8.0 MB).
- Phase 2 doc IDs corrected: `0xa40b2c/d` → `0xa40a2c/d` (in `phase2_iter4_situation.md`).
- Empirical struct sizes 168/2040 captured — supersede Phase 2's 144/1947 estimates.
- 4-codec regression: documented; mitigation decision pending (A/B/C).
- iter4 ready to advance to Phase 4 (plan-build) once device-path mitigation is chosen.