marfrit
56abe3d6a2
Captured on linux-fresnel-fourier 7.0-1 (post 6.19 decommission). VP9 baseline (kernel-direct via ffmpeg-v4l2request on rkvdec): - 5-frame SW reference PNG SHA256 anchors (criterion-4) - VIDIOC_S_EXT_CTRLS strace with full payload at -s 16384 - Empirical struct sizes 168 B (FRAME) / 2040 B (COMPRESSED_HDR) supersede Phase 2 estimates of 144 / 1947 - Probe pattern: count=1 (FRAME-only) then count=2 (FRAME + COMPRESSED_HDR) Phase 2 doc fix: control IDs corrected 0xa40b2c/d -> 0xa40a2c/d. 4-codec regression (H.264, MPEG-2, HEVC, VP8): all fall back to SW on default config because /dev/video0 is now rockchip-rga (RGB color converter), not a codec device. Fork hardcodes /dev/video0 in request.c:149. Env override LIBVA_V4L2_REQUEST_VIDEO_PATH / _MEDIA_PATH restores per-driver profile enumeration; mitigation A/B/C queued for user decision. New contract clauses surfaced: - Clause 11: uncompressed-header partial parse for lf_delta / base_q_idx (VAAPI doesn't expose these; keyframe ref_deltas non-zero for BBB so leave-at-zero is wrong) - Clause 12: compile-time sizeof asserts on the two control structs so future UAPI shifts fail loudly iter4_phase3.tgz: full Phase 3 artifact bundle (strace + PNG refs). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
138 lines
9.8 KiB
Markdown
138 lines
9.8 KiB
Markdown
# Iteration 4 — Phase 3 (baseline capture)
|
||
|
||
VP9 baseline anchor for iter4, plus 4-codec regression block under new substrate `linux-fresnel-fourier 7.0-1`. Captured 2026-05-09 22:05–22:13 CEST.
|
||
|
||
## Substrate fingerprint
|
||
|
||
```
|
||
Linux fresnel 7.0.0-fresnel-fourier #1 SMP PREEMPT Sat May 9 19:24:42 CEST 2026 aarch64
|
||
Driver: rkvdec @ /dev/video1 + /dev/media0 (advertises HEVC/H.264/VP9F)
|
||
Driver: hantro-vpu @ /dev/video2 (enc) + /dev/video3 (dec) + /dev/media1 (advertises MPEG-2/VP8)
|
||
Backend: /usr/lib/dri/v4l2_request_drv_video.so SHA256 0ab5b2ba22df19569be26228629968ee254c030cd3664ce7afd1bc0396c254ef
|
||
(built earlier 2026-05-09 01:04, before 7.0 install at 20:06)
|
||
Sample: ~/fourier-test/bbb_720p10s_vp9.webm — 3,433,189 bytes
|
||
```
|
||
|
||
Phase 2 doc had wrong control IDs `0xa40b2c`/`0xa40b2d` — corrected to **`0xa40a2c` (VP9_FRAME)** and **`0xa40a2d` (VP9_COMPRESSED_HDR)** in-doc and verified empirically.
|
||
|
||
## Anchor 1 — SW reference frames (criterion-4 anchor)
|
||
|
||
Generated via `mpv --hwdec=no --vo=image --vo-image-format=png --frames=5 ~/fourier-test/bbb_720p10s_vp9.webm`. Decoder: libavcodec.
|
||
|
||
```
|
||
sw_ref/00000001.png b055681b27f2b3cd4151ffdf632876cfcb18ff1e44696d9da4c3f164510b1047
|
||
sw_ref/00000002.png faa84e17aca0f908b9456eab1a1c04ac9832436fa2ff279096de24ae1b072c87
|
||
sw_ref/00000003.png f5d7ed402901052d8b8c0f6f421a164379e9d0da8c3be18429756ccb3998fadf
|
||
sw_ref/00000004.png 06ff497a2debe3f93b6038645976a093cbca5058fee0d121a438ec3f87863af2
|
||
sw_ref/00000005.png df3e23ae926ccbf9a8c1a3207f0c449365ba04bb9b26573bb18a7a5d5019e82a
|
||
```
|
||
|
||
These hashes are the `iter4` Phase 7 criterion-4 reference. Persisted at `fresnel:/tmp/iter4_phase3/sw_ref/`.
|
||
|
||
## Anchor 2 — Kernel-direct VP9 control payloads (`ffmpeg -hwaccel v4l2request`)
|
||
|
||
`strace -ff -tt -y -v -s 16384 -e trace=ioctl ffmpeg -hwaccel v4l2request -i bbb_720p10s_vp9.webm -frames:v 5 -f null -`
|
||
|
||
**Verbatim ioctl observation** (frame 1, keyframe submission, after S_FMT/REQBUFS):
|
||
|
||
```
|
||
ioctl(/dev/video1, VIDIOC_S_EXT_CTRLS,
|
||
ctrl_class=0xf010000, count=2,
|
||
controls=[
|
||
{ id=0xa40a2c (V4L2_CID_STATELESS_VP9_FRAME), size=168, ... },
|
||
{ id=0xa40a2d (V4L2_CID_STATELESS_VP9_COMPRESSED_HDR), size=2040, ... }
|
||
]) = 0
|
||
```
|
||
|
||
**Empirical struct sizes — Phase 2 doc estimates were off**:
|
||
|
||
| Control | Phase 2 estimate | Empirical 7.0 | Delta |
|
||
|---|---|---|---|
|
||
| `v4l2_ctrl_vp9_frame` | 144 B | **168 B** | +24 |
|
||
| `v4l2_ctrl_vp9_compressed_hdr` | 1947 B | **2040 B** | +93 |
|
||
|
||
The Phase 4 plan must allocate / memcpy / cast against the empirical sizes; Phase 5 review should re-cite from `<linux/v4l2-controls.h>` on a 7.0-installed host (fresnel) rather than a 6.19 working copy.
|
||
|
||
**Frame-1 keyframe FRAME payload (decoded prefix)**:
|
||
|
||
- `lf` (16 B): `ref_deltas={1,0,-1,-1}, mode_deltas={0,0}, level=3, sharpness=0, flags=3, reserved[7]=0`
|
||
- `quant` (8 B): `base_q_idx=0x2e=46, delta_q_*=0`
|
||
- `seg` (~80 B): all zeros (segmentation disabled for keyframe)
|
||
- (Full byte-by-byte decode deferred to Phase 4 mapping clauses)
|
||
|
||
**Strace + decode artifacts** persisted: `fresnel:/tmp/iter4_phase3/vp9_strace_full.*`. Tarball pulled to `noether:~/src/fresnel-fourier/iter4_phase3.tgz` (8.0 MB).
|
||
|
||
**Initial probe call observation**: ffmpeg first issues a `count=1` S_EXT_CTRLS with only `0xa40a2c` (no compressed-header) — that's the runtime probe to detect kernel CID support. rkvdec accepts (CID is registered). Subsequent submissions use `count=2`. iter4 backend should mirror this 1→2 pattern OR unconditionally send 2 (rkvdec mandatorily-requires COMPRESSED_HDR per `rkvdec-vp9.c:752`).
|
||
|
||
## Anchor 3 — VP9 mpv-vaapi engagement (negative baseline)
|
||
|
||
```
|
||
mpv -v --hwdec=vaapi --frames=2 ~/fourier-test/bbb_720p10s_vp9.webm
|
||
[vd] Opening decoder vp9
|
||
[vd] Looking at hwdec vp9-vaapi...
|
||
[vd] Using software decoding. ← expected: backend has no VP9 yet
|
||
[vd] Selected decoder: vp9 - Google VP9
|
||
```
|
||
|
||
**Expected**: SW fallback. iter4 backend not built. After Phase 6 install, this exact command should switch to "Selected decoder: vp9_vaapi" + non-SW selection — that is the engagement check per memory `feedback_hw_decode_engagement_check.md`.
|
||
|
||
## 4-codec regression block — REGRESSION CONFIRMED
|
||
|
||
Substrate change exposed a pre-existing fork bug. Pre-existing `iter1`/`iter2`/`iter3` PASS results captured on 6.19.9 do **not** reproduce on 7.0 with the env-defaulted device path:
|
||
|
||
| Codec | Driver/device pre-7.0 | Driver/device 7.0 | mpv-vaapi engages? (no env override) |
|
||
|---|---|---|---|
|
||
| H.264 | rkvdec (probably video0) | rkvdec at /dev/video1 | NO (SW fallback) |
|
||
| MPEG-2 | hantro-dec | hantro-dec at /dev/video3 | NO |
|
||
| HEVC | rkvdec | rkvdec at /dev/video1 | NO |
|
||
| VP8 | hantro-dec | hantro-dec at /dev/video3 | NO |
|
||
|
||
**Root cause** (`request.c:149`):
|
||
|
||
```c
|
||
video_path = getenv("LIBVA_V4L2_REQUEST_VIDEO_PATH");
|
||
if (!video_path)
|
||
video_path = "/dev/video0";
|
||
```
|
||
|
||
On 7.0, `/dev/video0` is now `rockchip-rga` (RGB color converter) — its OUTPUT formats are pure color, no compressed codec — so the backend enumerates 0 supported codec profiles. `vainfo` confirms: empty profile list under default env. With explicit env `LIBVA_V4L2_REQUEST_VIDEO_PATH=/dev/video1 LIBVA_V4L2_REQUEST_MEDIA_PATH=/dev/media0`, rkvdec advertises H.264×5 + HEVCMain. With `/dev/video3` + `/dev/media1`, hantro advertises MPEG-2×2 + VP8Version0_3.
|
||
|
||
**Why this didn't surface earlier**: 6.19.9's bind order put rkvdec or hantro at `video0` by default. 7.0's binding/probe order changed. The fork's `/dev/video0` default was always implicit-numbering-dependent; the kernel upgrade decommissioned the assumption.
|
||
|
||
**Mitigation options for iter4** (decide before Phase 4):
|
||
|
||
- **A** (cheapest, env-only): document that users must set `LIBVA_V4L2_REQUEST_VIDEO_PATH` + `LIBVA_V4L2_REQUEST_MEDIA_PATH` per codec class. Add to `~/.config/mpv/config` system-wide. No fork patch.
|
||
- **B** (in-iter4 fork patch): walk `/dev/video0..N`, query VIDIOC_QUERYCAP, pick first device whose driver name is in {`rkvdec`, `hantro-vpu`}. Adds ~30 LOC to `request.c`. Cleaner end-user experience, but expands iter4 scope beyond VP9.
|
||
- **C** (defer to iter5): document as known issue for now; ship iter4 with env-var workaround in regression tests.
|
||
|
||
**Note**: there is also an mpv-vaapi `Could not create device` failure mode visible at `[vd] Looking at hwdec ...` even with env vars set, suggesting a second issue (likely vaapi-DRM render-node path). This is potentially a separate fix from the device-path issue. Investigation deferred to Phase 6 — ffmpeg-vaapi vs mpv-vaapi may use different device discovery paths.
|
||
|
||
## Open questions resolved by this baseline
|
||
|
||
1. **Loop filter deltas**: keyframe `ref_deltas={1,0,-1,-1}, mode_deltas={0,0}` — non-zero. So for BBB the libva backend can't leave them at zero. Source: bitstream `loop_filter_level/lf_delta_enabled/lf_ref_delta` parsed by libavcodec (kernel-direct path uses VP9Context internal state). VAAPI `VADecPictureParameterBufferVP9` does NOT expose these — backend must parse the uncompressed header for them, OR fail BBB criterion-4. **Decision for Phase 4**: ADD an uncompressed-header partial parser for `lf_delta_enabled` + `lf_ref_delta[4]` + `lf_mode_delta[2]`.
|
||
2. **Quantization base_q_idx**: keyframe `base_q_idx=46`. VAAPI exposes `seg_param[0].luma_ac_quant_scale` only; need inverse mapping via VP9 spec quantization table at `[1][q]` — feasible but slow. **Decision**: also pull `base_q_idx` from uncompressed header parse (already needed for #1).
|
||
3. **Reference mode**: deferred — verbatim payload byte at offset for `reference_mode` field needs decode. Phase 4 plan will pick from explicit byte.
|
||
4. **Reset frame context, interpolation filter, segmentation feature mapping**: byte-decode + cross-validate against VAAPI fields — Phase 4 mapping clauses will cite the empirical bytes, not the FFmpeg-inferred mapping.
|
||
5. **mpv VP9 hwdec engagement**: SW fallback expected (no VP9 backend yet). After Phase 6 install, expect the engagement string to change.
|
||
6. **rkvdec readback non-zero**: deferred to Phase 6/7 — predicted yes (rkvdec passed for HEVC iter2, H.264 T4) but cannot test without engaging libva first, which the device-path issue blocks. Resolve in Phase 6 install with env override.
|
||
|
||
## New iter4 contract clauses surfaced from baseline
|
||
|
||
The Phase 4 plan must add:
|
||
|
||
- **Clause 11** (uncompressed-header partial parse): backend reads bytes 0..uncompressed_header_size from `surface_object->source_data`, runs a minimal-state VPX bool reader to extract `loop_filter_level`/`loop_filter_sharpness`/`lf_delta_enabled`/`lf_ref_delta[4]`/`lf_mode_delta[2]`/`base_q_idx`/`y_dc_delta_q`/`uv_dc_delta_q`/`uv_ac_delta_q`. ~40 LOC. Replaces the "leave loop-filter-deltas at zero" predicted approach.
|
||
- **Clause 12** (struct sizing): use empirical 168/2040 B sizes; assert `sizeof(struct v4l2_ctrl_vp9_frame) == 168 && sizeof(struct v4l2_ctrl_vp9_compressed_hdr) == 2040` at compile time so any future kernel UAPI shift fails loudly instead of silently corrupting.
|
||
|
||
## Decisions queued for the user
|
||
|
||
- **device-path mitigation**: A / B / C above. Affects iter4 LOC budget (B adds ~30 LOC; A/C none).
|
||
- **engagement test path**: `mpv-vaapi-copy` has TWO failure modes (device-path + Could-not-create-device); `ffmpeg-vaapi -hwaccel_output_format vaapi -vf hwdownload` has the device-create issue separately. Should iter4 verify HW=SW via libva at all, or use the kernel-direct-only transitive proof from iter3? (Predicted Phase 7 work.)
|
||
|
||
## Substrate state at Phase 3 close
|
||
|
||
- Phase 3 captures persisted on fresnel `/tmp/iter4_phase3/` and tar'd to `noether:~/src/fresnel-fourier/iter4_phase3.tgz` (8.0 MB).
|
||
- Phase 2 doc IDs corrected: `0xa40b2c/d` → `0xa40a2c/d` (in `phase2_iter4_situation.md`).
|
||
- Empirical struct sizes 168/2040 captured — supersede Phase 2's 144/1947 estimates.
|
||
- 4-codec regression: documented; mitigation decision pending (A/B/C).
|
||
- iter4 ready to advance to Phase 4 (plan-build) once device-path mitigation is chosen.
|