b3ba157cb4
Phase 2 source-read of the HEVC path post-iter1-close (fork master
229d6d1). Six bugs identified, all in libva backend; kernel + driver
path proven for HEVC in Phase 0 cross-validator sweep.
Substrate timing caveat: Phase 2 conducted against fresnel kernel
6.19.9-99. Operator-scheduled rolling pacman -Syyuu to linux-7
imminent. Phase 2 source-read findings are kernel-agnostic (fork
code + UAPI + FFmpeg reference); they carry forward across the
kernel jump unchanged. Phase 3 baselines will run on linux-7.
Bug 1 — src/config.c:64-69 HEVCMain falls through to default,
returns VA_STATUS_ERROR_UNSUPPORTED_PROFILE. Verbatim match for
iter1 Bug 1 pattern; fix is 3-line break addition.
Bug 2 — src/picture.c:204-206 explicit
case VAProfileHEVCMain: return UNSUPPORTED_PROFILE
with stale comment "Fourier-local: HEVC stripped, no HW support
on RK3566." (RK3566 is ohm context; fresnel is RK3399 where
rkvdec DOES support HEVC.) Fix: replace explicit reject with
dispatch to h265_set_controls() (mirrors MPEG-2 dispatch at
picture.c:186-191).
Bug 3 — src/h265.c uses staging-era CIDs:
V4L2_CID_MPEG_VIDEO_HEVC_PPS / _SPS / _SLICE_PARAMS
These don't exist on fresnel's 6.19 kernel headers (verified via
test-compile: gcc reports undeclared identifiers, suggests
V4L2_CID_MPEG_VIDEO_DEC_PTS as nearest match). Mainline kernel
UAPI splits HEVC stateless into 7 controls:
V4L2_CID_STATELESS_HEVC_{SPS,PPS,SLICE_PARAMS,SCALING_MATRIX,
DECODE_PARAMS,DECODE_MODE,START_CODE}
+ ENTRY_POINT_OFFSETS, EXT_SPS_ST_RPS, EXT_SPS_LT_RPS
(0xa40a90..0xa40a96 + extensions, V4L2_CID_CODEC_STATELESS_BASE
+ 400..407+).
Fix shape: rewrite h265.c against new split API. Substantially
larger than iter1's mpeg2.c rewrite (HEVC has 7 controls vs MPEG-2
3, + slice_params dynamic-array, + per-slice accumulation logic
needed).
Bug 4 — h265.c uses single-slice_params shape; new API is
dynamic-array. Fresnel rkvdec advertises:
hevc_slice_parameters 0xa40a92 elems=1 dims=[600] dynamic-array
Up to 600 slice_params entries per submission. Current
codec_store_buffer:115-135 OVERWRITES previous slice on
VASliceParameterBufferType arrival. Multi-slice frames need
APPEND-not-overwrite. FFmpeg reference v4l2_request_hevc.c:540-547
shows the pattern.
Fix shape: extend params.h265 to hold slice_params array (or
pointer+count); codec_store_buffer appends; h265_set_controls
flushes the array at end_picture as a single dynamic-array
S_EXT_CTRLS entry.
Bug 5 — h265.c missing controls: doesn't submit DECODE_PARAMS
(per-frame DPB info; new in modern API), SCALING_MATRIX (conditional
on iqmatrix_set + sps.scaling_list_enabled), DECODE_MODE+START_CODE
(device-wide menus, set once per context init).
Fix shape: add h265_fill_decode_params() (DPB ordering from VAAPI
ReferenceFrames[15] — preserve current extraction logic from
h265_fill_slice_params:269-315, route to new struct). Conditional
SCALING_MATRIX from VAIQMatrixBufferHEVC. Device-wide
DECODE_MODE+START_CODE either at first h265_set_controls call or
in extended context.c device-init block.
Bug 6 — src/meson.build comments out 'h265.c' (line 50) and
'h265.h' (line 73). Fix: uncomment both. Trivial.
Bug 7 (verify only) — include/hevc-ctrls.h is a 9-line shim that
just #include <linux/v4l2-controls.h>. Comment dates the
modernization to "linux-media 6.6+". Adds zero value; harmless.
Leave in place per iter1 Phase 5 Nit 6 lower-risk path.
Bug 8 (latent) — picture.c:287 params.h264.matrix_set=false
writes union byte 240. For HEVC: byte 240 lands inside
h265.picture (range [0..604), size 604) — different field than
MPEG-2's chroma_intra_quantiser_matrix. ffmpeg-vaapi's
per-frame VAPictureParameterBufferHEVC re-send overwrites the
corrupted byte before h265_set_controls reads. Latent for
clients that reuse a surface without re-sending picture params.
iter2+ Phase 4 cross-cutting backlog candidate; not iter2 scope.
Things verified NOT bugs:
- h265_fill_pps/sps/slice_params field extraction from VAAPI
structs is sound (just routes to wrong destination structs)
- NAL header parsing (data_bit_offset bit-search) is preserved
in new API — slice_params still has bit_size + data_bit_offset
- v4l2_set_controls batching API in place (used by H.264 + iter1
MPEG-2; iter2 uses same)
Substrate / kernel observation:
- Linux mainline 7.1.0-rc2 reference checkout has
drivers/staging/media/rkvdec/ with rkvdec.c, rkvdec-h264.c,
rkvdec-vp9.c — NO rkvdec_hevc.c. fresnel's HEVC support is
out-of-tree (Christian Hewitt patches per phase0_findings.md
external references). May land in stable 7.x.
- Phase 4 contract-before-code therefore can't cite kernel-side
HEVC handler source until/unless rkvdec_hevc.c lands in
mainline. UAPI doc + FFmpeg reference + Phase 3 cross-validator
bytes are the contract anchor.
Open questions tabled for Phase 3 (post-linux-7-upgrade):
1. iter1 + T4 references on linux-7 (regression check of closed
iter1 work)
2. SDDM watchpoint on linux-7
3. Cross-validator HEVC re-anchor (Baseline C equivalent for
HEVC) — verbatim payload bytes for SPS, PPS, DECODE_PARAMS,
SLICE_PARAMS array, SCALING_MATRIX
4. Pre-fix scratch test (Bug 1 + Bug 2 only, h265.c kept
commented out) — confirm collateral safe
5. Slice-count for bbb_720p10s_hevc.mp4 fixture
6. Whether linux-7 brings rkvdec_hevc.c into mainline
Predicted iter2 close shape: trivial Bugs 1+2+6 fixes + sizable
h265.c rewrite (~250-400 lines, ~3x iter1's mpeg2.c) + new
codec_store_buffer slice accumulation logic. If Phase 7 fails:
likely struct-size mismatch (run pahole), DPB ordering, or
slice_params array size encoding.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
318 lines
23 KiB
Markdown
318 lines
23 KiB
Markdown
# Iteration 2 — Phase 2 (situation analysis)
|
||
|
||
Phase 2 source-read of the HEVC path in libva-v4l2-request-fourier post-iter1-close (master tip `229d6d1`). Six bugs identified, all in the libva backend. Kernel + driver path was already proven for HEVC in Phase 0 cross-validator sweep.
|
||
|
||
**Caveat on substrate timing**: this Phase 2 was conducted against fresnel running `linux-eos-arm 6.19.9-99`. Phase 3 baselines + Phase 6 implementation + Phase 7 verification will run against an imminent linux-7 rolling upgrade (operator-scheduled). Phase 2 source-read findings are kernel-agnostic (fork code + kernel UAPI + FFmpeg reference + VAAPI buffers); they carry forward across the kernel jump unchanged.
|
||
|
||
## Reset context
|
||
|
||
```
|
||
$ git -C /home/mfritsche/src/fresnel-fourier status -sb
|
||
## master
|
||
$ git -C /home/mfritsche/src/fresnel-fourier log --oneline -3
|
||
6e8c970 iter2 Phase 0 + Phase 1 lock: HEVC Main on rkvdec
|
||
dc69378 iter1 Phase 8 close: 2/5 codecs passing
|
||
ec9133a iter1 Phase 7: verification — all 5 criteria GREEN
|
||
|
||
$ git -C /home/mfritsche/src/libva-multiplanar/libva-v4l2-request-fourier log --oneline -1
|
||
229d6d1 fresnel-fourier iter1 Phase 6 commit D: drop missed mpeg2-ctrls.h include from context.c
|
||
$ ssh fresnel 'uname -r'
|
||
6.19.9-99-eos-arm
|
||
```
|
||
|
||
Source files inspected:
|
||
|
||
| File | Lines |
|
||
|---|---|
|
||
| `src/h265.c` | 407 (currently excluded from build) |
|
||
| `src/picture.c` | 403 (HEVCMain reject at 204-206) |
|
||
| `src/config.c` | 255 (HEVCMain fall-through at 67) |
|
||
| `src/surface.h` | 182 (params.h265 union member) |
|
||
| `src/meson.build` | 98 (h265.c + h265.h commented out) |
|
||
| `include/hevc-ctrls.h` | 9 (thin shim, NOT a duplication header) |
|
||
|
||
Reference checkouts:
|
||
|
||
- Linux mainline at `~/src/libva-multiplanar/references/linux-mainline/` (7.1.0-rc2). **Note**: rkvdec_hevc.c does not exist in this checkout (rkvdec only has h264, vp9). The HEVC stateless API is in UAPI but the rkvdec driver-side HEVC handler is out-of-tree on fresnel's 6.19 (Christian Hewitt patch series). May land in stable 7.x.
|
||
- FFmpeg Kwiboo `v4l2-request-n8.1` branch — `libavcodec/v4l2_request_hevc.c` (752 lines, modern API).
|
||
- VAAPI headers on fresnel — `<va/va.h>` defines `VAPictureParameterBufferHEVC`, `VASliceParameterBufferHEVC`, `VAIQMatrixBufferHEVC`.
|
||
- Kernel UAPI on fresnel — `<linux/v4l2-controls.h>` lines 2090+ define new HEVC stateless controls.
|
||
|
||
## Bug 1 — `RequestCreateConfig` HEVCMain fall-through to default
|
||
|
||
`src/config.c:55-71`:
|
||
|
||
```c
|
||
case VAProfileMPEG2Simple:
|
||
case VAProfileMPEG2Main:
|
||
// fresnel-fourier iter1: MPEG-2 enabled. ...
|
||
break;
|
||
case VAProfileHEVCMain:
|
||
default:
|
||
return VA_STATUS_ERROR_UNSUPPORTED_PROFILE;
|
||
```
|
||
|
||
iter1 added the `break;` for MPEG-2 cases. HEVCMain still falls through to `default:` and returns `VA_STATUS_ERROR_UNSUPPORTED_PROFILE` (= 12). **Verbatim match for iter1 Bug 1's pattern**, applied to a different profile.
|
||
|
||
**Fix shape**: 3-line `break;` addition (analogous to iter1 Commit A pattern). Matches existing H.264 + MPEG-2 case structure. No profile-specific validation logic in `RequestCreateConfig` (validation happens at vaCreateContext / control submission time).
|
||
|
||
## Bug 2 — `picture.c::codec_set_controls` explicit HEVCMain reject
|
||
|
||
`src/picture.c:204-206`:
|
||
|
||
```c
|
||
case VAProfileHEVCMain:
|
||
/* Fourier-local: HEVC stripped, no HW support on RK3566. */
|
||
return VA_STATUS_ERROR_UNSUPPORTED_PROFILE;
|
||
```
|
||
|
||
Comment is stale — references RK3566 (PineTab2 / ohm-side context); fresnel-fourier targets RK3399 where rkvdec DOES support HEVC. Phase 0 cross-validator sweep confirmed `ffmpeg -hwaccel v4l2request` decodes the BBB HEVC fixture exit 0 on fresnel.
|
||
|
||
**Fix shape**: replace the explicit reject with a dispatch to `h265_set_controls()` (mirroring the MPEG-2 dispatch at `picture.c:186-191`). 5-line change. Comment updated to remove the stale RK3566 reference.
|
||
|
||
## Bug 3 — `src/h265.c` uses staging-era control IDs
|
||
|
||
`src/h265.c` calls `v4l2_set_control` with three control IDs:
|
||
|
||
- `V4L2_CID_MPEG_VIDEO_HEVC_PPS` (line 386)
|
||
- `V4L2_CID_MPEG_VIDEO_HEVC_SPS` (line 393)
|
||
- `V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS` (line 401)
|
||
|
||
These are staging-era — they don't exist on fresnel's 6.19 kernel headers. Empirical test:
|
||
|
||
```
|
||
$ gcc -c /tmp/test_compile_h265.c -I include
|
||
/tmp/test_compile_h265.c:3:29: error: 'V4L2_CID_MPEG_VIDEO_HEVC_PPS' undeclared (first use in this function);
|
||
did you mean 'V4L2_CID_MPEG_VIDEO_DEC_PTS'?
|
||
... (similar for _SPS and _SLICE_PARAMS)
|
||
```
|
||
|
||
So `h265.c` won't compile against the modern kernel UAPI. That's why `src/meson.build` comments it out (sources line 50, headers line 73).
|
||
|
||
The mainline kernel UAPI defines (`<linux/v4l2-controls.h>:2096-2100` and following):
|
||
|
||
```c
|
||
#define V4L2_CID_STATELESS_HEVC_SPS (V4L2_CID_CODEC_STATELESS_BASE + 400) /* 0xa40a90 */
|
||
#define V4L2_CID_STATELESS_HEVC_PPS (V4L2_CID_CODEC_STATELESS_BASE + 401) /* 0xa40a91 */
|
||
#define V4L2_CID_STATELESS_HEVC_SLICE_PARAMS (V4L2_CID_CODEC_STATELESS_BASE + 402) /* 0xa40a92 */
|
||
#define V4L2_CID_STATELESS_HEVC_SCALING_MATRIX (V4L2_CID_CODEC_STATELESS_BASE + 403) /* 0xa40a93 */
|
||
#define V4L2_CID_STATELESS_HEVC_DECODE_PARAMS (V4L2_CID_CODEC_STATELESS_BASE + 404) /* 0xa40a94 */
|
||
#define V4L2_CID_STATELESS_HEVC_DECODE_MODE (V4L2_CID_CODEC_STATELESS_BASE + 405) /* 0xa40a95 */
|
||
#define V4L2_CID_STATELESS_HEVC_START_CODE (V4L2_CID_CODEC_STATELESS_BASE + 406) /* 0xa40a96 */
|
||
#define V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSETS (V4L2_CID_CODEC_STATELESS_BASE + 407)
|
||
```
|
||
|
||
Plus EXT_SPS_ST_RPS, EXT_SPS_LT_RPS for HEVC range extensions (likely not needed on fresnel for Main profile).
|
||
|
||
The fresnel V4L2 inventory ([`phase0_evidence/2026-05-07/v4l2_inventory.txt`](phase0_evidence/2026-05-07/v4l2_inventory.txt)) confirms `/dev/video<rkvdec>` exposes the new IDs `0xa40a90/91/92/93/94/95/96` (matches `mpeg_2_*` style for MPEG-2 in iter1; here `hevc_*`).
|
||
|
||
**Fix shape**: rewrite `h265.c` against the new split API. Significantly larger than iter1's mpeg2.c rewrite — see Bug 4-6 for additional structural changes.
|
||
|
||
## Bug 4 — h265.c uses old struct shape (slice_params is now dynamic-array)
|
||
|
||
The current `src/h265.c` submits **one** slice_params per frame (one `v4l2_set_control` call with `&slice_params, sizeof(slice_params)`). The new mainline API treats slice_params as a **dynamic array** — one entry per slice in the frame.
|
||
|
||
Per Phase 0 inventory, fresnel's rkvdec advertises:
|
||
|
||
```
|
||
hevc_slice_parameters 0x00a40a92 (hevc-slice-params): elems=1 dims=[600] flags=has-payload, dynamic-array
|
||
```
|
||
|
||
So the kernel accepts up to 600 slice_params entries per submission. FFmpeg `libavcodec/v4l2_request_hevc.c:540-547`:
|
||
|
||
```c
|
||
if (ctx->max_slice_params && controls->num_slice_params) {
|
||
control[count++] = (struct v4l2_ext_control) {
|
||
.id = V4L2_CID_STATELESS_HEVC_SLICE_PARAMS,
|
||
.ptr = controls->frame_slice_params,
|
||
.size = sizeof(*controls->frame_slice_params) *
|
||
FFMIN(controls->num_slice_params, ctx->max_slice_params),
|
||
};
|
||
}
|
||
```
|
||
|
||
FFmpeg accumulates slice_params across multiple `decode_slice` callbacks and submits them all together at end_frame.
|
||
|
||
**Fix shape**: `h265_set_controls()` needs to iterate over all slices in the frame (the libva backend currently sees them as multiple `VARenderPicture(VASliceParameterBufferType)` calls — `codec_store_buffer:115-135` for HEVC currently does `memcpy(&surface->params.h265.slice, …)` which **overwrites** previous slice params with the latest one). For multi-slice frames, the libva backend must accumulate slice params into an array, not overwrite.
|
||
|
||
This is the largest behavioral change vs iter1 — the existing `codec_store_buffer` doesn't support per-slice accumulation. Phase 4 plan needs to extend `surface_object->params.h265` to hold a slice_params array (or use a separate accumulator), and `codec_store_buffer` needs to append rather than overwrite.
|
||
|
||
For the iter2 fixture (`bbb_720p10s_hevc.mp4`, libx265 ultrafast 1280×720), the typical slices-per-frame count is small (probably 1-4 depending on x265's slice configuration). Phase 3 baseline will measure the actual count via cross-validator strace.
|
||
|
||
## Bug 5 — h265.c missing controls (DECODE_PARAMS, SCALING_MATRIX, DECODE_MODE, START_CODE)
|
||
|
||
Current `h265.c` submits only PPS, SPS, SLICE_PARAMS. The new API has:
|
||
|
||
| Control | Required | Frequency | Source data |
|
||
|---|---|---|---|
|
||
| `STATELESS_HEVC_SPS` | yes | per-frame (or once per stream) | VAPictureParameterBufferHEVC |
|
||
| `STATELESS_HEVC_PPS` | yes | per-frame | VAPictureParameterBufferHEVC |
|
||
| `STATELESS_HEVC_SLICE_PARAMS` | yes | per-frame, dynamic-array | VASliceParameterBufferHEVC × N |
|
||
| `STATELESS_HEVC_SCALING_MATRIX` | conditional on `sps_scaling_list_data_present_flag` | per-frame | VAIQMatrixBufferHEVC |
|
||
| `STATELESS_HEVC_DECODE_PARAMS` | yes | per-frame | DPB info + POC, derived from VAPictureParameterBufferHEVC |
|
||
| `STATELESS_HEVC_DECODE_MODE` | once | device-wide at init | menu (FRAME_BASED on rkvdec) |
|
||
| `STATELESS_HEVC_START_CODE` | once | device-wide at init | menu (ANNEX_B on rkvdec) |
|
||
|
||
Per FFmpeg `v4l2_request_hevc.c:512-565` `v4l2_request_hevc_queue_decode`, the per-frame batch contains 3 mandatory + 3 conditional = up to 6 controls (no DECODE_MODE/START_CODE — those are device-wide menus set once at init).
|
||
|
||
The new **DECODE_PARAMS** control carries:
|
||
- `dpb[]` array of DPB entries (timestamp + flags per ref pic) — was inside slice_params in old API; moved to a per-frame DECODE_PARAMS struct in new API.
|
||
- `num_active_dpb_entries`, `num_poc_st_curr_before/after`, `num_poc_lt_curr` — same; moved to DECODE_PARAMS.
|
||
- POC info per frame.
|
||
|
||
**Fix shape**: `h265_fill_decode_params()` extracts DPB+POC info from VAAPI's ReferenceFrames[15] — the existing `h265_fill_slice_params` lines 269-315 already does this work, just into the wrong destination struct. New code populates `v4l2_ctrl_hevc_decode_params.dpb[]` instead of `slice_params.dpb[]`.
|
||
|
||
The old `dpb[]` location in the staging API was inside slice_params; new API moves it to decode_params (per-frame, sensible since DPB is shared across all slices in a frame). Phase 4 plan should preserve the existing extraction logic and route it into the new struct.
|
||
|
||
**SCALING_MATRIX**: only needed when `sps->scaling_list_enabled_flag` is set AND `pps->pps_scaling_list_data_present_flag` or SPS provides explicit lists. For the BBB-720p10s fixture, most likely default (flat) scaling — `iqmatrix_set` will be false, and we omit SCALING_MATRIX from the batch (kernel uses spec defaults).
|
||
|
||
**DECODE_MODE + START_CODE**: device-wide controls set once at context init, not per frame. Currently `src/context.c:142-155` sets H.264-specific device controls; iter2 should NOT add HEVC device init there because HEVC decoding shares the rkvdec device with H.264 (and possibly future VP9). The H.264 device-init currently in context.c works because the device is rkvdec; for HEVC we need to set HEVC's `_DECODE_MODE` and `_START_CODE` on the same device. Either extend context.c's device-init block or set the HEVC-specific controls inside `h265_set_controls()` once-per-context (with a flag to skip on subsequent frames).
|
||
|
||
## Bug 6 — `src/meson.build` excludes h265.c + h265.h
|
||
|
||
`src/meson.build:50` and `:73`:
|
||
|
||
```
|
||
sources = [
|
||
...
|
||
# 'h265.c'
|
||
]
|
||
|
||
headers = [
|
||
...
|
||
# 'h265.h'
|
||
]
|
||
```
|
||
|
||
Both commented out. iter2 fix: uncomment both. Trivial 2-line change in meson.build.
|
||
|
||
## Bug 7 (verify-only) — include/hevc-ctrls.h is a thin shim
|
||
|
||
Unlike `mpeg2-ctrls.h` (which iter1 deleted because it was a duplication header that masked kernel UAPI), `include/hevc-ctrls.h` is already a 9-line shim:
|
||
|
||
```c
|
||
/* Fourier-local override: HEVC controls are upstream since linux-media
|
||
* 6.6+, so defer to the kernel's linux/v4l2-controls.h instead of
|
||
* duplicating the struct definitions (duplication causes redefinition
|
||
* errors on newer linux-api-headers). */
|
||
#ifndef _LIBVA_V4L2_REQUEST_HEVC_CTRLS_H
|
||
#define _LIBVA_V4L2_REQUEST_HEVC_CTRLS_H
|
||
#include <linux/v4l2-controls.h>
|
||
#endif
|
||
```
|
||
|
||
The shim adds zero value — `<linux/v4l2-controls.h>` is already pulled transitively via `<linux/videodev2.h>` from any `.c` file that uses HEVC controls.
|
||
|
||
**Decision (defer to Phase 4)**: leave the shim in place (lower-risk path; iter1 Phase 5 Nit 6 deferral). Deletion is vestigial-cleanup, not iter2-scope. Verify in Phase 4 plan whether `src/h265.c` or any other file uniquely depends on `<hevc-ctrls.h>` being a separate include; if not, deletion is a 1-line follow-up cleanup but not blocking.
|
||
|
||
## Bug 8 (NEW, latent) — `picture.c:287` `params.h264.matrix_set = false` corrupts h265.picture
|
||
|
||
Per offsetof verification on fresnel via gcc + libva:
|
||
|
||
```
|
||
h264.matrix_set offset = 240
|
||
h265.picture range = [0 .. 604) size=604
|
||
h265.slice range = [604 .. 868) size=264
|
||
h265.iqmatrix range = [868 .. 1884) size=1016
|
||
h265.iqmatrix_set offset = 1884
|
||
```
|
||
|
||
The unconditional `params.h264.matrix_set = false` write at `picture.c:287` lands at union byte 240, which falls **inside h265.picture** (range 0..604). Specifically byte 240 of `VAPictureParameterBufferHEVC` — checking the va.h struct layout, byte 240 is somewhere in the middle (likely inside one of the bitfield sub-structs `pic_fields` or `slice_parsing_fields` or near `ReferenceFrames[]`).
|
||
|
||
For iter2's binding cells: same masking-by-RenderPicture-overwrite mechanism as iter1's MPEG-2 case. ffmpeg-vaapi sends `VAPictureParameterBufferHEVC` every frame via `vaRenderPicture(VAPictureParameterBufferType)`, which `codec_store_buffer:104-108` copies wholesale into `surface->params.h265.picture` (overwriting the corrupted byte 240). Net safe for iter2.
|
||
|
||
**Latent bug confirmed for HEVC too**: a VAAPI client that reuses a surface in BeginPicture without re-sending VAPictureParameterBufferType (legal VAAPI for surfaces with prior picture-param state) would see byte 240 corrupted. iter2 doesn't fix this; iter2+ Phase 4 cross-cutting backlog item B3 (BeginPicture profile-aware reset) covers it.
|
||
|
||
## Things verified NOT bugs
|
||
|
||
### h265.c field extraction logic from VAAPI is sound
|
||
|
||
The 407-line h265.c code:
|
||
|
||
- `h265_fill_pps()` (lines 48-102): extracts PPS fields from VAAPI `picture` (`VAPictureParameterBufferHEVC`) and `slice` (`VASliceParameterBufferHEVC`). Field-by-field source reads are correct against VAAPI's struct shape.
|
||
- `h265_fill_sps()` (lines 104-158): extracts SPS from VAAPI `picture`. Same shape; correct against VAAPI struct.
|
||
- `h265_fill_slice_params()` (lines 160-365): extracts slice params + DPB info + ref index lists + pred weight tables. The DPB extraction (lines 269-315) reads VAPictureHEVC ReferenceFrames[15] correctly.
|
||
- The data extraction targets the OLD struct layout (where DPB is in slice_params, where slice_params is single not dynamic-array, etc.). Phase 4 rewrite preserves the extraction logic and re-routes the destination to the new struct layout.
|
||
|
||
### h265.c parses NAL header from bitstream
|
||
|
||
Lines 184-209: extracts `nal_unit_type`, `nuh_temporal_id_plus1`, and `data_bit_offset` from the slice data buffer. The `data_bit_offset` calculation searches for the slice-segment-header start-code-prefix bit. **This logic is preserved** — the new V4L2 API still requires `slice_params.bit_size` and `slice_params.data_bit_offset` per slice. Field locations differ; computation is the same.
|
||
|
||
### v4l2.c batching API is in place
|
||
|
||
`src/v4l2.c:475-489` `v4l2_set_controls` accepts an `v4l2_ext_control[]` array. iter2's batched 6-control-per-frame submission uses this same API (single `VIDIOC_S_EXT_CTRLS` call), matching iter1's pattern.
|
||
|
||
## Open questions for Phase 3 baseline
|
||
|
||
Before Phase 4 plan, Phase 3 should capture (after the linux-7 kernel upgrade to anchor against the shipping kernel):
|
||
|
||
1. **iter1 + T4 references on linux-7** — re-run iter1 Phase 7 criterion 5 (T4 H.264) and criterion 4 (MPEG-2 +02s). If hashes differ, that's a substrate regression to investigate before iter2 proceeds. Criterion 1 (vainfo) regression is also worth checking.
|
||
2. **fresnel SDDM watchpoint** on linux-7 — verify the greeter still works after the kernel jump. If regression fires, run `~/.claude/plans/dynamic-forging-piglet.md` qFatal capture.
|
||
3. **Cross-validator HEVC re-anchor** — re-capture `ffmpeg -hwaccel v4l2request -i bbb_720p10s_hevc.mp4 -frames:v 5 -f null -` strace + ftrace on linux-7. Capture the VERBATIM payload bytes for SPS, PPS, DECODE_PARAMS, SLICE_PARAMS array entries, SCALING_MATRIX (if present). Phase 4 transcription anchor (per [`memory/feedback_review_empirical_over_theoretical.md`](../../.claude/projects/-home-mfritsche-src-fresnel-fourier/memory/feedback_review_empirical_over_theoretical.md)).
|
||
4. **Pre-fix scratch test** — apply Bug 1 (config break) + Bug 2 (picture.c reject removal) as a scratch patch on a throwaway branch; rebuild WITHOUT touching h265.c (keep it commented out). Verify mpv `--hwdec=vaapi` MPEG-2 + H.264 still pass (no collateral). Then attempt to compile h265.c uncommented to confirm the staging-era CID errors. Document the exact compile errors as Phase 3 baseline.
|
||
5. **Verify the slice-count for bbb_720p10s_hevc.mp4** via FFmpeg's slice count or by counting `decode_slice` entries — small (1-4 per frame) vs large affects whether iter2 needs to allocate slice_params arrays statically or dynamically.
|
||
6. **Determine if linux-7 brings rkvdec_hevc.c into mainline** — if yes, read the kernel-side HEVC handler for additional contract verification (especially around DECODE_PARAMS and slice_params dynamic-array semantics).
|
||
|
||
## Phase 4 plan inputs
|
||
|
||
Once Phase 3 baseline confirms the situation on linux-7, Phase 4 plan should specify:
|
||
|
||
1. **Diff scope**:
|
||
- `src/config.c:64-69` — add `break;` for VAProfileHEVCMain case (3-line change, mirrors iter1 Bug 1 fix shape).
|
||
- `src/picture.c:204-206` — replace explicit reject with dispatch to `h265_set_controls()`.
|
||
- `src/h265.c` — substantial rewrite (~250-400 lines):
|
||
- Replace V4L2_CID_MPEG_VIDEO_HEVC_* with V4L2_CID_STATELESS_HEVC_*.
|
||
- Restructure for slice_params dynamic-array (per-slice append in codec_store_buffer + flush at frame end).
|
||
- Add DECODE_PARAMS struct fill (split from slice_params).
|
||
- Add SCALING_MATRIX (conditional on iqmatrix_set + sps.scaling_list_enabled).
|
||
- Add device-wide DECODE_MODE + START_CODE at h265_set_controls first-call (or context init).
|
||
- Update flag bitmasks: collapse boolean SPS/PPS fields into u64 flags (analogous to MPEG-2 picture.flags collapse in iter1).
|
||
- `src/picture.c::codec_store_buffer` — add HEVC slice_params accumulation logic (append-not-overwrite for `VASliceParameterBufferType`).
|
||
- `src/surface.h::params.h265` — add a slice_params array (or pointer + count) to hold accumulated slice params.
|
||
- `src/meson.build` — uncomment `h265.c` (line 50) and `h265.h` (line 73).
|
||
- `include/hevc-ctrls.h` — keep as-is (not a duplication header; lower-risk path).
|
||
- `src/context.c` — possibly extend H.264 device-init block to also set HEVC's device-wide DECODE_MODE + START_CODE if h265_set_controls doesn't handle that itself.
|
||
|
||
2. **Contract anchor** (per `feedback_dev_process.md` Phase 6 contract-before-code):
|
||
- Cite verbatim from `<linux/v4l2-controls.h>:2090+` — the 7 stateless HEVC control struct definitions and flag constants.
|
||
- Cite verbatim from FFmpeg `libavcodec/v4l2_request_hevc.c:512-565` — the queue_decode batched submission shape.
|
||
- Cite Phase 3 Baseline C HEVC anchor's verbatim payload (re-captured on linux-7).
|
||
- If linux-7 has rkvdec_hevc.c in mainline, cite the driver's slice_params handling, DECODE_PARAMS handling.
|
||
|
||
3. **Phase 7 verification harness**: re-use iter1's 5-criterion shape with HEVC fixture substituted. Add bonus byte-compare of post-fix VIDIOC_S_EXT_CTRLS payload vs Baseline C HEVC anchor (per [`memory/feedback_review_empirical_over_theoretical.md`](../../.claude/projects/-home-mfritsche-src-fresnel-fourier/memory/feedback_review_empirical_over_theoretical.md) — empirical wins).
|
||
|
||
4. **Header deletion completeness check**: per [`memory/feedback_header_deletion_check.md`](../../.claude/projects/-home-mfritsche-src-fresnel-fourier/memory/feedback_header_deletion_check.md). For iter2: include/hevc-ctrls.h is preserved as the lower-risk path, so this rule mostly doesn't apply. But: any `<mpeg2-ctrls.h>` reference iter1 might have missed (defensive grep in Phase 6 — even though iter1's Commit D fixed the one we found in context.c, do another sweep).
|
||
|
||
## Predicted iter2 outcome
|
||
|
||
The bugs are well-bounded but the fix is structurally larger than iter1. Phase 4 plan will produce:
|
||
|
||
- Trivial diffs for Bugs 1 + 2 + 6 (config.c break, picture.c dispatch, meson uncomment).
|
||
- Substantial rewrite of h265.c for Bugs 3-5 (~350 lines, ~3× iter1's mpeg2.c rewrite).
|
||
- New per-slice accumulation logic in picture.c::codec_store_buffer for the slice_params dynamic-array semantics.
|
||
|
||
If Phase 7 misses a check, most likely culprits in priority order:
|
||
|
||
1. **`VIDIOC_S_EXT_CTRLS` returns EINVAL** because of a struct-size mismatch (HEVC structs are large; `v4l2_ctrl_hevc_sps` is ~70 bytes, `_pps` ~80 bytes, `_decode_params` ~100+ bytes, `_slice_params` ~150 bytes per slice). Mitigation: `pahole` / `sizeof` against kernel UAPI; compare to Baseline C verbatim sizes.
|
||
|
||
2. **Slice_params dynamic-array submission shape wrong** — kernel may reject if size is not a multiple of `sizeof(struct v4l2_ctrl_hevc_slice_params)`, or if num_slices doesn't match an internal kernel expectation. Phase 6 should test single-slice + multi-slice cases.
|
||
|
||
3. **DECODE_PARAMS DPB ordering** — VAAPI's ReferenceFrames[] uses POC-sorted order; kernel's DECODE_PARAMS.dpb[] may want a specific order (long-term first, then short-term-curr-before, etc.). FFmpeg reference shows the canonical population.
|
||
|
||
4. **SCALING_MATRIX presence/absence** — if BBB has flat scaling and we omit SCALING_MATRIX from the batch, kernel may default OK or may EINVAL if `pps.scaling_list_data_present_flag` is set. Phase 6 should test both branches if possible.
|
||
|
||
5. **Pixel hash mismatch HW vs SW** — many possible causes (DPB ref ordering, slice_params bit_offset wrong, NAL unit type mis-extracted). Mitigation: byte-compare per-control payload against cross-validator before debugging pixels.
|
||
|
||
If Phase 7 → Phase 4 loopback fires for any reason, the loopback target is bounded: contract is well-cited, mismatches localize to specific control struct fields.
|
||
|
||
## What this leaves Phase 3 with
|
||
|
||
Phase 3 baseline measurements should:
|
||
|
||
1. Wait for linux-7 kernel upgrade (substrate update) and re-verify iter1 + T4 + SDDM.
|
||
2. Re-capture the cross-validator HEVC anchor on linux-7 (Baseline C-equivalent for HEVC).
|
||
3. Apply Bug 1 + Bug 2 as scratch patches to confirm the next failure mode is the h265.c compile error (not some new surprise).
|
||
4. Capture VAAPI VAPicture/VASlice/VAIQMatrix HEVC payload via LIBVA_TRACE for one frame's decode (to confirm field values our backend reads from VAAPI match what we expect to populate into V4L2 controls).
|
||
|
||
After Phase 3 closes, Phase 4 plan can lock the implementation diff against the captured contract.
|