fresnel-fourier/phase4_iter39_subprofile_plan.md

# Phase 4 — iter39 sub-profile support plan

**Status:** Phase 6 LANDED at backend `662f887` on gitea master (pushed 2026-05-17).
Phase 5 review (sonnet-architect, 3 mandatory amendments + 1 corrected claim) folded in below.
Phase 7 test rig at `phase7_iter39_test_rig.sh`; blocked on fresnel power-on.



## FR

PRE_COMPACT_HANDOFF.md "Open items" #2 — H264 Hi10P, HEVC Main10, VP9 Profile 2 are advertised as HW-capable on RK3399 but the libva backend has no entries. Drop them in.

## Phase 0/2 findings (locked from linux-mmind-v7.0 rkvdec source on boltzmann)

`drivers/media/platform/rockchip/rkvdec/rkvdec.c` ctrl tables, with `rk3399_rkvdec_variant` binding `rkvdec_coded_fmts`:

| VAProfile         | rkvdec HW | V4L2 OUTPUT pix_fmt   | rkvdec CAPTURE pix_fmt | notes |
|-------------------|-----------|------------------------|------------------------|-------|
| `H264High10`      | ✅ yes    | `H264_SLICE`           | `NV15` (4:2:0 10-bit)  | ctrl `cfg.max=HIGH_422_INTRA`, `bit_depth_luma_minus8==2` path live in `rkvdec-h264-common.c:196` |
| `HEVCMain10`      | ✅ yes    | `HEVC_SLICE`           | `NV15`                 | ctrl `cfg.max=MAIN_10`, rkvdec-hevc.c:514 "only 8-bit and 10-bit are supported" |
| `VP9Profile2`     | ❌ no     | n/a                    | n/a                    | rkvdec-vp9.c:670 "We only support profile 0"; ctrl `cfg.max=PROFILE_0`. **EXCLUDED FROM SCOPE.** |

Bonus / aside: H264 4:2:2 profiles also supported by HW (NV16/NV20) but VAAPI's `VAProfileH264High422` is non-standard and most consumers won't use it. **OUT OF SCOPE.**

## Architectural ripple — NV15 ↔ VA-standard pixel format

The hard part. RK3399 rkvdec emits 10-bit frames as **NV15**: 4 × 10-bit values packed in 5 bytes per element, no padding. VAAPI's standard 10-bit fourcc is **P010**: 2 bytes per pixel, 10 high bits used, 6 low bits zero. Mapping requires a bit unpack pass.

`/usr/include/va/va.h`:
- `VA_FOURCC_P010 = 0x30313050` (defined, standard)
- `VA_FOURCC_NV15` (not defined — would need inline `VA_FOURCC('N','V','1','5')`)
- `VA_RT_FORMAT_YUV420_10 = 0x100` (defined)

ffmpeg-v4l2-request kdirect path already handles NV15 → P010 internally inside `libavcodec/v4l2_request_hevc.c` family for the kdirect-test-rig codepath — so kdirect output for a Main10 fixture lands in P010 buffers already. Our libva-vs-kdirect bit-exact contract can still hold if we surface P010 too.

### Three scope options

**Option A — Enumerate-only.** Wire profile lists, pixelformat_for_profile, picture.c, synthetic SPS bit_depth. Skip the unpack. vainfo will list Hi10P / Main10 but actual decode emits NV15 in raw buffers that no standard VAAPI consumer understands. Misleading — not recommended.

**Option B — NV15-as-FOURCC expose.** Surface decoded NV15 with a non-standard `VA_FOURCC('N','V','1','5')`. Mesa/ffmpeg-vaapi will reject in their `vaCreateImage` path. Only useful for `vaExportSurfaceHandle` (DRM-PRIME) consumers that understand DRM_FORMAT_NV15 modifiers — Mesa panfrost-Midgard support unclear.

**Option C — Userspace unpack to P010.** Add `nv15_to_p010()` in surface.c / image.c, run in `copy_surface_to_image`. ~150 LOC bit-twiddling. Adds 1× decoded-frame-size memcpy + bit unpack per `vaDeriveImage` / `vaGetImage` call. Standard VAAPI consumers work as expected.

**Option D — Skeleton only / documented partial.** Profiles enumerated, decode goes through, but `vaDeriveImage` returns `VA_STATUS_ERROR_UNIMPLEMENTED` for 10-bit surfaces with a clear log. Real downstream usage broken; flagged as known limitation in README + memory.

## Recommended plan: Option C (full P010 unpack) — LOCKED

Reasoning: PRE_COMPACT_HANDOFF.md marks this as "open polish item" — partial-work options A/B/D would resurface as a TODO. Sub-profile support is only useful end-to-end. Option C is the only one that lets a Main10 fixture round-trip through `ffmpeg -hwaccel vaapi -i x.hevc.10b.mp4 ...` with our backend.

NV15 packing per `Documentation/userspace-api/media/v4l/pixfmt-nv15.rst` (linux-mmind-v7.0): 4×10-bit values packed in 5 consecutive bytes — `A[9:2] A[1:0]B[9:4] B[3:0]C[7:6] C[5:0]D[9:8] D[7:0]`. Unpack one 5-byte group → 4 P010 words (2B each, value in bits [15:6], zeros in [5:0]).

**Stride gotcha (per Phase 5 review)**: NV15 row stride is `ceil(width/4)*5` bytes, NOT `width*2`. The kernel's `G_FMT` returns the NV15 stride in `destination_bytesperlines[0]`. The unpack must use NV15 stride for source iteration and compute P010 stride independently as `width * 2`.

**kdirect does NOT unpack (Phase 5 correction)**: ffmpeg-v4l2-request's hwcontext_v4l2request.c maps NV15 as `AV_PIX_FMT_YUV420P10` with `DRM_FORMAT_NV15` modifier and exports raw DRM_PRIME. The downstream `av_frame_copy` uses libswscale's unpack — kdirect itself emits raw NV15 in DRM-PRIME buffers. **The libva backend cannot use that trick** — `vaGetImage` consumers receive a `VAImage` buffer (no AVFrame, no libswscale call). The Option C userspace unpack is the only path.

## Code changes (Option C, amended after Phase 5 review)

### libva-v4l2-request-fourier (gitea master, claude-noether identity)

**src/codec.c** — `pixelformat_for_profile`:
- Add `case VAProfileH264High10:` → `V4L2_PIX_FMT_H264_SLICE` (same OUTPUT slice as 8-bit H.264 — bit depth is signaled via SPS contents not OUTPUT pix_fmt)
- Add `case VAProfileHEVCMain10:` → `V4L2_PIX_FMT_HEVC_SLICE`

**src/config.c**:
- `RequestCreateConfig` switch: add the 2 profile cases (no-op validation, same shape as siblings)
- `RequestQueryConfigProfiles`: append `VAProfileH264High10` after existing H264 block (guard `-5` → `-6`), `VAProfileHEVCMain10` after `HEVCMain` (guard `-1` → `-2`). Bump `V4L2_REQUEST_MAX_PROFILES` from 11 → 13 in `request.h`.
- `RequestQueryConfigEntrypoints`: add the 2 profile cases
- `RequestGetConfigAttributes` + the inline assignment in `RequestCreateConfig` (line 122): branch on profile to return `VA_RT_FORMAT_YUV420_10` instead of `VA_RT_FORMAT_YUV420` for 10-bit profiles.

**src/context.c** — `RequestCreateContext`:
- Line 111-131 CAPTURE-probe block: extend to try NV15 first for 10-bit profiles (else NULL `video_format` → NULL-deref at line 135). Profile-gated branch.
- Line 178 `capture_pixelformat = V4L2_PIX_FMT_NV12` → branch on profile to set `V4L2_PIX_FMT_NV15` for 10-bit profiles.
- Synthetic SPS (line 235+): add Hi10P / Main10 cases with `bit_depth_luma_minus8 = 2`, `bit_depth_chroma_minus8 = 2`. H264 `profile_idc=110` is benign-but-unnecessary per Phase 5 review (kernel ignores in `get_image_fmt`); HEVC SPS has no profile_idc field at all. Image_fmt resolution is purely on `bit_depth_luma_minus8` (==2) + `chroma_format_idc` (==1).

**src/video.c — CRITICAL (Phase 5 amendment 1)**:
- `formats[]` table (line 37) currently has only NV12 entry. **Add NV15 entry.** Without this, `video_format_find(V4L2_PIX_FMT_NV15)` at context.c:117 returns NULL → `v4l2_type_video_capture()` at context.c:135 NULL-derefs.

**src/picture.c** — 5 switch blocks (lines 102, 123, 165, 210, 277). Add the new cases routing to the same per-codec function as the 8-bit profile siblings.

**src/h264.c / src/h265.c** — verify `bit_depth_luma_minus8 != 0` paths exist (they do, mapped to V4L2_CTRL field). No change expected.

**src/surface.c — CRITICAL (Phase 5 amendment 2)**:
- Line 185: `if (format != VA_RT_FORMAT_YUV420) return UNSUPPORTED;` — extend to `(format != YUV420 && format != YUV420_10)`. Without this fix the 10-bit `vaCreateSurfaces` aborts before context creation.
- `copy_surface_to_image`: branch on surface fourcc — if NV15 source, call unpack into P010 destination
- `RequestExportSurfaceHandle` (line ~685): emit `DRM_FORMAT_P010` (copy path) or pass-through `DRM_FORMAT_NV15` for PRIME path. Ship copy path only for v1; PRIME path is follow-up.

**src/image.c — CRITICAL (Phase 5 amendment 3)**:
- `RequestDeriveImage` line ~272: hardcoded `format.fourcc = VA_FOURCC_NV12` + `bits_per_pixel = 12`. **Branch on the underlying surface's bit depth** — emit `VA_FOURCC_P010` + `bits_per_pixel = 24` for 10-bit surfaces.
- `RequestQueryImageFormats` line ~326: currently advertises NV12 only. Extend to also advertise P010 when the active session is 10-bit. Requires per-session `is_10bit` flag on `driver_data` or a config lookup.

**src/request.h or struct request_data**:
- Add `bool is_10bit` (or equivalent — could derive from active config), set in `RequestCreateContext` based on `config_object->profile`, used by image.c branches.

**src/nv15.c + src/nv15.h (new files)**:
- `nv15_to_p010(const uint8_t *src, uint16_t *dst, unsigned int width, unsigned int height, unsigned int src_stride)` — pure C bit unpack. ~30-40 LOC for the function itself. `dst_stride = width * 2` (computed inline). `src_stride = ceil(width/4)*5` from kernel G_FMT.
- Two calls: luma plane (full size), chroma plane (UV interleaved, half-height).

### Diff size estimate (revised)
- codec.c, config.c, picture.c, context.c, video.c: ~80 LOC
- surface.c + image.c branches: ~50 LOC
- NV15→P010 unpack (nv15.c new): ~50 LOC
- Total: ~180 LOC (Phase 5 corrected pessimistic 150→100 for unpack itself; offset by extra video.c + image.c + surface.c amendments)

## Test plan (Phase 7)

**Fixtures (acquire on fresnel):**
```bash
# Re-encode BBB into Main10 + Hi10P
ffmpeg -i ~/measurements/source/bbb_720p.mov -t 10 -c:v libx265 -preset fast -crf 28 \
    -pix_fmt yuv420p10le -profile:v main10 ~/measurements/encoded/bbb_main10.mp4
ffmpeg -i ~/measurements/source/bbb_720p.mov -t 10 -c:v libx264 -preset medium -crf 23 \
    -pix_fmt yuv420p10le -profile:v high10 ~/measurements/encoded/bbb_hi10p.mp4
```

**Criteria:**
1. `vainfo` lists `VAProfileHEVCMain10` + `VAProfileH264High10`.
2. `ffmpeg -hwaccel vaapi -i bbb_main10.mp4 -vf hwdownload,format=p010le -frames:v 10 -f rawvideo /tmp/L_main10.yuv` succeeds without error.
3. SHA matches between libva and kdirect for 10 frames each codec — **using the same `-vf hwdownload,format=p010le` on BOTH paths** (kdirect emits NV15 via DRM-PRIME and libswscale unpacks via the format filter; libva emits P010 directly via our new unpack). The format filter normalizes both into P010 byte stream.
4. SSIM vs libavcodec SW reference ≥ 0.999 against `-pix_fmt yuv420p10le` SW decode (Main10 SW reference encoded above; convert P010 to YUV420P10 in the compare step or compare SSIM_Y after conversion).
5. No regression — 5/5 PASS still holds on the existing 5-codec smoke (run after changes).

## Open / pending decisions

- **PRIME path vs copy path for 10-bit**: ship copy-only first (P010 derived/created images), defer PRIME path for a follow-up. Many consumers actually use `vaPutImage` rather than `vaExportSurface` so this covers most cases.
- **VP9 Profile 2**: confirmed HW-unsupported on RK3399; do NOT add to enumeration. Add an explicit `/* not on RK3399 rkvdec */` comment in config.c near the VP9 line to prevent future "completeness" PRs adding it.