User question 2026-05-23: 'Wayland does need a conversion of NV12 to
its output format. Could we cram that in?'
Yes — trivially. Added Stage 5 to the pipeline doc with:
- 5-line per-pixel compute shader (BT.709 limited-range example
given; matrix selected from H.264 VUI at runtime)
- explicit OPT-IN flag, off by default
- rationale for default-off: most consumers (V4L2 stateless,
Wayland zwp_linux_dmabuf NV12 passthrough, Firefox/mpv VAAPI
paths) want NV12 because compositors convert during composition
essentially for free. RGBA8 is 4x the bandwidth of NV12 — not
worth burning DMA + electrons when no downstream needs it
- colourspace metadata plumbing requirement: SPS vui_parameters
(colour_primaries, transfer_characteristics, matrix_coefficients,
video_full_range_flag) MUST flow through to the shader; default
BT.709 limited-range with warning if VUI absent
Updated the new-shader inventory to include v3d_h264_yuv_to_rgba.
Total dispatches/frame remains ~190-200; Stage 5 adds one.
Path C of the 2026-05-23 architecture decision after the daedalus-
fourier substitution arc's per-block QPU dispatch was measured to be
>600x slower than NEON in production. Root cause: per-block synchronous
Vulkan dispatch from inside libavcodec's per-MB loops, paying ~50us of
queue-submit/wait round-trip per ~30ns of NEON-equivalent arithmetic.
NVDEC and Vulkan Video escape this by dispatching at picture-level.
Pi 5 has no dedicated H.264 hardware decode block and Mesa V3DV does
not implement VK_KHR_video_decode_h264; this project builds the same
*shape* (one submit per frame, one fence wait per frame, encoded
bitstream in, NV12 out) using V3D7 Vulkan compute as the substrate.
DESIGN.md covers:
- architecture sketch (CPU side keeps entropy decode + descriptors;
GPU runs 4-stage compute pipeline per frame)
- per-MB descriptor layout (frame-shaped SSBO, ~8160 entries for 1080p)
- inter-stage dependencies (vkCmdPipelineBarrier within one command
buffer)
- intra prediction wavefront (~187 dispatches per frame on diagonals)
- libavcodec intercept point (macroblock-level, evolves the
substitution shim from "dispatch now" to "append to frame buffer")
- shader inventory (existing daedalus-fourier reuse + ~14 new ones)
- 4-phase plan, 4-6 months total budget
- 7 open questions including DPB allocation, qpel parameterization,
daemon integration shape
- explicit out-of-scope: VP9 / AV1 / HEVC / 10-bit / interlaced
This is design only. No code beyond README.md and DESIGN.md. User
review + redirect expected before Phase 1 implementation begins.
claude-noether