From e0acc334559e040df760221f03b5b2c7e69f84c2 Mon Sep 17 00:00:00 2001 From: Markus Fritsche Date: Sat, 25 Apr 2026 22:41:54 +0000 Subject: [PATCH] =?UTF-8?q?STUDY.md:=20phase=202=20finding=20=E2=80=94=20l?= =?UTF-8?q?ibva=20surface=20stack=20works;=20Brave=20wall=20is=20chromeos?= =?UTF-8?q?=20pipeline?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit mpv --hwdec=vaapi successfully probes our driver end-to-end: RequestQueryImageFormats, QueryConfigEntrypoints, CreateConfig, QuerySurfaceAttributes, CreateSurfaces2, DeriveImage, CreateImage, CreateBuffer, ExportSurfaceHandle all run clean across all seven enumerated profiles. mpv then falls back to SW for actual decode (drops match the SW baseline) because our decode-submission path isn't there yet — but the libva entry-point surface is largely done. Brave's "failed Initialize()ing the frame pool" turns out to be in chromium's chromeos pipeline (PickDecoderOutputFormat → ImageProcessor init in media/gpu/chromeos/video_decoder_pipeline.cc), not in our driver. No more libva calls happen between our successful CreateContext and the failure; chromium bails on the chromeos-specific V4L2 ImageProcessor it expects on real ChromeOS but doesn't find on a plain Linux Wayland system. Fix is on the Chromium build side, not here. Remaining real work in this library: decode submission path (Begin/ Render/EndPicture → V4L2 stateless queue/dequeue with controls attached), and proper STREAMON ordering on hantro. STUDY.md now documents both. Co-Authored-By: Claude Opus 4.7 (1M context) --- STUDY.md | 63 ++++++++++++++++++++++++++++++++++++++++++++++---------- 1 file changed, 52 insertions(+), 11 deletions(-) diff --git a/STUDY.md b/STUDY.md index a57bc29..50e10e1 100644 --- a/STUDY.md +++ b/STUDY.md @@ -74,19 +74,60 @@ MPEG-2 profiles. ### Failure mode reached now (2026-04-26) -`vainfo` works fully. Brave on ohm with the patched `.so` reaches deeper: +`vainfo` works fully. **mpv `--hwdec=vaapi` probes our driver end-to-end +successfully** (seven profiles enumerated, `RequestQueryImageFormats / +QueryConfigEntrypoints / CreateConfig / QuerySurfaceAttributes / +CreateSurfaces2 / DeriveImage / CreateImage / CreateBuffer / +ExportSurfaceHandle` all run clean) — then falls back to libavcodec's +software H.264 decoder for the actual decode. mpv's drop pattern matches +the SW baseline (≈16 drops/s through KWin gpu-next), so the SW path is +in use, not ours. -- Init: detects NV12 multi-plane CAPTURE format ✓ -- vaCreateContext: profile=H264Main 1920×1088, S_FMT + CREATE_BUFS pass ✓ -- vaCreateContext: STREAMON OUTPUT returned EINVAL ← deferred to unblock -- Now stuck at: `failed Initialize()ing the frame pool` from - `vaapi_video_decoder.cc` +Brave's failure is **not** in our driver. The verbose log shows: -That's the next-phase boundary. `vaCreateSurfaces2` (frame-pool init) is -where Chromium discovers buffer geometry for output frames; the library -needs to do CAPTURE-side `S_FMT` + `REQBUFS` + `EXPBUF` without single- -plane assumptions on a freshly-created context, and the V4L2 stateless -protocol's source-change-event dance probably needs implementing too. +``` +ApplyResolutionChangeWithScreenSizes() +PickDecoderOutputFormat(): Initializing ImageProcessor; max buffers: 16 +ERROR: failed Initialize()ing the frame pool +``` + +`PickDecoderOutputFormat` from `media/gpu/chromeos/video_decoder_pipeline.cc` +is the **ChromeOS** pipeline trying to init a V4L2 ImageProcessor (a +ChromeOS-specific concept — separate V4L2 m2m chip block for color +conversion / scaling). Brave-on-Linux runs this code path because the +build doesn't gate it on `is_chromeos`, but on a plain Linux Wayland +system there's nothing for the ImageProcessor to bind to and it bails +before any libva call lands in our driver. **No code change in +libva-v4l2-request-fourier will fix Brave**; the lever is on the +Chromium / Brave build side (skip the chromeos pipeline, or supply the +expected V4L2 image processor device). + +### What still needs to happen for actual decode + +The libva entry-point surface is largely done (probing works); the +remaining gap is the **decode submission path**: + +- `RequestBeginPicture` / `RequestRenderPicture` / `RequestEndPicture` + need to map to V4L2 stateless: queue OUTPUT buffer with the encoded + slice + the SPS/PPS/SLICE_PARAMS/PRED_WEIGHTS/DECODE_PARAMS/ + SCALING_MATRIX controls via the request fd, then trigger decode and + dequeue a CAPTURE buffer. +- The `STREAMON` ordering needs proper sequencing on hantro: the + current WIP defer in `RequestCreateContext` (commit `44a7327`) bypasses + the EINVAL but doesn't actually enable the queue. The real fix is to + set both queue formats up front, queue the first buffer with controls + attached, then `STREAMON` both queues. +- Source-change-event (`V4L2_EVENT_SOURCE_CHANGE`) handling is probably + needed for resolution-change streams; not strictly required for the + fixed-resolution Big Buck Bunny clip we test with. + +Once that lands, `mpv --hwdec=vaapi` should switch from "probe then SW +decode" to "actual HW decode through our path", and the user-facing +recipe matches what FFmpeg `-hwaccel v4l2request -hwaccel_output_format +drm_prime` already delivers (14 % CPU realtime). + +Brave / Chromium specifically remains parked behind the chromeos +pipeline issue, independent of this library's progress. ## Port plan