marfrit
a4e7d8ab90
panvk-bifrost campaigns (r1..r4 Vulkan compositor + r5.video1 Vulkan
video decode) shipped before this repo existed; the deliverable
patches live in marfrit-packages, but the reasoning chain, phase docs,
and source-state evidence lived only in local working trees on the
development host.
This retrofit imports:
- mesa-panvk-bifrost/ — r1..r4 era phase docs (iter1..iter18)
(libmali stub blobs at iter18/blob/ excluded
— 109MB of RE artifacts replaced with a README
pointer)
- mesa-panvk-bifrost-video/ — sibling campaign phase docs + probe
- evidence/ — frozen .tgz source snapshots at each milestone
(basis for the 0005 patch diff generation)
Future iterations should branch off here from day one, so each iter is
a commit rather than a snapshot. See [[feedback-session-local-process-pins]]
for the process drift this retrofit closes.
Total: 1.9 MB across 124 files.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
93 lines
4.0 KiB
Markdown
93 lines
4.0 KiB
Markdown
# panvk-bifrost-video
|
||
|
||
Successor campaign to **panvk-bifrost** (closed 2026-05-21). New
|
||
deliverable: extend the panvk Vulkan driver for Mali-Bifrost so that
|
||
it exposes the Khronos `VK_KHR_video_decode_*` extension surface,
|
||
backed under the hood by the SoC's separate V4L2-stateless VPU
|
||
(hantro on RK3566 / VDPU381 on RK3588), not by the Mali GPU itself
|
||
— which has no video unit.
|
||
|
||
## Why this exists
|
||
|
||
The closing observation of panvk-bifrost was: **Brave on aarch64 is a
|
||
closed binary that won't VAAPI-route on its own.** iter14 documented
|
||
this as a permanent wall. chromium-fourier closes it for *Chromium*
|
||
(by rebuilding from source with the V4L2 path forced open), but Brave
|
||
is unmodifiable as-shipped.
|
||
|
||
The operator-stated lever — "this is the exact reason I want the
|
||
Vulkan driver — so brave does not just use vulkan to draw buttons, but
|
||
to actively use the features to offload, create buffers that kwin can
|
||
understand, yadda yadda younameit" — points at the *driver* side of
|
||
the boundary, not the browser. If we present Brave's existing Vulkan
|
||
dispatch with a competent `VK_KHR_video_decode_h264` implementation,
|
||
its media path engages through that without source modification.
|
||
|
||
## Goal
|
||
|
||
Make `vk-video-samples` (Khronos's canonical Vulkan video test client)
|
||
decode an H.264 BBB clip end-to-end on ohm (RK3566 / PineTab2 / Mali-G52)
|
||
using `mesa-panvk-bifrost-video` as the Vulkan ICD. Frames must be
|
||
hantro-decoded (not software-fallback), and the round-trip must be
|
||
provably zero-copy at the VkImage layer.
|
||
|
||
Brave engagement is a **subsequent** milestone; Phase 1 locks against
|
||
vk-video-samples as the test client because it isolates the driver
|
||
work from browser-binary unknowns.
|
||
|
||
## Why this is novel
|
||
|
||
Every Mesa VK_KHR_video implementation today (Anv = Intel, RADV = AMD,
|
||
NVK = NVIDIA) assumes the GPU has an integrated video engine. The
|
||
extension API is shaped around that: a queue family with
|
||
`VK_QUEUE_VIDEO_DECODE_BIT_KHR` on the same device as the graphics
|
||
queue, command buffers carrying both graphics and video ops to the
|
||
same physical engine.
|
||
|
||
Our SoC topology is different. The hantro VPU is a separate IP block
|
||
exposed only via V4L2 (`/dev/video1`); it has no relationship to the
|
||
Mali GPU other than sharing DMA-capable system memory through IOMMU /
|
||
dmabuf. This campaign is the first time (to my knowledge) Mesa would
|
||
bridge VK_KHR_video to a V4L2-stateless backend.
|
||
|
||
The architectural pattern, if it works, generalizes to every ARM SoC
|
||
where a Vulkan-capable GPU and a V4L2-only VPU live on the same SoC —
|
||
which is most of them.
|
||
|
||
## Scope (locked 2026-05-21)
|
||
|
||
- **Codec**: H.264 only initially. H.265 deferred (RK3588 hardware
|
||
not yet substrate-anchored).
|
||
- **Package**: New `mesa-panvk-bifrost-video` sibling to
|
||
`mesa-panvk-bifrost`. Separated so users who don't want the V4L2 /
|
||
libva runtime dep graph can opt out.
|
||
- **Phase 1 validation target**: `vk-video-samples` Khronos test
|
||
client decodes BBB H.264 on ohm. Brave integration becomes a later
|
||
iteration milestone.
|
||
|
||
## Inherits
|
||
|
||
- `mesa-panvk-bifrost` r4 (campaign-close 2026-05-21).
|
||
`/usr/lib/panvk-bifrost/libvulkan_panfrost.so` is the active Vulkan
|
||
ICD on ohm via `VK_ICD_FILENAMES`.
|
||
- `libva-v4l2-request-fourier` on ohm — proves the V4L2 stateless
|
||
H.264 decode path on hantro works at 1.16× realtime (lower-stack
|
||
measurement). Reference for the V4L2 ↔ H.264 mapping. **Device
|
||
ownership question lives here** (only one userspace can hold
|
||
`/dev/video1` at a time).
|
||
- 9(+1)-phase Claude-Assisted Development Process (see
|
||
`~/.claude/projects/-home-mfritsche-src/memory/feedback_dev_process.md`).
|
||
|
||
## Non-goals (explicit)
|
||
|
||
- No Mesa upstream MR (permanent rule: never upstream).
|
||
- No H.265, no AV1, no VP9 in this campaign.
|
||
- No Brave-side modification.
|
||
- No rebuild of Brave from source (chromium-fourier exists for the
|
||
open-source case; this campaign exists *because* Brave isn't).
|
||
- No re-implementation of libva — `libva-v4l2-request-fourier` stays
|
||
the libva backend, this is the Vulkan backend, they coexist via
|
||
device-arbitration policy (TBD in Phase 0).
|
||
|
||
— claude-noether, 2026-05-21
|