test0r
3b453de45a
Full state-of-play matrix (browser × SoC × codec) plus per-SoC ranked-by-tractability YouTube delivery paths, drafted by the Plan subagent with web research synthesizing LibreELEC/CoreELEC patterns, Mozilla bugzilla, Collabora's mainline Rockchip status, and chromium-dev threads. Sources cited at the end. Headline: - fresnel ships today via enhanced-h264ify forcing YouTube to serve H.264 (rkvdec already validated bbb_1080p30 RDD 78% to 5%) - ohm needs VP9 hwaccel verification against hantro before claims - ampere blocked on Linux 7.0 / VDPU381 rkvdec2 driver kernel rebase - mpv-as-YouTube is the LibreELEC-inspired browser-bypass path Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
133 lines
11 KiB
Markdown
133 lines
11 KiB
Markdown
# YouTube Hardware Decode Delivery Plan — Fourier Campaign
|
||
|
||
Drafted 2026-04-29 by the Plan subagent ("Sonnet Architect") with web
|
||
research synthesizing LibreELEC/CoreELEC patterns, Mozilla bugzilla,
|
||
Collabora's mainline progress reports, and chromium-dev threads.
|
||
Audience: future-Claude after the next compaction, plus the user
|
||
reading async.
|
||
|
||
## 1. State-of-play matrix (browser × SoC × codec, 2026-04-29)
|
||
|
||
Legend: ✓ shipping · 🚧 in flight · ⚠ known gap · ✗ infeasible on this hardware · — not advertised by SoC
|
||
|
||
### Firefox (firefox-fourier 150.0.1-16, libavcodec v4l2_request route)
|
||
|
||
| SoC / VPU | H.264 | HEVC | VP9 | AV1 |
|
||
|---|---|---|---|---|
|
||
| **fresnel** RK3399 / rkvdec | ✓ validated bbb_1080p30, RDD 78%→5% | ✗ no HW | ✗ no HW | ✗ no HW |
|
||
| **ohm** RK3566 / hantro | 🚧 should work, untested | ⚠ framework only — bug 1969297 parser/slicer pieces missing | ⚠ fourcc registered, hwaccel path unverified | — hantro lacks AV1 |
|
||
| **ampere** RK3588 / rkvdec2 | 🚧 needs rkvdec2 driver (VDPU381 merged for ~Linux 7.0) | 🚧 same — VDPU381 H.265 controls just merged | ⚠ Collabora "future work", not yet in mainline | ⚠ rkvdec2 AV1 patches floated by Gaignard, RK3588 playback "not played well" per chromium-dev |
|
||
|
||
### Chromium (chromium-fourier, V4L2 stateless via `media/gpu/v4l2/stateless/`)
|
||
|
||
| SoC | H.264 | HEVC | VP9 | AV1 |
|
||
|---|---|---|---|---|
|
||
| **fresnel** RK3399 | 🚧 install-v5 not yet deployed here | ✗ | ✗ | ✗ |
|
||
| **ohm** RK3566 | ✓ install-v5 manual deploy, decode path active | ⚠ untested for YouTube clips | ⚠ untested | — |
|
||
| **ampere** RK3588 | 🚧 driver dependency | 🚧 driver dependency | ⚠ Chromium has stateless VP9 code; needs rkvdec2 mainline | ⚠ Chromium has stateless AV1; rkvdec2 instability reported |
|
||
|
||
### mpv (`--hwdec=v4l2request`, dmabuf-wayland VO)
|
||
|
||
| SoC | H.264 | HEVC | VP9 | AV1 |
|
||
|---|---|---|---|---|
|
||
| fresnel | ✓ file playback works | ✗ | ✗ | ✗ |
|
||
| ohm | ✓ | ✓ likely | 🚧 | — |
|
||
| ampere | 🚧 | 🚧 | 🚧 | 🚧 |
|
||
|
||
## 2. Per-SoC ranked-by-tractability YouTube delivery paths
|
||
|
||
### fresnel (RK3399) — H.264-only VPU, must coerce YouTube codec
|
||
|
||
1. **Force H.264 via enhanced-h264ify in firefox-fourier.** Most tractable. Ship the extension preinstalled or as a documented one-click add-on. We already validated H.264 HW decode end-to-end. *Trade-off:* H.264 1080p stream is ~40-50% larger bandwidth than VP9 same-quality, and YouTube caps H.264 at 1080p (no 1440p/2160p H.264). For a 1080p Pinebook Pro panel, perfect fit.
|
||
2. **Same trick in chromium-fourier** once the package finishes building (data:CT220 job in flight). Use enhanced-h264ify or set `?html5=1` cookie variants.
|
||
3. **mpv-as-YouTube fallback** for fullscreen viewing: `mpv --ytdl-format='bv*[vcodec~=avc1]+ba/b[vcodec~=avc1]' --hwdec=v4l2request --vo=dmabuf-wayland`. Bypass browser entirely, use as desktop launcher / Plasma activity.
|
||
4. **Don't bother with VP9/AV1** — it's CPU-bound forever on this SoC.
|
||
|
||
### ohm (RK3566) — hantro covers H.264 + HEVC + VP8 + VP9
|
||
|
||
1. **Validate VP9 stateless end-to-end first.** Smallest unit of work that unlocks the bulk of YouTube. Tasks: (a) confirm `v4l2-ctl --list-formats` on `/dev/video*` shows `VP9F`; (b) ffmpeg sample clip via `ffmpeg -hwaccel v4l2request -i vp9.webm -f null -` to confirm libavcodec→hantro path works at the CLI; (c) only then trust firefox-fourier's 0001/0003 patches with VP9.
|
||
2. **If (1) is broken in libavcodec's v4l2_request VP9 hwaccel,** pivot to chromium-fourier's stateless code path — it's better-exercised on hantro (ChromeOS lineage). This may make Chromium the recommended default browser on ohm.
|
||
3. **Force-H.264 fallback** via enhanced-h264ify if VP9 path proves flaky — same recipe as fresnel, lossy on resolution but no stutter.
|
||
4. **HEVC (S265)** is not a YouTube codec, defer behind bug 1969297.
|
||
|
||
### ampere (RK3588) — rkvdec2 mainline arrived for H.264/HEVC, VP9/AV1 still cooking
|
||
|
||
1. **Track Linux 7.0 / Collabora VDPU381 merge** (just landed Feb 2026). Get a kernel build with this merged onto ampere ASAP — H.264 + HEVC HW decode follows. This is the single highest-leverage step.
|
||
2. **Force-H.264 in browser** as bridge: gives stutter-free 1080p YouTube the day the kernel module probes.
|
||
3. **VP9 on rkvdec2** is "future work" per Collabora — track the patches, do not commit code. Until then VP9 on ampere = SW.
|
||
4. **AV1 on rkvdec2** has out-of-tree patches floating; chromium-dev thread reports "video is not played well". Don't ship; revisit Q3 2026.
|
||
5. **panvk Vulkan video decode (`VK_KHR_video_decode_*`)** is not implemented in panvk yet (Mesa 26.0/26.1 release notes do not mention it). Not a near-term path. Mark as "watch the mesa-dev list".
|
||
|
||
## 3. LibreELEC inspiration — what to copy, what to skip
|
||
|
||
LibreELEC is the canonical "make ARM SoC video decode work end-to-end" project. Their model:
|
||
|
||
- **Kernel patch stack at `LibreELEC.tv/projects/Rockchip/patches`** — mix of upstream-pending and WIP. Specifically `linux-2000-v4l2-wip-rkvdec-hevc.patch` and the iep-driver patches show the bridging glue between mainline V4L2 stateless skeleton and what userspace actually needs. Worth diffing against our kernel build to see what we silently lack.
|
||
- **ffmpeg fork is Kwiboo's `ffmpeg-v4l2-request`** — the same fork we already package. LibreELEC's "explanation of the ffmpeg patches" forum thread documents which patches are about request API plumbing vs DRM PRIME exporters vs codec-specific slicer fixes. Use this to audit which of our own patches we really need.
|
||
- **Compromise points to copy:** LibreELEC ships a curated kernel + curated ffmpeg + curated media player (Kodi); they do NOT try to make every browser work. **Application surface is Kodi only.** This is the single biggest architectural lesson — they bypass the browser problem entirely. Our analogue is the **mpv-as-YouTube** path.
|
||
- **What NOT to copy:** they ship a read-only OS image. We're a rolling Arch derivative — different distribution model. We can't pin kernel + ffmpeg + Kodi together as a single artifact.
|
||
- **Specific things to mine:**
|
||
- their `package/mediacenter/kodi/patches/` for V4L2 request hwaccel integration patterns (relevant to our ffmpeg/mpv config, not browsers)
|
||
- their iommu / dma-buf workarounds — overlap with our vb2-dma-resv RFC
|
||
- LibreELEC forum thread #25198 "Hardware accelerated video in 2022" for the full architecture writeup
|
||
|
||
## 4. Decision criteria — when do we stop
|
||
|
||
**Per-SoC target bar (ship gate):**
|
||
|
||
> 1080p30 YouTube playback at <30% total CPU, zero dropped frames over a 60-second window, on the codec YouTube serves by default for that device class.
|
||
|
||
**Per-SoC accept conditions:**
|
||
|
||
- **fresnel:** 1080p30 H.264 YouTube via firefox-fourier + enhanced-h264ify, <30% CPU on 1× big-cluster core. Already 80% there based on bbb_1080p30 result.
|
||
- **ohm:** EITHER (a) 1080p30 VP9 native via hantro <30% CPU, OR (b) 1080p30 H.264 forced <30% CPU. Either qualifies.
|
||
- **ampere:** 1080p30 H.264 via rkvdec2 once kernel lands, <20% CPU (this SoC is overpowered, raise the bar). Stretch goal: 4K HEVC test clip via Kodi/mpv to validate the kernel/ffmpeg path even though YouTube won't serve HEVC.
|
||
|
||
**Anti-goals (explicit non-targets):**
|
||
|
||
- 4K YouTube on any SoC. None of these displays are 4K; YouTube serves VP9/AV1 at 4K; rkvdec2 VP9 isn't ready. Drop.
|
||
- AV1 anywhere. Not worth the 2026 engineering cost.
|
||
- HEVC in browsers. YouTube doesn't serve it. Track bug 1969297 passively.
|
||
|
||
## 5. Risks and dependencies
|
||
|
||
**Kernel-side blockers:**
|
||
- **rkvdec2 mainline driver for ampere** — VDPU381 H.264/HEVC merged (Linux 7.0). Need to actually rebase ampere's kernel onto this. This is the single biggest lever; until then ampere is a SW-decode device.
|
||
- **rkvdec2 VP9** — Collabora "future work", no timeline. Do not block on this.
|
||
- **vb2 dma_resv producer fences (our RFC)** — required for KWin's `watchDmaBuf` to actually wait on real fences. Already submitted; track linux-media response. If rejected, KWin patch is the workaround.
|
||
|
||
**Userspace-side blockers:**
|
||
- **Mozilla bug 1969297 (HEVC stateless)** — David Turner mid-implementation, last activity 2026-01-13. Our 0003 patch handles framework; HEVC parser/slicer integration into hwaccel ctx is the missing piece. Not on YouTube critical path; deprioritize.
|
||
- **libavcodec VP9 v4l2_request hwaccel correctness** on hantro — unverified. Highest single risk to ohm story. Test before shipping VP9 claims.
|
||
- **Chromium V4L2 stateless VP9/AV1 path** — Chromium has the code (ChromeOS lineage) but RK3588 reports are rough. For ohm/hantro VP9, Chromium's path may actually be more battle-tested than libavcodec's; consider declaring Chromium the default browser on ohm.
|
||
- **chromium-fourier PKGBUILD build** — currently in flight on data:CT220 (~4-8h). If it fails, we have install-v5 only as a manual deploy and ohm shipping story partially regresses.
|
||
|
||
**Distribution risks:**
|
||
- Rolling release means kernel/ffmpeg/Firefox can desync. Need explicit pin or version-coupling logic in the marfrit repo for the stack to stay coherent.
|
||
- KWin MR 9157 (watchDmaBuf) needs to land upstream eventually or we carry the patch forever.
|
||
|
||
## Critical Files for Implementation
|
||
|
||
- `arch/firefox-fourier/patches/0001-gfxinfo-v4l2-stateless-fourccs.patch`
|
||
- `arch/firefox-fourier/patches/0003-ffmpegvideo-v4l2-request-route.patch`
|
||
- `arch/firefox-fourier/patches/0005-rdd-sandbox-v4l2-media-ctl.patch`
|
||
- `arch/chromium-fourier/patches/enable-v4l2-decoder-default.patch`
|
||
- `arch/ffmpeg-v4l2-request-git/PKGBUILD` + `0001-libudev-bypass-fallback.patch`
|
||
|
||
## Sources
|
||
|
||
- [RK3588 / RK3576 video decoders merged (Collabora)](https://www.collabora.com/news-and-blog/news-and-events/rk3588-and-rk3576-video-decoders-support-merged-in-the-upstream-linux-kernel.html)
|
||
- [Rockchip RK3588/RK3576 H.264/H.265 mainline (CNX, 2026-02-27)](https://www.cnx-software.com/2026/02/27/rockchip-rk3588-rk3576-h-264-and-h-265-video-decoders-mainline-linux/)
|
||
- [Mozilla Bug 1969297 — HEVC v4l2-stateless](https://bugzilla.mozilla.org/show_bug.cgi?id=1969297)
|
||
- [Mozilla Bug 1833354 — V4L2-M2M HW decode](https://bugzilla.mozilla.org/show_bug.cgi?id=1833354)
|
||
- [LibreELEC Rockchip patches tree](https://github.com/LibreELEC/LibreELEC.tv/tree/master/projects/Rockchip/patches)
|
||
- [LibreELEC forum: getting HEVC decoding working](https://forum.libreelec.tv/thread/26796-getting-hevc-decoding-working-with-libreelec-patches/)
|
||
- [LibreELEC forum: explanation of the ffmpeg patches](https://forum.libreelec.tv/thread/27915-explanation-of-the-ffmpeg-patches/)
|
||
- [LibreELEC forum: HW accelerated video in 2022](https://forum.libreelec.tv/thread/25198-hardware-accelerated-video-in-2022/)
|
||
- [PINE64 wiki: Mainline Hardware Decoding](https://wiki.pine64.org/wiki/Mainline_Hardware_Decoding)
|
||
- [chromium-dev: V4L2 AV1 decoding on rk3588 linux](https://groups.google.com/a/chromium.org/g/chromium-dev/c/Dq5by1PeJXA)
|
||
- [enhanced-h264ify Firefox addon](https://addons.mozilla.org/en-US/firefox/addon/enhanced-h264ify/)
|
||
- [PanVK Mesa 26.1 extension sprint](https://christian-gmeiner.info/2026-04-20-panvk-extensions/)
|
||
- [Rockchip RK3588 mainline status 2025 (CNX)](https://www.cnx-software.com/2024/12/21/rockchip-rk3588-mainline-linux-support-current-status-and-future-work-for-2025/)
|
||
- [Hantro G1 RK3588 patches v7 (lore)](https://lore.kernel.org/linux-kernel/20240430024002.708227-1-liujianfeng1994@gmail.com/T/)
|