marfrit/marfrit-packages
0
0
Fork 1
Code Issues 2 Pull Requests Actions Packages Projects Releases Wiki Activity

upstream-submissions/fourier-campaign: YouTube HW decode delivery plan

Browse Source 
Full state-of-play matrix (browser × SoC × codec) plus per-SoC
ranked-by-tractability YouTube delivery paths, drafted by the Plan
subagent with web research synthesizing LibreELEC/CoreELEC patterns,
Mozilla bugzilla, Collabora's mainline Rockchip status, and
chromium-dev threads. Sources cited at the end.

Headline:
- fresnel ships today via enhanced-h264ify forcing YouTube to
  serve H.264 (rkvdec already validated bbb_1080p30 RDD 78% to 5%)
- ohm needs VP9 hwaccel verification against hantro before claims
- ampere blocked on Linux 7.0 / VDPU381 rkvdec2 driver kernel rebase
- mpv-as-YouTube is the LibreELEC-inspired browser-bypass path

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Markus Fritsche
2026-04-29 21:56:12 +00:00
parent e9b5252314
commit 3b453de45a
1 changed files with 132 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
upstream-submissions/fourier-campaign/youtube-hw-decode-plan.md
+132
View File
@@ -0,0 +1,132 @@
# YouTube Hardware Decode Delivery Plan — Fourier Campaign
Drafted 2026-04-29 by the Plan subagent ("Sonnet Architect") with web
research synthesizing LibreELEC/CoreELEC patterns, Mozilla bugzilla,
Collabora's mainline progress reports, and chromium-dev threads.
Audience: future-Claude after the next compaction, plus the user
reading async.
## 1. State-of-play matrix (browser × SoC × codec, 2026-04-29)
Legend: ✓ shipping · 🚧 in flight · ⚠ known gap · ✗ infeasible on this hardware · — not advertised by SoC
### Firefox (firefox-fourier 150.0.1-16, libavcodec v4l2_request route)
| SoC / VPU | H.264 | HEVC | VP9 | AV1 |
|---|---|---|---|---|
| **fresnel** RK3399 / rkvdec | ✓ validated bbb_1080p30, RDD 78%→5% | ✗ no HW | ✗ no HW | ✗ no HW |
| **ohm** RK3566 / hantro | 🚧 should work, untested | ⚠ framework only — bug 1969297 parser/slicer pieces missing | ⚠ fourcc registered, hwaccel path unverified | — hantro lacks AV1 |
| **ampere** RK3588 / rkvdec2 | 🚧 needs rkvdec2 driver (VDPU381 merged for ~Linux 7.0) | 🚧 same — VDPU381 H.265 controls just merged | ⚠ Collabora "future work", not yet in mainline | ⚠ rkvdec2 AV1 patches floated by Gaignard, RK3588 playback "not played well" per chromium-dev |
### Chromium (chromium-fourier, V4L2 stateless via `media/gpu/v4l2/stateless/`)
| SoC | H.264 | HEVC | VP9 | AV1 |
|---|---|---|---|---|
| **fresnel** RK3399 | 🚧 install-v5 not yet deployed here | ✗ | ✗ | ✗ |
| **ohm** RK3566 | ✓ install-v5 manual deploy, decode path active | ⚠ untested for YouTube clips | ⚠ untested | — |
| **ampere** RK3588 | 🚧 driver dependency | 🚧 driver dependency | ⚠ Chromium has stateless VP9 code; needs rkvdec2 mainline | ⚠ Chromium has stateless AV1; rkvdec2 instability reported |
### mpv (`--hwdec=v4l2request`, dmabuf-wayland VO)
| SoC | H.264 | HEVC | VP9 | AV1 |
|---|---|---|---|---|
| fresnel | ✓ file playback works | ✗ | ✗ | ✗ |
| ohm | ✓ | ✓ likely | 🚧 | — |
| ampere | 🚧 | 🚧 | 🚧 | 🚧 |
## 2. Per-SoC ranked-by-tractability YouTube delivery paths
### fresnel (RK3399) — H.264-only VPU, must coerce YouTube codec
1. **Force H.264 via enhanced-h264ify in firefox-fourier.** Most tractable. Ship the extension preinstalled or as a documented one-click add-on. We already validated H.264 HW decode end-to-end. *Trade-off:* H.264 1080p stream is ~40-50% larger bandwidth than VP9 same-quality, and YouTube caps H.264 at 1080p (no 1440p/2160p H.264). For a 1080p Pinebook Pro panel, perfect fit.
2. **Same trick in chromium-fourier** once the package finishes building (data:CT220 job in flight). Use enhanced-h264ify or set `?html5=1` cookie variants.
3. **mpv-as-YouTube fallback** for fullscreen viewing: `mpv --ytdl-format='bv*[vcodec~=avc1]+ba/b[vcodec~=avc1]' --hwdec=v4l2request --vo=dmabuf-wayland`. Bypass browser entirely, use as desktop launcher / Plasma activity.
4. **Don't bother with VP9/AV1** — it's CPU-bound forever on this SoC.
### ohm (RK3566) — hantro covers H.264 + HEVC + VP8 + VP9
1. **Validate VP9 stateless end-to-end first.** Smallest unit of work that unlocks the bulk of YouTube. Tasks: (a) confirm `v4l2-ctl --list-formats` on `/dev/video*` shows `VP9F`; (b) ffmpeg sample clip via `ffmpeg -hwaccel v4l2request -i vp9.webm -f null -` to confirm libavcodec→hantro path works at the CLI; (c) only then trust firefox-fourier's 0001/0003 patches with VP9.
2. **If (1) is broken in libavcodec's v4l2_request VP9 hwaccel,** pivot to chromium-fourier's stateless code path — it's better-exercised on hantro (ChromeOS lineage). This may make Chromium the recommended default browser on ohm.
3. **Force-H.264 fallback** via enhanced-h264ify if VP9 path proves flaky — same recipe as fresnel, lossy on resolution but no stutter.
4. **HEVC (S265)** is not a YouTube codec, defer behind bug 1969297.
### ampere (RK3588) — rkvdec2 mainline arrived for H.264/HEVC, VP9/AV1 still cooking
1. **Track Linux 7.0 / Collabora VDPU381 merge** (just landed Feb 2026). Get a kernel build with this merged onto ampere ASAP — H.264 + HEVC HW decode follows. This is the single highest-leverage step.
2. **Force-H.264 in browser** as bridge: gives stutter-free 1080p YouTube the day the kernel module probes.
3. **VP9 on rkvdec2** is "future work" per Collabora — track the patches, do not commit code. Until then VP9 on ampere = SW.
4. **AV1 on rkvdec2** has out-of-tree patches floating; chromium-dev thread reports "video is not played well". Don't ship; revisit Q3 2026.
5. **panvk Vulkan video decode (`VK_KHR_video_decode_*`)** is not implemented in panvk yet (Mesa 26.0/26.1 release notes do not mention it). Not a near-term path. Mark as "watch the mesa-dev list".
## 3. LibreELEC inspiration — what to copy, what to skip
LibreELEC is the canonical "make ARM SoC video decode work end-to-end" project. Their model:
- **Kernel patch stack at `LibreELEC.tv/projects/Rockchip/patches`** — mix of upstream-pending and WIP. Specifically `linux-2000-v4l2-wip-rkvdec-hevc.patch` and the iep-driver patches show the bridging glue between mainline V4L2 stateless skeleton and what userspace actually needs. Worth diffing against our kernel build to see what we silently lack.
- **ffmpeg fork is Kwiboo's `ffmpeg-v4l2-request`** — the same fork we already package. LibreELEC's "explanation of the ffmpeg patches" forum thread documents which patches are about request API plumbing vs DRM PRIME exporters vs codec-specific slicer fixes. Use this to audit which of our own patches we really need.
- **Compromise points to copy:** LibreELEC ships a curated kernel + curated ffmpeg + curated media player (Kodi); they do NOT try to make every browser work. **Application surface is Kodi only.** This is the single biggest architectural lesson — they bypass the browser problem entirely. Our analogue is the **mpv-as-YouTube** path.
- **What NOT to copy:** they ship a read-only OS image. We're a rolling Arch derivative — different distribution model. We can't pin kernel + ffmpeg + Kodi together as a single artifact.
- **Specific things to mine:**
- their `package/mediacenter/kodi/patches/` for V4L2 request hwaccel integration patterns (relevant to our ffmpeg/mpv config, not browsers)
- their iommu / dma-buf workarounds — overlap with our vb2-dma-resv RFC
- LibreELEC forum thread #25198 "Hardware accelerated video in 2022" for the full architecture writeup
## 4. Decision criteria — when do we stop
**Per-SoC target bar (ship gate):**
> 1080p30 YouTube playback at <30% total CPU, zero dropped frames over a 60-second window, on the codec YouTube serves by default for that device class.
**Per-SoC accept conditions:**
- **fresnel:** 1080p30 H.264 YouTube via firefox-fourier + enhanced-h264ify, <30% CPU on 1× big-cluster core. Already 80% there based on bbb_1080p30 result.
- **ohm:** EITHER (a) 1080p30 VP9 native via hantro <30% CPU, OR (b) 1080p30 H.264 forced <30% CPU. Either qualifies.
- **ampere:** 1080p30 H.264 via rkvdec2 once kernel lands, <20% CPU (this SoC is overpowered, raise the bar). Stretch goal: 4K HEVC test clip via Kodi/mpv to validate the kernel/ffmpeg path even though YouTube won't serve HEVC.
**Anti-goals (explicit non-targets):**
- 4K YouTube on any SoC. None of these displays are 4K; YouTube serves VP9/AV1 at 4K; rkvdec2 VP9 isn't ready. Drop.
- AV1 anywhere. Not worth the 2026 engineering cost.
- HEVC in browsers. YouTube doesn't serve it. Track bug 1969297 passively.
## 5. Risks and dependencies
**Kernel-side blockers:**
- **rkvdec2 mainline driver for ampere** — VDPU381 H.264/HEVC merged (Linux 7.0). Need to actually rebase ampere's kernel onto this. This is the single biggest lever; until then ampere is a SW-decode device.
- **rkvdec2 VP9** — Collabora "future work", no timeline. Do not block on this.
- **vb2 dma_resv producer fences (our RFC)** — required for KWin's `watchDmaBuf` to actually wait on real fences. Already submitted; track linux-media response. If rejected, KWin patch is the workaround.
**Userspace-side blockers:**
- **Mozilla bug 1969297 (HEVC stateless)** — David Turner mid-implementation, last activity 2026-01-13. Our 0003 patch handles framework; HEVC parser/slicer integration into hwaccel ctx is the missing piece. Not on YouTube critical path; deprioritize.
- **libavcodec VP9 v4l2_request hwaccel correctness** on hantro — unverified. Highest single risk to ohm story. Test before shipping VP9 claims.
- **Chromium V4L2 stateless VP9/AV1 path** — Chromium has the code (ChromeOS lineage) but RK3588 reports are rough. For ohm/hantro VP9, Chromium's path may actually be more battle-tested than libavcodec's; consider declaring Chromium the default browser on ohm.
- **chromium-fourier PKGBUILD build** — currently in flight on data:CT220 (~4-8h). If it fails, we have install-v5 only as a manual deploy and ohm shipping story partially regresses.
**Distribution risks:**
- Rolling release means kernel/ffmpeg/Firefox can desync. Need explicit pin or version-coupling logic in the marfrit repo for the stack to stay coherent.
- KWin MR 9157 (watchDmaBuf) needs to land upstream eventually or we carry the patch forever.
## Critical Files for Implementation
- `arch/firefox-fourier/patches/0001-gfxinfo-v4l2-stateless-fourccs.patch`
- `arch/firefox-fourier/patches/0003-ffmpegvideo-v4l2-request-route.patch`
- `arch/firefox-fourier/patches/0005-rdd-sandbox-v4l2-media-ctl.patch`
- `arch/chromium-fourier/patches/enable-v4l2-decoder-default.patch`
- `arch/ffmpeg-v4l2-request-git/PKGBUILD` + `0001-libudev-bypass-fallback.patch`
## Sources
- [RK3588 / RK3576 video decoders merged (Collabora)](https://www.collabora.com/news-and-blog/news-and-events/rk3588-and-rk3576-video-decoders-support-merged-in-the-upstream-linux-kernel.html)
- [Rockchip RK3588/RK3576 H.264/H.265 mainline (CNX, 2026-02-27)](https://www.cnx-software.com/2026/02/27/rockchip-rk3588-rk3576-h-264-and-h-265-video-decoders-mainline-linux/)
- [Mozilla Bug 1969297 — HEVC v4l2-stateless](https://bugzilla.mozilla.org/show_bug.cgi?id=1969297)
- [Mozilla Bug 1833354 — V4L2-M2M HW decode](https://bugzilla.mozilla.org/show_bug.cgi?id=1833354)
- [LibreELEC Rockchip patches tree](https://github.com/LibreELEC/LibreELEC.tv/tree/master/projects/Rockchip/patches)
- [LibreELEC forum: getting HEVC decoding working](https://forum.libreelec.tv/thread/26796-getting-hevc-decoding-working-with-libreelec-patches/)
- [LibreELEC forum: explanation of the ffmpeg patches](https://forum.libreelec.tv/thread/27915-explanation-of-the-ffmpeg-patches/)
- [LibreELEC forum: HW accelerated video in 2022](https://forum.libreelec.tv/thread/25198-hardware-accelerated-video-in-2022/)
- [PINE64 wiki: Mainline Hardware Decoding](https://wiki.pine64.org/wiki/Mainline_Hardware_Decoding)
- [chromium-dev: V4L2 AV1 decoding on rk3588 linux](https://groups.google.com/a/chromium.org/g/chromium-dev/c/Dq5by1PeJXA)
- [enhanced-h264ify Firefox addon](https://addons.mozilla.org/en-US/firefox/addon/enhanced-h264ify/)
- [PanVK Mesa 26.1 extension sprint](https://christian-gmeiner.info/2026-04-20-panvk-extensions/)
- [Rockchip RK3588 mainline status 2025 (CNX)](https://www.cnx-software.com/2024/12/21/rockchip-rk3588-mainline-linux-support-current-status-and-future-work-for-2025/)
- [Hantro G1 RK3588 patches v7 (lore)](https://lore.kernel.org/linux-kernel/20240430024002.708227-1-liujianfeng1994@gmail.com/T/)