Fourier — Hardware-Assisted Video Decoding on the Rockchip Fleet
One umbrella project to bring up working HW video decode on all four Rockchip devices in Markus' fleet. Named 2026-04-24.
Target fleet
| Host | SoC | Role | Parent umbrella |
|---|---|---|---|
| fresnel | RK3399 | Pinebook Pro fleet laptop | — |
| ohm | RK3566 | PineTab2 tablet | — |
| boltzmann | RK3588 | Rock 5 ITX+ (always on) | Volta |
| ampere | RK3588 | CoolPi GenBook laptop | Coulomb |
Current priority: ohm. The other three follow in the order of easiest-win (fresnel, mature mainline) → incremental (ampere, RK3588 landed Feb 2026) → hardest (boltzmann, currently on vendor kernel).
Related Rockchip projects
Fourier touches hardware that already has its own umbrellas. Keep display/boot concerns in those projects; keep video decode in Fourier.
- Coulomb — ampere / CoolPi GenBook stack. Subprojects:
- Bin — u-boot, display bringup (eDP + keyboard)
- MegabitChip — DDR init blob matching-decomp / HIL
- RockHard — mainline kernel (Collabora TF-A, LP5-3200 OC)
- Volta — boltzmann / Rock 5 ITX+ stack. Subprojects:
- Quark — edk2-rk3588 UEFI firmware
- Neutron — mainline / vendor kernel, UEFI-booted
- fresnel — Pinebook Pro, custom overclocked kernel package (no umbrella)
- ohm — PineTab2, DanctNIX base, UART capture rig for ampere DDR work
Cross-cutting notes in /home/mfritsche/.claude/projects/-home-mfritsche-claude/memory/
— see MEMORY.md for the index.
Meet His
When Fourier needs infra muscle (wake a sleeping host, reach a VPN peer, bump DHCP on the Fritz, find the right lmcp token, set up distcc for a kernel build), summon His — the Home Infrastructure Specialist.
- As a subagent:
Agent(subagent_type: "his", prompt: "...")— takes over a task end-to-end and returns a report. - As a skill:
Skill(skill: "his")— loads the runbook cheatsheet into the current session.
His knows the distcc workers (tesla, CT108, dcc1), the Fritz!DECT plug blast
radii (careful with Himbeere + Office), the /opt/herding/ layout on hertz,
the lmcp endpoint token map, and the VPN naming (<host>.vpn via shannon's
dnsmasq). Canonical runbook: /home/mfritsche/claude/CLAUDE.md on noether.
For Fourier specifically, expect to lean on His for: waking ohm over LAN + VPN, confirming ampere is at .vpn (not the stale .88.x LAN IP), setting up cross-builds of patched ffmpeg via distcc, and nudging kernel packages into the marfrit repo.
Two kernel paths
Rockchip hardware decode has two incompatible kernel driver families. Userspace config diverges accordingly — don't mix flags.
Path A — mainline V4L2 stateless
- Kernel:
rkvdec/rkvdec2stateless V4L2 drivers (mainline or out-of-tree patch series). - /dev node: V4L2 m2m
/dev/videoNwith compressed H.264/HEVC/VP9 queues + uncompressed output queue. - FFmpeg:
ffmpeg-v4l2-request-gitfrom AUR (Jernej's patchset). Upstream v2 patchset posted on ffmpeg-devel 2024-08, not yet merged as of 2026-04-24. - GStreamer:
v4l2codecsplugin in gst-plugins-bad. 1.26 added Rockchip buffer formats; 1.28 added the new HEVC UAPI controls. - mpv:
mpv --hwdec=drm --vo=gpu-next(wayland compositor) or--vo=dmabuf-wayland(wlroots). - Kodi: native V4L2-request API support, independent of FFmpeg.
- VA-API:
libva-v4l2-requestbridge for Firefox / Chromium paths.
Path B — Rockchip MPP vendor
- Kernel:
mpp_rkvdec/mpp_rkvdec2from Rockchip BSP (the rkr* kernel series). - /dev node:
/dev/mpp_servicechardev, not V4L2. - FFmpeg: build with
--enable-rkmppagainstlibrockchip-mpp. - GStreamer: Rockchip-specific plugins (
rkximagesink,mppvideodec, …). - Present today on boltzmann (kernel 6.1.75-rkr3).
rkvdec mainline status (2026-04-24)
| SoC | Block | Mainline status | Codecs (mainline) |
|---|---|---|---|
| RK3399 | Hantro G1/G2 | mature | MPEG-2, VP8, H.264, HEVC, VP9 |
| RK3566 | rkvdec2 (VDPU346) | not yet (Collabora TODO) | rkvdec1: MPEG-2/VP8/H.264; HEVC/VP9 need out-of-tree |
| RK3588 | VDPU381 | merged 2026-02-27 | H.264, H.265 (no VP9/AV1 upstream yet) |
| RK3576 | VDPU383 | merged 2026-02-27 | H.264, H.265 |
The RK3588 / RK3576 landing was the Feb 2026 Collabora drop. VDPU346 for the RK356x family is on the roadmap but not merged; ohm depends on DanctNIX carrying out-of-tree rkvdec2 patches if we want HEVC/VP9.
ohm — first priority, known recipe
From Martin Chang's blog (clehaxze.tw, 2023, still mirrored by DanctNIX):
sudo pacman -S mpv
yay -S ffmpeg-v4l2-request-git
mpv --hwdec=drm --vo=gpu-next --wayland-disable-vsync=yes input.mp4
# wlroots compositor variant:
mpv --hwdec=drm --vo=dmabuf-wayland input.mp4
2023 coverage: MPEG-2, VP8, H.264. 1080p60 H.264 hit ~80% of one CPU core — decode was on the hardware; the CPU was in the compositor path.
Open question: does the current linux-pinetab2 6.15-danctnix2 carry out-of-tree rkvdec2 patches for HEVC/VP9, and if so, does the 2023 recipe still work verbatim? Next live recon on ohm answers this.
Plan (tasks #18–24 in the noether task list)
- Live recon — kernel,
/dev/video*,v4l2-ctl --list-devices, dmesg rkvdec,/lib/firmware/rockchip/, installed ffmpeg/gstreamer/mpv/kodi - SW baseline — 1080p H.264 / HEVC / VP9 benchmark with ffmpeg and mpv hwdec=no, record CPU% + fps
- Driver binding — confirm V4L2 stateless decode node exposed; if not, diagnose kconfig / DT / firmware
- GStreamer v4l2codecs — cleanest proof;
v4l2slh264decpipeline withfakesink, then with display sink - FFmpeg v4l2-request — install
ffmpeg-v4l2-request-git(AUR) or verify a DanctNIX-shipped ffmpeg with v4l2-request enabled - Kodi + mpv validation — 1080p HEVC at <30% of one core target
- Document — freeze the final recipe in this README
Current blocker: ohm offline. Last known VPN state — only nc connected;
LAN gives "No route to host" for ohm.fritz.box (192.168.88.168). Ask
Markus to wake / reach out when ready.
Acceptance criterion
1080p @ 30 fps, no dropped frames. Applies to every device and every
codec we claim support for. Same bar everywhere — don't grade RK3566 on a
curve vs RK3588. Dropped-frame count comes from mpv --msg-level=all=info
or ffmpeg -f null -; "no dropped frames" means zero across a 60 s clip.
Test corpus
Big Buck Bunny is the canonical open test clip (Blender Foundation 2008, CC-BY), ubiquitous in HW-decode testing across a decade of embedded silicon. Various encodes readily available at peach.blender.org and mirrors.
Plan: park a fixed set on doppler under /moviedata/fourier-test/ so Gerbera
re-indexes it and every fleet device on LAN + VPN streams identical bits.
Canonical encodes:
| File | Codec | Res | Notes |
|---|---|---|---|
| bbb_1080p30_h264.mp4 | H.264 | 1080p | covers all four devices |
| bbb_1080p30_hevc.mp4 | HEVC | 1080p | ampere/boltzmann full; ohm pending rkvdec2 |
| bbb_1080p30_vp9.webm | VP9 | 1080p | ohm pending; RK3588 pending upstream |
| bbb_1080p30_mpeg2.ts | MPEG-2 | 1080p | Path A fresnel + rkvdec1 baseline |
| bbb_1080p30_vp8.webm | VP8 | 1080p | fresnel + RK356x rkvdec1 baseline |
| bbb_2160p60_hevc.mp4 | HEVC | 4K60 | RK3588 stretch goal (ampere/boltzmann only) |
Stretch: add Tears of Steel (2012, also CC-BY) for HDR / high-bitrate HEVC once the stretch goal is viable.
After ohm
- ampere (RK3588) — once RockHard kernel tracks a tree with VDPU381
merged (>= Linux 6.14-ish with the Feb 2026 backport), add
v4l2codecsandffmpeg-v4l2-requestpackaging to RockHard's output. Should be a small step. - boltzmann (RK3588) — dual-path decision. Short-term: use Path B with rkmpp (the kernel already exposes it). Long-term: migrate Neutron to a mainline kernel with VDPU381 and move to Path A for symmetry with ampere.
- fresnel (RK3399) — Hantro is mature upstream; likely only needs userspace install + recipe validation. Endeavour OS package names TBD.
Patches upstreamed along the way (v4l2-request to FFmpeg, VDPU346 to linux-media) count double — they benefit the whole fleet.
Packaging — marfrit-packages
New fleet hosts should get Fourier userspace via pacman -S / apt install,
not per-device AUR rebuilds. Mirror plan in marfrit-packages (layout:
arch/<pkg>/PKGBUILD, debian/<pkg>/build-deb.sh + debian/):
| Package | Source | CI runner | Target |
|---|---|---|---|
ffmpeg-v4l2-request-git |
fork of AUR (Jernej's patchset on ffmpeg tip) | fermi (Arch ARM aarch64) | Path A: ohm, fresnel, ampere |
ffmpeg-v4l2-request |
same patches against Debian ffmpeg src | feynman (Debian aarch64) | Path A Debian hosts |
gst-plugins-bad-fourier |
only if stock distro package is <1.28 | fermi / feynman | 1.28 HEVC UAPI uplift |
libva-v4l2-request |
upstream tag | fermi / feynman | VA-API bridge |
The Arch package is the priority (ohm + fresnel are Arch-based). Debian
package comes second (relevant if a fleet host ever runs Debian — kepler or
similar). Both are provides=(ffmpeg) conflicts=(ffmpeg), so install
deliberately replaces the distro ffmpeg on a given host.
CI trigger on each push. Artifacts land in packages.reauktion.de
(reference_marfrit_repo_bootstrap.md in noether memory has the client
bootstrap).
Out of scope: rkmpp-based FFmpeg (Path B). Boltzmann stays on rkr* until Neutron migrates; that host doesn't need a marfrit-packages entry until migration.
Working agreements
Standing rules for how we run this project — inherited from the broader
collaboration canon (feedback_* / project_* in noether's memory system),
captured here so a cold-start Fourier session doesn't re-learn them.
ReCAP — ReContextualization After Pruning
Claude's context gets compacted when long sessions hit the window limit, and any live-session state not in durable storage vanishes. Our counter:
- Memory files (
/home/mfritsche/.claude/projects/-home-mfritsche-claude/memory/,MEMORY.mdis the index) are the long-term substrate. Don't keep load-bearing facts only in conversation. - Project READMEs (this file) carry the research dossier + current plan. Update them when state changes, not later.
- Task list on noether carries active work. Status transitions are cheap — mark in_progress when starting, completed when done.
- After a
/compact, re-read relevant memory files + README + recent git log before reasoning. Don't infer from conversation alone.
See project_recap.md in memory for the full protocol.
Commit per experiment
Every experiment that touches the tree — a kconfig change, a DT tweak, a ffmpeg build flag, a test clip benchmark — gets its own commit on a WIP branch, with a short message naming what changed and what was observed.
- Dirty trees are tech debt. If the tree is dirty for >30 min, commit.
- Include benchmark numbers / dmesg excerpts / observation in the commit body, not just the code diff.
- Rebase / squash later if a clean series matters; don't delay the commit waiting for it.
See feedback_commit_cadence.md in memory.
Ask before flash / reboot
This project spans fleet laptops and always-on hosts. We have real blast radius.
- Never flash anything without a verified, tested backup. Fritz!Box 7490 was
bricked by flashing with an empty backup on 2026-04-12 — we don't repeat
that class of mistake. See
feedback_flash_critical.md. - Shared hardware reboots pause the user's other work. Ask before rebooting
data (Proxmox), hertz (home-LAN spine), boltzmann (always-on build host).
See
feedback_no_bulldoze_reboots.md. - For kernel / u-boot / firmware changes: simulate first where possible
(QEMU, module-style load) before flashing silicon. See
feedback_simulate_first.md. - Never run
update-initramfsor equivalent on a remote host that boots from a non-standard root (ZFS, btrfs-with-subvols). Seefeedback_initramfs.md.
Off-machine backups
Fleet laptops back up to data via backintime (Anacron mode, daily attempts,
VPN-guarded, 30-day staleness alert by email). Hertz backs up to data
weekly / quarterly via backup-hertz.sh (vitruvius dashboard shows
progress). Data itself lives on ZFS RAIDZ2 + snapshots.
- Before any invasive change on a fleet laptop, confirm its last successful
backintime snapshot in
/opt/herding/var/backintime-status/<host>on hertz. - New per-device state (kernel patch series, ffmpeg build tree) lives in a Gitea repo + gets backed up with the rest of the working tree — don't leave unique work on a single disk.
- External photos / family data on hertz is mirrored via the hertz backup to
data. Restore recipe is in the two-step
restore-hertz-step1-sd.sh/restore-hertz-step2-lxd.shon data.
See also — broader feedback canon
- Test the observer first (
feedback_observer_first.md) — before drawing decode-performance conclusions, confirm the rig (v4l2-ctl reports sensible caps? ffmpeg actually routing through v4l2request?). - Three strikes then verify (
feedback_three_strikes.md) — after two failed fixes, stop guessing; verify the binary / wire / protocol. - TRM or nothing (
feedback_trm_or_nothing.md) — register writes need documentation backing. Relevant for any DT or driver patches we ship. - Trust Markus' eyes (
feedback_trust_user_eyes.md) — if he reports "it plays smoothly", that's primary evidence. Don't over-qualify. - No budget framing (
feedback_no_budget_framing.md) — don't pre-shrink scope citing "session cost"; Markus sets pace.
References
Mainline kernel state
- Rockchip RK3588 / RK3576 H.264 and H.265 video decoders gain mainline Linux support (CNX Software, 2026-02-27)
- RK3588 and RK3576 video decoders support merged in the upstream Linux Kernel (Collabora)
- Upstream support for Rockchip's RK3588: Progress and future plans (Collabora)
- media: rkvdec: Add support for VDPU381 and VDPU383 (LWN)
- RKVDEC2 Driver Posted For Accelerated Video Decoding On Newer Rockchip SoCs (Phoronix)
FFmpeg V4L2 request API
- PATCH v2 0/8 Add V4L2 Request API hwaccels for MPEG2, H.264 and HEVC (ffmpeg-devel, 2024-08)
- Miouyouyou/FFmpeg-V4L2-Request (build script)
- ffmpeg v4l2 requests 4.4.3 patchset (artemis.sh, 2023-03)
GStreamer v4l2codecs
- GStreamer v4l2codecs plugin docs
- Adding VP9 and MPEG2 stateless support in v4l2codecs for GStreamer (Collabora, 2021)
- GStreamer 1.26 — improved hardware efficiency (Collabora)