marfrit/fourier
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity
3 Commits 1 Branch 0 Tags
56a3aa28894cd5e025ce471df5f6f7397a8537b7
T
Clone
Open with VS Code Open with VSCodium Open with Intellij IDEA
Download ZIP Download TAR.GZ Download BUNDLE

README.md

Fourier — Hardware-Assisted Video Decoding on the Rockchip Fleet

One umbrella project to bring up working HW video decode on all four Rockchip devices in Markus' fleet. Named 2026-04-24.

Target fleet

Host SoC Role Parent umbrella
fresnel RK3399 Pinebook Pro fleet laptop —
ohm RK3566 PineTab2 tablet —
boltzmann RK3588 Rock 5 ITX+ (always on) Volta
ampere RK3588 CoolPi GenBook laptop Coulomb

Current priority: ohm. The other three follow in the order of easiest-win (fresnel, mature mainline) → incremental (ampere, RK3588 landed Feb 2026) → hardest (boltzmann, currently on vendor kernel).

Related Rockchip projects

Fourier touches hardware that already has its own umbrellas. Keep display/boot concerns in those projects; keep video decode in Fourier.

  • Coulomb — ampere / CoolPi GenBook stack. Subprojects:
    • Bin — u-boot, display bringup (eDP + keyboard)
    • MegabitChip — DDR init blob matching-decomp / HIL
    • RockHard — mainline kernel (Collabora TF-A, LP5-3200 OC)
  • Volta — boltzmann / Rock 5 ITX+ stack. Subprojects:
    • Quark — edk2-rk3588 UEFI firmware
    • Neutron — mainline / vendor kernel, UEFI-booted
  • fresnel — Pinebook Pro, custom overclocked kernel package (no umbrella)
  • ohm — PineTab2, DanctNIX base, UART capture rig for ampere DDR work

Cross-cutting notes in /home/mfritsche/.claude/projects/-home-mfritsche-claude/memory/ — see MEMORY.md for the index.

Meet His

When Fourier needs infra muscle (wake a sleeping host, reach a VPN peer, bump DHCP on the Fritz, find the right lmcp token, set up distcc for a kernel build), summon His — the Home Infrastructure Specialist.

  • As a subagent: Agent(subagent_type: "his", prompt: "...") — takes over a task end-to-end and returns a report.
  • As a skill: Skill(skill: "his") — loads the runbook cheatsheet into the current session.

His knows the distcc workers (tesla, CT108, dcc1), the Fritz!DECT plug blast radii (careful with Himbeere + Office), the /opt/herding/ layout on hertz, the lmcp endpoint token map, and the VPN naming (<host>.vpn via shannon's dnsmasq). Canonical runbook: /home/mfritsche/claude/CLAUDE.md on noether.

For Fourier specifically, expect to lean on His for: waking ohm over LAN + VPN, confirming ampere is at .vpn (not the stale .88.x LAN IP), setting up cross-builds of patched ffmpeg via distcc, and nudging kernel packages into the marfrit repo.

Two kernel paths

Rockchip hardware decode has two incompatible kernel driver families. Userspace config diverges accordingly — don't mix flags.

Path A — mainline V4L2 stateless

  • Kernel: rkvdec / rkvdec2 stateless V4L2 drivers (mainline or out-of-tree patch series).
  • /dev node: V4L2 m2m /dev/videoN with compressed H.264/HEVC/VP9 queues + uncompressed output queue.
  • FFmpeg: ffmpeg-v4l2-request-git from AUR (Jernej's patchset). Upstream v2 patchset posted on ffmpeg-devel 2024-08, not yet merged as of 2026-04-24.
  • GStreamer: v4l2codecs plugin in gst-plugins-bad. 1.26 added Rockchip buffer formats; 1.28 added the new HEVC UAPI controls.
  • mpv: mpv --hwdec=drm --vo=gpu-next (wayland compositor) or --vo=dmabuf-wayland (wlroots).
  • Kodi: native V4L2-request API support, independent of FFmpeg.
  • VA-API: libva-v4l2-request bridge for Firefox / Chromium paths.

Path B — Rockchip MPP vendor

  • Kernel: mpp_rkvdec / mpp_rkvdec2 from Rockchip BSP (the rkr* kernel series).
  • /dev node: /dev/mpp_service chardev, not V4L2.
  • FFmpeg: build with --enable-rkmpp against librockchip-mpp.
  • GStreamer: Rockchip-specific plugins (rkximagesink, mppvideodec, …).
  • Present today on boltzmann (kernel 6.1.75-rkr3).

rkvdec mainline status (2026-04-24)

SoC Block Mainline status Codecs (mainline)
RK3399 Hantro G1/G2 mature MPEG-2, VP8, H.264, HEVC, VP9
RK3566 rkvdec2 (VDPU346) not yet (Collabora TODO) rkvdec1: MPEG-2/VP8/H.264; HEVC/VP9 need out-of-tree
RK3588 VDPU381 merged 2026-02-27 H.264, H.265 (no VP9/AV1 upstream yet)
RK3576 VDPU383 merged 2026-02-27 H.264, H.265

The RK3588 / RK3576 landing was the Feb 2026 Collabora drop. VDPU346 for the RK356x family is on the roadmap but not merged; ohm depends on DanctNIX carrying out-of-tree rkvdec2 patches if we want HEVC/VP9.

ohm — first priority, known recipe

From Martin Chang's blog (clehaxze.tw, 2023, still mirrored by DanctNIX):

sudo pacman -S mpv
yay -S ffmpeg-v4l2-request-git
mpv --hwdec=drm --vo=gpu-next --wayland-disable-vsync=yes input.mp4
# wlroots compositor variant:
mpv --hwdec=drm --vo=dmabuf-wayland input.mp4

2023 coverage: MPEG-2, VP8, H.264. 1080p60 H.264 hit ~80% of one CPU core — decode was on the hardware; the CPU was in the compositor path.

Open question: does the current linux-pinetab2 6.15-danctnix2 carry out-of-tree rkvdec2 patches for HEVC/VP9, and if so, does the 2023 recipe still work verbatim? Next live recon on ohm answers this.

Plan (tasks #18–24 in the noether task list)

  1. Live recon — kernel, /dev/video*, v4l2-ctl --list-devices, dmesg rkvdec, /lib/firmware/rockchip/, installed ffmpeg/gstreamer/mpv/kodi
  2. SW baseline — 1080p H.264 / HEVC / VP9 benchmark with ffmpeg and mpv hwdec=no, record CPU% + fps
  3. Driver binding — confirm V4L2 stateless decode node exposed; if not, diagnose kconfig / DT / firmware
  4. GStreamer v4l2codecs — cleanest proof; v4l2slh264dec pipeline with fakesink, then with display sink
  5. FFmpeg v4l2-request — install ffmpeg-v4l2-request-git (AUR) or verify a DanctNIX-shipped ffmpeg with v4l2-request enabled
  6. Kodi + mpv validation — 1080p HEVC at <30% of one core target
  7. Document — freeze the final recipe in this README

Current blocker: ohm offline. Last known VPN state — only nc connected; LAN gives "No route to host" for ohm.fritz.box (192.168.88.168). Ask Markus to wake / reach out when ready.

Acceptance criterion

1080p @ 30 fps, no dropped frames. Applies to every device and every codec we claim support for. Same bar everywhere — don't grade RK3566 on a curve vs RK3588. Dropped-frame count comes from mpv --msg-level=all=info or ffmpeg -f null -; "no dropped frames" means zero across a 60 s clip.

Test corpus

Big Buck Bunny is the canonical open test clip (Blender Foundation 2008, CC-BY), ubiquitous in HW-decode testing across a decade of embedded silicon. Various encodes readily available at peach.blender.org and mirrors.

Plan: park a fixed set on doppler under /moviedata/fourier-test/ so Gerbera re-indexes it and every fleet device on LAN + VPN streams identical bits. Canonical encodes:

File Codec Res Notes
bbb_1080p30_h264.mp4 H.264 1080p covers all four devices
bbb_1080p30_hevc.mp4 HEVC 1080p ampere/boltzmann full; ohm pending rkvdec2
bbb_1080p30_vp9.webm VP9 1080p ohm pending; RK3588 pending upstream
bbb_1080p30_mpeg2.ts MPEG-2 1080p Path A fresnel + rkvdec1 baseline
bbb_1080p30_vp8.webm VP8 1080p fresnel + RK356x rkvdec1 baseline
bbb_2160p60_hevc.mp4 HEVC 4K60 RK3588 stretch goal (ampere/boltzmann only)

Stretch: add Tears of Steel (2012, also CC-BY) for HDR / high-bitrate HEVC once the stretch goal is viable.

After ohm

  • ampere (RK3588) — once RockHard kernel tracks a tree with VDPU381 merged (>= Linux 6.14-ish with the Feb 2026 backport), add v4l2codecs and ffmpeg-v4l2-request packaging to RockHard's output. Should be a small step.
  • boltzmann (RK3588) — dual-path decision. Short-term: use Path B with rkmpp (the kernel already exposes it). Long-term: migrate Neutron to a mainline kernel with VDPU381 and move to Path A for symmetry with ampere.
  • fresnel (RK3399) — Hantro is mature upstream; likely only needs userspace install + recipe validation. Endeavour OS package names TBD.

Patches upstreamed along the way (v4l2-request to FFmpeg, VDPU346 to linux-media) count double — they benefit the whole fleet.

Working agreements

Standing rules for how we run this project — inherited from the broader collaboration canon (feedback_* / project_* in noether's memory system), captured here so a cold-start Fourier session doesn't re-learn them.

ReCAP — ReContextualization After Pruning

Claude's context gets compacted when long sessions hit the window limit, and any live-session state not in durable storage vanishes. Our counter:

  • Memory files (/home/mfritsche/.claude/projects/-home-mfritsche-claude/memory/, MEMORY.md is the index) are the long-term substrate. Don't keep load-bearing facts only in conversation.
  • Project READMEs (this file) carry the research dossier + current plan. Update them when state changes, not later.
  • Task list on noether carries active work. Status transitions are cheap — mark in_progress when starting, completed when done.
  • After a /compact, re-read relevant memory files + README + recent git log before reasoning. Don't infer from conversation alone.

See project_recap.md in memory for the full protocol.

Commit per experiment

Every experiment that touches the tree — a kconfig change, a DT tweak, a ffmpeg build flag, a test clip benchmark — gets its own commit on a WIP branch, with a short message naming what changed and what was observed.

  • Dirty trees are tech debt. If the tree is dirty for >30 min, commit.
  • Include benchmark numbers / dmesg excerpts / observation in the commit body, not just the code diff.
  • Rebase / squash later if a clean series matters; don't delay the commit waiting for it.

See feedback_commit_cadence.md in memory.

Ask before flash / reboot

This project spans fleet laptops and always-on hosts. We have real blast radius.

  • Never flash anything without a verified, tested backup. Fritz!Box 7490 was bricked by flashing with an empty backup on 2026-04-12 — we don't repeat that class of mistake. See feedback_flash_critical.md.
  • Shared hardware reboots pause the user's other work. Ask before rebooting data (Proxmox), hertz (home-LAN spine), boltzmann (always-on build host). See feedback_no_bulldoze_reboots.md.
  • For kernel / u-boot / firmware changes: simulate first where possible (QEMU, module-style load) before flashing silicon. See feedback_simulate_first.md.
  • Never run update-initramfs or equivalent on a remote host that boots from a non-standard root (ZFS, btrfs-with-subvols). See feedback_initramfs.md.

Off-machine backups

Fleet laptops back up to data via backintime (Anacron mode, daily attempts, VPN-guarded, 30-day staleness alert by email). Hertz backs up to data weekly / quarterly via backup-hertz.sh (vitruvius dashboard shows progress). Data itself lives on ZFS RAIDZ2 + snapshots.

  • Before any invasive change on a fleet laptop, confirm its last successful backintime snapshot in /opt/herding/var/backintime-status/<host> on hertz.
  • New per-device state (kernel patch series, ffmpeg build tree) lives in a Gitea repo + gets backed up with the rest of the working tree — don't leave unique work on a single disk.
  • External photos / family data on hertz is mirrored via the hertz backup to data. Restore recipe is in the two-step restore-hertz-step1-sd.sh / restore-hertz-step2-lxd.sh on data.

See also — broader feedback canon

  • Test the observer first (feedback_observer_first.md) — before drawing decode-performance conclusions, confirm the rig (v4l2-ctl reports sensible caps? ffmpeg actually routing through v4l2request?).
  • Three strikes then verify (feedback_three_strikes.md) — after two failed fixes, stop guessing; verify the binary / wire / protocol.
  • TRM or nothing (feedback_trm_or_nothing.md) — register writes need documentation backing. Relevant for any DT or driver patches we ship.
  • Trust Markus' eyes (feedback_trust_user_eyes.md) — if he reports "it plays smoothly", that's primary evidence. Don't over-qualify.
  • No budget framing (feedback_no_budget_framing.md) — don't pre-shrink scope citing "session cost"; Markus sets pace.

References

Mainline kernel state

FFmpeg V4L2 request API

GStreamer v4l2codecs

PineTab2 specific

Reference in New Issue View Git Blame Copy Permalink
S
Description
HW-assisted video decoding across Rockchip fleet (Pinebook Pro, PineTab2, Rock 5 ITX+, CoolPi GenBook)
Readme 237 KiB