From c51760dae1516fe1c11e97f4c05755485c085182 Mon Sep 17 00:00:00 2001 From: Markus Fritsche Date: Fri, 24 Apr 2026 18:46:07 +0000 Subject: [PATCH] =?UTF-8?q?Fourier=20=E2=80=94=20initial=20dossier=20+=20p?= =?UTF-8?q?lan?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Umbrella for HW-assisted video decode across fresnel/ohm/boltzmann/ampere. Captures Path A (mainline V4L2 stateless) vs Path B (Rockchip MPP vendor), rkvdec mainline status as of 2026-04-24 (VDPU381/383 landed Feb 2026; VDPU346 for RK356x still pending), the 2023 working recipe from clehaxze.tw, plan for ohm as first priority, and working-agreement reminders (ReCAP, commit-per-experiment, ask-before-flash/reboot, off-machine backups). --- README.md | 262 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 262 insertions(+) create mode 100644 README.md diff --git a/README.md b/README.md new file mode 100644 index 0000000..643dda4 --- /dev/null +++ b/README.md @@ -0,0 +1,262 @@ +# Fourier — Hardware-Assisted Video Decoding on the Rockchip Fleet + +One umbrella project to bring up working HW video decode on all four Rockchip +devices in Markus' fleet. Named 2026-04-24. + +## Target fleet + +| Host | SoC | Role | Parent umbrella | +|-----------|--------|---------------------------|-----------------| +| fresnel | RK3399 | Pinebook Pro fleet laptop | — | +| ohm | RK3566 | PineTab2 tablet | — | +| boltzmann | RK3588 | Rock 5 ITX+ (always on) | Volta | +| ampere | RK3588 | CoolPi GenBook laptop | Coulomb | + +Current priority: **ohm**. The other three follow in the order of easiest-win +(fresnel, mature mainline) → incremental (ampere, RK3588 landed Feb 2026) → +hardest (boltzmann, currently on vendor kernel). + +## Related Rockchip projects + +Fourier touches hardware that already has its own umbrellas. Keep display/boot +concerns in those projects; keep video decode in Fourier. + +- **Coulomb** — ampere / CoolPi GenBook stack. Subprojects: + - **Bin** — u-boot, display bringup (eDP + keyboard) + - **MegabitChip** — DDR init blob matching-decomp / HIL + - **RockHard** — mainline kernel (Collabora TF-A, LP5-3200 OC) +- **Volta** — boltzmann / Rock 5 ITX+ stack. Subprojects: + - **Quark** — edk2-rk3588 UEFI firmware + - **Neutron** — mainline / vendor kernel, UEFI-booted +- **fresnel** — Pinebook Pro, custom overclocked kernel package (no umbrella) +- **ohm** — PineTab2, DanctNIX base, UART capture rig for ampere DDR work + +Cross-cutting notes in `/home/mfritsche/.claude/projects/-home-mfritsche-claude/memory/` +— see `MEMORY.md` for the index. + +## Meet His + +When Fourier needs infra muscle (wake a sleeping host, reach a VPN peer, bump +DHCP on the Fritz, find the right lmcp token, set up distcc for a kernel +build), summon **His** — the Home Infrastructure Specialist. + +- As a subagent: `Agent(subagent_type: "his", prompt: "...")` — takes over a + task end-to-end and returns a report. +- As a skill: `Skill(skill: "his")` — loads the runbook cheatsheet into the + current session. + +His knows the distcc workers (tesla, CT108, dcc1), the Fritz!DECT plug blast +radii (careful with Himbeere + Office), the `/opt/herding/` layout on hertz, +the lmcp endpoint token map, and the VPN naming (`.vpn` via shannon's +dnsmasq). Canonical runbook: `/home/mfritsche/claude/CLAUDE.md` on noether. + +For Fourier specifically, expect to lean on His for: waking ohm over LAN + +VPN, confirming ampere is at .vpn (not the stale .88.x LAN IP), setting up +cross-builds of patched ffmpeg via distcc, and nudging kernel packages into +the marfrit repo. + +## Two kernel paths + +Rockchip hardware decode has **two incompatible kernel driver families**. +Userspace config diverges accordingly — don't mix flags. + +### Path A — mainline V4L2 stateless + +- Kernel: `rkvdec` / `rkvdec2` stateless V4L2 drivers (mainline or out-of-tree + patch series). +- /dev node: V4L2 m2m `/dev/videoN` with compressed H.264/HEVC/VP9 queues + + uncompressed output queue. +- FFmpeg: `ffmpeg-v4l2-request-git` from AUR (Jernej's patchset). Upstream + v2 patchset posted on ffmpeg-devel 2024-08, **not yet merged** as of + 2026-04-24. +- GStreamer: `v4l2codecs` plugin in gst-plugins-bad. 1.26 added Rockchip + buffer formats; 1.28 added the new HEVC UAPI controls. +- mpv: `mpv --hwdec=drm --vo=gpu-next` (wayland compositor) or + `--vo=dmabuf-wayland` (wlroots). +- Kodi: native V4L2-request API support, independent of FFmpeg. +- VA-API: `libva-v4l2-request` bridge for Firefox / Chromium paths. + +### Path B — Rockchip MPP vendor + +- Kernel: `mpp_rkvdec` / `mpp_rkvdec2` from Rockchip BSP (the rkr* kernel + series). +- /dev node: `/dev/mpp_service` chardev, not V4L2. +- FFmpeg: build with `--enable-rkmpp` against `librockchip-mpp`. +- GStreamer: Rockchip-specific plugins (`rkximagesink`, `mppvideodec`, …). +- Present today on **boltzmann** (kernel 6.1.75-rkr3). + +## rkvdec mainline status (2026-04-24) + +| SoC | Block | Mainline status | Codecs (mainline) | +|---------|--------------------|------------------------------|------------------------------------------------------| +| RK3399 | Hantro G1/G2 | mature | MPEG-2, VP8, H.264, HEVC, VP9 | +| RK3566 | rkvdec2 (VDPU346) | **not yet** (Collabora TODO) | rkvdec1: MPEG-2/VP8/H.264; HEVC/VP9 need out-of-tree | +| RK3588 | VDPU381 | **merged 2026-02-27** | H.264, H.265 (no VP9/AV1 upstream yet) | +| RK3576 | VDPU383 | merged 2026-02-27 | H.264, H.265 | + +The RK3588 / RK3576 landing was the Feb 2026 Collabora drop. VDPU346 for the +RK356x family is on the roadmap but not merged; ohm depends on DanctNIX +carrying out-of-tree rkvdec2 patches if we want HEVC/VP9. + +## ohm — first priority, known recipe + +From Martin Chang's blog (clehaxze.tw, 2023, still mirrored by DanctNIX): + +```sh +sudo pacman -S mpv +yay -S ffmpeg-v4l2-request-git +mpv --hwdec=drm --vo=gpu-next --wayland-disable-vsync=yes input.mp4 +# wlroots compositor variant: +mpv --hwdec=drm --vo=dmabuf-wayland input.mp4 +``` + +2023 coverage: **MPEG-2, VP8, H.264**. 1080p60 H.264 hit ~80% of one CPU core +— decode was on the hardware; the CPU was in the compositor path. + +**Open question**: does the current linux-pinetab2 6.15-danctnix2 carry +out-of-tree rkvdec2 patches for HEVC/VP9, and if so, does the 2023 recipe +still work verbatim? Next live recon on ohm answers this. + +### Plan (tasks #18–24 in the noether task list) + +1. **Live recon** — kernel, `/dev/video*`, `v4l2-ctl --list-devices`, dmesg + rkvdec, `/lib/firmware/rockchip/`, installed ffmpeg/gstreamer/mpv/kodi +2. **SW baseline** — 1080p H.264 / HEVC / VP9 benchmark with ffmpeg and mpv + hwdec=no, record CPU% + fps +3. **Driver binding** — confirm V4L2 stateless decode node exposed; if not, + diagnose kconfig / DT / firmware +4. **GStreamer v4l2codecs** — cleanest proof; `v4l2slh264dec` pipeline with + `fakesink`, then with display sink +5. **FFmpeg v4l2-request** — install `ffmpeg-v4l2-request-git` (AUR) or + verify a DanctNIX-shipped ffmpeg with v4l2-request enabled +6. **Kodi + mpv validation** — 1080p HEVC at <30% of one core target +7. **Document** — freeze the final recipe in this README + +**Current blocker**: ohm offline. Last known VPN state — only `nc` connected; +LAN gives "No route to host" for `ohm.fritz.box` (192.168.88.168). Ask +Markus to wake / reach out when ready. + +## After ohm + +- **ampere** (RK3588) — once RockHard kernel tracks a tree with VDPU381 + merged (>= Linux 6.14-ish with the Feb 2026 backport), add `v4l2codecs` and + `ffmpeg-v4l2-request` packaging to RockHard's output. Should be a small + step. +- **boltzmann** (RK3588) — dual-path decision. Short-term: use Path B with + rkmpp (the kernel already exposes it). Long-term: migrate Neutron to a + mainline kernel with VDPU381 and move to Path A for symmetry with ampere. +- **fresnel** (RK3399) — Hantro is mature upstream; likely only needs + userspace install + recipe validation. Endeavour OS package names TBD. + +Patches upstreamed along the way (v4l2-request to FFmpeg, VDPU346 to +linux-media) count double — they benefit the whole fleet. + +## Working agreements + +Standing rules for how we run this project — inherited from the broader +collaboration canon (`feedback_*` / `project_*` in noether's memory system), +captured here so a cold-start Fourier session doesn't re-learn them. + +### ReCAP — ReContextualization After Pruning + +Claude's context gets compacted when long sessions hit the window limit, and +any live-session state not in durable storage vanishes. Our counter: + +- Memory files (`/home/mfritsche/.claude/projects/-home-mfritsche-claude/memory/`, + `MEMORY.md` is the index) are the long-term substrate. Don't keep + load-bearing facts only in conversation. +- Project READMEs (this file) carry the research dossier + current plan. + Update them when state changes, not later. +- Task list on noether carries active work. Status transitions are cheap — + mark in_progress when starting, completed when done. +- After a `/compact`, re-read relevant memory files + README + recent git log + before reasoning. Don't infer from conversation alone. + +See `project_recap.md` in memory for the full protocol. + +### Commit per experiment + +Every experiment that touches the tree — a kconfig change, a DT tweak, a +ffmpeg build flag, a test clip benchmark — gets its own commit on a WIP +branch, with a short message naming what changed and what was observed. + +- Dirty trees are tech debt. If the tree is dirty for >30 min, commit. +- Include benchmark numbers / dmesg excerpts / observation in the commit + body, not just the code diff. +- Rebase / squash later if a clean series matters; don't delay the commit + waiting for it. + +See `feedback_commit_cadence.md` in memory. + +### Ask before flash / reboot + +This project spans fleet laptops and always-on hosts. We have **real blast +radius**. + +- Never flash anything without a verified, tested backup. Fritz!Box 7490 was + bricked by flashing with an empty backup on 2026-04-12 — we don't repeat + that class of mistake. See `feedback_flash_critical.md`. +- Shared hardware reboots pause the user's other work. Ask before rebooting + data (Proxmox), hertz (home-LAN spine), boltzmann (always-on build host). + See `feedback_no_bulldoze_reboots.md`. +- For kernel / u-boot / firmware changes: simulate first where possible + (QEMU, module-style load) before flashing silicon. See + `feedback_simulate_first.md`. +- **Never** run `update-initramfs` or equivalent on a remote host that boots + from a non-standard root (ZFS, btrfs-with-subvols). See + `feedback_initramfs.md`. + +### Off-machine backups + +Fleet laptops back up to data via backintime (Anacron mode, daily attempts, +VPN-guarded, 30-day staleness alert by email). Hertz backs up to data +weekly / quarterly via `backup-hertz.sh` (vitruvius dashboard shows +progress). Data itself lives on ZFS RAIDZ2 + snapshots. + +- Before any invasive change on a fleet laptop, confirm its last successful + backintime snapshot in `/opt/herding/var/backintime-status/` on + hertz. +- New per-device state (kernel patch series, ffmpeg build tree) lives in a + Gitea repo + gets backed up with the rest of the working tree — don't + leave unique work on a single disk. +- External photos / family data on hertz is mirrored via the hertz backup to + data. Restore recipe is in the two-step `restore-hertz-step1-sd.sh` / + `restore-hertz-step2-lxd.sh` on data. + +### See also — broader feedback canon + +- **Test the observer first** (`feedback_observer_first.md`) — before + drawing decode-performance conclusions, confirm the rig (v4l2-ctl reports + sensible caps? ffmpeg actually routing through v4l2request?). +- **Three strikes then verify** (`feedback_three_strikes.md`) — after two + failed fixes, stop guessing; verify the binary / wire / protocol. +- **TRM or nothing** (`feedback_trm_or_nothing.md`) — register writes need + documentation backing. Relevant for any DT or driver patches we ship. +- **Trust Markus' eyes** (`feedback_trust_user_eyes.md`) — if he reports + "it plays smoothly", that's primary evidence. Don't over-qualify. +- **No budget framing** (`feedback_no_budget_framing.md`) — don't pre-shrink + scope citing "session cost"; Markus sets pace. + +## References + +### Mainline kernel state +- [Rockchip RK3588 / RK3576 H.264 and H.265 video decoders gain mainline Linux support (CNX Software, 2026-02-27)](https://www.cnx-software.com/2026/02/27/rockchip-rk3588-rk3576-h-264-and-h-265-video-decoders-mainline-linux/) +- [RK3588 and RK3576 video decoders support merged in the upstream Linux Kernel (Collabora)](https://www.collabora.com/news-and-blog/news-and-events/rk3588-and-rk3576-video-decoders-support-merged-in-the-upstream-linux-kernel.html) +- [Upstream support for Rockchip's RK3588: Progress and future plans (Collabora)](https://www.collabora.com/news-and-blog/news-and-events/rockchip-rk3588-upstream-support-progress-future-plans.html) +- [media: rkvdec: Add support for VDPU381 and VDPU383 (LWN)](https://lwn.net/Articles/1053556/) +- [RKVDEC2 Driver Posted For Accelerated Video Decoding On Newer Rockchip SoCs (Phoronix)](https://www.phoronix.com/news/RKVDEC2-Rockchip-Video-Decode) + +### FFmpeg V4L2 request API +- [PATCH v2 0/8 Add V4L2 Request API hwaccels for MPEG2, H.264 and HEVC (ffmpeg-devel, 2024-08)](https://ffmpeg.org/pipermail/ffmpeg-devel/2024-August/332034.html) +- [Miouyouyou/FFmpeg-V4L2-Request (build script)](https://github.com/Miouyouyou/FFmpeg-V4L2-Request) +- [ffmpeg v4l2 requests 4.4.3 patchset (artemis.sh, 2023-03)](https://artemis.sh/2023/03/06/ffmpeg-v4l2-requests-4.4.3.html) + +### GStreamer v4l2codecs +- [GStreamer v4l2codecs plugin docs](https://gstreamer.freedesktop.org/documentation/v4l2codecs/index.html) +- [Adding VP9 and MPEG2 stateless support in v4l2codecs for GStreamer (Collabora, 2021)](https://www.collabora.com/news-and-blog/blog/2021/06/23/adding-vp9-and-mpeg2-stateless-support-in-v4l2codecs-for-gstreamer/) +- [GStreamer 1.26 — improved hardware efficiency (Collabora)](https://www.collabora.com/news-and-blog/news-and-events/gstreamer-126-improved-hardware-efficiency.html) + +### PineTab2 specific +- [Hardware accelerated playback on PineTab 2 (clehaxze.tw, 2023-09)](https://clehaxze.tw/gemlog/2023/09-17-hardware-accelerated-playback-on-pinetab2.gmi) +- [PINE64 Mainline Hardware Decoding wiki](https://wiki.pine64.org/wiki/Mainline_Hardware_Decoding) +- [dreemurrs-embedded/linux-pinetab2 releases (6.15.2-danctnix2 latest)](https://github.com/dreemurrs-embedded/linux-pinetab2/releases)