docs: genericize host references to machine type (Rock 5 ITX+)

Replace the local host name "boltzmann" with the machine type
(Rock 5 ITX+, RK3588, 32 GB) throughout so the replication docs read
for anyone on the same hardware, not just this fleet. Rename
fleet/boltzmann.yaml -> fleet/rock5-itx-plus.yaml.
This commit is contained in:
marfrit
2026-07-15 09:59:00 +02:00
parent 27df00a201
commit c77be48151
9 changed files with 43 additions and 44 deletions
+7 -8
View File
@@ -5,8 +5,7 @@ hardware neural network (400 photocells, stepper-motor-tunable analog weights).
This project lights up the RK3588 NPU on mainline Linux, so the OSS world
finally owns the silicon-side of inference on that chip.
**Scope (Phase 1):** small LLM running CPU + NPU mix on `boltzmann` (Rock 5
ITX+, RK3588, 32 GB DDR4). Backend: `llama.cpp` with a new `rknpu` device that
**Scope (Phase 1):** small LLM running CPU + NPU mix on the Rock 5 ITX+ (RK3588, 32 GB DDR4). Backend: `llama.cpp` with a new `rknpu` device that
offloads the heavy GEMM (matmul in attention + FFN) to the NPU's INT8 path
while leaving dequant / RoPE / softmax / sampling / embedding lookup on the
A76 NEON cores.
@@ -35,10 +34,10 @@ point; the open path to it is.
| # | Phase | Deliverable |
|---|---|---|
| 1 | **Substrate** | Audit mainline NPU driver state (Tomeu Vizoso's rknpu / DRM-accel series); `/dev/accel/*` probe on boltzmann; running kernel + module inventory. `docs/npu-mainline-status.md` snapshot. |
| 1 | **Substrate** | Audit mainline NPU driver state (Tomeu Vizoso's rknpu / DRM-accel series); `/dev/accel/*` probe on the Rock 5 ITX+; running kernel + module inventory. `docs/npu-mainline-status.md` snapshot. |
| 2 | Formulate | Pick the exact matmul shape that fits the NPU's tile-MAC array. Identify the smallest-possible op-set llama.cpp can offload. |
| 3 | Analyze | Read the RKNPU2 SDK + Tomeu's rknpu uAPI to learn: register layout, DMA tensor format, INT8 quant scheme. Don't lift code — extract the spec. |
| 4 | Baseline | llama.cpp pure-CPU tok/s on boltzmann for qwen-1.5B Q4_K_M. Three runs, median. Reproducible bench script in `benchmarks/`. |
| 4 | Baseline | llama.cpp pure-CPU tok/s on the Rock 5 ITX+ for qwen-1.5B Q4_K_M. Three runs, median. Reproducible bench script in `benchmarks/`. |
| 5 | Plan | rknpu backend interface design — where it plugs into ggml's compute graph; memory mapping strategy (dmabuf vs userptr); fallback path. |
| 6 | Review | Janet (ARM/DRM specialist agent) reviews the NPU register-write + DMA fence strategy. Cold-eyes pass. |
| 7 | Implement | rknpu ggml backend skeleton + first INT8 matmul. Bit-exact against CPU reference (Q4_K dequant + fp32 matmul). |
@@ -66,7 +65,7 @@ rosenblatt/
│ ├── npu-probe/ smallest-possible "open device + run trivial matmul" sanity
│ └── llm-runtime/ llama.cpp fork with rknpu backend
├── fleet/
│ └── boltzmann.yaml host manifest (kernel + NPU driver pin, baseline measurement)
│ └── rock5-itx-plus.yaml host manifest (kernel + NPU driver pin, baseline measurement)
└── benchmarks/ reproducible bench scripts + recorded results (JSON + plots)
```
@@ -74,13 +73,13 @@ rosenblatt/
## Host
Primary: **boltzmann** (Rock 5 ITX+, RK3588, 32 GB DDR4-2666, NVMe rootfs).
Primary: **Rock 5 ITX+** (RK3588, 32 GB DDR4-2666, NVMe rootfs).
- Already runs mainline ~v7.0 with most peripheral drivers working.
- Has the Quark UEFI / Neutron kernel stack — NPU is the next missing peripheral.
- Other RK3588 hosts (`ampere` = CoolPi GenBook) come later for port-validation.
Why not `ampere`: laptop, intermittent power, in-use for other campaigns.
Boltzmann is always-on with 32 GB headroom — right substrate for kernel
The Rock 5 ITX+ is always-on with 32 GB headroom — right substrate for kernel
hacking with serial-console fallback (when [Quark](https://git.reauktion.de/marfrit/quark) exposes one).
---
@@ -109,7 +108,7 @@ We're writing the second half.
Phase-1 closeout: `docs/phases.md` + `docs/npu-mainline-status.md`.
Headline: mainline driver name is **`rocket`** (not `rknpu`); it's
already shipped in boltzmann's kernel as a built module. Phase-2
already shipped in the Rock 5 ITX+'s kernel as a built module. Phase-2
unblock is small (DT enable + IOMMU v1.0 mitigation + modprobe),
not a driver port.
+2 -2
View File
@@ -2,7 +2,7 @@
Run `llama.cpp` LLM inference on the **RK3588 NPU via the mainline `rocket`
DRM-accel driver** — no vendor `librknnrt` blob — at 1 GHz. Measured on
`boltzmann` (Radxa Rock 5 ITX+, RK3588, 32 GB): gemma-4-E2B-it-Q8_0
the Rock 5 ITX+ (RK3588, 32 GB): gemma-4-E2B-it-Q8_0
**pp512 ≈ 46.7 t/s**, within ~9 % of the vendor closed stack (51.4), both
DDR-bandwidth-bound.
@@ -150,7 +150,7 @@ output — but pp512-neutral, being bandwidth-bound).
---
## 3. Results & findings (boltzmann, gemma-4-E2B-it-Q8_0, NPU @ 1 GHz)
## 3. Results & findings (the Rock 5 ITX+, gemma-4-E2B-it-Q8_0, NPU @ 1 GHz)
| backend | pp512 (t/s) | tg128 |
|---|---|---|
+8 -8
View File
@@ -6,7 +6,7 @@ Rolling punch-list. Older items at bottom (move done → DONE.md when noisy).
## Phase 1 — substrate audit
- [ ] On boltzmann: `uname -r` → record in `fleet/boltzmann.yaml:kernel.running_version`
- [ ] On the Rock 5 ITX+: `uname -r` → record in `fleet/rock5-itx-plus.yaml:kernel.running_version`
- [ ] `find / -path '*accel*' -name '*.ko' 2>/dev/null` — check if accel framework is built
- [ ] `ls /dev/accel/ /dev/dri/` — what's exposed?
- [ ] `lsmod | grep -iE 'rknpu|accel'` — what's loaded?
@@ -56,11 +56,11 @@ clear answer to "do we drive via accel uAPI or write our own MMIO driver."
## Phase 4 — baseline
- [ ] Build vanilla llama.cpp on boltzmann (mainline branch)
- [ ] Build vanilla llama.cpp on the Rock 5 ITX+ (mainline branch)
- [ ] Pull qwen2.5-1.5b-instruct Q4_K_M GGUF
- [ ] `llama-bench -m qwen2.5-1.5b -p 512 -n 128` × 3 runs
- [ ] Capture JSON to `benchmarks/$(date +%F)_boltzmann_qwen1.5b_cpu_baseline.json`
- [ ] Record into `fleet/boltzmann.yaml:baseline_measurement`
- [ ] Capture JSON to `benchmarks/$(date +%F)_rock5-itx-plus_qwen1.5b_cpu_baseline.json`
- [ ] Record into `fleet/rock5-itx-plus.yaml:baseline_measurement`
---
@@ -70,7 +70,7 @@ Phase-1 audit (2026-05-19) reframed Phase-2 from "design rknpu backend
interface" to a concrete bringup sequence. See
`docs/npu-mainline-status.md` for full context.
- [ ] Patch boltzmann board DTS / overlay to flip
- [ ] Patch the Rock 5 ITX+ board DTS / overlay to flip
`npu@fdab0000`, `npu@fdac0000`, `npu@fdad0000` from
`status = "disabled"``"okay"`. Rebuild DTB.
- [ ] **Mitigate IOMMU v1.0 hazard before first NPU job** (32 GB host).
@@ -102,8 +102,8 @@ interface" to a concrete bringup sequence. See
- [ ] Mirror Tomeu's branch — superseded: code is now in-tree.
Keep `git.kernel.org/.../torvalds/linux.git` checkout pinned to
the boltzmann kernel rev for in-tree reading.
- [ ] Set up serial console on boltzmann for kernel-panic recovery
the the Rock 5 ITX+ kernel rev for in-tree reading.
- [ ] Set up serial console on the Rock 5 ITX+ for kernel-panic recovery
(Quark umbrella; check current state) — **becomes load-bearing
once we start poking IOMMU code.**
- [x] Add `project_rosenblatt_overview.md` + `project_rocket_upstream_state.md`
@@ -111,7 +111,7 @@ interface" to a concrete bringup sequence. See
- [ ] Decide repo home: marfrit/rosenblatt on git.reauktion.de
(probably yes, after Phase-1 substrate is captured).
- [ ] **Resolve board-name discrepancy.** README and
`fleet/boltzmann.yaml` say boltzmann is a "Rock 5 ITX+" /
`fleet/rock5-itx-plus.yaml` say the manifest records the board type /
`rock-5-itx-plus`; the running DT reports
`model = "Radxa ROCK 5 ITX"`, `compatible = "radxa,rock-5-itx"`.
Confirm physical board model (Radxa sells both SKUs) and
+1 -1
View File
@@ -74,7 +74,7 @@ Two paths:
Phase-1 choice: (2) — straightforward, makes the NPU path the only
thing happening at inference time, easier to profile. The memory cost
on 1.5B is ~1.5 GB INT8 weights vs ~900 MB Q4_K_M — boltzmann has
on 1.5B is ~1.5 GB INT8 weights vs ~900 MB Q4_K_M — the Rock 5 ITX+ has
32 GB, this isn't the constraint.
Phase-2+: revisit (1) if we go for larger models or if INT8 turns out
+10 -10
View File
@@ -6,11 +6,11 @@
> blocker hits.
**Audit date:** 2026-05-19
**Kernel under audit:** boltzmann, `7.0.0-rc3-ARCH+ #2 SMP PREEMPT_DYNAMIC Wed Apr 29 11:16:17 CEST 2026 aarch64` (image `linux-rk3588-marfrit-A1`)
**Kernel under audit:** the Rock 5 ITX+, `7.0.0-rc3-ARCH+ #2 SMP PREEMPT_DYNAMIC Wed Apr 29 11:16:17 CEST 2026 aarch64` (image `linux-rk3588-marfrit-A1`)
---
## Local-host findings (boltzmann)
## Local-host findings (the Rock 5 ITX+)
The custom `linux-rk3588-marfrit-A1` build is mainline-rc3-based and **already
ships Tomeu Vizoso's upstream in-tree NPU accel driver** as a loadable module:
@@ -67,15 +67,15 @@ forward.
Tomeu Vizoso's RK3588 NPU work landed in **Linux 6.18 (Nov 2025)** under
the codename **`rocket`** (not `rknpu`). The matching userspace shipped
in **Mesa 25.3** (Rocket Gallium driver + Teflon TFLite delegate).
Boltzmann at 7.0.0-rc3 is one major release past the merge.
The Rock 5 ITX+ at 7.0.0-rc3 is one major release past the merge.
| Component | Mainline state | URL | Notes |
|---|---|---|---|
| `drivers/accel/rocket/` | **in-tree since 6.18** | <https://github.com/torvalds/linux/tree/master/drivers/accel/rocket> · [kernel docs](https://docs.kernel.org/accel/rocket/index.html) | Author Tomeu Vizoso. Files: `rocket_core.c`, `_device.c`, `_drv.c`, `_gem.c`, `_job.c`, `_registers.h`. Kconfig `DRM_ACCEL_ROCKET`. |
| DT binding `rockchip,rk3588-rknn-core.yaml` | in-tree (6.18 pull) | [v8 06/10 patch](https://www.mail-archive.com/dri-devel@lists.freedesktop.org/msg554406.html) | Path `Documentation/devicetree/bindings/npu/`. Requires 3 reg blocks (pc/cna/core), 4 clocks, IOMMU, power-domain, resets. |
| `pd_npu` power-domain in `rk3588-base.dtsi` | in-tree (6.18 pull) | [v8 07/10](https://www.mail-archive.com/dri-devel@lists.freedesktop.org/msg554409.html) | Label added so board files can wire a regulator instead of leaving the domain permanently on. |
| `npu@fdab/c/d_0000` nodes in `rk3588.dtsi` | in-tree (boltzmann's DT has all 3) | observed at `/sys/firmware/devicetree/base/npu@fdab0000` etc. | All `compatible = "rockchip,rk3588-rknn-core"`, **`status = "disabled"`** in the DT boltzmann is booting. |
| Per-board enable on Rock 5 ITX+ (boltzmann) | **not done** | n/a | Three `rknn-core` nodes ship disabled; need a board overlay or DTS edit to set `status = "okay"` before `rocket` can probe. |
| `npu@fdab/c/d_0000` nodes in `rk3588.dtsi` | in-tree (the Rock 5 ITX+'s DT has all 3) | observed at `/sys/firmware/devicetree/base/npu@fdab0000` etc. | All `compatible = "rockchip,rk3588-rknn-core"`, **`status = "disabled"`** in the DT the Rock 5 ITX+ is booting. |
| Per-board enable on the Rock 5 ITX+ | **not done** | n/a | Three `rknn-core` nodes ship disabled; need a board overlay or DTS edit to set `status = "okay"` before `rocket` can probe. |
| Userspace consumer (Mesa Rocket + Teflon) | merged in **Mesa 25.3** | [Tomeu blog "we are in mainline"](https://blog.tomeuvizoso.net/2025/07/rockchip-npu-update-6-we-are-in-mainline.html) · [Mesa Teflon docs](https://docs.mesa3d.org/teflon.html) | TFLite delegate; the authoritative reference for how jobs are encoded and submitted over the rocket uAPI. |
| v8 series cover letter (final revision) | **merged via DRM-misc-next → 6.18** | [v8 cover](https://www.mail-archive.com/dri-devel@lists.freedesktop.org/msg554401.html) · [Patchew](https://patchew.org/linux/20250713-6-10-rocket-v8-0-64fa3115e910@tomeuvizoso.net/) · [LWN coverage](https://lwn.net/Articles/1029800/) | 10 patches. |
| BSP `rockchip-linux/rknpu2` (vendor) | out-of-tree, deprecated, superseded by `airockchip/rknn-toolkit2` | <https://github.com/rockchip-linux/rknpu2> · <https://github.com/rockchip-linux/kernel> | BSP `rknpu` driver under `drivers/rknpu/` in 5.10/6.1 BSP branches. Register-map partly readable from source; TRM fills gaps. Use as spec-extraction reference per `feedback_megabitchip_semantic_match`. |
@@ -89,7 +89,7 @@ Boltzmann at 7.0.0-rc3 is one major release past the merge.
The Phase-1 exit question — "accel uAPI vs own MMIO driver" — answers
itself with the upstream state we just inventoried:
- `rocket.ko` is already on disk on boltzmann
- `rocket.ko` is already on disk on the Rock 5 ITX+
(`/lib/modules/7.0.0-rc3-ARCH+/kernel/drivers/accel/rocket/rocket.ko`,
`intree: Y`) and aliased to the exact `rockchip,rk3588-rknn-core`
compatible string the DT uses.
@@ -99,7 +99,7 @@ itself with the upstream state we just inventoried:
reimplementing the (already-reviewed) `pd_npu` power-domain sequencing,
and owning forward-compatibility burden with zero upstream path.
**Unblock path:** patch the boltzmann board DTS (or apply an overlay) to
**Unblock path:** patch the the Rock 5 ITX+ board DTS (or apply an overlay) to
flip the three `npu@fdab/c/d_0000` nodes to `status = "okay"`, rebuild
the DTB, reboot, `modprobe rocket`, expect `/dev/accel/accel0..2` to
appear and corresponding `dmesg` probe lines. That's a Phase-2 task.
@@ -145,7 +145,7 @@ VOP2 uses v2.0. Both map 40-bit physical pages, but **v1.0 does not
support placing the first-level page table (DTE) above 4 GB**.
The current DT binding `rockchip,rk3568-iommu` does **not** distinguish
the two — same compat for both IP versions. On `boltzmann` all three
the two — same compat for both IP versions. On the Rock 5 ITX+ all three
NPU IOMMUs are bound to this compat (with `rockchip,rk3588-iommu`
fallback), so they will be driven by the v2.0 code path:
@@ -155,7 +155,7 @@ fallback), so they will be driven by the v2.0 code path:
/sys/firmware/devicetree/base/iommu@fdada000 compatible: rockchip,rk3588-iommu rockchip,rk3568-iommu status: (none → okay default)
```
With `MemTotal = 32 GB` on boltzmann, two failure modes are reachable
With `MemTotal = 32 GB` on the Rock 5 ITX+, two failure modes are reachable
the moment we enable the NPU:
1. **DTE allocated above `0x100000000`** → v1.0 hardware silently
@@ -193,7 +193,7 @@ discriminator-compat approach is correct but not yet upstream.
**Implication for Rosenblatt Phase 2:** the README's Phase-2 unblock
plan ("enable DT nodes → `modprobe rocket``/dev/accel/accel0..2`")
will **almost certainly trigger one of the two failure modes** on
32 GB boltzmann. Three viable workarounds:
32 GB the Rock 5 ITX+. Three viable workarounds:
- **A. Boot with `mem=4GB`** (degraded but valid Phase-2 path; lets us
validate the rocket bringup end-to-end without addressing the IOMMU
+4 -4
View File
@@ -20,7 +20,7 @@ for RK3588 NPU on mainline`.
**Closed:** 2026-05-19
**Deliverable:** `docs/npu-mainline-status.md` table fully populated;
`fleet/boltzmann.yaml` kernel/NPU-driver block filled with live data;
`fleet/rock5-itx-plus.yaml` kernel/NPU-driver block filled with live data;
clear answer to the "accel uAPI vs. own MMIO driver" question.
### Findings
@@ -30,7 +30,7 @@ clear answer to the "accel uAPI vs. own MMIO driver" question.
Kconfig `DRM_ACCEL_ROCKET`. Driver author + history kept the same
shape, but the **upstream name is `rocket`, not `rknpu`**
searching for `rknpu` in mainline misses everything.
2. **Boltzmann already ships the driver**`linux-rk3588-marfrit-A1`
2. **The Rock 5 ITX+ already ships the driver**`linux-rk3588-marfrit-A1`
(7.0.0-rc3-ARCH+) is post-6.18 and contains `rocket.ko` at
`/lib/modules/.../drivers/accel/rocket/rocket.ko`, marked
`intree: Y`. Module aliases to `rockchip,rk3588-rknn-core`,
@@ -65,7 +65,7 @@ clear answer to the "accel uAPI vs. own MMIO driver" question.
8. **IOMMU v1.0 hazard surfaced from `linux-rockchip` thread**
(Midgy BALON / Simon Xue, 2026-04-03). The NPU IOMMU is v1.0 IP
bound to generic `rockchip,rk3568-iommu` — driven via the v2.0
code path. v1.0 can't allocate its DTE above 4 GB. Boltzmann
code path. v1.0 can't allocate its DTE above 4 GB. The Rock 5 ITX+
has 32 GB. Naive enable will silently fault. Discriminator-compat
patch series planned but **not landed in mainline master as of
2026-05-19** (verified via cgit on
@@ -88,7 +88,7 @@ list is short: DT enable + IOMMU mitigation + `modprobe rocket`.
Captured in `TODO.md` "Phase-2 unblock" section. Highlights:
- Apply `kernel/dt-overlays/rk3588-rosenblatt-npu-enable.dtso` (or
equivalent board-DTS patch) to boltzmann.
equivalent board-DTS patch) to the Rock 5 ITX+.
- Mitigate IOMMU v1.0 hazard before first NPU job: `mem=4G` boot or
local discriminator-compat carry.
- `modprobe rocket`, confirm `/dev/accel/accel0`, no IOMMU faults.
@@ -1,14 +1,14 @@
# rosenblatt fleet manifest — boltzmann (Rock 5 ITX+, RK3588)
# rosenblatt fleet manifest — rock5-itx-plus (Rock 5 ITX+, RK3588)
#
# Phase-1 audit host. Always-on, 32 GB DDR4, NVMe rootfs. NPU silicon
# present + accessible via Rockchip-BSP vendor module today; mainline
# path TBD (see docs/npu-mainline-status.md).
host: boltzmann
host: rock5-itx-plus
arch: arm64
soc: rockchip/rk3588
board: rock-5-itx-plus
distro: archlinuxarm # ALARM aarch64; boltzmann is the umbrella RK3588 host
distro: archlinuxarm # ALARM aarch64; rock5-itx-plus is the umbrella RK3588 host
role: primary-development # not yet primary-target (laptop targets land later)
hardware:
@@ -57,7 +57,7 @@ baseline_measurement:
target: |
llama.cpp pure-CPU tok/s on qwen2.5-1.5b-instruct-q4_k_m.gguf,
3 runs, median wallclock. Use llama-bench from llama.cpp/build/bin.
ground_truth_file: benchmarks/2026-XX-XX_boltzmann_qwen1.5b_cpu_baseline.json
ground_truth_file: benchmarks/2026-XX-XX_rock5-itx-plus_qwen1.5b_cpu_baseline.json
bringup_sequence:
1: substrate audit (docs/npu-mainline-status.md table filled)
@@ -71,6 +71,6 @@ bringup_sequence:
backup_host: ampere # CoolPi GenBook — port-validation target. Phase-2+ scope.
reverse_dependencies:
- Quark (boltzmann UEFI) — must stay bootable across kernel-rev experiments
- Neutron (boltzmann kernel build) — provides the kernel we tweak for rknpu
- Volta (boltzmann umbrella) — Rosenblatt is the third Volta-child after Quark + Neutron
- Quark (rock5-itx-plus UEFI) — must stay bootable across kernel-rev experiments
- Neutron (rock5-itx-plus kernel build) — provides the kernel we tweak for rknpu
- Volta (rock5-itx-plus umbrella) — Rosenblatt is the third Volta-child after Quark + Neutron
+2 -2
View File
@@ -2,7 +2,7 @@
Phase-1 audit closed the "do we need a driver?" question: Tomeu
Vizoso's `rocket` driver is in mainline since Linux 6.18, and
boltzmann's `linux-rk3588-marfrit-A1` build (7.0.0-rc3-ARCH+)
the Rock 5 ITX+'s `linux-rk3588-marfrit-A1` build (7.0.0-rc3-ARCH+)
already ships it as a module. See `docs/npu-mainline-status.md` for
the full audit.
@@ -29,4 +29,4 @@ Anticipated next additions:
Patch handling routes through the `kernel-agent` subagent (per
`feedback_agent_routing.md`); board / host wiring lives in
`fleet/boltzmann.yaml`.
`fleet/rock5-itx-plus.yaml`.
@@ -3,11 +3,11 @@
* Rosenblatt: enable all three RK3588 RKNN cores + their IOMMUs.
*
* Apply on top of any rk3588 board DT that uses the mainline
* rk3588-base.dtsi labels. Verified against boltzmann
* rk3588-base.dtsi labels. Verified against the Rock 5 ITX+
* (model "Radxa ROCK 5 ITX", compatible "radxa,rock-5-itx").
*
* Pre-conditions before applying on a system with >4 GB RAM
* (boltzmann has 32 GB):
* (the Rock 5 ITX+ has 32 GB):
* - The IOMMU v1.0 hazard MUST be mitigated first.
* See docs/npu-mainline-status.md "IOMMU v1.0 hazard".
* Either boot with `mem=4G`, OR carry the discriminator-compat