marfrit/rosenblatt
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: genericize host references to machine type (Rock 5 ITX+)

Browse Source 
Replace the local host name "boltzmann" with the machine type
(Rock 5 ITX+, RK3588, 32 GB) throughout so the replication docs read
for anyone on the same hardware, not just this fleet. Rename
fleet/boltzmann.yaml -> fleet/rock5-itx-plus.yaml.
This commit is contained in:
marfrit
2026-07-15 09:59:00 +02:00
parent 27df00a201
commit c77be48151
9 changed files with 43 additions and 44 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+7 -8
View File
@@ -5,8 +5,7 @@ hardware neural network (400 photocells, stepper-motor-tunable analog weights).
This project lights up the RK3588 NPU on mainline Linux, so the OSS world
finally owns the silicon-side of inference on that chip.
**Scope (Phase 1):** small LLM running CPU + NPU mix on `boltzmann` (Rock 5
ITX+, RK3588, 32 GB DDR4). Backend: `llama.cpp` with a new `rknpu` device that
**Scope (Phase 1):** small LLM running CPU + NPU mix on the Rock 5 ITX+ (RK3588, 32 GB DDR4). Backend: `llama.cpp` with a new `rknpu` device that
offloads the heavy GEMM (matmul in attention + FFN) to the NPU's INT8 path
while leaving dequant / RoPE / softmax / sampling / embedding lookup on the
A76 NEON cores.
@@ -35,10 +34,10 @@ point; the open path to it is.
| # | Phase | Deliverable |
|---|---|---|
| 1 | **Substrate** | Audit mainline NPU driver state (Tomeu Vizoso's rknpu / DRM-accel series); `/dev/accel/*` probe on boltzmann; running kernel + module inventory. `docs/npu-mainline-status.md` snapshot. |
| 1 | **Substrate** | Audit mainline NPU driver state (Tomeu Vizoso's rknpu / DRM-accel series); `/dev/accel/*` probe on the Rock 5 ITX+; running kernel + module inventory. `docs/npu-mainline-status.md` snapshot. |
| 2 | Formulate | Pick the exact matmul shape that fits the NPU's tile-MAC array. Identify the smallest-possible op-set llama.cpp can offload. |
| 3 | Analyze | Read the RKNPU2 SDK + Tomeu's rknpu uAPI to learn: register layout, DMA tensor format, INT8 quant scheme. Don't lift code — extract the spec. |
| 4 | Baseline | llama.cpp pure-CPU tok/s on boltzmann for qwen-1.5B Q4_K_M. Three runs, median. Reproducible bench script in `benchmarks/`. |
| 4 | Baseline | llama.cpp pure-CPU tok/s on the Rock 5 ITX+ for qwen-1.5B Q4_K_M. Three runs, median. Reproducible bench script in `benchmarks/`. |
| 5 | Plan | rknpu backend interface design — where it plugs into ggml's compute graph; memory mapping strategy (dmabuf vs userptr); fallback path. |
| 6 | Review | Janet (ARM/DRM specialist agent) reviews the NPU register-write + DMA fence strategy. Cold-eyes pass. |
| 7 | Implement | rknpu ggml backend skeleton + first INT8 matmul. Bit-exact against CPU reference (Q4_K dequant + fp32 matmul). |
@@ -66,7 +65,7 @@ rosenblatt/
│ ├── npu-probe/ smallest-possible "open device + run trivial matmul" sanity
│ └── llm-runtime/ llama.cpp fork with rknpu backend
├── fleet/
│ └── boltzmann.yaml host manifest (kernel + NPU driver pin, baseline measurement)
│ └── rock5-itx-plus.yaml host manifest (kernel + NPU driver pin, baseline measurement)
└── benchmarks/ reproducible bench scripts + recorded results (JSON + plots)
```
@@ -74,13 +73,13 @@ rosenblatt/
## Host
Primary: **boltzmann** (Rock 5 ITX+, RK3588, 32 GB DDR4-2666, NVMe rootfs).
Primary: **Rock 5 ITX+** (RK3588, 32 GB DDR4-2666, NVMe rootfs).
- Already runs mainline ~v7.0 with most peripheral drivers working.
- Has the Quark UEFI / Neutron kernel stack — NPU is the next missing peripheral.
- Other RK3588 hosts (`ampere` = CoolPi GenBook) come later for port-validation.
Why not `ampere`: laptop, intermittent power, in-use for other campaigns.
Boltzmann is always-on with 32 GB headroom — right substrate for kernel
The Rock 5 ITX+ is always-on with 32 GB headroom — right substrate for kernel
hacking with serial-console fallback (when [Quark](https://git.reauktion.de/marfrit/quark) exposes one).
---
@@ -109,7 +108,7 @@ We're writing the second half.
Phase-1 closeout: `docs/phases.md` + `docs/npu-mainline-status.md`.
Headline: mainline driver name is **`rocket`** (not `rknpu`); it's
already shipped in boltzmann's kernel as a built module. Phase-2
already shipped in the Rock 5 ITX+'s kernel as a built module. Phase-2
unblock is small (DT enable + IOMMU v1.0 mitigation + modprobe),
not a driver port.