docs: record live validation of a7ace5d (30B MoE cross-domain fixed, 16.70 tok/s)

a7ace5d's commit message says "NOT YET LIVE-VALIDATED" -- it now is.
Markus rebooted boltzmann on the a7ace5d build (md5 29714de2) and
re-ran the 30B prefill that previously wedged the NPU permanently:

  Qwen3-30B-A3B-Instruct-2507 UD-Q4_K_XL (16.47 GiB, ~5 domains):
  pp512 = 16.70 tok/s, exit 0, clean. Fence signals, job completes.
  Zero SError/panic/hung-task/VA-insert in dmesg for the whole boot.

The pm_lock-around-domain-switch hypothesis in a7ace5d was correct:
the D-state that looked like a permanent hang was the uninterruptible
NPU-job wait now actually completing instead of wedging forever.

Full scoreboard on a7ace5d, all on the same boot: 4B-Q8 unchanged
(single-domain, the pre-existing production path), 3B-f16 = 76.75
tok/s native (b831a2d), 30B-A3B = 16.70 tok/s native (a7ace5d). The
~4GiB single-domain ceiling is broken and MoE cross-domain access
works. a7ace5d is production.

Deliberately a new commit rather than amending a7ace5d: it's already
pushed to github+gitea, and rewriting pushed history over a doc update
isn't worth the force-push risk. README updated with the same numbers
for anyone landing here without the commit log.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EWpfhDgYNA21tETDP9ueBE
This commit is contained in:
Markus Fritsche
2026-07-20 05:35:29 +02:00
parent a7ace5da8e
commit 7e32abd9eb
+13
View File
@@ -149,6 +149,19 @@ rising interrupt counts for the three `npu` IRQs in `/proc/interrupts`.
three cores run (an early version let drm_sched pick a core and left one idle).
- Board-to-board differences are **DDR-bandwidth**-bound, not clock: a GenBook (LPDDR5)
runs ~49.5 tok/s vs a Rock 5 ITX (LPDDR5-6400) at ~51.5, both with the NPU at 1 GHz.
- **Multi-domain (>4GiB NPU-resident models), Rock 5 ITX, live-validated:** rkopnu honours
the vendor `iommu_domain_id` (client-side `librknnrt`/RKNN-matmul callers, e.g.
ggml-rknpu2's `IOMMUDomainManager`, already spread allocations across up to 16
independent IOMMU paging domains — each domain gets its own ~4GiB IOVA aperture, the
real ceiling coming from the vendor RKNN matmul API's int32 B-matrix offsets, not IOMMU
hardware). pp512 prefill: Qwen2.5-3B-Instruct f16 (5.75 GiB, ≥2 domains) = 76.75 tok/s
native (no INT8 fallback needed); Qwen3-30B-A3B-Instruct-2507 UD-Q4_K_XL (16.47 GiB,
~5 domains, MoE cross-domain expert routing) = 16.70 tok/s, exit 0, zero
SError/panic/hung-task/VA-insert. A domain switch on one core's IOMMU submodule shares
clock/PMU infrastructure with the cores' own power-domain resume (see
`rocket_device_pm_get()`'s device-global `pm_lock`, used for exactly this class of
cross-core race elsewhere in this driver); serializing IOMMU attach/detach against that
same lock was what took the 30B case from a permanent D-state hang to a clean run.
## Limitations / notes