Markus Fritscherocky-vulkan-llama
Making the RK3588 Mali-G610 GPU a real LLM prefill accelerator — via the open Vulkan stack (panthor kernel + Mesa panvk + llama.cpp), no vendor blobs.
Two upstreamable fixes take large-batch prefill on a Radxa ROCK 5 ITX+ (Rockchip RK3588) from crashing to 4.8× faster, beating the CPU — on a GPU with no matrix cores.
TL;DR results (Qwen2.5-3B-Q4_K_M, pp512, Mali-G610 MC4, Mesa 26.1.3 panvk)
| prefill t/s | note | |
|---|---|---|
| stock llama.cpp Vulkan @ ub≥128 | crash | vk::CommandBuffer::end: ErrorOutOfDeviceMemory |
| + driver fix | 5.8 | runs, but the desktop-tuned shader is register-spilling |
| + shader tune (this repo) | 28.1 | 4.8×, spill-free, ~35 % of roofline, output-verified |
| (CPU reference, i8mm-off) | ~22 |
The ceiling was never the hardware — it was a desktop-tuned shader thrashing Mali's tiny register file, and a Mesa allocator that never shared per-command-buffer scratch. Same shape as unlocking hardware video decode: fix the whole stack (kernel → Mesa → the consumer), not just one layer.
The two fixes
-
Mesa panvk — priv-heap exhaustion (
patches/mesa-panvk-shared-tls-wls.patch) CSF panvk allocates a per-command-buffer TLS stack (up to 304 MB) and per-dispatch WLS scratch from a device-wide sub-4 GB priv VA heap, never shared. A pipelined LLM graph keeps ~12 command buffers in flight → ~4 GB → OOM at any ubatch > 64. Fix: one device-shared TLS scratch reused across the serially-executing compute queue, plus grid-independent WLS reuse. Peak priv-VA 3985 MB → 908 MB; no crash from ub 64 to 1024. Affects any large-workgroup compute on Mali/panvk, not just LLMs. -
llama.cpp — Mali-tuned
mul_mm/mul_mmqwarptile (patches/llama.cpp-mali-mmq-warptile.patch) Per-thread matmul accumulators =WM·WN/WARP; the stock L-variant is 128 → spills 5328 B/thread → occupancy collapses on Mali. Fix (an ARM/panvk warptile branch): shrink to 16 accumulators (spill → 96 B), enlarge the workgroup at fixed registers for shared-memory occupancy, and scopeBK_STEPto the L pipeline via a spec constant (so decode/M-variant are untouched). Correctness verified by GPU-vs-CPU generation match.
Apply on top of Mesa 26.1.3 (panvk build: -Dvulkan-drivers=panfrost, minimal) and llama.cpp
389ff61; run on a panthor kernel with the GPU bound to panthor (not the vendor mali blob).
How it was found
Instrument-driven, with a second model ("Fable") implementing the shader and a hardware-in-the-loop
measure loop. The panvk TLS-stack size is the register-spill metric, which made every tile change
a single-variable experiment — including the informative negatives (bigger tiles → occupancy-bound
not compute-bound; BK_STEP=1 → barrier crossover; f16-accumulators → refused with proof). Full
trail in docs/ (root-cause*.md → shader-tuning.md → RESULTS.md).
Repo layout
patches/— the two final patches (Mesa + llama.cpp), pluspatches/iterations/(the tuning trail).docs/— root-cause analysis, the review notes, the iteration log, benchmarks, and the fix plan.
Not fixed here
- Decode stays memory-bandwidth-bound (~28 GB/s LPDDR5) — the GPU can't help it; that's a speculation/MoE problem, not a shader one.
- Gemma models produce garbage on llama.cpp's Vulkan backend (a pre-existing gemma-op bug, independent of these patches — they're correct on CPU). Qwen-family works.
- The last ~15 % to roofline is the separate q8_1 activation-quantize dispatch, not this shader.
License
MIT (see LICENSE). The patches are against Mesa and llama.cpp, both MIT-licensed.