diff --git a/docs/shader-tuning.md b/docs/shader-tuning.md index 5a4f816..5f72421 100644 --- a/docs/shader-tuning.md +++ b/docs/shader-tuning.md @@ -23,3 +23,14 @@ Spill collapsed 55×; large ubatch now beats the ub64/M ceiling AND the CPU. Con was the shader, NOT the hardware. ~24 t/s ≈ 25–30% of roofline (~80–150 t/s ideal) → headroom left. Patch: `patches/llama.cpp-0003-mali-warptile-iter1.patch` (adds VK_VENDOR_ID_ARM + is_mali warptile branch, L tuples only). Iteration 2: grow the tile toward the register ceiling for arithmetic reuse. + +## Iteration 2 — grow tile to 32 acc (WM=32,WN=16) — REGRESSED +| ubatch | iter1 | iter2 | tls_size | +|--------|-------|-------|----------| +| 128 | 23.4 | **9.7 t/s (2.4× slower)** | 96 → 624 | +| 256 | 23.8 | **10.1 t/s (2.4× slower)** | 96 → 624 | + +Bigger tile improved arithmetic intensity (AI 1.6→2.67) but tanked throughput. **We're +OCCUPANCY-bound, not AI-bound**: 8704 B shared + 624 B spill cut resident workgroups per core. +"Smaller is better" wins the head-to-head — **iter1 (16 acc) stays champion at ~23.5 t/s**. +Iteration 3: go smaller / maximize concurrent workgroups, not bigger.