diff --git a/docs/gemma-gpu-kernel-bringup-2026-07-12.md b/docs/gemma-gpu-kernel-bringup-2026-07-12.md index 435c7bc..759f838 100644 --- a/docs/gemma-gpu-kernel-bringup-2026-07-12.md +++ b/docs/gemma-gpu-kernel-bringup-2026-07-12.md @@ -118,3 +118,36 @@ still unknown but moot — use plain mainline). GPU wedges if you pkill a job mi **Next:** port the champion shader onto `build-vulkan` and re-measure gemma-4-E2B prefill on GPU; the correctness question is now closed (works). + +## CHAMPION SHADER PORTED TO JULY BUILD + honest CPU comparison (2026-07-12) + +Ported the champion `mul_mmq` warptile onto stock July mainline via a REDUCED patch: the 4 +essential ggml-vulkan hunks (VK_VENDOR_ID_ARM, GGML_VK_CMD_CLEANUP_FREQ env, is_mali warptile +override {256,64,64,...}, Mali wg_denoms/align) + the mul_mmq.comp BK_STEP spec-constant. Dropped +the 2 submit-cadence hunks (they hit July's restructured `ggml_backend_vk_graph_compute`; secondary). +July already had the BK_STEP scaffolding + `l_warptile_mmq_int` variants upstream, and warptile is +`std::vector` so the 12th element (BK_STEP=2) compiles + plumbs as spec-constant 11. +Reduced patch applies clean; build OK; correctness preserved (gemma-E2B -> ' Paris.'). + +### Measured (gemma-4-E2B, 242-tok prompt, warm) +| path | prefill | decode | +|------|---------|--------| +| GPU stock July (no champion) | 0.9 t/s | 3.2 t/s | +| **GPU champion shader** | **26.5 t/s** | 4.3 t/s | +| **CPU (4x A76)** | **28.1 t/s** | **23.7 t/s** | + +### VERDICT — the champion shader works, but it does NOT beat CPU on this box +- Champion vs stock GPU: **29x prefill** (0.9 -> 26.5). The shader port is a real success and hits + its documented ~28 t/s. Bug reproduced-and-fixed on the July build. +- Champion GPU vs CPU: **prefill is a TIE** (26.5 vs 28.1), and **CPU wins decode decisively** + (23.7 vs 4.3, the unified-DDR bandwidth wall the GPU can't beat). +- **Correction to an earlier optimistic claim:** the champion's documented "4.8x" is stock-GPU -> + champion-GPU, NOT vs CPU. The real result is GPU reaches PARITY with CPU on prefill, not a win. +- The only way GPU prefill would beat CPU is a model big enough for the tile efficiency to pull + ahead — but those don't fit on the Mali (sub-4GB VA heaps). The models that FIT are too small + for the GPU to win. Squeeze: fits => too small to win; big enough to win => doesn't fit. +- Net on boltzmann: **CPU (llama.cpp, Q4 QAT) remains the right path** for gemma. The GPU champion + is a correct, impressive shader that lands at prefill-parity / decode-loss — not an upgrade. + +Op note: SIGKILL of even an IDLE GPU llama-server wedges the panvk context (next GPU init hangs +at 0% CPU) -> reboot to clear. Confirmed twice.