rocky-vulkan-llama/docs/root-cause-final.md

Root cause — FULLY characterized (2026-07-11, live panthor instrumentation)

The ub64 crash cliff is priv-VA-heap exhaustion from large per-command-buffer tls-pool allocations accumulating across ~12 command buffers, with TWO consumers:

1. WLS (workgroup local storage) — FIXED (partial contribution)

• Per-dispatch, grid-dependent instances → sizes climb (155 KB … 19 MB).
• Fix applied & verified: grid-independent instances (pan_calc_wls_instances(...,NULL)) + grow-only reuse. Instrumentation confirms it works: 644 REUSE / 46 ALLOC at ub128 (vs 326 without) — collapses to ~4 allocs per cmdbuf. Cut WLS VA roughly in half (326→690 survivable dispatches). See patches/mesa-panvk-0001-*, mesa-panvk-0002-*.

2. TLS stack (register spill) — the DOMINANT remaining consumer

• emit_tls() (once per cmdbuf, at vkEndCommandBuffer — exactly where the OOM fires) allocates pan_get_total_stack_size(tls_size, thread_tls_alloc, core_id_range) from the tls pool.
• Measured tls_size per thread: ub64 (M shader) = 544 B → 38 MB stack; ub128 (L shader) = 5328 B → 304 MB stack. The L-variant matmul shader spills ~10× more registers.
• core_id_range = 19 (sparse core mask 0x50005, real cores = 4) → ~5× overallocation baked into every WLS and TLS size.
• ~12 command buffers per pp512 run, each retaining a 150–304 MB TLS stack BO in the pool (freed only at panvk_reset_cmdbuf) → ~3 GB of TLS alone, + ~1 GB WLS → over the 4064 MB priv heap.

Why it's ub64-sharp

llama.cpp switches M→L matmul shader at n=64 (ggml_vk_guess_matmul_pipeline). L's register pressure raises tls_size 10× → the per-cmdbuf TLS stack jumps 38 MB→304 MB. WLS doubling (16→32 KB shared mem) is the smaller half. Together they cross the heap ceiling at ub>64.

Remaining fix options (for the TLS-stack term)

1. Reuse the TLS stack BO the same way as WLS (grow-only, in emit_tls) — but it's already one-per-cmdbuf; the real accumulation is CROSS-cmdbuf (the pool isn't freed between the ~12 cmdbufs). So within-cmdbuf reuse won't fix this term.
2. Free tls-pool big BOs after submit (not just at reset) — Fable's constraint is the CS holds WLS/TLS VA until submit, so freeing is safe post-submit. This is the structural fix: recycle the pool per submit so cross-cmdbuf accumulation stops. Needs care in panvk_pool/submit path.
3. Cut the ×19 → ×4 core overallocation (sparse core-id remap) — ~5× smaller WLS+TLS. Biggest single lever but HW-indexed by core id; needs a remap table. Would bring 304 MB→64 MB, ~3 GB→ ~640 MB, likely under the ceiling by itself.
4. Reduce L-shader register spill (llama.cpp side) — out of scope for the driver.

Status

Mechanism fully characterized and instrumented. WLS half fixed & verified. The TLS-stack half needs a pool-lifecycle change (option 2) and/or the core-overallocation fix (option 3) — a larger, upstream-worthy change. Not a guess anymore: every number above is measured on the live GPU.

Phase 1 executed (2026-07-11) — consumer-side levers FALSIFIED

Instrument-driven test on panthor (patched panvk w/ priv-VA counter + GGML_VK_CMD_CLEANUP_FREQ):

• cleanup_frequency 10→2: no effect. Peak priv-VA ~3800 MB at both; still OOM. resetCommandPool doesn't reduce live VA — the tls pool RECYCLES big BOs into a free-list (panvk_pool_reset, big_bo_pool branch) instead of freeing; varying 155 KB–304 MB sizes never match → VA grows monotonically (0 frees in 128 allocs).
• free-on-reset (tls_pool big_bo_pool→NULL) + freq=3: still OOM. Only 1 free / 174 allocs. The cmdbufs are IN-FLIGHT (submitted, fences pending till graph end) so they can't be reset; post-hoc freeing can't touch the in-flight peak.
• Confirmed peak: 12 live 304 MB TLS-stack BOs ≈ 3.65 GB → OOM. The failing alloc is a 304 MB TLS stack at used=3.98 GB.

Conclusion: the accumulation is IN-FLIGHT command buffers (~12 per pp512, from pipelined ubatch forwards), each carrying a 304 MB TLS stack. Neither cleanup_frequency nor free-on-reset helps. The only viable levers reduce the in-flight peak: (a) cut the ×19 sparse-core overallocation (304→64 MB; 12×64=768 MB → fits) or (b) reduce in-flight cmdbuf depth in llama.cpp (fewer/bigger submits per graph, or submit+sync granularity). Phase 4.