marfrit/rocky-vulkan-llama
1
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Releases Wiki Activity

Phase 1: consumer-side levers falsified; accumulation is in-flight cmdbufs x 304MB TLS

Browse Source 
cleanup_frequency (10->2) and free-on-reset both no-effect: cmdbufs are in-flight so can't be
reset. ~12 live 304MB TLS-stack BOs = 3.65GB. Real levers: cut x19 core overalloc, or reduce
in-flight depth. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Claude (noether)
2026-07-11 16:00:09 +02:00
parent 881b118158
commit bec204e375
1 changed files with 18 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/root-cause-final.md
+18
View File
@@ -41,3 +41,21 @@ shared mem) is the smaller half. Together they cross the heap ceiling at ub>64.
Mechanism fully characterized and instrumented. WLS half fixed & verified. The TLS-stack half needs
a pool-lifecycle change (option 2) and/or the core-overallocation fix (option 3) — a larger,
upstream-worthy change. Not a guess anymore: every number above is measured on the live GPU.
## Phase 1 executed (2026-07-11) — consumer-side levers FALSIFIED
Instrument-driven test on panthor (patched panvk w/ priv-VA counter + `GGML_VK_CMD_CLEANUP_FREQ`):
- **`cleanup_frequency` 10→2: no effect.** Peak priv-VA ~3800 MB at both; still OOM. `resetCommandPool`
doesn't reduce live VA — the tls pool RECYCLES big BOs into a free-list (`panvk_pool_reset`,
`big_bo_pool` branch) instead of freeing; varying 155 KB–304 MB sizes never match → VA grows
monotonically (**0 frees in 128 allocs**).
- **free-on-reset (tls_pool big_bo_pool→NULL) + freq=3: still OOM.** Only **1 free / 174 allocs**.
The cmdbufs are **IN-FLIGHT** (submitted, fences pending till graph end) so they can't be reset;
post-hoc freeing can't touch the in-flight peak.
- **Confirmed peak: 12 live 304 MB TLS-stack BOs ≈ 3.65 GB → OOM.** The failing alloc is a 304 MB
TLS stack at used=3.98 GB.
**Conclusion:** the accumulation is IN-FLIGHT command buffers (~12 per pp512, from pipelined
ubatch forwards), each carrying a 304 MB TLS stack. Neither cleanup_frequency nor free-on-reset
helps. The only viable levers reduce the *in-flight peak*: (a) **cut the ×19 sparse-core
overallocation** (304→64 MB; 12×64=768 MB → fits) or (b) **reduce in-flight cmdbuf depth** in
llama.cpp (fewer/bigger submits per graph, or submit+sync granularity). Phase 4.