Claude (noether)
e74bae69fe
Instrument-first panthor boot confirms Fable's diagnosis: ~326 per-dispatch WLS allocs exhaust the sub-4GB priv VA heap at ub>=128. Grow-only single-max reuse (fix v1) halves the pressure (326->715 dispatches) but WLS sizes vary 128x (155KB-19MB) so a single buffer is insufficient. Refined fix pending 2nd review. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1.6 KiB
1.6 KiB
Benchmarks & WLS instrumentation (panthor boot 2026-07-11)
Patched panvk (WLS-reuse, env-gated) loaded via VK_ICD_FILENAMES; Qwen2.5-3B-Q4_K_M, pp512.
Confirmed: the mechanism is per-dispatch WLS exhausting panvk's sub-4 GB priv VA heap
| Case | ubatch | WLS allocs/graph | Result |
|---|---|---|---|
| no-reuse | 64 | (freed per reset) | 21.0 t/s ✓ |
| no-reuse | 128 | 326 before OOM | crash CommandBuffer::end: OutOfDeviceMemory |
| reuse v1 | 128 | 715 before OOM | still crash (survives ~2× longer) |
| reuse v1 | 256 | 690 | still crash |
| reuse v1 | 512 | 432 | still crash |
WLS per-dispatch sizes vary 128× (155 KB … 19 MB; dominant 9.5 MB), summing ~2 GB/graph without reuse. driver_set/tsd sibling allocs are tiny (descriptors) — not contributors.
Status of the fix
- fix v1 (grow-only single-max WLS buffer,
patches/mesa-panvk-0001-*): confirmed the mechanism and halved the pressure, but insufficient — the 128×-varying WLS sizes defeat a single-max buffer (the original TODO warned about exactly this). Refined fix in progress (2nd Fable pass). - ub64 remains the only crash-free mode at 21 t/s (overhead-bound; ties an i8mm-off CPU).
How to reproduce (on panthor kernel, patched panvk)
export XDG_RUNTIME_DIR=/run/user/$(id -u) VK_ICD_FILENAMES=$HOME/rocky-panvk-icd.json
cd ~/src/llama.cpp/build-vulkan/bin
PANVK_WLS_LOG=1 ./llama-bench -m MODEL -p 512 -n 0 -ngl 99 -b 512 -ub 128 # crash, ~326 allocs
PANVK_WLS_REUSE=1 PANVK_WLS_LOG=1 ./llama-bench -m MODEL -p 512 -n 0 -ngl 99 -b 512 -ub 128 # v1: still crash