rknn_create_mem returns null when the RK3588 NPU's ~4 GiB IOVA aperture is
full -- this backend maps every weight tensor into a single 4 GiB IOMMU window
(rkopnu currently ignores iommu_domain_id, so the IOMMUDomainManager's 2 GiB
domain-spreading is a no-op and everything lands in one 4 GiB window). The old
GGML_ASSERT reported a cryptic "Failed to allocate tensor memory via RKNN API"
with no guidance.
Replace it with a GGML_ABORT that names the aperture and points at the fix:
use a quantized GGUF, or set RKNPU_HYBRID=W8A8_STANDARD for on-the-fly INT8
weights (which fits >4 GiB dense f16 models -- e.g. Qwen2.5-3B-f16 runs at
~130 t/s pp512 that way). Also null-check the grow re-alloc path, which was a
latent copy of the same crash.
No behaviour change for models that fit; only the failure message improves.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EWpfhDgYNA21tETDP9ueBE
mfritsche
396159c9d5