marfrit/rk-llama.cpp
1
0
Fork 0
Code Issues Pull Requests Actions 33 Packages Projects Releases Wiki Activity

ggml-rocket: make ROCKET_MIN_BATCH runtime-tunable via GGML_ROCKET_MIN_BATCH

Browse Source 
The offload gate (M < 32 -> CPU) was compile-time only. Reading the env var
(default 32 unchanged) gives a clean NPU on/off toggle: GGML_ROCKET_MIN_BATCH=999999
forces CPU-only for benchmarking without rmmod or a CPU-only build. Used for the
2026-07-18 ampere NPU-vs-CPU gemma prefill benchmark.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EWpfhDgYNA21tETDP9ueBE
This commit is contained in:
Claude (noether)
2026-07-18 15:21:55 +02:00
parent 288d70413a
commit 67316774a0
1 changed files with 2 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ggml/src/ggml-rocket/ggml-rocket.cpp
+2 -1
View File
@@ -384,7 +384,8 @@ static bool ggml_backend_rocket_device_supports_op(ggml_backend_dev_t dev, const
const int64_t K = src1->ne[0]; // == src0->ne[0]
const int64_t M = op->ne[1]; // batch rows (tokens)
const char *why = NULL;
if (M < ROCKET_MIN_BATCH) why="M<min_batch";
static const int min_batch = (getenv("GGML_ROCKET_MIN_BATCH") ? atoi(getenv("GGML_ROCKET_MIN_BATCH")) : ROCKET_MIN_BATCH);
if (M < min_batch) why="M<min_batch";
else if (src1->type != GGML_TYPE_F32) why="src1!=F32";
else if (!ggml_is_contiguous(src0)) why="src0 noncontig";
else if (!ggml_is_contiguous(src1)) why="src1 noncontig";