docs: remove local infrastructure references from gemma4-optimized.md

Replace the hostname with the hardware it actually is (Radxa ROCK 5 ITX+,
RK3588), and swap hardcoded home paths for the systemd %h specifier so the
unit stays correct while being user-agnostic.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EWpfhDgYNA21tETDP9ueBE
This commit is contained in:
mfritsche
2026-07-10 07:26:03 +02:00
parent 3258224a57
commit 3cd6e67b55
+12 -7
View File
@@ -1,7 +1,9 @@
# Gemma 4 12B on the RK3588 NPU — optimized serving config
Branch `mtp-b9549` (upstream b9570 + RKNPU2 backend + perf work). Host: boltzmann, RK3588,
31 GiB RAM, 4× A76 (cpu4-7) + 4× A55 (cpu0-3).
Branch `mtp-b9549` (upstream b9570 + RKNPU2 backend + perf work).
Hardware: Radxa ROCK 5 ITX+ (Rockchip RK3588), 32 GB RAM, 4× Cortex-A76 (cpu4-7) +
4× Cortex-A55 (cpu0-3), vendor NPU via `librknnrt`.
## Models
@@ -25,10 +27,10 @@ in-process alongside the matching target.
Type=simple
LimitNOFILE=65536
CPUAffinity=4-7
WorkingDirectory=/home/mfritsche/npu
ExecStart=/home/mfritsche/src/rk-llama.cpp/build-mtp/bin/llama-server \
-m /home/mfritsche/models/gemma-4-12b-it-Q8_0.gguf \
-md /home/mfritsche/models/mtp-gemma-4-12b-it-Q8_0.gguf \
WorkingDirectory=%h
ExecStart=%h/src/rk-llama.cpp/build-mtp/bin/llama-server \
-m %h/models/gemma-4-12b-it-Q8_0.gguf \
-md %h/models/mtp-gemma-4-12b-it-Q8_0.gguf \
--spec-type draft-mtp --spec-draft-n-max 4 \
--alias gemma-4-12b-q8 \
-c 131072 -t 4 \
@@ -37,6 +39,8 @@ Restart=on-failure
TimeoutStartSec=300
```
(`%h` is the systemd specifier for the service user home directory.)
Build: `cmake -B build-mtp -DLLAMA_RKNPU2=ON -DGGML_NATIVE=ON -DGGML_OPENMP=ON`.
Non-obvious, all load-bearing:
@@ -105,4 +109,5 @@ prompt-lookup — it reads a static cache file (`-lcs`) and drafts from an empty
`max_tokens` returns an empty `content` — that is not a failure. Benchmarks of copy behaviour
need `--reasoning-budget 0`.
- Tool calls work without `--jinja` (the gguf carries a template with tool support).
- Rollback: `perf-rknpu2` + `build/` + `llama-server-ornith-npu.service` are preserved.
- Rollback: branch `perf-rknpu2` and a non-MTP `build/` tree are the fallback; the backend
there is identical minus speculation.