Files
rk-llama.cpp/ggml
mfritsche 447672d647 rknpu2: saturate int8 activation quantization
quantize_fp32_to_int8 cast roundf(v) straight to int8_t. With scale = amax/127
the value should never exceed 127, but one ULP of error in iscale can round to
128, which wraps to -128 and injects a sign-flipped activation outlier. The
int4 path already clamped to +/-7; int8 did not.

Latent, not active: greedy output on gemma-4-12b-it-Q8_0 is byte-identical
before and after (sha 11b2174379e0c2040246, 708 B). This is insurance.

Found by a source review of the NPU math path.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EWpfhDgYNA21tETDP9ueBE
2026-07-10 09:01:57 +02:00
..