Implemented the prepA/collectC <-> rknn_matmul_run overlap as an M-chunk
software pipeline (correct, byte-identical). Flat at every chunk depth.
Timing the CPU-prep section with the NPU running vs idle showed it runs +43%
slower under overlap (1973 -> 2827 us): the "idle" cores contend for DDR with
the NPU streaming weights. rknn_matmul_run does not busy-poll (cores 96-100%
idle) but the bus is saturated. No-go; not merged.
Campaign through-line: decode, GEMM ceiling (30.5%), and prefill overlap are
all DDR-bandwidth-bound. Only levers left are fewer bytes (int4, broken) or
fewer passes (MTP, shipped).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EWpfhDgYNA21tETDP9ueBE
mfritsche
2635371317