rknpu2: guard OMP parallel regions with if(M>=8) for decode

Experiment: decode runs M=1; opening+joining an OpenMP parallel region per row (A-prep and C-collect loops) is pure fork/join overhead. Guard both pragmas with if(M>=8) so decode runs serially while prefill (M>=300) stays parallel. Byte-identical output. tg128 2.57 -> 2.73-2.80 (+6-9%), tg256 2.65; pp unchanged (35.8/41.6/40.6 for 300/512/700). Zero risk (same math, threading policy only).
This commit is contained in:
mfritsche
2026-07-09 01:25:39 +02:00
parent c5a47492ff
commit 8adda8448c
+2 -2
View File
@@ -647,7 +647,7 @@ static enum ggml_status ggml_backend_rknpu_graph_compute(ggml_backend_t backend,
const int row_stride = (int)(src1->nb[1] / sizeof(float));
void* dst_base = mem_A_shared->virt_addr;
#pragma omp parallel for
#pragma omp parallel for if(M >= 8)
for (int m = 0; m < M; ++m) {
const float* src_row = x + (size_t)m * row_stride;
std::vector<float> ready_row(K_seg_op);
@@ -738,7 +738,7 @@ static enum ggml_status ggml_backend_rknpu_graph_compute(ggml_backend_t backend,
const float hadamard_divisor = pipeline->use_hadamard ? (float)K_op : 1.0f;
#pragma omp parallel for
#pragma omp parallel for if(M >= 8)
for (int m = 0; m < M; m++) {
// Handling types and quantizations
switch (pipeline->npu_type_c) {