Cataract: NPU (RKNPU2) and open GPU (panvk/panthor) can't share a kernel #1

New Issue

2026-07-11T18:46:15Z

marfrit commented

2026-07-11 18:46:15 +00:00

Cataract's core idea — run prefill concurrently on the NPU (RKNPU2) and the open GPU (panvk) — has a kernel-coexistence blocker to solve before a single token flows.

The conflict

RKNPU2 backend (rk-llama.cpp, the 4.4x prefill engine) needs the vendor 6.1.75-npu-port kernel — where the mali kbase blob owns the GPU and the RKNPU driver exposes the NPU (/dev/dri/renderD129).
panvk (this repo — the open GPU path) needs the mainline panthor kernel, where the GPU is bound to panthor and the vendor mali blob is absent.

These are two different kernels. Today you get NPU xor open-GPU, not both.

Open questions

Does the panthor (mainline 7.x) kernel expose the RKNPU DRM node at all, or is RKNPU a vendor-only driver? (On panthor we saw card0/renderD128 -> rockchip-drm, card1/renderD129 -> RKNPU on the vendor kernel; need to confirm what mainline exposes.)
Can the vendor rknpu kmod be built/loaded against a mainline panthor kernel (out-of-tree), so RKNPU2 + panvk share one kernel?
Or: does mainline have an open NPU path (rocket/accel) usable for GEMM, so we drop the vendor blob entirely and run NPU+GPU both on panthor? (rocket exists but is CNN-shaped; see the RK3588 GPU-compute notes.)
Fallback: is CPU-orchestrated GPU-only prefill (panvk) + NPU-skipped an acceptable Cataract v0, adding the NPU later once coexistence is solved?

Why it matters

Without coexistence, "NPU + GPU concurrent prefill" collapses to "pick one engine." The GPU champion (this repo, 28 t/s Qwen-3B) and the NPU (42 t/s ornith-9B) each work in isolation on their own kernel; Cataract needs them on the same boot.

Blocks: the Cataract heterogeneous-prefill benchmark.

Cataract's core idea — run **prefill concurrently on the NPU (RKNPU2) and the open GPU (panvk)** — has a kernel-coexistence blocker to solve before a single token flows. ## The conflict - **RKNPU2 backend** (rk-llama.cpp, the 4.4x prefill engine) needs the **vendor `6.1.75-npu-port` kernel** — where the `mali` kbase blob owns the GPU and the `RKNPU` driver exposes the NPU (`/dev/dri/renderD129`). - **panvk** (this repo — the open GPU path) needs the **mainline `panthor` kernel**, where the GPU is bound to `panthor` and the vendor `mali` blob is absent. These are two different kernels. Today you get **NPU xor open-GPU**, not both. ## Open questions 1. Does the panthor (mainline 7.x) kernel expose the RKNPU DRM node at all, or is `RKNPU` a vendor-only driver? (On panthor we saw `card0/renderD128 -> rockchip-drm`, `card1/renderD129 -> RKNPU` on the *vendor* kernel; need to confirm what mainline exposes.) 2. Can the vendor `rknpu` kmod be built/loaded against a mainline panthor kernel (out-of-tree), so RKNPU2 + panvk share one kernel? 3. Or: does mainline have an open NPU path (rocket/accel) usable for GEMM, so we drop the vendor blob entirely and run NPU+GPU both on panthor? (rocket exists but is CNN-shaped; see the RK3588 GPU-compute notes.) 4. Fallback: is CPU-orchestrated **GPU-only prefill (panvk) + NPU-skipped** an acceptable Cataract v0, adding the NPU later once coexistence is solved? ## Why it matters Without coexistence, "NPU + GPU concurrent prefill" collapses to "pick one engine." The GPU champion (this repo, 28 t/s Qwen-3B) and the NPU (42 t/s ornith-9B) each work in isolation on their own kernel; Cataract needs them on the same boot. Blocks: the Cataract heterogeneous-prefill benchmark.

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: marfrit/rocky-vulkan-llama#1