---
cycle: 5
phase: 4
status: draft, awaiting Phase 5 review
date_opened: 2026-05-18
parent: k5_cdef_phase3.md
predicted_R: 0.02-0.05 (deep RED)
---

# Cycle 5, Phase 4 — QPU CDEF shader plan

Plan a Vulkan compute shader for the AV1 CDEF primary+secondary
8×8 luma filter on V3D 7.1. Predicted **deep RED** (R₅ = 0.02-0.05);
plan + build it anyway because:
- Confirms the prediction with measured data (or refutes it).
- Provides the dispatch path needed for Phase 8 V4L2 wrapper.
- Closes cycle 5 (Phases 1-7 all on the record).

## Kernel shape (NEON reference: 263 ns/block)

Per 8×8 output block: 8 directions table, 2 offsets each. For
each output pixel:

- 2 primary taps (off1, -off1) using `dir`
- 4 secondary taps (off2, -off2, off3, -off3) using `(dir+2)%8` and `(dir-2+8)%8`
- For each of 2 k-rounds (different tap weights)
- 12 `constrain()` ops per pixel × 64 pixels = **768 constrain ops per block**
- Plus min/max bookkeeping for iclip

The constrain math:
```
diff = p - px;
adiff = abs(diff);
clip = max(0, threshold - (adiff >> shift));
constrained = sign(diff) * min(adiff, clip);
sum += tap * constrained;
```

Output: `dst[r,c] = clamp(px + ((sum - (sum<0) + 8) >> 4), min, max);`

## V3D substrate fit (phase0 constraints)

- **No DP4A**: each constrain is scalar int math; no vector packing
  helps (per cycle 3 MC finding). Predicted instruction count
  proportional to ops.
- **16KB shared**: not needed — each pixel computes independently;
  no row sharing in compute side (tmp is read-only input).
- **subgroupSize=16**: 1 pixel per lane × 16 lanes/sg = 16 pixels
  per sg. Block of 64 pixels = 4 sg slots. Better: 2 blocks per
  WG of 256 invocations (16 sg) → 256 pixels = 4 blocks per WG.
  Following cycle-2 pattern: aim for **64 blocks/WG**? Too high
  — 64 × 64 = 4096 pixels/WG → 256 lanes × 16 pixels/lane.
  Wait — 256 lanes total, 1 pixel/lane → 256 pixels = 4 blocks/WG.
  Settle on **4 blocks/WG**, 256 invocations.
- **≤8 SSBO**: need 3 (meta, tmp, dst). Comfortable.
- **No shaderFloat16/Int8 ALU**: int math everywhere. uint8 dst
  via storageBuffer8BitAccess (cycle-1 v4 pattern).

## SSBO layout (post Phase 5 RED-1 fix)

- `Meta[i]`: `uvec4(dst_off_bytes, params0, tmp_off_u16, dir)` —
  i.e. `m.x` = dst_off, `m.y` = params (pri | sec << 8 |
  damping << 16), `m.z` = tmp block-origin u16-element offset,
  `m.w` = dir (3 bits used). **Pseudo-code below uses this
  layout consistently.**
- `Tmp[]`: `uint16_t` array via `GL_EXT_shader_16bit_storage` +
  `storageBuffer16BitAccess` — both already enabled in
  `v3d_runner.c` and used by cycle 1 IDCT shader. No uncertainty.
- `Dst[]`: `uint8_t` array via `GL_EXT_shader_8bit_storage` (per
  cycle-1 v4 pattern).

## Lane decomposition

256 invocations / WG, 4 blocks/WG:
- `lane_in_wg = 0..255`
- `block_in_wg = lane_in_wg / 64` (0..3)
- `pixel_in_block = lane_in_wg & 63` (0..63 → row=>>3, col=&7)
- `block_idx = wg_id * 4 + block_in_wg`

No barrier needed; each pixel computes independently.

## Push constants

```glsl
layout(push_constant) uniform PC {
    uint n_blocks;
    uint tmp_stride_u16;   // = 16
    uint dst_stride_u8;
    uint _pad;
} pc;
```

## Directions table (post Phase 5 RED-3 fix)

Use `const ivec2 dirs[14]` (8 directions + 6 wrap copies), each
entry = `(off_k0, off_k1)`. Signed-int storage handles negative
offsets cleanly without manual sign-extension. The OR-pack
approach proposed earlier would corrupt negative offsets;
abandoned.

Values from `tests/cdef_ref.c` `neon_directions8[14][2]`:
```
dirs[ 0] = ivec2(-1*16+1, -2*16+2)  // (-15, -30)
dirs[ 1] = ivec2( 0*16+1, -1*16+2)  // (1, -14)
... (etc.)
```

## Shader pseudo-code

```glsl
void main() {
    uint gid = gl_GlobalInvocationID.x;
    uint wg_id = gid / 256u;
    uint block_in_wg = (gid & 255u) >> 6;   // 0..3
    uint px_idx = gid & 63u;                 // 0..63
    uint row = px_idx >> 3;                  // 0..7
    uint col = px_idx & 7u;                  // 0..7

    uint block_idx = wg_id * 4u + block_in_wg;
    if (block_idx >= pc.n_blocks) return;

    uvec4 m = u_meta.meta[block_idx];
    uint dst_off = m.x + row * pc.dst_stride_u8 + col;
    uint tmp_off = m.z + row * pc.tmp_stride_u16 + col;   // m.z = tmp block-origin u16 offset
    int pri = int(m.y & 0xffu);
    int sec = int((m.y >> 8) & 0xffu);
    int damping = int((m.y >> 16) & 0xffu);
    int dir = int(m.w & 7u);

    int px = int(u_tmp.tmp[tmp_off]);
    int sum = 0;
    int mn = px, mx = px;

    int pri_shift = max(0, damping - ulog2(pri));
    int sec_shift = max(0, damping - ulog2(sec));  // RED-2: NEON uqsub saturates to 0; GLSL >> by negative is UB.

    // pri_tap[k] for k=0,1 = 4-(pri&1), then (tap & 3) | 2
    int pri_tap0 = 4 - (pri & 1);
    int pri_tap1 = (pri_tap0 & 3) | 2;

    int pri_idx = dir;
    int sec1_idx = (dir + 2) & 7;
    int sec2_idx = (dir + 6) & 7;

    // k=0
    {
        int off = dirs_off1[pri_idx];
        int p0 = int(u_tmp.tmp[tmp_off + off]);
        int p1 = int(u_tmp.tmp[tmp_off - off]);
        sum += pri_tap0 * constrain(p0 - px, pri, pri_shift);
        sum += pri_tap0 * constrain(p1 - px, pri, pri_shift);
        mn = min(min(mn, p0), p1); mx = max(max(mx, p0), p1);
        // ... 4 secondary taps the same way for off2, off3
    }
    // k=1: same with off2 versions

    int adj = (sum - int(sum < 0) + 8) >> 4;
    int out = clamp(px + adj, mn, mx);
    u_dst.dst[dst_off] = uint8_t(out);
}
```

Note: dirs_off1/dirs_off2 are per-k-round offsets. For k=0 use
`*[idx][0]` (the "+1 row" component); for k=1 use `*[idx][1]`
(the "+2 rows" component).

## Throughput prediction

NEON 1-core: 3.81 Mblock/s = 262 ns/block.
V3D 7.1 compute estimate (per cycle 3 MC pattern):
- 12 constrain ops × 8 SMUL24+ADD per constrain = ~96 instructions per pixel
- 64 pixels per block, 4 blocks/WG → 256 lanes work in parallel
- Per-block QPU latency ≈ instruction count / lanes × cycle time
- Predicted: ~5000-8000 ns per block → 0.125-0.2 Mblock/s
- R₅ = 0.125 / 3.81 = **0.033** (deep RED, matches prediction)

shaderdb prediction:
- ~800-1200 instructions (similar shape to cycle 1 IDCT, more
  ops though)
- 2-4 threads (if uniform count stays < 144 per phase5''' finding 2)
- uniform count: 14 entries × 2 offsets = 28; + tap weights 4
  = small. Should stay well below threshold. Predict 4 threads.

## Phase 5 review applied (2026-05-18, Sonnet)

REDs fixed inline above:
- RED-1: meta field layout — `m.z = tmp_off`, `m.w = dir` (was swapped).
- RED-2: `sec_shift = max(0, ...)` to match NEON's `uqsub` saturation.
- RED-3: directions table is `const ivec2 dirs[14]`, not packed.

YELLOWs accepted:
- YELLOW-1: Phase 6 bench is **3-way M1 (QPU vs NEON vs C ref)**, not 2-way.
- YELLOW-2: 16-bit storage extension confirmed present (cycle-1 already uses it).
- YELLOW-3: `sec_tap0 = 2, sec_tap1 = 1` made explicit in shader.
- YELLOW-4: use `gl_WorkGroupID.x` directly, not `gid / 256u`.

**Also**: also clamp `sec_shift` in `tests/cdef_ref.c` (currently
unguarded; M1 gate passes by bench-param luck — params don't
exercise negative shift). Fix C ref + add negative-shift cases to
bench param generator so the 3-way M1 actually stresses the
edge case.

## Phase 5 review focus

Particular review items for the Phase 5 second-model audit:

1. **Sentinel handling**: when reading from tmp halo, raw uint16
   values could be 0x8000 (INT16_MIN sentinel from padding) for
   real picture-boundary blocks. Our cycle 5 bench uses random
   pixel values (no sentinels), but a production deployment would
   pass through padded blocks. The constrain() math naturally
   handles INT16_MIN-as-uint16=32768 (clip becomes 0), BUT the
   `min(mn, p)` should use UNSIGNED compare and `max(mx, p)`
   should use SIGNED compare to match NEON. GLSL's `min`/`max`
   on `int` is signed; need separate `umin` (or cast to uint).

   Concretely: `mn = int(min(uint(mn), uint(p)))`,
   `mx = max(mx, int(int16_t(p)))`.

2. **OOB read on direction taps**: for blocks near the picture
   edge, the direction offsets reach into the halo. Our bench
   uses random pixels there (valid uint8). For deployment with
   sentinels, we need to either (a) zero-out halo values that are
   sentinels before reading or (b) accept the constrain-math-
   handles-it argument.

3. **Tmp stride**: must equal 16 (stride_u16=16) to match the
   directions table that's baked at stride 16. push constant
   `tmp_stride_u16` should be const or asserted = 16 in bench.

4. **dst_stride_u8**: cycle-2 LPF used dst_stride_u8 = 8 (for
   isolated blocks). Same here. Production deployment with real
   picture strides (e.g. 1920) would need re-validation.

5. **Push-constant meta size**: m.z carries dir (only 3 bits used);
   could be packed into params0. But current layout simple, leave
   as-is.

## Acceptance criteria

- shaderdb predicted ≤ 1200 inst, ≥ 2 threads, ≤ 30 uniforms, no
  spills.
- M1 bit-exact (use the same bench setup as Phase 3 but compare
  QPU output vs NEON output).
- M2 captured (any number, even deep RED).
- M4 measured against pure-NEON-4 baseline (expected: negative,
  per same-kernel pattern); cross-reference Issue 003 V1/V2 for
  the mixed-kernel context.

## Estimated effort

2-3 hours for the shader; 30 min for the M2 bench; 30 min for
M4. Total: ~4 hours, then Phase 7 closure.