From: Markus Fritsche <fritsche.markus@gmail.com>
Date: 2026-05-01
Subject: [PATCH] h264: omit per-slice controls in FRAME_BASED mode

Identified by cross-reference against GStreamer's
gst-plugins-bad/sys/v4l2codecs/gstv4l2codech264dec.c (upstream commit
9e3e775). At lines 1263-1304, GStreamer gates SLICE_PARAMS and
PRED_WEIGHTS submission on is_slice_based(self):

    if (is_slice_based (self)) {
        control[num_controls].id = V4L2_CID_STATELESS_H264_SLICE_PARAMS;
        ...
        control[num_controls].id = V4L2_CID_STATELESS_H264_PRED_WEIGHTS;
        ...
    }

In V4L2_STATELESS_H264_DECODE_MODE_FRAME_BASED, the kernel parses the
bitstream itself from the OUTPUT-queue payload; per-slice controls in
the request trigger cluster-validation EINVAL at error_idx=count
(observed on RK3568 hantro-vpu, kernel 6.19.10).

This patch:
  - Reorders controls[] so FRAME_BASED-required entries come first
    (SPS, PPS, SCALING_MATRIX, DECODE_PARAMS at indices 0..3) and the
    SLICE_BASED-only entries come last (SLICE_PARAMS, PRED_WEIGHTS at
    indices 4..5).
  - Defaults num_controls=4 (FRAME_BASED), expanding to 5 for
    SLICE_BASED and 6 when V4L2_H264_CTRL_PRED_WEIGHTS_REQUIRED.
  - Hardcodes slice_based=false for now since patch 0002 sets the
    device to FRAME_BASED unconditionally. A TODO marks the spot for
    the planned probe-then-set commit, which will populate
    context->decode_mode at CreateContext via VIDIOC_QUERYCTRL/
    G_EXT_CTRLS and replace the hardcoded false with a runtime check.

Diagnosis chain:
  - patch 0005 reduced one EINVAL per frame on PRED_WEIGHTS
    submission, but cluster-level rejection persisted at error_idx=5
    (count) — meaning kernel walked all 5 controls cleanly but
    rejected the request as a whole.
  - dmesg silent → rejection in V4L2 core (v4l2-ctrls-request.c /
    v4l2-h264.c), not in hantro driver where it could log.
  - GStreamer reference confirmed FRAME_BASED contract: only 4
    sequence-and-frame-level controls go in the per-request batch.

After this patch the kernel should accept the per-request controls
and actually decode the bitstream into the CAPTURE buffer.

Signed-off-by: Markus Fritsche <fritsche.markus@gmail.com>
---
--- a/src/h264.c	2026-05-01 20:30:02.632190563 +0000
+++ b/src/h264.c	2026-05-01 20:49:46.937497317 +0000
@@ -531,6 +531,21 @@
 
 	sps.profile_idc = h264_profile_to_idc(profile);
 
+	/*
+	 * Per-request control batch, ordered so the controls REQUIRED in
+	 * V4L2_STATELESS_H264_DECODE_MODE_FRAME_BASED come first
+	 * (indices 0..3) and the SLICE_BASED-only controls come last
+	 * (indices 4..5).
+	 *
+	 * Cross-reference: GStreamer gst-plugins-bad
+	 * sys/v4l2codecs/gstv4l2codech264dec.c (commit 9e3e775,
+	 * lines 1263-1304) gates SLICE_PARAMS and PRED_WEIGHTS on
+	 * is_slice_based(self); under FRAME_BASED only SPS/PPS/
+	 * SCALING_MATRIX/DECODE_PARAMS are submitted. The kernel
+	 * parses the bitstream itself in FRAME_BASED mode; submitting
+	 * per-slice controls in that mode triggers cluster-validation
+	 * EINVAL at error_idx=count.
+	 */
 	struct v4l2_ext_control controls[6] = {
 		{
 			.id = V4L2_CID_STATELESS_H264_SPS,
@@ -545,14 +560,14 @@
 			.p_h264_scaling_matrix = &matrix,
 			.size = sizeof(matrix),
 		}, {
-			.id = V4L2_CID_STATELESS_H264_SLICE_PARAMS,
-			.p_h264_slice_params = &slice,
-			.size = sizeof(slice),
-		}, {
 			.id = V4L2_CID_STATELESS_H264_DECODE_PARAMS,
 			.p_h264_decode_params = &decode,
 			.size = sizeof(decode),
 		}, {
+			.id = V4L2_CID_STATELESS_H264_SLICE_PARAMS,
+			.p_h264_slice_params = &slice,
+			.size = sizeof(slice),
+		}, {
 			.id = V4L2_CID_STATELESS_H264_PRED_WEIGHTS,
 			.ptr = &weights,
 			.size = sizeof(weights),
@@ -560,20 +575,24 @@
 	};
 
 	/*
-	 * PRED_WEIGHTS is conditionally required per kernel UAPI:
-	 * V4L2_H264_CTRL_PRED_WEIGHTS_REQUIRED(pps, slice) is only
-	 * true when explicit weighted prediction applies (P/SP slice
-	 * with WEIGHTED_PRED flag, or B slice with weighted_bipred_idc
-	 * == 1). Submitting it unconditionally on a frame that does
-	 * not need it triggers EINVAL at error_idx=5 on hantro and
-	 * other drivers that strictly enforce the spec.
+	 * Decode-mode dispatch. Patch 0002 unconditionally sets the
+	 * device to FRAME_BASED, so we hardcode that here. When the
+	 * planned probe-then-set commit lands, slice_based becomes
+	 *     context->decode_mode == V4L2_STATELESS_H264_DECODE_MODE_SLICE_BASED
+	 * with context->decode_mode populated at CreateContext via
+	 * VIDIOC_QUERYCTRL/G_EXT_CTRLS.
 	 *
-	 * controls[5] is PRED_WEIGHTS (last in array); narrow the
-	 * submission count to exclude it when not required.
+	 * FRAME_BASED:    4 controls (SPS, PPS, SCALING_MATRIX, DECODE_PARAMS).
+	 * SLICE_BASED:   +SLICE_PARAMS (always), +PRED_WEIGHTS (when
+	 *                V4L2_H264_CTRL_PRED_WEIGHTS_REQUIRED).
 	 */
-	unsigned int num_controls = 6;
-	if (!V4L2_H264_CTRL_PRED_WEIGHTS_REQUIRED(&pps, &slice))
+	const bool slice_based = false; /* TODO: probe via context->decode_mode */
+	unsigned int num_controls = 4;
+	if (slice_based) {
 		num_controls = 5;
+		if (V4L2_H264_CTRL_PRED_WEIGHTS_REQUIRED(&pps, &slice))
+			num_controls = 6;
+	}
 
 	rc = v4l2_set_controls(driver_data->video_fd, surface->request_fd,
 			       controls, num_controls);