h264: omit per-slice controls in FRAME_BASED mode

Identified by cross-reference against GStreamer's
gst-plugins-bad/sys/v4l2codecs/gstv4l2codech264dec.c (upstream commit
9e3e775). At lines 1263-1304, GStreamer gates SLICE_PARAMS and
PRED_WEIGHTS submission on is_slice_based(self):

    if (is_slice_based (self)) {
        control[num_controls].id = V4L2_CID_STATELESS_H264_SLICE_PARAMS;
        ...
        control[num_controls].id = V4L2_CID_STATELESS_H264_PRED_WEIGHTS;
        ...
    }

In V4L2_STATELESS_H264_DECODE_MODE_FRAME_BASED, the kernel parses the
bitstream itself from the OUTPUT-queue payload; per-slice controls in
the request trigger cluster-validation EINVAL at error_idx=count
(observed on RK3568 hantro-vpu, kernel 6.19.10).

This patch:
  - Reorders controls[] so FRAME_BASED-required entries come first
    (SPS, PPS, SCALING_MATRIX, DECODE_PARAMS at indices 0..3) and the
    SLICE_BASED-only entries come last (SLICE_PARAMS, PRED_WEIGHTS at
    indices 4..5).
  - Defaults num_controls=4 (FRAME_BASED), expanding to 5 for
    SLICE_BASED and 6 when V4L2_H264_CTRL_PRED_WEIGHTS_REQUIRED.
  - Hardcodes slice_based=false for now since patch 0002 sets the
    device to FRAME_BASED unconditionally. A TODO marks the spot for
    the planned probe-then-set commit, which will populate
    context->decode_mode at CreateContext via VIDIOC_QUERYCTRL/
    G_EXT_CTRLS and replace the hardcoded false with a runtime check.

Diagnosis chain:
  - patch 0005 reduced one EINVAL per frame on PRED_WEIGHTS
    submission, but cluster-level rejection persisted at error_idx=5
    (count) — meaning kernel walked all 5 controls cleanly but
    rejected the request as a whole.
  - dmesg silent → rejection in V4L2 core (v4l2-ctrls-request.c /
    v4l2-h264.c), not in hantro driver where it could log.
  - GStreamer reference confirmed FRAME_BASED contract: only 4
    sequence-and-frame-level controls go in the per-request batch.

After this patch the kernel should accept the per-request controls
and actually decode the bitstream into the CAPTURE buffer.

Signed-off-by: Markus Fritsche <fritsche.markus@gmail.com>
This commit is contained in:
2026-05-01 12:00:00 +00:00
parent e382c63e20
commit 4246d5d537
+34 -15
View File
@@ -531,6 +531,21 @@ int h264_set_controls(struct request_data *driver_data,
sps.profile_idc = h264_profile_to_idc(profile);
/*
* Per-request control batch, ordered so the controls REQUIRED in
* V4L2_STATELESS_H264_DECODE_MODE_FRAME_BASED come first
* (indices 0..3) and the SLICE_BASED-only controls come last
* (indices 4..5).
*
* Cross-reference: GStreamer gst-plugins-bad
* sys/v4l2codecs/gstv4l2codech264dec.c (commit 9e3e775,
* lines 1263-1304) gates SLICE_PARAMS and PRED_WEIGHTS on
* is_slice_based(self); under FRAME_BASED only SPS/PPS/
* SCALING_MATRIX/DECODE_PARAMS are submitted. The kernel
* parses the bitstream itself in FRAME_BASED mode; submitting
* per-slice controls in that mode triggers cluster-validation
* EINVAL at error_idx=count.
*/
struct v4l2_ext_control controls[6] = {
{
.id = V4L2_CID_STATELESS_H264_SPS,
@@ -544,14 +559,14 @@ int h264_set_controls(struct request_data *driver_data,
.id = V4L2_CID_STATELESS_H264_SCALING_MATRIX,
.p_h264_scaling_matrix = &matrix,
.size = sizeof(matrix),
}, {
.id = V4L2_CID_STATELESS_H264_SLICE_PARAMS,
.p_h264_slice_params = &slice,
.size = sizeof(slice),
}, {
.id = V4L2_CID_STATELESS_H264_DECODE_PARAMS,
.p_h264_decode_params = &decode,
.size = sizeof(decode),
}, {
.id = V4L2_CID_STATELESS_H264_SLICE_PARAMS,
.p_h264_slice_params = &slice,
.size = sizeof(slice),
}, {
.id = V4L2_CID_STATELESS_H264_PRED_WEIGHTS,
.ptr = &weights,
@@ -560,20 +575,24 @@ int h264_set_controls(struct request_data *driver_data,
};
/*
* PRED_WEIGHTS is conditionally required per kernel UAPI:
* V4L2_H264_CTRL_PRED_WEIGHTS_REQUIRED(pps, slice) is only
* true when explicit weighted prediction applies (P/SP slice
* with WEIGHTED_PRED flag, or B slice with weighted_bipred_idc
* == 1). Submitting it unconditionally on a frame that does
* not need it triggers EINVAL at error_idx=5 on hantro and
* other drivers that strictly enforce the spec.
* Decode-mode dispatch. Patch 0002 unconditionally sets the
* device to FRAME_BASED, so we hardcode that here. When the
* planned probe-then-set commit lands, slice_based becomes
* context->decode_mode == V4L2_STATELESS_H264_DECODE_MODE_SLICE_BASED
* with context->decode_mode populated at CreateContext via
* VIDIOC_QUERYCTRL/G_EXT_CTRLS.
*
* controls[5] is PRED_WEIGHTS (last in array); narrow the
* submission count to exclude it when not required.
* FRAME_BASED: 4 controls (SPS, PPS, SCALING_MATRIX, DECODE_PARAMS).
* SLICE_BASED: +SLICE_PARAMS (always), +PRED_WEIGHTS (when
* V4L2_H264_CTRL_PRED_WEIGHTS_REQUIRED).
*/
unsigned int num_controls = 6;
if (!V4L2_H264_CTRL_PRED_WEIGHTS_REQUIRED(&pps, &slice))
const bool slice_based = false; /* TODO: probe via context->decode_mode */
unsigned int num_controls = 4;
if (slice_based) {
num_controls = 5;
if (V4L2_H264_CTRL_PRED_WEIGHTS_REQUIRED(&pps, &slice))
num_controls = 6;
}
rc = v4l2_set_controls(driver_data->video_fd, surface->request_fd,
controls, num_controls);