surface: don't VIDIOC_S_FMT the CAPTURE queue

The hantro stateless decoder derives the CAPTURE format from the
SPS attached to the per-request OUTPUT controls. Calling
VIDIOC_S_FMT on the CAPTURE queue at vaCreateSurfaces2 time can
leave the driver's vb2 state in an inconsistent configuration
where the queue accepts buffers and DQBUF returns successfully but
the kernel never actually writes decoded pixels into them.

Cross-reference: GStreamer's gst-plugins-bad/sys/v4l2codecs/
gstv4l2decoder.c only calls VIDIOC_G_FMT on the CAPTURE side
(via gst_v4l2_decoder_negotiate_src_format and friends). The
same code path produces correctly-decoded NV12 frames on the
same RK3568 hantro-vpu where libva-v4l2-request-with-S_FMT
emits flat-green zeroed CAPTURE buffers.

The v4l2_get_format() call immediately after this block already
gives us the bytesperline / sizes the driver chose; nothing else
in this file consumed the explicit S_FMT side-effects.

Empirical hypothesis test for the lingering "kernel decodes
without errors but emits zeroed CAPTURE" bug. If post-patch
output shows actual picture content, this confirms the
diagnosis: explicit CAPTURE format mutation breaks hantro's
internal state. If output remains flat-green, the bug is
elsewhere and we resume hex-dump-grade instrumentation.

Signed-off-by: Markus Fritsche <fritsche.markus@gmail.com>
This commit is contained in:
2026-05-01 12:00:00 +00:00
parent 86a8545146
commit 597e896594
+14 -4
View File
@@ -118,10 +118,20 @@ VAStatus RequestCreateSurfaces2(VADriverContextP context, unsigned int format,
capture_type = v4l2_type_video_capture(video_format->v4l2_mplane);
rc = v4l2_set_format(driver_data->video_fd, capture_type,
video_format->v4l2_format, width, height);
if (rc < 0)
return VA_STATUS_ERROR_OPERATION_FAILED;
/*
* Do not VIDIOC_S_FMT on the CAPTURE queue. The hantro
* stateless decoder derives the CAPTURE format from the
* SPS attached to the OUTPUT request; explicitly setting
* it here can put the driver into an inconsistent state.
* GStreamer's v4l2slh264dec only G_FMTs CAPTURE (see
* gst-plugins-bad/sys/v4l2codecs/gstv4l2decoder.c::
* gst_v4l2_decoder_negotiate_src_format), and that
* variant produces correct decoded NV12 on the same
* hardware where this driver currently emits zeros.
*
* v4l2_get_format() below queries the driver's current
* state and gives us the bytesperline/sizes we need.
*/
} else {
video_format = driver_data->video_format;
capture_type = v4l2_type_video_capture(video_format->v4l2_mplane);