The hantro stateless decoder derives the CAPTURE format from the
SPS attached to the per-request OUTPUT controls. Calling
VIDIOC_S_FMT on the CAPTURE queue at vaCreateSurfaces2 time can
leave the driver's vb2 state in an inconsistent configuration
where the queue accepts buffers and DQBUF returns successfully but
the kernel never actually writes decoded pixels into them.
Cross-reference: GStreamer's gst-plugins-bad/sys/v4l2codecs/
gstv4l2decoder.c only calls VIDIOC_G_FMT on the CAPTURE side
(via gst_v4l2_decoder_negotiate_src_format and friends). The
same code path produces correctly-decoded NV12 frames on the
same RK3568 hantro-vpu where libva-v4l2-request-with-S_FMT
emits flat-green zeroed CAPTURE buffers.
The v4l2_get_format() call immediately after this block already
gives us the bytesperline / sizes the driver chose; nothing else
in this file consumed the explicit S_FMT side-effects.
Empirical hypothesis test for the lingering "kernel decodes
without errors but emits zeroed CAPTURE" bug. If post-patch
output shows actual picture content, this confirms the
diagnosis: explicit CAPTURE format mutation breaks hantro's
internal state. If output remains flat-green, the bug is
elsewhere and we resume hex-dump-grade instrumentation.
Signed-off-by: Markus Fritsche <fritsche.markus@gmail.com>
Fourier's h264_va_picture_to_v4l2 only populated four fields of the
struct v4l2_ctrl_h264_decode_params: dpb (via h264_fill_dpb),
nal_ref_idc, top_field_order_cnt, bottom_field_order_cnt, and the
IDR_PIC flag. Many other required-by-spec fields were left at zero-
init (frame_num, idr_pic_id, pic_order_cnt_lsb, delta_pic_order_cnt_*,
dec_ref_pic_marking_bit_size, pic_order_cnt_bit_size,
slice_group_change_cycle, FIELD_PIC and BOTTOM_FIELD flags).
For an IDR (first frame) on hantro-vpu RK3568, the kernel parses
the bitstream from the OUTPUT buffer and uses these fields to drive
its bitstream-element offset tracking. Empirically the kernel
returned a successfully-decoded but ZEROED CAPTURE buffer — flat
dark-green frames in mpv output, no errors logged.
This patch fills every field VAAPI exposes:
- frame_num: from VAPicture->frame_num.
- FIELD_PIC flag: from VAPicture->pic_fields.bits.field_pic_flag.
- BOTTOM_FIELD flag: from
VAPicture->CurrPic.flags & VA_PICTURE_H264_BOTTOM_FIELD.
Also corrects the IDR_PIC flag to use |= instead of = so the new
field flags don't clobber it.
Fields NOT derivable from VAAPI's pre-parsed structures —
idr_pic_id, pic_order_cnt_lsb, delta_pic_order_cnt_*,
dec_ref_pic_marking_bit_size, pic_order_cnt_bit_size,
slice_group_change_cycle — require a slice_header() bit-level
parse. libva-v4l2-request does not currently do this. They remain
at zero-init.
Empirical question this patch answers: does hantro tolerate the
bit_size fields being zero for IDR frames, or does it strictly
require them? If post-patch CAPTURE is still zeroed, a slice-header
parser is required. If CAPTURE shows real picture data, hantro
fills in the bit-positions itself when no hint is supplied.
Cross-reference: gstv4l2codech264dec.c::
gst_v4l2_codec_h264_dec_fill_decoder_params (commit 9e3e775,
lines 632-678).
Signed-off-by: Markus Fritsche <fritsche.markus@gmail.com>
Patch 0002 sets V4L2_CID_STATELESS_H264_START_CODE to ANNEX_B on the
device, telling the kernel that OUTPUT-buffer payloads will contain
0x00 0x00 0x01 NAL start codes. picture.c::codec_store_buffer has
the prepend logic guarded by `if (context->h264_start_code)`, but
that boolean is set ONLY inside h264_get_controls() — a function
that exists but is never called.
Result: device expects ANNEX_B, libva-v4l2-request feeds raw NAL
payloads with no start codes, kernel cannot find slice boundaries,
hantro emits a zeroed CAPTURE buffer. mpv reports successful decode
because the V4L2 round-trip succeeds (no EINVAL); the visual output
is a flat dark-green frame (NV12 zero through BT.709).
Identified via:
- Patch 0006 cleared the EINVAL cluster-rejection (128 → 0 on
bbb_1080p30) but visual output remained flat green.
- GStreamer reference (gstv4l2codech264dec.c:1363-1377) confirms
start codes are required when ANNEX_B is selected.
- Source-archaeology of fourier's picture.c:67-74 showed the gate
on context->h264_start_code.
Fix: in context.c::RequestCreateContext, immediately after patch
0002's device-control block, set context_object->h264_start_code =
true to match the ANNEX_B mode we just programmed. Hardcoded for
now (matches 0002's hardcoded set); replaced with a runtime probe
in the planned probe-then-set commit.
Signed-off-by: Markus Fritsche <fritsche.markus@gmail.com>
Identified by cross-reference against GStreamer's
gst-plugins-bad/sys/v4l2codecs/gstv4l2codech264dec.c (upstream commit
9e3e775). At lines 1263-1304, GStreamer gates SLICE_PARAMS and
PRED_WEIGHTS submission on is_slice_based(self):
if (is_slice_based (self)) {
control[num_controls].id = V4L2_CID_STATELESS_H264_SLICE_PARAMS;
...
control[num_controls].id = V4L2_CID_STATELESS_H264_PRED_WEIGHTS;
...
}
In V4L2_STATELESS_H264_DECODE_MODE_FRAME_BASED, the kernel parses the
bitstream itself from the OUTPUT-queue payload; per-slice controls in
the request trigger cluster-validation EINVAL at error_idx=count
(observed on RK3568 hantro-vpu, kernel 6.19.10).
This patch:
- Reorders controls[] so FRAME_BASED-required entries come first
(SPS, PPS, SCALING_MATRIX, DECODE_PARAMS at indices 0..3) and the
SLICE_BASED-only entries come last (SLICE_PARAMS, PRED_WEIGHTS at
indices 4..5).
- Defaults num_controls=4 (FRAME_BASED), expanding to 5 for
SLICE_BASED and 6 when V4L2_H264_CTRL_PRED_WEIGHTS_REQUIRED.
- Hardcodes slice_based=false for now since patch 0002 sets the
device to FRAME_BASED unconditionally. A TODO marks the spot for
the planned probe-then-set commit, which will populate
context->decode_mode at CreateContext via VIDIOC_QUERYCTRL/
G_EXT_CTRLS and replace the hardcoded false with a runtime check.
Diagnosis chain:
- patch 0005 reduced one EINVAL per frame on PRED_WEIGHTS
submission, but cluster-level rejection persisted at error_idx=5
(count) — meaning kernel walked all 5 controls cleanly but
rejected the request as a whole.
- dmesg silent → rejection in V4L2 core (v4l2-ctrls-request.c /
v4l2-h264.c), not in hantro driver where it could log.
- GStreamer reference confirmed FRAME_BASED contract: only 4
sequence-and-frame-level controls go in the per-request batch.
After this patch the kernel should accept the per-request controls
and actually decode the bitstream into the CAPTURE buffer.
Signed-off-by: Markus Fritsche <fritsche.markus@gmail.com>
Per kernel UAPI (include/uapi/linux/v4l2-controls.h),
V4L2_CID_STATELESS_H264_PRED_WEIGHTS is a conditional control:
V4L2_H264_CTRL_PRED_WEIGHTS_REQUIRED(pps, slice) :=
((pps->flags & V4L2_H264_PPS_FLAG_WEIGHTED_PRED) &&
(slice_type == P || slice_type == SP)) ||
(pps->weighted_bipred_idc == 1 && slice_type == B)
Submitting PRED_WEIGHTS on a frame where the macro evaluates false
triggers VIDIOC_S_EXT_CTRLS to return EINVAL at error_idx=5 (the
6th, last control in the per-request batch) on hantro-vpu and any
other driver that strictly enforces the spec.
Smoke trace from RK3568 hantro on bbb_1080p30 (Main profile, no
weighted prediction): every per-frame batch fails identically, 13
EINVALs over a 10-frame run. Without this fix, ffmpeg's vaapi-copy
falls back to software decode for every frame.
Fix: narrow num_controls to 5 (excluding PRED_WEIGHTS at index 5)
when the macro returns false; keep at 6 when it returns true.
Defect found and fixed via Phase 6 Step 1 ohm smoke testing. Not
part of Sonnet's six-commit upstreamable plan; slotted in as patch
0005 ahead of the planned probe-then-set / FRAME_BASED commits
because it unblocks per-frame submission on every backing driver,
not just hantro.
Signed-off-by: Markus Fritsche <fritsche.markus@gmail.com>
Commit 3 of the upstreamable plan (upstreamable_design.md §1, §5).
Replaces the prior per-surface OUTPUT-buffer ownership model with a
small driver-wide pool sized by codec pipeline depth (4 H.264 frames
in flight), allocated unconditionally regardless of caller's
num_render_targets.
Prior art (kernel UAPI dev-stateless-decoder.rst, ffmpeg
v4l2_request.c, Chromium V4L2StatelessVideoDecoder, GStreamer
v4l2slh264dec) all decouple OUTPUT and CAPTURE pool sizing. fourier's
"output_count == surfaces_count" model was a category error: OUTPUT
buffers are request-time bitstream slots, CAPTURE buffers are
picture-time DPB slots; their lifecycles and sizing are independent.
Changes:
* NEW src/request_pool.{c,h} (~200 LoC):
- request_pool_init(): CREATE_BUFS + per-slot QUERYBUF + mmap.
- request_pool_destroy(): munmap all, idempotent.
- request_pool_acquire(): round-robin claim; returns V4L2 buffer
index of an unused slot or -1.
- request_pool_release(): mark slot free for reuse.
- request_pool_slot(): accessor for ptr/size given a buffer index.
* src/request.h: add struct request_pool output_pool to request_data.
* src/context.c::RequestCreateContext: replace the per-surface
OUTPUT loop with a single request_pool_init() call (count=4,
independent of surfaces_count). Drop the now-unused locals
(length, offset, source_data, output_buffers_count, index,
index_base, i, surface_object). DELETES patch 0002's
"output_buffers_count = ... ? ... : 4" hack inline — the pool's
own count parameter supersedes it.
* src/picture.c::RequestBeginPicture: borrow a pool slot at frame
start, write its mmap pointer/size/index into the surface's
transient source_* fields. The fields stay (still useful as
a borrow handle that the existing codec_store_buffer memcpys
target), but no longer represent surface-permanent ownership.
Reset slices_size/slices_count here too (was implicit on first
Render).
* src/surface.c::RequestSyncSurface: after VIDIOC_DQBUF returns
the OUTPUT buffer, release the pool slot and clear the surface's
borrow handle. Fixes the segv on second-frame submission.
* src/surface.c::RequestDestroySurfaces: remove the munmap of
source_data — pool owns the mmap.
* src/request.c::RequestTerminate: call request_pool_destroy()
before close(video_fd) so munmaps still target a valid fd.
* src/meson.build: add request_pool.c and request_pool.h to the
sources/headers lists.
This commit removes 0002's OUTPUT-pool hack inline (the
"floor to 4" line is gone). The DECODE_MODE/START_CODE block in 0002
remains until commit 4 lands.
Build-verified clean on aarch64.
Signed-off-by: Markus Fritsche <fritsche.markus@gmail.com>
Pure utility additions, no behaviour change. Three helpers in
src/v4l2.{c,h}:
- v4l2_query_ext_ctrl(): wraps VIDIOC_QUERY_EXT_CTRL by CID.
Returns 0 if the control exists, -1 if not. Caller passes NULL
qec to test existence only.
- v4l2_query_menu(): wraps VIDIOC_QUERYMENU at a given index.
Returns 0 if a menu item exists at that index, -1 otherwise.
- v4l2_ctrl_menu_has_value(): convenience layered on the above.
For a menu/intmenu-type control, walks all menu items between
minimum and maximum and returns true iff `value` is a valid
entry. Used by callers that ask "does this driver accept menu
value X for this CID?" without caring about iteration details.
These unblock commit 3 (request_pool — needs ext-ctrl probing for
codec-ops dispatch) and commit 4 (probe-then-set DECODE_MODE/
START_CODE — replaces 0002's unconditional set with a real probe)
of the upstreamable design's six-commit series.
Forward-declarations in v4l2.h keep the header lean: existing
prototypes already use opaque struct v4l2_ext_control * pointers
without including <linux/videodev2.h>; we follow the same
convention for struct v4l2_query_ext_ctrl and struct v4l2_querymenu.
No call sites added in this commit. Compile-only verification:
the .so links cleanly with three new exports.
Signed-off-by: Markus Fritsche <fritsche.markus@gmail.com>
Two related fixes that surfaced during the first hantro-vpu (RK3568)
smoke test of the multiplanar build:
1. **OUTPUT queue must be non-empty at STREAMON.** Hantro's
vb2_start_streaming rejects an empty queue with EINVAL. Some
VA-API callers (notably ffmpeg's vaapi-copy path) call
vaCreateContext with num_render_targets=0 and allocate render
targets lazily. The OUTPUT (bitstream-input) pool must NOT be
sized off surfaces_count alone — it is a request-time resource,
not per-surface. Quick fix: floor the pool to 4 buffers when
the caller passes 0. (A proper decoupling of OUTPUT pool from
surface lifecycle is documented in upstreamable_design.md.)
2. **Device-wide stateless H.264 controls before STREAMON.** The
V4L2 stateless framework requires V4L2_CID_STATELESS_H264_
DECODE_MODE and START_CODE be set on the device fd
(request_fd=-1) before stream start. Per-request controls
(SPS/PPS/SLICE_PARAMS/etc.) attached to a request_fd come
later via h264_set_controls(). hantro-vpu accepts only
DECODE_MODE_FRAME_BASED; START_CODE_ANNEX_B matches what the
existing slice-assembly path emits.
This is set unconditionally for now (errors silently ignored)
to keep cedrus and other backends compatible — they may
default to SLICE_BASED and not expose DECODE_MODE at all.
Probe-then-set via VIDIOC_QUERYCTRL is the upstream-correct
approach (see upstreamable_design.md §3).
After this patch, vainfo still enumerates as before, but the first
mpv vaapi-copy attempt advances past STREAMON and into actual
decode submission.
Signed-off-by: Markus Fritsche <fritsche.markus@gmail.com>
Fourier's local patch already wired multiplanar plumbing through
src/v4l2.c (helpers v4l2_type_video_{output,capture}() at lines 59-69,
struct v4l2_plane planes[] threading in QUERYBUF/QBUF/DQBUF, per-plane
EXPBUF loop at line 411) and through src/context.c, src/buffer.c,
src/picture.c via the v4l2_type_video_{output,capture}(video_format
->v4l2_mplane) helper calls.
The remaining gap: the NV12 entry in src/video.c was hardcoded to
v4l2_mplane=false, and the bootstrap path in src/surface.c was
hardcoded to singleplanar literals before video_format is populated.
This patch flips the NV12 entry to v4l2_mplane=true and updates the
two singleplanar literals in src/surface.c to their MPLANE variants:
- src/video.c:42 v4l2_mplane=false -> true (NV12 only;
Sunxi-tiled NV12 left at false for cedrus compatibility)
- src/surface.c:84 output_type = v4l2_type_video_output(true)
- src/surface.c:109 v4l2_find_format(..., CAPTURE_MPLANE, NV12)
Empirically, hantro-vpu (RK3568 mainline) advertises NV12 only under
V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE; querying the singleplanar type
returns no match (verified via VIDIOC_ENUM_FMT in Phase 3 GStreamer
strace baseline).
Trade-off accepted: legacy sunxi-cedrus singleplanar NV12 paths are
left unchanged via the SUNXI_TILED_NV12 entry (still mplane=false,
__arm__ only). Pure-NV12 cedrus on aarch64 would regress, but the
known userbase here is RK3566/RK3568 hantro.
Signed-off-by: Markus Fritsche <fritsche.markus@gmail.com>
Compound patch carrying the fork's pre-Step-1 substrate, originally
authored by Jernej Škrabec / fourier on top of bootlin's a3c2476:
- src/h264.c + src/picture.c: V4L2_CID_MPEG_VIDEO_H264_* renamed to
V4L2_CID_STATELESS_H264_*, struct shapes tracked to mainline
(V4L2_CID_STATELESS_H264_DECODE_MODE/_START_CODE added to the
passthrough shim).
- include/hevc-ctrls.h: redirect shim to <linux/v4l2-controls.h>
(kernel-side HEVC controls now live in the canonical UAPI header).
- src/meson.build: src/h265.c / src/h265.h commented out — HEVC
build path is excluded from this fork (RK3568 hantro G1/G2 has
no HEVC, and the kernel-side HEVC controls have a separate
rework in flight upstream).
- src/tiled_yuv.S: aarch64 stub for tiled_to_planar (assembly
source was sunxi-cedrus armv7-only; aarch64 needs a stub to keep
the build linking).
- include/h264-ctrls.h: removed (dead post-fourier — no source
includes it; the passthrough shim's CID aliases live in the
kernel header now).
Functionally equivalent to the prior fork master commits:
c1f5108 V4L2_PIX_FMT_H264_SLICE rename
4ccbfe9 Strip HEVC build path
da9f2a5 include/h264-ctrls.h passthrough + CID aliases
fc4bb10 src/h264.c track upstream UAPI shape
13e9b64 src/h264.c drop num_slices field
4d14ffb src/tiled_yuv.S aarch64 stub
1b02c9b src/h264.c include utils.h
Folded into one commit during 2026-05-04 Step 1 reconciliation
(see ../phase0_evidence/2026-05-04/findings.md). Per-patch history
of the early fork commits preserved on the pre-step1 branch.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Distritubions may install libraries under architecture-specific
sub-directories, to support multiple architectures on the same
system. In addition, the user may not wish to install the library
with the default prefix.
Use the libdir variable when setting the install path. This
allows both specifying different sub-directories, and a different
prefix.
The system normally has kernel headers shipped with the distribution.
These typically lag behind actual kernel releases. Thus they would not
have the latest API additions, such as the V4L2 request API this driver
uses.
However, it is also bad practice to just install new kernel headers into
the system wide default location, as there may be some differences
between it and what the C library was built against.
Add an option to specify a path to a set of up-to-date kernel headers.
This would allow the user to build this project in a safe but working
environment.
Signed-off-by: Chen-Yu Tsai <wens@csie.org>
We where first copying the image structure and then setting the pitches
and offets, so this information was lost. This fixes vaDerivedImage and
vaGetImage implementation.
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
This enables raw playback within GStreamer. This is useful for testing
even if slower then DMABuf. This is a partial implementation since we
don't implement partial copy of the surface.
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
The vaCreateSurface2 may be called multiple times, setting the format
again would lead to EBUSY being returned as you cannot change the
format if you have buffers allocated.
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reference frames are now identified using their timestamp:
set the timestamp when queuing the output buffer and use it to identify
the frame later on.
Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
Drop the per-codec options while at it, since we'll soon include a copy
of the associated headers.
Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
Commit 7ff2543e64 ("Add support for the single-planar V4L2 API")
missed the VIDIOC_QBUF bytesused parameter. The kernel will
warn loudly if bytesused is not properly defined for an OUTPUT buffer.
Signed-off-by: Ezequiel Garcia <ezequiel@collabora.com>
Apparently, pixelformats are expected to be settable although
the reason is not exactly clear to me.
However, intel vaapi driver sets all its pixelformats as
settable, and gstreamer-vaapi expects that as well.
Signed-off-by: Ezequiel Garcia <ezequiel@collabora.com>
H.264 and H.265 support is still not supported upstream,
so it makes sense to autodetect each codec and only
enable those that are supported.
Signed-off-by: Ezequiel Garcia <ezequiel@collabora.com>
Albin did not had an account on Kickstarter initially, so he was
registered as GuestXYZ. Upon his request, add his real name.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
As promised by Bootlin's Kickstarter campaign, all contributors above
16 EUR would get their name in the CREDITS file. This commit
implements the promised CREDITS file.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Since our use of the v4l2 API has some assumptions on the available
userspace APIs, check the capabilities reported by the driver to make
sure they are supported.
Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
This adds a helper function to query v4l2 capabilities, returning
either the device caps when available or the general device caps
otherwise.
Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
This adds support for MPEG2 quantization matrices, which are optional
given that fallback default matrices are used (on the kernel side) when
no such matrix is provided.
Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
Because there might be more than a single call to CreateSurfaces,
we cannot assume that the index relative to the number of surfaces
requested in a single call matches the v4l2 index.
Grab the base index (as returned by the kernel) when allocating
buffers and use it for memory mapping and addressing them in v4l2.
This avoids memory-mapping the first (index 0) buffer multiple times
in that scenario instead of the n-th allocated buffer (in the n-th
call in the sequence).
Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
The V4L2 API does not currently provide a way to liberate allocated
buffers one by one (which would fit well with DestroySurfaces in
VAAPI). Moreover, streaming needs to be off before liberating
buffers is allowed.
As a result, output an capture buffers can only be liberated when
destroying the decoding context, all at once, such as implemented
in this patch.
Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
Although this is not needed when using the combination of CREATE_BUFS and
QUERYBUF V4L2 ioctls (as currently done) to allocate and prepare buffers,
the REQBUF ioctl is useful to liberate the buffers after use.
This introduces a helper for this purpose.
Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
test0r
Chen-Yu Tsai