reauktion/daedalus-v4l2
2
0
Fork 0
Code Issues 3 Pull Requests Actions Packages Projects Releases Wiki Activity

kernel + daemon: H.264 B-frame display reorder fix (issue #6)

Browse Source 
H.264 streams with B-frames showed visibly pair-swapped output in
mpv / Firefox playback through the libva → daedalus_v4l2 path —
"frames went 2 1 4 3 6 5 instead of 1 2 3 4 5 6".  Reproduced in mpv
with --hwdec=vaapi-copy at 720p (bypassing Firefox's compositor),
confirming the bug was in this daemon pipeline, not downstream.

Root cause
----------
libavcodec's H.264 decoder internally reorders output to DISPLAY
order before returning from avcodec_receive_frame.  The daemon
previously called send_packet → receive_frame ONCE per REQ_DECODE
and shipped the resulting pixels in a RESP_FRAME tagged with the
SAME cookie.  For B-frames this is wrong: the frame returned from
receive_frame may belong to an EARLIER bitstream (libavcodec held
it for display-order release).  Cookie N's CAPTURE buffer therefore
got cookie N-2's pixels, while cookie N-2's CAPTURE buffer got
silently marked VB2_BUF_STATE_ERROR (the daemon returned
DAEDALUS_DECODE_NO_FRAME for the cookie whose pixels were held).

Fix shape
---------
Decouple kernel cookie identity (decode-order routing) from
libavcodec's display-ordered output.  Wire-protocol changes:

  REQ_DECODE  + __u64 src_pts        (= src_buf->vb2_buf.timestamp)
  RESP_FRAME  + __u32 flags          (HAS_PIXELS | SRC_CONSUMED)
              + __u64 output_src_pts (= frame->pts on drain)

PROTO_VERSION bumped 0 → 1.  Lock-step rebuild required.

Kernel
------
device_run now mirrors src_buf->vb2_buf.timestamp into req->src_pts
before sending REQ_DECODE, and stores it on the inflight item so
the completion path can stamp dst_buf.timestamp explicitly when
src/dst lifecycles decouple (V4L2_BUF_FLAG_TIMESTAMP_COPY's auto-
pairing no longer applies).

daedalus_complete_resp_frame splits into:

  HAS_PIXELS:    pack pixels into THIS cookie's CAPTURE buffer,
                 stamp dst timestamp from inflight->src_pts,
                 v4l2_m2m_buf_done(dst, DONE/ERROR).
                 No job_finish here.

  SRC_CONSUMED:  release the bound media_request, run
                 v4l2_m2m_buf_done(src) + v4l2_m2m_job_finish so
                 the scheduler can dispatch the next REQ.  dst_buf
                 may still be parked at this point.

Inflight entry is removed and freed only when BOTH src_buf and
dst_buf have been cleared.  Combined HAS_PIXELS|SRC_CONSUMED RESPs
(steady-state VP9/AV1 with no reorder lag) collapse to the prior
1:1 behaviour for free.

Daemon
------
daedalus_decoder_run_request split into three primitives:

  daedalus_decoder_submit       — set pkt->pts = req->src_pts,
                                  avcodec_send_packet.
  daedalus_decoder_drain_one    — avcodec_receive_frame, populate
                                  resp meta + output_src_pts (= the
                                  frame's pts, carried back from
                                  the bitstream that produced it).
  daedalus_decoder_pack_current — pack current AVFrame into the
                                  caller-mapped CAPTURE planes.

chardev_client maintains a small (src_pts → cookie, cached_req)
table indexed linearly (≤64 entries; bounded by V4L2 client buffer
pool depth).  On each REQ_DECODE:

  1. Register (src_pts → cookie) in the table.
  2. submit().
  3. Drain loop: for each frame returned, look up its owner cookie
     via pending_lookup(frame->pts), GET_DMABUF for THAT cookie,
     pack pixels, emit RESP_FRAME(owner_cookie, HAS_PIXELS,
     output_src_pts=frame->pts).  Combine with SRC_CONSUMED when
     owner_cookie equals the current REQ's cookie.
  4. If the current REQ's cookie wasn't drained inside the loop
     (libavcodec held the frame), emit a standalone SRC_CONSUMED
     RESP so the kernel runs job_finish + dispatches the next REQ;
     dst_buf for this cookie stays parked until a future drain
     produces its pixels.

VP9 / AV1 paths are unchanged in behaviour: one frame per REQ,
HAS_PIXELS|SRC_CONSUMED in one combined RESP.

Verified
--------
Builds clean cross-compiled on higgs against 6.18.29+rpt-rpi-2712
(Pi CM5).  Frame-size warning in device_run is pre-existing
(unchanged by this commit).
This commit is contained in:
Claude (noether)
2026-05-21 12:32:47 +02:00
parent f0d41867f6
commit 15fc2aba14
6 changed files with 725 additions and 288 deletions
Download Patch File Download Diff File Expand all files Collapse all files
daemon/src/decoder.h
+57 -22
View File
@@ -56,33 +56,68 @@ int daedalus_decoder_init(struct daedalus_decoder *dec,
void daedalus_decoder_cleanup(struct daedalus_decoder *dec);
/**
* daedalus_decoder_run_request - decode one REQ_DECODE payload
* daedalus_decoder_submit - send one REQ_DECODE's bitstream into libavcodec
* @dec: initialised decoder
* @req: REQ_DECODE prefix (from the wire)
* @req: REQ_DECODE prefix (from the wire); src_pts is stamped on
* the AVPacket so libavcodec returns frame->pts == src_pts
* when it eventually outputs the matching frame in display
* order (daedalus-v4l2#6).
* @bitstream: bitstream blob (req->bitstream_len bytes)
* @h264_meta: optional H.264 SPS/PPS metadata; non-NULL only when
* codec_id == H264 and the kernel set DAEDALUS_REQ_FLAG_
* H264_META. Used to synthesise the AnnexB SPS+PPS NALs
* libavcodec needs before any slice (libva-v4l2-request
* passes only the slice in @bitstream per the V4L2
* stateless API contract). NULL for VP9/AV1 paths.
* @resp: caller-allocated RESP_FRAME output (zeroed by callee)
* @planes: mapped CAPTURE planes (Phase 8.6 dmabuf path). If
* NULL or planes->nr == 0, the decoder runs but
* writes no pixels — caller still gets dims + digest.
* H264_META. See decoder.c for the AnnexB synthesis.
*
* Populates @resp with the decode outcome and writes decoded
* pixels (NV12 layout: Y to plane 0, interleaved CbCr to plane
* 1) directly into the mapped dmabuf planes. Always returns
* 0; decode-level failures are reported via @resp->status so
* the kernel sees a structured response rather than a dropped
* request.
* Calls avcodec_send_packet on the codec's per-codec AVCodecContext.
* Returns 0 on success; one of DAEDALUS_DECODE_ERR_* on failure
* (which the caller should propagate as the RESP_FRAME status for
* the cookie of this REQ). Does NOT call avcodec_receive_frame —
* use daedalus_decoder_drain_one for that.
*/
int daedalus_decoder_run_request(struct daedalus_decoder *dec,
const struct daedalus_req_decode *req,
const uint8_t *bitstream,
const struct daedalus_h264_meta *h264_meta,
struct daedalus_resp_frame *resp,
const struct daedalus_capture_planes *planes);
int daedalus_decoder_submit(struct daedalus_decoder *dec,
const struct daedalus_req_decode *req,
const uint8_t *bitstream,
const struct daedalus_h264_meta *h264_meta);
/**
* daedalus_decoder_drain_one - pop the next display-ordered frame, if any
* @dec: initialised decoder
* @codec_id: which codec context to drain (matches the REQ that just
* called submit). VP9/AV1/H264 use independent contexts.
* @resp: caller-allocated RESP_FRAME output (zeroed by callee).
* On a successful drain (return 0), resp's status / width /
* height / pix_fmt / luma_len / chroma_len / fnv1a_yuv /
* output_src_pts are populated; flags is left at 0 (caller
* adds HAS_PIXELS / SRC_CONSUMED). On EAGAIN, resp is
* zeroed.
*
* Return: 0 on a frame returned, -EAGAIN if libavcodec needs more
* input (display-order frame held inside DPB), <0 on a hard codec
* error (resp->status set).
*
* After a successful drain, the dec's internal AVFrame holds the
* decoded picture. Caller may immediately call
* daedalus_decoder_pack_current(planes) to write that picture into
* a CAPTURE buffer's dmabuf-mapped planes. Subsequent calls to
* drain_one (without another submit) try to pull additional frames
* from libavcodec's DPB.
*/
int daedalus_decoder_drain_one(struct daedalus_decoder *dec,
uint32_t codec_id,
struct daedalus_resp_frame *resp);
/**
* daedalus_decoder_pack_current - pack the last drained frame into planes
* @dec: initialised decoder; must have a frame from drain_one
* @planes: mapped CAPTURE planes (open via GET_DMABUF using the
* cookie that owns the frame's output_src_pts).
* @capture_pix_fmt: V4L2 fourcc on the CAPTURE side (NV12M, NV12,
* P010).
*
* Return: 0 on success, <0 on a pack failure (kernel sees only the
* metadata, not pixels — typical when a format isn't wired yet).
*/
int daedalus_decoder_pack_current(struct daedalus_decoder *dec,
const struct daedalus_capture_planes *planes,
uint32_t capture_pix_fmt);
#endif /* DAEDALUS_V4L2_DECODER_H */