Phase 8.5: full V4L2 m2m driver, VP9 decode via QBUF/DQBUF
Replaces the Phase 8.4 debugfs-triggered chardev path with a
real V4L2 m2m driver. Userspace clients now drive decoding the
standard way — S_FMT / REQBUFS / QBUF on the OUTPUT (bitstream)
queue, DQBUF on the CAPTURE (NV12M) queue. Kernel device_run
packs the bitstream into REQ_DECODE; daemon decodes via FFmpeg;
RESP_FRAME's inline NV12 pixel payload lands in the CAPTURE
buffer. Phase 8.6 swaps the inline payload for dmabuf so big
frames stop being capped at 64 KiB.
Kernel (daedalus_v4l2_main.c, rewritten + main.h added):
- Per-open struct daedalus_ctx: v4l2_fh, m2m_ctx, ctrl_handler,
per-queue v4l2_pix_format_mplane.
- Two vb2_queues (vb2_vmalloc_memops for both — no DMA needed
yet; 8.6 switches CAPTURE to dma_contig for dmabuf-export):
OUTPUT = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE, VP9_FRAME
CAPTURE = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE, NV12M
- Full v4l2_ioctl_ops table: querycap, enum_fmt, g/s/try_fmt
for both queues, reqbufs/querybuf/qbuf/dqbuf/create_bufs/
prepare_buf/expbuf/streamon/streamoff via v4l2_m2m_ioctl_*
helpers.
- v4l2_m2m_ops.device_run: peeks next OUTPUT buf, builds
REQ_DECODE inline with the bitstream bytes, enqueues with an
auto-incrementing cookie, stores {ctx, src_buf, dst_buf} in
a per-device inflight list. Job stays open until RESP_FRAME.
- daedalus_complete_resp_frame(): pops the inflight entry,
memcpys inline NV12 pixels into the CAPTURE buffer (Y plane
+ interleaved CbCr), finishes via
v4l2_m2m_buf_done_and_job_finish — NOT plain buf_done +
job_finish, which leaves the src buf on the m2m queue and
causes device_run to immediately re-run on the same input
(caught on first run; second REQ_DECODE for same bitstream +
eventual oops in stop_streaming on teardown).
Kernel (daedalus_v4l2_chardev.c):
- RESP_FRAME handler now hands inline pixel payload to
daedalus_complete_resp_frame so it lands in the CAPTURE
vb2 buffer. Existing PONG and debugfs test_decode paths still
work; the latter produces a harmless ratelimited "unknown
cookie" since it bypasses V4L2 m2m.
Daemon (decoder.c, decoder.h):
- daedalus_decoder_run_request signature extended with
(nv12_out, nv12_cap, nv12_used). After the FNV-1a digest the
decoder packs YUV420P into NV12 in the caller's buffer: Y
plane line-by-line stripped of stride padding; Cb/Cr
interleaved into a single chroma plane. Truncation silent —
kernel only memcpys what fits in the CAPTURE plane.
Daemon (chardev_client.c):
- handle_req_decode allocates a response buffer sized for the
full chardev payload, lets decoder fill the pixel area
after the resp_frame struct, sends the full payload via the
existing send_response.
Test client (tools/test_m2m_decode.c, new):
- Minimal V4L2 m2m client: S_FMT both queues, REQBUFS 1 each,
mmap+fill OUTPUT, QBUF both, STREAMON, poll, DQBUF, dump
CAPTURE planes to a raw NV12 file. ~250 LOC; verifies the
whole flow without needing v4l2-ctl framing.
Roadmap update (docs/roadmap.md):
- Phase 8.4 retitled "daemon ↔ kernel decode round-trip"
to reflect what actually shipped (vs. the original V4L2-
ioctl-driven plan which moved here).
- Phase 8.5 retitled "full V4L2 m2m driver" with closure
status.
- Phase 8.6 reshaped to two tracks: dmabuf + AV1/H.264/
stateless controls + media controller. Adds the punch list
of v4l2-compliance failures (DECODER_CMD, S_FMT colorspace)
that 8.6 will fix.
Verification on hertz (Pi 5, 6.12.75+rpt-rpi-2712):
Kernel + daemon build clean (-Wall -Wextra clean both sides).
Test harness drives one VP9 keyframe end-to-end:
OUTPUT REQBUFS -> 2
CAPTURE REQBUFS -> 2
QBUF OUTPUT[0] bytesused=1566
QBUF CAPTURE[0]; STREAMON both
poll revents=0x5
DQBUF OUTPUT[0] flags=0x4001 (DONE)
DQBUF CAPTURE[0] flags=0x4000 payloads=[12288, 6144]
wrote 12288 Y + 6144 UV bytes to /tmp/out_m2m.nv12
Pixel correctness vs reference:
ffmpeg -i vp9_small.ivf -pix_fmt nv12 -f rawvideo -y ref.nv12
cmp /tmp/out_m2m.nv12 /tmp/ref.nv12 → match ✓
Byte-for-byte identical to FFmpeg's stock CPU decode.
v4l2-compliance: detected as Stateless Decoder; most ioctls
pass; two expected fails documented in closure doc
(DECODER_CMD/media controller, S_FMT colorspace).
Clean teardown: SIGTERM the daemon, rmmod the module, no
oops/WARN in dmesg.
Per correctness-before-speed:
- Real V4L2 ioctl table (not stubs); uses v4l2-core helpers
where they exist instead of reinventing.
- v4l2_m2m_buf_done_and_job_finish (not the manual sequence)
to keep scheduler state consistent.
- Bit-exact reference comparison, not just "looks right."
- Documented every compliance failure with the planned fix.
- All resource paths (kmalloc/kfree, inflight list cleanup,
src/dst buf removal in stop_streaming) handled on every
error branch.
Phase 8.6 next: dmabuf-export for CAPTURE (removes 64 KiB
frame-size cap), add AV1+H.264 codecs, add V4L2 stateless
controls + media controller binding, fix the colorspace +
cookie-namespace compliance issues.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,291 @@
|
||||
/* SPDX-License-Identifier: BSD-2-Clause */
|
||||
/*
|
||||
* test_m2m_decode — minimal V4L2 m2m stateless decoder client.
|
||||
*
|
||||
* Drives /dev/videoNN through one full QBUF/DQBUF round-trip:
|
||||
* 1. open the m2m device
|
||||
* 2. S_FMT on OUTPUT (VP9_FRAME) and CAPTURE (NV12M)
|
||||
* 3. REQBUFS 1 on both queues
|
||||
* 4. mmap the OUTPUT buffer, copy bitstream into it, QBUF
|
||||
* 5. QBUF the CAPTURE buffer
|
||||
* 6. STREAMON both queues
|
||||
* 7. poll for completion
|
||||
* 8. DQBUF capture
|
||||
* 9. mmap+dump the CAPTURE buffer to a file
|
||||
*
|
||||
* Phase 8.5 verification harness — confirms the kernel's m2m
|
||||
* wiring works end-to-end against a real bitstream.
|
||||
*
|
||||
* Usage:
|
||||
* test_m2m_decode <vp9_keyframe.bin> <out.nv12> [w] [h]
|
||||
* defaults: w=128 h=96
|
||||
*/
|
||||
#include <stdint.h>
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
#include <errno.h>
|
||||
#include <fcntl.h>
|
||||
#include <unistd.h>
|
||||
#include <sys/ioctl.h>
|
||||
#include <sys/mman.h>
|
||||
#include <sys/stat.h>
|
||||
#include <poll.h>
|
||||
#include <linux/videodev2.h>
|
||||
|
||||
#define V4L2_DEV "/dev/video0"
|
||||
#define POLL_TIMEOUT_MS 5000
|
||||
|
||||
static void die(const char *msg)
|
||||
{
|
||||
perror(msg);
|
||||
exit(1);
|
||||
}
|
||||
|
||||
static void *read_file(const char *path, size_t *out_len)
|
||||
{
|
||||
struct stat st;
|
||||
void *buf;
|
||||
int fd;
|
||||
ssize_t n;
|
||||
|
||||
fd = open(path, O_RDONLY);
|
||||
if (fd < 0)
|
||||
die("open bitstream");
|
||||
if (fstat(fd, &st) < 0)
|
||||
die("fstat");
|
||||
buf = malloc(st.st_size);
|
||||
if (!buf)
|
||||
die("malloc bitstream");
|
||||
n = read(fd, buf, st.st_size);
|
||||
if (n != st.st_size)
|
||||
die("read bitstream short");
|
||||
close(fd);
|
||||
*out_len = (size_t) st.st_size;
|
||||
return buf;
|
||||
}
|
||||
|
||||
int main(int argc, char **argv)
|
||||
{
|
||||
const char *bitstream_path, *out_path;
|
||||
void *bitstream;
|
||||
size_t bs_len;
|
||||
uint32_t w = 128, h = 96;
|
||||
|
||||
struct v4l2_format fmt;
|
||||
struct v4l2_requestbuffers reqbuf;
|
||||
struct v4l2_buffer buf;
|
||||
struct v4l2_plane planes[2];
|
||||
struct v4l2_exportbuffer expbuf;
|
||||
|
||||
int fd, rc, i;
|
||||
void *out_map;
|
||||
void *cap_y, *cap_uv;
|
||||
size_t cap_y_size, cap_uv_size;
|
||||
uint32_t out_buf_offset;
|
||||
enum v4l2_buf_type t;
|
||||
|
||||
if (argc < 3) {
|
||||
fprintf(stderr,
|
||||
"usage: %s <vp9_keyframe.bin> <out.nv12> [w] [h]\n",
|
||||
argv[0]);
|
||||
return 2;
|
||||
}
|
||||
bitstream_path = argv[1];
|
||||
out_path = argv[2];
|
||||
if (argc >= 5) {
|
||||
w = (uint32_t) atoi(argv[3]);
|
||||
h = (uint32_t) atoi(argv[4]);
|
||||
}
|
||||
|
||||
bitstream = read_file(bitstream_path, &bs_len);
|
||||
printf("loaded bitstream: %zu bytes\n", bs_len);
|
||||
|
||||
fd = open(V4L2_DEV, O_RDWR);
|
||||
if (fd < 0)
|
||||
die("open " V4L2_DEV);
|
||||
|
||||
/* --- S_FMT OUTPUT (compressed) --- */
|
||||
memset(&fmt, 0, sizeof(fmt));
|
||||
fmt.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
|
||||
fmt.fmt.pix_mp.width = w;
|
||||
fmt.fmt.pix_mp.height = h;
|
||||
fmt.fmt.pix_mp.pixelformat = V4L2_PIX_FMT_VP9_FRAME;
|
||||
if (ioctl(fd, VIDIOC_S_FMT, &fmt) < 0)
|
||||
die("S_FMT OUTPUT");
|
||||
printf("OUTPUT sizeimage = %u\n", fmt.fmt.pix_mp.plane_fmt[0].sizeimage);
|
||||
|
||||
/* --- S_FMT CAPTURE (NV12M) --- */
|
||||
memset(&fmt, 0, sizeof(fmt));
|
||||
fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
|
||||
fmt.fmt.pix_mp.width = w;
|
||||
fmt.fmt.pix_mp.height = h;
|
||||
fmt.fmt.pix_mp.pixelformat = V4L2_PIX_FMT_NV12M;
|
||||
if (ioctl(fd, VIDIOC_S_FMT, &fmt) < 0)
|
||||
die("S_FMT CAPTURE");
|
||||
printf("CAPTURE planes = %u, [0].sizeimage=%u [1].sizeimage=%u\n",
|
||||
fmt.fmt.pix_mp.num_planes,
|
||||
fmt.fmt.pix_mp.plane_fmt[0].sizeimage,
|
||||
fmt.fmt.pix_mp.plane_fmt[1].sizeimage);
|
||||
cap_y_size = fmt.fmt.pix_mp.plane_fmt[0].sizeimage;
|
||||
cap_uv_size = fmt.fmt.pix_mp.plane_fmt[1].sizeimage;
|
||||
|
||||
/* --- REQBUFS OUTPUT --- */
|
||||
memset(&reqbuf, 0, sizeof(reqbuf));
|
||||
reqbuf.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
|
||||
reqbuf.memory = V4L2_MEMORY_MMAP;
|
||||
reqbuf.count = 1;
|
||||
if (ioctl(fd, VIDIOC_REQBUFS, &reqbuf) < 0)
|
||||
die("REQBUFS OUTPUT");
|
||||
printf("OUTPUT REQBUFS -> %u\n", reqbuf.count);
|
||||
|
||||
/* --- REQBUFS CAPTURE --- */
|
||||
memset(&reqbuf, 0, sizeof(reqbuf));
|
||||
reqbuf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
|
||||
reqbuf.memory = V4L2_MEMORY_MMAP;
|
||||
reqbuf.count = 1;
|
||||
if (ioctl(fd, VIDIOC_REQBUFS, &reqbuf) < 0)
|
||||
die("REQBUFS CAPTURE");
|
||||
printf("CAPTURE REQBUFS -> %u\n", reqbuf.count);
|
||||
|
||||
/* --- QUERYBUF OUTPUT[0] + mmap + fill --- */
|
||||
memset(&buf, 0, sizeof(buf));
|
||||
memset(planes, 0, sizeof(planes));
|
||||
buf.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
|
||||
buf.memory = V4L2_MEMORY_MMAP;
|
||||
buf.index = 0;
|
||||
buf.m.planes = planes;
|
||||
buf.length = 1;
|
||||
if (ioctl(fd, VIDIOC_QUERYBUF, &buf) < 0)
|
||||
die("QUERYBUF OUTPUT");
|
||||
out_buf_offset = planes[0].m.mem_offset;
|
||||
out_map = mmap(NULL, planes[0].length, PROT_READ | PROT_WRITE,
|
||||
MAP_SHARED, fd, out_buf_offset);
|
||||
if (out_map == MAP_FAILED)
|
||||
die("mmap OUTPUT");
|
||||
if (bs_len > planes[0].length) {
|
||||
fprintf(stderr, "bitstream too big: %zu > %u\n",
|
||||
bs_len, planes[0].length);
|
||||
return 1;
|
||||
}
|
||||
memcpy(out_map, bitstream, bs_len);
|
||||
planes[0].bytesused = (uint32_t) bs_len;
|
||||
|
||||
if (ioctl(fd, VIDIOC_QBUF, &buf) < 0)
|
||||
die("QBUF OUTPUT");
|
||||
printf("QBUF OUTPUT[0] bytesused=%zu\n", bs_len);
|
||||
|
||||
/* --- QBUF CAPTURE[0] --- */
|
||||
memset(&buf, 0, sizeof(buf));
|
||||
memset(planes, 0, sizeof(planes));
|
||||
buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
|
||||
buf.memory = V4L2_MEMORY_MMAP;
|
||||
buf.index = 0;
|
||||
buf.m.planes = planes;
|
||||
buf.length = 2;
|
||||
if (ioctl(fd, VIDIOC_QBUF, &buf) < 0)
|
||||
die("QBUF CAPTURE");
|
||||
printf("QBUF CAPTURE[0]\n");
|
||||
|
||||
/* --- STREAMON both --- */
|
||||
t = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
|
||||
if (ioctl(fd, VIDIOC_STREAMON, &t) < 0)
|
||||
die("STREAMON OUTPUT");
|
||||
t = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
|
||||
if (ioctl(fd, VIDIOC_STREAMON, &t) < 0)
|
||||
die("STREAMON CAPTURE");
|
||||
printf("STREAMON both\n");
|
||||
|
||||
/* --- poll for CAPTURE completion --- */
|
||||
{
|
||||
struct pollfd p = { .fd = fd, .events = POLLIN | POLLOUT };
|
||||
rc = poll(&p, 1, POLL_TIMEOUT_MS);
|
||||
if (rc < 0)
|
||||
die("poll");
|
||||
if (rc == 0) {
|
||||
fprintf(stderr, "poll timeout\n");
|
||||
return 1;
|
||||
}
|
||||
printf("poll revents=0x%x\n", p.revents);
|
||||
}
|
||||
|
||||
/* --- DQBUF OUTPUT (return the bitstream buffer) --- */
|
||||
memset(&buf, 0, sizeof(buf));
|
||||
memset(planes, 0, sizeof(planes));
|
||||
buf.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
|
||||
buf.memory = V4L2_MEMORY_MMAP;
|
||||
buf.m.planes = planes;
|
||||
buf.length = 1;
|
||||
if (ioctl(fd, VIDIOC_DQBUF, &buf) < 0)
|
||||
die("DQBUF OUTPUT");
|
||||
printf("DQBUF OUTPUT[%u] flags=0x%x\n", buf.index, buf.flags);
|
||||
|
||||
/* --- DQBUF CAPTURE (get the decoded frame) --- */
|
||||
memset(&buf, 0, sizeof(buf));
|
||||
memset(planes, 0, sizeof(planes));
|
||||
buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
|
||||
buf.memory = V4L2_MEMORY_MMAP;
|
||||
buf.m.planes = planes;
|
||||
buf.length = 2;
|
||||
if (ioctl(fd, VIDIOC_DQBUF, &buf) < 0)
|
||||
die("DQBUF CAPTURE");
|
||||
printf("DQBUF CAPTURE[%u] flags=0x%x payloads=[%u, %u]\n",
|
||||
buf.index, buf.flags,
|
||||
planes[0].bytesused, planes[1].bytesused);
|
||||
if (buf.flags & V4L2_BUF_FLAG_ERROR) {
|
||||
fprintf(stderr, "CAPTURE buffer flagged ERROR\n");
|
||||
return 1;
|
||||
}
|
||||
|
||||
/* --- mmap CAPTURE plane 0 + 1 and dump --- */
|
||||
{
|
||||
struct v4l2_buffer qb;
|
||||
struct v4l2_plane pl[2];
|
||||
|
||||
memset(&qb, 0, sizeof(qb));
|
||||
memset(pl, 0, sizeof(pl));
|
||||
qb.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
|
||||
qb.memory = V4L2_MEMORY_MMAP;
|
||||
qb.index = 0;
|
||||
qb.m.planes = pl;
|
||||
qb.length = 2;
|
||||
if (ioctl(fd, VIDIOC_QUERYBUF, &qb) < 0)
|
||||
die("QUERYBUF CAPTURE");
|
||||
cap_y = mmap(NULL, pl[0].length, PROT_READ, MAP_SHARED, fd,
|
||||
pl[0].m.mem_offset);
|
||||
if (cap_y == MAP_FAILED)
|
||||
die("mmap cap Y");
|
||||
cap_uv = mmap(NULL, pl[1].length, PROT_READ, MAP_SHARED, fd,
|
||||
pl[1].m.mem_offset);
|
||||
if (cap_uv == MAP_FAILED)
|
||||
die("mmap cap UV");
|
||||
}
|
||||
|
||||
{
|
||||
FILE *of = fopen(out_path, "wb");
|
||||
size_t y_actual = planes[0].bytesused
|
||||
? planes[0].bytesused : cap_y_size;
|
||||
size_t uv_actual = planes[1].bytesused
|
||||
? planes[1].bytesused : cap_uv_size;
|
||||
if (!of)
|
||||
die("fopen out");
|
||||
fwrite(cap_y, 1, y_actual, of);
|
||||
fwrite(cap_uv, 1, uv_actual, of);
|
||||
fclose(of);
|
||||
printf("wrote %zu Y + %zu UV bytes to %s\n",
|
||||
y_actual, uv_actual, out_path);
|
||||
}
|
||||
|
||||
/* --- STREAMOFF, cleanup --- */
|
||||
t = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
|
||||
ioctl(fd, VIDIOC_STREAMOFF, &t);
|
||||
t = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
|
||||
ioctl(fd, VIDIOC_STREAMOFF, &t);
|
||||
|
||||
close(fd);
|
||||
free(bitstream);
|
||||
printf("OK\n");
|
||||
(void) expbuf;
|
||||
(void) i;
|
||||
return 0;
|
||||
}
|
||||
Reference in New Issue
Block a user