From: Markus Fritsche Subject: [PATCH 3/4] dom/media/platforms/ffmpeg: route through libavcodec v4l2_request hwaccel for V4L2 stateless boards Date: 2026-04-27 Background ---------- Firefox's existing V4L2 init (`InitV4L2Decoder`) finds the codec by suffix lookup (`FindVideoHardwareAVCodec(mLib, mCodecID)`), which on Linux resolves to the **stateful** V4L2-M2M wrapper codec (`h264_v4l2m2m` etc). Mainline-Linux Rockchip boards (RK3399 rkvdec, RK3566/RK3588 hantro, RK3588 rkvdec2) only expose **stateless** V4L2 fourccs (`S264`, `S265`, `VP9F`); the stateful wrapper codec fails to open, Firefox falls all the way through to software. This patch adds a sibling init path, `InitV4L2RequestDecoder`, that: * looks up the codec via two complementary mechanisms libavcodec uses for v4l2_request: - **named codec** (`h264_v4l2request`, `vp8_v4l2request`, etc.): the legacy AVCodec-per-hwaccel registration. ALARM, Debian, and most distros building with --enable-v4l2-request expose this (avcodec_find_decoder_by_name lookup). - **generic codec + hw_configs walk**: the modern hwaccel registration. Accepts EITHER AV_HWDEVICE_TYPE_DRM (legacy ffmpeg-v4l2-request-fork output prior to FFmpeg 7.1) OR AV_HWDEVICE_TYPE_V4L2REQUEST (FFmpeg 7.1+ dedicated enum, value 13 on Kwiboo's no-AMF tree, 14 on upstream-AMF tree). Probes named-codec first (explicit, portable) and falls back to walking the generic codec's `hw_configs` for either device type; * creates an hwdevice context of the matched type bound to `/dev/dri/renderD128` via the `av_hwdevice_ctx_create` wrapper (patch 2/4) and attaches it to the codec context; * reuses the existing `ChooseV4L2PixelFormat` get-format callback (already returns `AV_PIX_FMT_DRM_PRIME`) and the existing `apply_cropping = 0` constraint. `InitV4L2RequestDecoder` is invoked **before** `InitV4L2Decoder` in `InitHWDecoderIfAllowed`. On Rockchip mainline it succeeds via either mechanism (ALARM uses the named codec). On Pi4 / Mediatek / vendor-MPP-stateful boards neither mechanism is registered for the codec, the function bails out, and the existing stateful `InitV4L2Decoder` runs as before. No regression of stateful boards. `mV4L2RequestDeviceContext` is unconditionally `av_buffer_unref`'d in `ProcessShutdown` (no-op when null). Gated behind `media.ffmpeg.v4l2-request.enabled` from patch 4/4. Bug 1969297. diff --git a/dom/media/platforms/ffmpeg/FFmpegVideoDecoder.h b/dom/media/platforms/ffmpeg/FFmpegVideoDecoder.h --- a/dom/media/platforms/ffmpeg/FFmpegVideoDecoder.h 2026-03-18 19:22:14.000000000 +0000 +++ b/dom/media/platforms/ffmpeg/FFmpegVideoDecoder.h 2026-04-27 20:43:39.347992674 +0000 @@ -225,7 +225,13 @@ bool IsLinuxHDR() const; MediaResult InitVAAPIDecoder(); MediaResult InitV4L2Decoder(); + // firefox-fourier: V4L2 stateless (request API) decode path. Uses + // libavcodec's v4l2_request hwaccel, which it surfaces via either + // AV_HWDEVICE_TYPE_DRM (legacy) or AV_HWDEVICE_TYPE_V4L2REQUEST + // (FFmpeg 7.1+ dedicated enum). + MediaResult InitV4L2RequestDecoder(); bool CreateVAAPIDeviceContext(); + bool CreateV4L2RequestDeviceContext(int aDeviceType); bool GetVAAPISurfaceDescriptor(VADRMPRIMESurfaceDescriptor* aVaDesc); void AddAcceleratedFormats(nsTArray& aCodecList, AVCodecID aCodecID, AVVAAPIHWConfig* hwconfig); @@ -239,7 +245,10 @@ void AdjustHWDecodeLogging(); AVBufferRef* mVAAPIDeviceContext = nullptr; + // firefox-fourier: hwdevice ctx for the v4l2_request hwaccel. + AVBufferRef* mV4L2RequestDeviceContext = nullptr; bool mUsingV4L2 = false; + bool mUsingV4L2Request = false; // If video overlay is used we want to upload SW decoded frames to // DMABuf and present it as a external texture to rendering pipeline. bool mUploadSWDecodeToDMABuf = false; diff --git a/dom/media/platforms/ffmpeg/FFmpegVideoDecoder.cpp b/dom/media/platforms/ffmpeg/FFmpegVideoDecoder.cpp --- a/dom/media/platforms/ffmpeg/FFmpegVideoDecoder.cpp 2026-04-27 16:09:10.000000000 +0200 +++ b/dom/media/platforms/ffmpeg/FFmpegVideoDecoder.cpp 2026-04-29 00:10:00.098884335 +0200 @@ -403,6 +403,148 @@ return NS_OK; } +// firefox-fourier: V4L2 stateless (request API) hwdevice context. +// libavcodec's v4l2_request hwaccel binds via either AV_HWDEVICE_TYPE_DRM +// (legacy) or AV_HWDEVICE_TYPE_V4L2REQUEST (FFmpeg 7.1+, enum value 13 +// on Kwiboo's no-AMF tree, 14 on upstream-AMF tree). Caller passes the +// matched type as an int (Mozilla's bundled libavutil headers may not +// have the V4L2REQUEST enumerator). +bool FFmpegVideoDecoder::CreateV4L2RequestDeviceContext( + int aDeviceType) { + if (!mLib->av_hwdevice_ctx_create) { + FFMPEG_LOG(" av_hwdevice_ctx_create not available (libavutil too old)"); + return false; + } + const char* drmDevice = "/dev/dri/renderD128"; + if (mLib->av_hwdevice_ctx_create(&mV4L2RequestDeviceContext, + (enum AVHWDeviceType)aDeviceType, + drmDevice, nullptr, 0) < 0) { + FFMPEG_LOG(" av_hwdevice_ctx_create(type=%d, %s) failed", + aDeviceType, drmDevice); + return false; + } + mCodecContext->hw_device_ctx = mLib->av_buffer_ref(mV4L2RequestDeviceContext); + FFMPEG_LOG(" v4l2_request hwdevice ctx created (type=%d) on %s", + aDeviceType, drmDevice); + return true; +} + +// firefox-fourier: try V4L2 stateless decode via libavcodec's +// v4l2_request hwaccel. Distinct from InitV4L2Decoder which uses the +// stateful h264_v4l2m2m wrapper codec. On Rockchip mainline boards +// (rkvdec / hantro / rkvdec2) only the stateless path exists. +MediaResult FFmpegVideoDecoder::InitV4L2RequestDecoder() { + FFMPEG_LOG("Initialising V4L2 stateless (request API) FFmpeg decoder"); + + StaticMutexAutoLock mon(sMutex); + + // libavcodec exposes V4L2 stateless decoders through one of two + // mechanisms depending on the build: + // (a) Named AVCodec entry (h264_v4l2request, vp8_v4l2request, + // etc.) — the legacy mechanism. Each hwaccel is a standalone + // AVCodec, looked up by name. Some forks expose this. + // (b) Modern hwaccel registration: the generic codec advertises + // AV_HWDEVICE_TYPE_DRM (legacy) or AV_HWDEVICE_TYPE_V4L2REQUEST + // (FFmpeg 7.1+ dedicated type, enum value 13 or 14 depending + // on whether AMF is in the enum) in its hw_configs array, and + // setting hw_device_ctx on the codec context binds v4l2_request + // internally. + // Probe (a) first — explicit, distro-portable lookup. + // Fall back to (b) when the named entry isn't registered. + const char* requestName = nullptr; + switch (mCodecID) { + case AV_CODEC_ID_H264: requestName = "h264_v4l2request"; break; + case AV_CODEC_ID_HEVC: requestName = "hevc_v4l2request"; break; + case AV_CODEC_ID_VP8: requestName = "vp8_v4l2request"; break; + case AV_CODEC_ID_VP9: requestName = "vp9_v4l2request"; break; + case AV_CODEC_ID_AV1: requestName = "av1_v4l2request"; break; + case AV_CODEC_ID_MPEG2VIDEO: requestName = "mpeg2_v4l2request"; break; + default: + FFMPEG_LOG(" no v4l2_request mapping for codec ID %d", mCodecID); + return NS_ERROR_NOT_AVAILABLE; + } + + // AV_HWDEVICE_TYPE_V4L2REQUEST integer values across known builds. + // Kwiboo's v4l2-request-n7.1.3 tree (no AMF) puts it at 13; + // upstream FFmpeg 8.x with AMF puts it at 14. + const int V4L2REQUEST_KWIBOO = 13; + const int V4L2REQUEST_UPSTREAM = 14; + + AVCodec* codec = nullptr; + int matchedType = AV_HWDEVICE_TYPE_DRM; + + AVCodec* named = mLib->avcodec_find_decoder_by_name(requestName); + if (named) { + codec = named; + FFMPEG_LOG(" using named v4l2_request codec %s", requestName); + } else { + AVCodec* generic = mLib->avcodec_find_decoder(mCodecID); + if (generic) { + for (int i = 0;; i++) { + const AVCodecHWConfig* cfg = mLib->avcodec_get_hw_config(generic, i); + if (!cfg) break; + if (!(cfg->methods & AV_CODEC_HW_CONFIG_METHOD_HW_DEVICE_CTX)) { + continue; + } + if (cfg->device_type == AV_HWDEVICE_TYPE_DRM || + (int)cfg->device_type == V4L2REQUEST_KWIBOO || + (int)cfg->device_type == V4L2REQUEST_UPSTREAM) { + codec = generic; + matchedType = (int)cfg->device_type; + FFMPEG_LOG(" using generic codec %s with v4l2_request hwaccel " + "(device_type=%d)", codec->name, matchedType); + break; + } + } + } + } + + if (!codec) { + FFMPEG_LOG(" no v4l2_request path for codec ID %d — neither named " + "codec %s nor generic codec with DRM/V4L2REQUEST hwaccel " + "available (libavcodec built without --enable-v4l2-request?)", + mCodecID, requestName); + return NS_ERROR_NOT_AVAILABLE; + } + FFMPEG_LOG(" V4L2 stateless: codec %s : %s", codec->name, codec->long_name); + + if (!(mCodecContext = mLib->avcodec_alloc_context3(codec))) { + FFMPEG_LOG(" couldn't init HW ffmpeg context"); + return NS_ERROR_OUT_OF_MEMORY; + } + mCodecContext->opaque = this; + + // Reuse the existing V4L2 init helpers: pixel-format selector returns + // AV_PIX_FMT_DRM_PRIME, cropping disabled (FFmpeg can't crop opaque + // DRM buffers). Same constraints as the stateful V4L2 path. + InitHWCodecContext(ContextType::V4L2); + mCodecContext->apply_cropping = 0; + + auto releaseDecoder = MakeScopeExit( + [&]() MOZ_NO_THREAD_SAFETY_ANALYSIS { ReleaseCodecContext(); }); + + if (!CreateV4L2RequestDeviceContext(matchedType)) { + return NS_ERROR_NOT_AVAILABLE; + } + + MediaResult ret = AllocateExtraData(); + if (NS_FAILED(ret)) { + return ret; + } + + if (mLib->avcodec_open2(mCodecContext, codec, nullptr) < 0) { + FFMPEG_LOG(" Couldn't open V4L2 stateless decoder"); + return NS_ERROR_DOM_MEDIA_FATAL_ERR; + } + + if (mAcceleratedFormats.IsEmpty()) { + mAcceleratedFormats.AppendElement(mCodecID); + } + + AdjustHWDecodeLogging(); + + FFMPEG_LOG(" V4L2 stateless FFmpeg init successful"); + mUsingV4L2 = true; + mUsingV4L2Request = true; + releaseDecoder.release(); + return NS_OK; +} + MediaResult FFmpegVideoDecoder::InitV4L2Decoder() { FFMPEG_LOG("Initialising V4L2-DRM FFmpeg decoder"); @@ -656,6 +798,16 @@ # endif // MOZ_ENABLE_VAAPI # ifdef MOZ_ENABLE_V4L2 + // firefox-fourier: try V4L2 stateless (request API) first. On + // mainline-Linux Rockchip boards (RK3399 rkvdec, RK3566/RK3588 + // hantro, RK3588 rkvdec2) the kernel exposes only stateless + // fourccs, so the stateful path below would fail anyway. On + // stateful boards (Pi4 / vendor MPP) this gracefully falls + // through (no DRM hwaccel registered for the codec). + if (StaticPrefs::media_ffmpeg_v4l2_request_enabled() && + NS_SUCCEEDED(InitV4L2RequestDecoder())) { + return; + } // VAAPI didn't work or is disabled, so try V4L2 with DRM if (NS_SUCCEEDED(InitV4L2Decoder())) { return; @@ -2046,6 +2198,11 @@ if (IsHardwareAccelerated()) { mLib->av_buffer_unref(&mVAAPIDeviceContext); } + // firefox-fourier: release the hwdevice ctx for the v4l2_request + // hwaccel. Always safe — av_buffer_unref no-ops on a null pointer. + if (mV4L2RequestDeviceContext) { + mLib->av_buffer_unref(&mV4L2RequestDeviceContext); + } #endif #ifdef MOZ_ENABLE_D3D11VA if (IsHardwareAccelerated()) {