From: Markus Fritsche Subject: [PATCH 3/4] dom/media/platforms/ffmpeg: route through libavcodec v4l2_request hwaccel for V4L2 stateless boards Date: 2026-04-27 Background ---------- Firefox's existing V4L2 init (`InitV4L2Decoder`) finds the codec by suffix lookup (`FindVideoHardwareAVCodec(mLib, mCodecID)`), which on Linux resolves to the **stateful** V4L2-M2M wrapper codec (`h264_v4l2m2m` etc). Mainline-Linux Rockchip boards (RK3399 rkvdec, RK3566/RK3588 hantro, RK3588 rkvdec2) only expose **stateless** V4L2 fourccs (`S264`, `S265`, `VP9F`); the stateful wrapper codec fails to open, Firefox falls all the way through to software. This patch adds a sibling init path, `InitV4L2RequestDecoder`, that: * looks up the codec via two complementary mechanisms libavcodec uses for v4l2_request: - **named codec** (`h264_v4l2request`, `vp8_v4l2request`, etc.): the legacy AVCodec-per-hwaccel registration. - **generic codec + hw_configs walk**: the modern hwaccel registration. Accepts EITHER AV_HWDEVICE_TYPE_DRM (legacy ffmpeg-v4l2-request-fork output prior to FFmpeg 7.1) OR AV_HWDEVICE_TYPE_V4L2REQUEST (FFmpeg 7.1+ dedicated enum, value 13 on Kwiboo's no-AMF tree, 14 on upstream-AMF tree). Mozilla's bundled libavutil headers may not have the V4L2REQUEST enumerator, so the test is on the integer value via `(int)cast`. Probes named-codec first (explicit, portable) and falls back to walking the generic codec's `hw_configs` for either device type; * creates an hwdevice context bound to `/dev/dri/renderD128`. Uses integer 13 (V4L2REQUEST as defined by Kwiboo's v4l2-request-n7.1.3 tree, what our libavcodec61-fourier emits) cast to enum AVHWDeviceType for the av_hwdevice_ctx_create call; * reuses the existing `ChooseV4L2PixelFormat` get-format callback (already returns `AV_PIX_FMT_DRM_PRIME`) and the existing `apply_cropping = 0` constraint. `InitV4L2RequestDecoder` is invoked **before** `InitV4L2Decoder` in `InitHWDecoderIfAllowed`. On Rockchip mainline it succeeds via either mechanism. On Pi4 / Mediatek / vendor-MPP-stateful boards neither mechanism is registered for the codec, the function bails out, and the existing stateful `InitV4L2Decoder` runs as before. No regression of stateful boards. `mDRMDeviceContext` is unconditionally `av_buffer_unref`'d in `ProcessShutdown` (no-op when null). Gated behind `media.ffmpeg.v4l2-request.enabled` from patch 4/4. Bug 1969297. --- a/dom/media/platforms/ffmpeg/FFmpegVideoDecoder.h 2026-05-21 04:57:59.570946601 +0000 +++ b/dom/media/platforms/ffmpeg/FFmpegVideoDecoder.h 2026-05-21 04:57:59.876488776 +0000 @@ -225,7 +225,12 @@ bool IsLinuxHDR() const; MediaResult InitVAAPIDecoder(); MediaResult InitV4L2Decoder(); + // firefox-fourier: V4L2 stateless (request API) decode path. Uses + // libavcodec's v4l2_request hwaccel, which it surfaces via + // AV_HWDEVICE_TYPE_DRM rather than a dedicated _V4L2REQUEST type. + MediaResult InitV4L2RequestDecoder(); bool CreateVAAPIDeviceContext(); + bool CreateV4L2RequestDeviceContext(); bool GetVAAPISurfaceDescriptor(VADRMPRIMESurfaceDescriptor* aVaDesc); void AddAcceleratedFormats(nsTArray& aCodecList, AVCodecID aCodecID, AVVAAPIHWConfig* hwconfig); @@ -239,7 +244,10 @@ void AdjustHWDecodeLogging(); AVBufferRef* mVAAPIDeviceContext = nullptr; + // firefox-fourier: DRM hwdevice ctx for the v4l2_request hwaccel. + AVBufferRef* mDRMDeviceContext = nullptr; bool mUsingV4L2 = false; + bool mUsingV4L2Request = false; // If video overlay is used we want to upload SW decoded frames to // DMABuf and present it as a external texture to rendering pipeline. bool mUploadSWDecodeToDMABuf = false; --- a/dom/media/platforms/ffmpeg/FFmpegVideoDecoder.cpp 2026-05-21 04:57:59.566685221 +0000 +++ b/dom/media/platforms/ffmpeg/FFmpegVideoDecoder.cpp 2026-05-21 04:58:00.136004159 +0000 @@ -403,6 +403,129 @@ return NS_OK; } +// firefox-fourier: V4L2 stateless (request API) DRM hwdevice context. +// libavcodec's v4l2_request hwaccel binds via AV_HWDEVICE_TYPE_DRM — +// no dedicated _V4L2REQUEST type exists upstream. +bool FFmpegVideoDecoder::CreateV4L2RequestDeviceContext() { + if (!mLib->av_hwdevice_ctx_create) { + FFMPEG_LOG(" av_hwdevice_ctx_create not available (libavutil too old)"); + return false; + } + const char* drmDevice = "/dev/dri/renderD128"; + if (mLib->av_hwdevice_ctx_create(&mDRMDeviceContext, + (enum AVHWDeviceType)13, drmDevice, + nullptr, 0) < 0) { + FFMPEG_LOG(" av_hwdevice_ctx_create(DRM, %s) failed", drmDevice); + return false; + } + mCodecContext->hw_device_ctx = mLib->av_buffer_ref(mDRMDeviceContext); + FFMPEG_LOG(" DRM hwdevice ctx created on %s", drmDevice); + return true; +} + +// firefox-fourier: try V4L2 stateless decode via libavcodec's +// v4l2_request hwaccel. Distinct from InitV4L2Decoder which uses the +// stateful h264_v4l2m2m wrapper codec. On Rockchip mainline boards +// (rkvdec / hantro / rkvdec2) only the stateless path exists. +MediaResult FFmpegVideoDecoder::InitV4L2RequestDecoder() { + FFMPEG_LOG("Initialising V4L2 stateless (request API) FFmpeg decoder"); + + StaticMutexAutoLock mon(sMutex); + + // libavcodec exposes V4L2 stateless decoders through one of two + // mechanisms depending on the build: + // (a) Named AVCodec entry (h264_v4l2request, vp8_v4l2request, + // etc.) — the legacy mechanism. Each hwaccel is a standalone + // AVCodec, looked up by name. ALARM, Debian, and most distros + // building with --enable-v4l2-request expose this. + // (b) Modern hwaccel registration: the generic codec advertises + // AV_HWDEVICE_TYPE_DRM in its hw_configs array, and setting + // hw_device_ctx on the codec context binds v4l2_request + // internally. Some upstream-only builds expose this. + // Probe (a) first — it is the explicit, distro-portable lookup. + // Fall back to (b) when the named entry isn't registered. + const char* requestName = nullptr; + switch (mCodecID) { + case AV_CODEC_ID_H264: requestName = "h264_v4l2request"; break; + case AV_CODEC_ID_HEVC: requestName = "hevc_v4l2request"; break; + case AV_CODEC_ID_VP8: requestName = "vp8_v4l2request"; break; + case AV_CODEC_ID_VP9: requestName = "vp9_v4l2request"; break; + case AV_CODEC_ID_AV1: requestName = "av1_v4l2request"; break; + case AV_CODEC_ID_MPEG2VIDEO: requestName = "mpeg2_v4l2request"; break; + default: + FFMPEG_LOG(" no v4l2_request mapping for codec ID %d", mCodecID); + return NS_ERROR_NOT_AVAILABLE; + } + + AVCodec* codec = mLib->avcodec_find_decoder_by_name(requestName); + if (codec) { + FFMPEG_LOG(" using named v4l2_request codec %s", requestName); + } else { + AVCodec* generic = mLib->avcodec_find_decoder(mCodecID); + if (generic) { + for (int i = 0;; i++) { + const AVCodecHWConfig* cfg = mLib->avcodec_get_hw_config(generic, i); + if (!cfg) break; + if (cfg->device_type == AV_HWDEVICE_TYPE_DRM || (int)cfg->device_type == 13 || (int)cfg->device_type == 14) { + codec = generic; + FFMPEG_LOG(" using generic codec %s with DRM hwaccel", codec->name); + break; + } + } + } + } + + if (!codec) { + FFMPEG_LOG(" no v4l2_request path for codec ID %d \u2014 neither named " + "codec %s nor generic codec with DRM hwaccel available " + "(libavcodec built without --enable-v4l2-request?)", + mCodecID, requestName); + return NS_ERROR_NOT_AVAILABLE; + } + FFMPEG_LOG(" V4L2 stateless: codec %s : %s", codec->name, codec->long_name); + + if (!(mCodecContext = mLib->avcodec_alloc_context3(codec))) { + FFMPEG_LOG(" couldn't init HW ffmpeg context"); + return NS_ERROR_OUT_OF_MEMORY; + } + mCodecContext->opaque = this; + + // Reuse the existing V4L2 init helpers: pixel-format selector returns + // AV_PIX_FMT_DRM_PRIME, cropping disabled (FFmpeg can't crop opaque + // DRM buffers). Same constraints as the stateful V4L2 path. + InitHWCodecContext(ContextType::V4L2); + mCodecContext->apply_cropping = 0; + + auto releaseDecoder = MakeScopeExit( + [&]() MOZ_NO_THREAD_SAFETY_ANALYSIS { ReleaseCodecContext(); }); + + if (!CreateV4L2RequestDeviceContext()) { + return NS_ERROR_NOT_AVAILABLE; + } + + MediaResult ret = AllocateExtraData(); + if (NS_FAILED(ret)) { + return ret; + } + + if (mLib->avcodec_open2(mCodecContext, codec, nullptr) < 0) { + FFMPEG_LOG(" Couldn't open V4L2 stateless decoder"); + return NS_ERROR_DOM_MEDIA_FATAL_ERR; + } + + if (mAcceleratedFormats.IsEmpty()) { + mAcceleratedFormats.AppendElement(mCodecID); + } + + AdjustHWDecodeLogging(); + + FFMPEG_LOG(" V4L2 stateless FFmpeg init successful"); + mUsingV4L2 = true; + mUsingV4L2Request = true; + releaseDecoder.release(); + return NS_OK; +} + MediaResult FFmpegVideoDecoder::InitV4L2Decoder() { FFMPEG_LOG("Initialising V4L2-DRM FFmpeg decoder"); @@ -656,6 +779,16 @@ # endif // MOZ_ENABLE_VAAPI # ifdef MOZ_ENABLE_V4L2 + // firefox-fourier: try V4L2 stateless (request API) first. On + // mainline-Linux Rockchip boards (RK3399 rkvdec, RK3566/RK3588 + // hantro, RK3588 rkvdec2) the kernel exposes only stateless + // fourccs, so the stateful path below would fail anyway. On + // stateful boards (Pi4 / vendor MPP) this gracefully falls + // through (no DRM hwaccel registered for the codec). + if (StaticPrefs::media_ffmpeg_v4l2_request_enabled() && + NS_SUCCEEDED(InitV4L2RequestDecoder())) { + return; + } // VAAPI didn't work or is disabled, so try V4L2 with DRM if (NS_SUCCEEDED(InitV4L2Decoder())) { return; @@ -2046,6 +2179,11 @@ if (IsHardwareAccelerated()) { mLib->av_buffer_unref(&mVAAPIDeviceContext); } + // firefox-fourier: release the DRM hwdevice ctx for the v4l2_request + // hwaccel. Always safe — av_buffer_unref no-ops on a null pointer. + if (mDRMDeviceContext) { + mLib->av_buffer_unref(&mDRMDeviceContext); + } #endif #ifdef MOZ_ENABLE_D3D11VA if (IsHardwareAccelerated()) {