libva-multiplanar/firefox-fourier/README.md

firefox-fourier — Firefox patch for V4L2 stateless RDD sandbox

Lets Firefox 150 hardware-decode H.264 video on Linux V4L2-stateless decoders (Rockchip hantro G1/G2, Allwinner cedrus, Sun8i, etc.) without disabling Firefox's RDD sandbox.

The problem

Stock Firefox 150 (and earlier) blocks V4L2 stateless decoders in three places — the broker (1 location, two gates) and the seccomp filter (two process classes):

1. Broker policy (security/sandbox/linux/broker/SandboxBrokerPolicyFactory.cpp): AddV4l2Dependencies() filters /dev/video* by VIDEO_M2M | VIDEO_M2M_MPLANE capability. Stateless decoders advertise CAPTURE_MPLANE + OUTPUT_MPLANE + STREAMING but typically not M2M, so /dev/video1 is silently dropped. And GetRDDPolicy() never references /dev/media* at all — the V4L2 request API (MEDIA_REQUEST_IOC_QUEUE et al), required for stateless decode, lives on /dev/media* nodes the broker won't open from RDD.

2. RDD seccomp (security/sandbox/linux/SandboxFilter.cpp, RDDSandboxPolicy::EvaluateSyscall): the ioctl handler allowlists ioctl magic byte 'V' (V4L2) but not '|' (<linux/media.h>). Even after the broker permits the open, the kernel ioctl path is filtered, returning ENOSYS (silent — Mozilla's seccomp uses SECCOMP_RET_ERRNO, not SIGSYS).

3. Utility seccomp (security/sandbox/linux/SandboxFilter.cpp, UtilitySandboxPolicy::EvaluateSyscall): since Firefox ~114, much of the VAAPI decode work runs in the Utility process, not RDD. UtilitySandboxPolicy does not override __NR_ioctl and falls through to SandboxPolicyCommon, which blocks DRM ('d'), DMA-Buf ('b'), V4L2 ('V'), and media-controller ('|') magic bytes — all four are needed for stateless decode end-to-end. Empirically: MEDIA_IOC_REQUEST_ALLOC (_IOR('|', 0x05, int)) returns ENOSYS even with RDD fully patched.

Existing M2M-stateful decoders work (per Mozilla bugs 1833354 / 1965646); stateless never did.

Two ways to use HW decode

Easy path — env var (sandbox disabled)

Stock Firefox 150 + this env var:

LIBVA_DRIVER_NAME=v4l2_request \
LIBVA_V4L2_REQUEST_VIDEO_PATH=/dev/video1 \
LIBVA_V4L2_REQUEST_MEDIA_PATH=/dev/media0 \
MOZ_DISABLE_RDD_SANDBOX=1 \
firefox <video-url>

MOZ_DISABLE_RDD_SANDBOX=1 disables the RDD process sandbox entirely. HW decode works because the broker policy isn't enforced; you also lose the RDD process's defense-in-depth boundary.

OK for personal-machine use; not OK for production / hostile environments.

Proper path — patched Firefox (sandbox kept on)

Apply 0001-rdd-allow-stateless-v4l2-request-api.patch to firefox-150.0.1 source.

The patch (six hunks across two files):

1. Widens AddV4l2Dependencies cap filter to admit nodes with (CAPTURE_MPLANE & OUTPUT_MPLANE & STREAMING) for stateless decoders.
2. Adds a new AddV4l2RequestApiDependencies() that enumerates /dev/media* and adds each rdwr to the RDD broker policy.
3. Adds ioctl magic byte '|' (linux/media.h) to RDDSandboxPolicy's seccomp allowlist alongside the existing 'V'.
4. Adds an explicit case __NR_ioctl: to UtilitySandboxPolicy::EvaluateSyscall mirroring RDD's allowlist ('d' DRM, 'b' DMA-Buf, 'V' V4L2, '|' linux/media.h) — required because FF150 routes VAAPI work to the Utility process and the common policy blocks all four magic bytes.

Tested on hantro G1 (PineTab2 — Rockchip RK3566 silicon, hantro driver via the rockchip,rk3568-vpu DT compatible) running bbb_1080p30 H.264 with full sandbox enabled. ENETDOWN gone, libva initializes in the Utility process, MEDIA_IOC_REQUEST_ALLOC succeeds, decode reaches end-of-stream.

Build instructions (Arch / ALARM)

The repo ships an Arch-style PKGBUILD overlay against upstream Arch's firefox PKGBUILD. See PKGBUILD-overlay.md for the full reproducible recipe — verified working on aarch64 (firefox 150.0.1-1.1, single-pass non-PGO build, ~169 MB libxul).

TL;DR:

# In an Arch-ish container or host with build deps:
~/firefox-fourier/bootstrap.sh
makepkg --syncdeps --skippgpcheck --noconfirm
sudo pacman -U firefox-150.0.1-1.1-aarch64.pkg.tar.xz

The bootstrap.sh script:

• Fetches upstream Arch firefox PKGBUILD + companion source files
• Drops in the 0001-...patch as 0005-rdd-allow-stateless-v4l2-request-api.patch
• Applies five PKGBUILD overlay edits (pkgrel bump to 1.1, add aarch64 to arch=, wire patch into source=() + prepare(), strip onnxruntime which isn't in ALARM)
• Runs updpkgsums

Required deps NOT in upstream ALARM extra (install before running bootstrap, see PKGBUILD-overlay.md):

• wasi-libc, wasi-compiler-rt, wasi-libc++, wasi-libc++abi — pull from upstream Arch x86_64 directly (arch=any, identical artifact) since ALARM's are 4 years stale.

Build instructions (other distros)

Generic shape:

# 1. Get firefox-150.0.1 source from https://archive.mozilla.org/pub/firefox/releases/150.0.1/source/
# 2. Apply the patch:
cd firefox-150.0.1
patch -Np1 < /path/to/0001-rdd-allow-stateless-v4l2-request-api.patch
# 3. Build per Mozilla's standard instructions, e.g.:
./mach build && ./mach package

The patch applies cleanly to firefox-150.0.1 source. It uses unified-diff with line-anchored hunks; minor whitespace drift in newer Firefox versions may need re-anchoring (see firefox-fourier/PKGBUILD-overlay.md for how the campaign repo handles this on top of upstream Arch).

What the libva-side needs

The Firefox patch alone isn't sufficient. You also need a working V4L2-stateless libva backend. The companion repo marfrit/libva-v4l2-request-fourier carries the libva backend with iter1+iter2+iter3+iter4+iter5 fixes (DPB FFmpeg semantics matching, fresh request_fd per frame, B-slice L1 reflist, multi-context safety, debug instrumentation removed).

# Clone, build, install
git clone https://git.reauktion.de:2222/marfrit/libva-v4l2-request-fourier.git
cd libva-v4l2-request-fourier
meson setup build --buildtype=release
ninja -C build
sudo cp build/src/v4l2_request_drv_video.so /usr/lib/dri/

Then either patched Firefox OR MOZ_DISABLE_RDD_SANDBOX=1 will work for HW decode.

YouTube codec note

YouTube negotiates the highest codec the browser advertises support for. Without forcing avc1, FF150 picks AV1 from YouTube on most modern hardware and SW-decodes — the v4l2_request driver only handles H.264, so libva isn't engaged. To exercise HW decode on YouTube, install Enhancer for YouTube or enhanced-h264ify and configure it to force h264 codec.

Upstream status

Not upstreamed at this writing (campaign discipline: no PR/MR without explicit operator instruction). The patch is Mozilla-bug-and-PR-ready in shape and ~50 lines across two files. Whoever picks it up to file with Mozilla should reference the existing Bug 1833354 and Bug 1965646 (V4L2-M2M precedent) as related work; this is the V4L2-stateless analogue.

File map

File	What it is
0001-rdd-allow-stateless-v4l2-request-api.patch	The patch. Apply with patch -Np1.
bootstrap.sh	Reproducible PKGBUILD overlay script for Arch/ALARM container builds.
PKGBUILD-overlay.md	Detailed step-by-step build recipe (Arch convention).
README.md	This file.