libva-multiplanar/firefox-fourier

firefox-fourier — Firefox patch for V4L2 stateless RDD sandbox

Lets Firefox 150 hardware-decode H.264 video on Linux V4L2-stateless decoders (Rockchip hantro G1/G2, Allwinner cedrus, Sun8i, etc.) without disabling Firefox's RDD sandbox.

The problem

Stock Firefox 150 (and earlier) blocks V4L2 stateless decoders in three places — the broker (1 location, two gates) and the seccomp filter (two process classes):

1. Broker policy (security/sandbox/linux/broker/SandboxBrokerPolicyFactory.cpp): AddV4l2Dependencies() filters /dev/video* by VIDEO_M2M | VIDEO_M2M_MPLANE capability. Stateless decoders advertise CAPTURE_MPLANE + OUTPUT_MPLANE + STREAMING but typically not M2M, so /dev/video1 is silently dropped. And GetRDDPolicy() never references /dev/media* at all — the V4L2 request API (MEDIA_REQUEST_IOC_QUEUE et al), required for stateless decode, lives on /dev/media* nodes the broker won't open from RDD.

2. RDD seccomp (security/sandbox/linux/SandboxFilter.cpp, RDDSandboxPolicy::EvaluateSyscall): the ioctl handler allowlists ioctl magic byte 'V' (V4L2) but not '|' (<linux/media.h>). Even after the broker permits the open, the kernel ioctl path is filtered, returning ENOSYS (silent — Mozilla's seccomp uses SECCOMP_RET_ERRNO, not SIGSYS).

3. Utility seccomp (security/sandbox/linux/SandboxFilter.cpp, UtilitySandboxPolicy::EvaluateSyscall): since Firefox ~114, much of the VAAPI decode work runs in the Utility process, not RDD. UtilitySandboxPolicy does not override __NR_ioctl and falls through to SandboxPolicyCommon, which blocks DRM ('d'), DMA-Buf ('b'), V4L2 ('V'), and media-controller ('|') magic bytes — all four are needed for stateless decode end-to-end. Empirically: MEDIA_IOC_REQUEST_ALLOC (_IOR('|', 0x05, int)) returns ENOSYS even with RDD fully patched.

Existing M2M-stateful decoders work (per Mozilla bugs 1833354 / 1965646); stateless never did.

Two ways to use HW decode

Easy path — env var (sandbox disabled)

Stock Firefox 150 + this env var:

LIBVA_DRIVER_NAME=v4l2_request \
LIBVA_V4L2_REQUEST_VIDEO_PATH=/dev/video1 \
LIBVA_V4L2_REQUEST_MEDIA_PATH=/dev/media0 \
MOZ_DISABLE_RDD_SANDBOX=1 \
firefox <video-url>

MOZ_DISABLE_RDD_SANDBOX=1 disables the RDD process sandbox entirely. HW decode works because the broker policy isn't enforced; you also lose the RDD process's defense-in-depth boundary.

OK for personal-machine use; not OK for production / hostile environments.

Proper path — patched Firefox (sandbox kept on)

Apply 0001-rdd-allow-stateless-v4l2-request-api.patch to firefox-150.0.1 source.

The patch (six hunks across two files):

1. Widens AddV4l2Dependencies cap filter to admit nodes with (CAPTURE_MPLANE & OUTPUT_MPLANE & STREAMING) for stateless decoders.
2. Adds a new AddV4l2RequestApiDependencies() that enumerates /dev/media* and adds each rdwr to the RDD broker policy.
3. Adds ioctl magic byte '|' (linux/media.h) to RDDSandboxPolicy's seccomp allowlist alongside the existing 'V'.
4. Adds an explicit case __NR_ioctl: to UtilitySandboxPolicy::EvaluateSyscall mirroring RDD's allowlist ('d' DRM, 'b' DMA-Buf, 'V' V4L2, '|' linux/media.h) — required because FF150 routes VAAPI work to the Utility process and the common policy blocks all four magic bytes.

Tested on hantro G1 (Rockchip RK3568 / PineTab2) running bbb_1080p30 H.264 with full sandbox enabled. ENETDOWN gone, libva initializes in the Utility process, MEDIA_IOC_REQUEST_ALLOC succeeds, decode reaches end-of-stream.

Build instructions (Arch / ALARM)

The repo ships an Arch-style PKGBUILD overlay against upstream Arch's firefox PKGBUILD. See PKGBUILD-overlay.md for the full reproducible recipe — verified working on aarch64 (firefox 150.0.1-1.1, single-pass non-PGO build, ~169 MB libxul).

TL;DR:

# In an Arch-ish container or host with build deps:
~/firefox-fourier/bootstrap.sh
makepkg --syncdeps --skippgpcheck --noconfirm
sudo pacman -U firefox-150.0.1-1.1-aarch64.pkg.tar.xz

The bootstrap.sh script:

• Fetches upstream Arch firefox PKGBUILD + companion source files
• Drops in the 0001-...patch as 0005-rdd-allow-stateless-v4l2-request-api.patch
• Applies five PKGBUILD overlay edits (pkgrel bump to 1.1, add aarch64 to arch=, wire patch into source=() + prepare(), strip onnxruntime which isn't in ALARM)
• Runs updpkgsums

Required deps NOT in upstream ALARM extra (install before running bootstrap, see PKGBUILD-overlay.md):

• wasi-libc, wasi-compiler-rt, wasi-libc++, wasi-libc++abi — pull from upstream Arch x86_64 directly (arch=any, identical artifact) since ALARM's are 4 years stale.

Build instructions (other distros)

Generic shape:

# 1. Get firefox-150.0.1 source from https://archive.mozilla.org/pub/firefox/releases/150.0.1/source/
# 2. Apply the patch:
cd firefox-150.0.1
patch -Np1 < /path/to/0001-rdd-allow-stateless-v4l2-request-api.patch
# 3. Build per Mozilla's standard instructions, e.g.:
./mach build && ./mach package

The patch applies cleanly to firefox-150.0.1 source. It uses unified-diff with line-anchored hunks; minor whitespace drift in newer Firefox versions may need re-anchoring (see firefox-fourier/PKGBUILD-overlay.md for how the campaign repo handles this on top of upstream Arch).

What the libva-side needs

The Firefox patch alone isn't sufficient. You also need a working V4L2-stateless libva backend. The companion repo marfrit/libva-v4l2-request-fourier carries the libva backend with iter1+iter2+iter3+iter4+iter5 fixes (DPB FFmpeg semantics matching, fresh request_fd per frame, B-slice L1 reflist, multi-context safety, debug instrumentation removed).

# Clone, build, install
git clone https://git.reauktion.de:2222/marfrit/libva-v4l2-request-fourier.git
cd libva-v4l2-request-fourier
meson setup build --buildtype=release
ninja -C build
sudo cp build/src/v4l2_request_drv_video.so /usr/lib/dri/

Then either patched Firefox OR MOZ_DISABLE_RDD_SANDBOX=1 will work for HW decode.

Upstream status

Not upstreamed at this writing (campaign discipline: no PR/MR without explicit operator instruction). The patch is Mozilla-bug-and-PR-ready in shape and ~50 lines across two files. Whoever picks it up to file with Mozilla should reference the existing Bug 1833354 and Bug 1965646 (V4L2-M2M precedent) as related work; this is the V4L2-stateless analogue.

File map

File	What it is
0001-rdd-allow-stateless-v4l2-request-api.patch	The patch. Apply with patch -Np1.
bootstrap.sh	Reproducible PKGBUILD overlay script for Arch/ALARM container builds.
PKGBUILD-overlay.md	Detailed step-by-step build recipe (Arch convention).
README.md	This file.