libva-multiplanar/firefox-fourier/0001-rdd-allow-stateless-v4l2-request-api.patch

From: claude-noether <claude@reauktion.de>
Date: 2026-05-05
Subject: [PATCH] sandbox/linux: allow V4L2 stateless request-API decoders in RDD + Utility

Firefox's RDD AND Utility process sandboxes block hardware video decode
for V4L2 stateless decoders (hantro G1/G2 on RK35xx, cedrus on Allwinner,
etc.). Since Firefox ~114 the VAAPI decode work has migrated from RDD to
the Utility process for many pipelines; both classes need the same
ioctl allowlist for V4L2-stateless to work.

Three distinct gates close the door:

  1. Broker policy: AddV4l2Dependencies() filters /dev/video* by
     VIDEO_M2M / VIDEO_M2M_MPLANE capability. Stateless decoders
     advertise CAPTURE_MPLANE + OUTPUT_MPLANE + STREAMING but typically
     not M2M, so /dev/video1 (the hantro device) is silently dropped.

  2. Broker policy: GetRDDPolicy() never references /dev/media*. The
     V4L2 request API lives on /dev/media* nodes that the broker won't
     open from RDD.

  3. Seccomp policy: RDDSandboxPolicy and UtilitySandboxPolicy do not
     allow ioctl magic byte '|' (linux/media.h). Even after broker
     permits the open, the kernel ioctl path is filtered, returning
     ENOSYS to userspace and causing libva to abandon decode.
     Empirically: "Sandbox: seccomp sandbox violation: ... syscall 29"
     where syscall 29 is ioctl on aarch64 and the filtered request is
     MEDIA_IOC_REQUEST_ALLOC (_IOR('|', 0x05, int)).

iter5 amendment: extends the original iter3 RDD seccomp fix to also
cover UtilitySandboxPolicy (which previously fell through to
SandboxPolicyCommon's restrictive ioctl handler). Required because
PID-by-PID inspection on Firefox 150 shows VAAPI work happening in
the Utility process, not RDD.

Tested: libva-v4l2-request-fourier on PineTab2 (RK3568, hantro G1)
playing bbb_1080p30 H.264 via Firefox 150 without
MOZ_DISABLE_RDD_SANDBOX=1.

---
--- a/security/sandbox/linux/broker/SandboxBrokerPolicyFactory.cpp
+++ b/security/sandbox/linux/broker/SandboxBrokerPolicyFactory.cpp
@@ -901,8 +901,16 @@
     }

     if ((cap.device_caps & V4L2_CAP_VIDEO_M2M) ||
-        (cap.device_caps & V4L2_CAP_VIDEO_M2M_MPLANE)) {
-      // This is an M2M device (i.e. not a webcam), so allow access
+        (cap.device_caps & V4L2_CAP_VIDEO_M2M_MPLANE) ||
+        // V4L2 stateless decoders (hantro G1/G2 on Rockchip, cedrus on
+        // Allwinner, etc.) report CAPTURE_MPLANE + OUTPUT_MPLANE +
+        // STREAMING but do not set the M2M caps. They use the request API
+        // via /dev/media* (see AddV4l2RequestApiDependencies below).
+        ((cap.device_caps & V4L2_CAP_VIDEO_CAPTURE_MPLANE) &&
+         (cap.device_caps & V4L2_CAP_VIDEO_OUTPUT_MPLANE) &&
+         (cap.device_caps & V4L2_CAP_STREAMING))) {
+      // This is an M2M or stateless decode device (i.e. not a webcam),
+      // so allow access
       policy->AddPath(rdwr, path.get());
     }

@@ -913,6 +921,32 @@
   // FFmpeg V4L2 needs to list /dev to find V4L2 devices.
   policy->AddPath(rdonly, "/dev");
 }
+
+// V4L2 stateless decoders submit per-frame decode requests via the
+// media-controller framework on /dev/media* nodes (ioctls in the
+// MEDIA_REQUEST_IOC_* family, magic byte '|', defined in <linux/media.h>).
+// These are required alongside /dev/video* for any request-API decoder.
+// We allow rdwr access to all /dev/media* nodes; the kernel's
+// media-controller layer enforces device-level access control.
+// This mirrors the model AddV4l2Dependencies uses for /dev/video*.
+static void AddV4l2RequestApiDependencies(SandboxBroker::Policy* policy) {
+  DIR* dir = opendir("/dev");
+  if (!dir) {
+    SANDBOX_LOG("Couldn't list /dev for media-controller nodes");
+    return;
+  }
+
+  struct dirent* dir_entry;
+  while ((dir_entry = readdir(dir))) {
+    if (strncmp(dir_entry->d_name, "media", 5)) {
+      continue;
+    }
+    nsCString path = "/dev/"_ns;
+    path += nsDependentCString(dir_entry->d_name);
+    policy->AddPath(rdwr, path.get());
+  }
+  closedir(dir);
+}
 #endif  // MOZ_ENABLE_V4L2

 /* static */ UniquePtr<SandboxBroker::Policy>
@@ -979,6 +1013,7 @@

 #ifdef MOZ_ENABLE_V4L2
   AddV4l2Dependencies(policy.get());
+  AddV4l2RequestApiDependencies(policy.get());
 #endif  // MOZ_ENABLE_V4L2

   // Bug 1903688: NVIDIA Tegra hardware decoding from Linux4Tegra
--- a/security/sandbox/linux/SandboxFilter.cpp
+++ b/security/sandbox/linux/SandboxFilter.cpp
@@ -2067,6 +2067,11 @@
         // Type 'V' for V4L2, used for hw accelerated decode
         static constexpr unsigned long kVideoType =
             static_cast<unsigned long>('V') << _IOC_TYPESHIFT;
+        // Type '|' for the V4L2 request API on /dev/media* nodes
+        // (MEDIA_REQUEST_IOC_QUEUE et al, defined in <linux/media.h>).
+        // Required by V4L2 stateless decoders such as hantro/cedrus/sun*.
+        static constexpr unsigned long kMediaType =
+            static_cast<unsigned long>('|') << _IOC_TYPESHIFT;
 #endif
         // nvidia non-tegra uses some ioctls from this range (but not actual
         // fbdev ioctls; nvidia uses values >= 200 for the NR field
@@ -2088,6 +2093,7 @@
             .ElseIf(shifted_type == kDmaBufType, Allow())
 #ifdef MOZ_ENABLE_V4L2
             .ElseIf(shifted_type == kVideoType, Allow())
+            .ElseIf(shifted_type == kMediaType, Allow())
 #endif
         // NVIDIA decoder from Linux4Tegra, this is specific to Tegra ARM64 SoC
 #if defined(__aarch64__)
@@ -2378,6 +2384,35 @@
       case __NR_set_mempolicy:
         return Error(ENOSYS);

+      // VAAPI hardware video decode in the Utility process. As of Firefox
+      // ~114 the VAAPI decode work moved from RDD to Utility for many
+      // pipelines; this filter mirrors the RDD ioctl allowlist so V4L2
+      // stateless decoders (hantro G1/G2 on Rockchip, cedrus on
+      // Allwinner, etc.) can issue MEDIA_REQUEST_IOC_* ioctls (magic
+      // byte '|', linux/media.h) and VIDIOC_* ioctls (magic byte 'V'),
+      // alongside the DRM/DMA-Buf surface-management calls.
+      case __NR_ioctl: {
+        Arg<unsigned long> request(1);
+        auto shifted_type = request & kIoctlTypeMask;
+        static constexpr unsigned long kDrmType =
+            static_cast<unsigned long>('d') << _IOC_TYPESHIFT;
+        static constexpr unsigned long kDmaBufType =
+            static_cast<unsigned long>('b') << _IOC_TYPESHIFT;
+#ifdef MOZ_ENABLE_V4L2
+        static constexpr unsigned long kVideoType =
+            static_cast<unsigned long>('V') << _IOC_TYPESHIFT;
+        static constexpr unsigned long kMediaType =
+            static_cast<unsigned long>('|') << _IOC_TYPESHIFT;
+#endif
+        return If(shifted_type == kDrmType, Allow())
+            .ElseIf(shifted_type == kDmaBufType, Allow())
+#ifdef MOZ_ENABLE_V4L2
+            .ElseIf(shifted_type == kVideoType, Allow())
+            .ElseIf(shifted_type == kMediaType, Allow())
+#endif
+            .Else(SandboxPolicyCommon::EvaluateSyscall(sysno));
+      }
+
       // Pass through the common policy.
       default:
         return SandboxPolicyCommon::EvaluateSyscall(sysno);