marfrit/fourier
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

ffmpeg -hwaccel vulkan falls back to SW on mesa-panvk-bifrost-video — SYNC_FD external semaphore handle unsupported #1

New Issue
Open
opened 2026-05-22 09:07:57 +00:00 by marfrit · 0 comments
marfrit commented 2026-05-22 09:07:57 +00:00
Owner
Copy Link
Copy Source

Summary

mesa-panvk-bifrost-video (panvk-bifrost-video campaign, sibling of mesa-panvk-bifrost-r4) ships VK_KHR_video_decode_h264 byte-exact-correct on Mali Bifrost SBCs (verified 2026-05-21: 48/48 BBB display frames byte-identical to ffmpeg+libva-v4l2-request-fourier on the same hantro VPU). But ffmpeg's own Vulkan hwaccel falls back to software decode when wired against this driver — the decoded image can't cross the API boundary to ffmpeg's downstream code.

Repro on ohm (PineTab2, RK3566, Mali-G52 r1 MC1)

sudo pacman -S mesa-panvk-bifrost-video
export PAN_I_WANT_A_BROKEN_VULKAN_DRIVER=1
export VK_ICD_FILENAMES=/usr/lib/panvk-bifrost-video/icd.json
export MESA_VK_VERSION_OVERRIDE=1.3
ffmpeg -hwaccel vulkan -i /tmp/bbb_1080p30.h264 -frames:v 5 -pix_fmt yuv420p -y /tmp/out.yuv

Observed:

Failed to create semaphore: VK_ERROR_INVALID_EXTERNAL_HANDLE
[h264 @ ...] Failed setup for format vulkan: hwaccel initialisation returned error.
... (ffmpeg falls back to SW h264 decode silently)

Diagnosis

The Vulkan device DOES advertise the external-sync extensions ffmpeg looks for:

VK_KHR_external_semaphore         : extension revision 1
VK_KHR_external_semaphore_fd      : extension revision 1
VK_KHR_external_memory            : extension revision 1
VK_KHR_external_memory_fd         : extension revision 1
VK_KHR_external_fence             : extension revision 1
VK_KHR_external_fence_fd          : extension revision 1
VK_EXT_external_memory_dma_buf    : extension revision 1

So the gap isn't extension availability. It's the handle-type granularity inside the extensions: ffmpeg's hwcontext_vulkan.c requests VK_EXTERNAL_SEMAPHORE_HANDLE_TYPE_SYNC_FD_BIT for the post-decode cross-API handoff (sync_file / drm-prime), and vkGetPhysicalDeviceExternalSemaphoreProperties on panvk-bifrost reports it unsupported. Hence VK_ERROR_INVALID_EXTERNAL_HANDLE at semaphore-create time.

panvk likely exposes VK_EXTERNAL_SEMAPHORE_HANDLE_TYPE_OPAQUE_FD_BIT (the Mesa-side common minimum), but not SYNC_FD. The latter is what Linux drm-prime consumers want.

What works vs what doesn't (same device, same driver)

Consumer Result
Khronos vulkan-video-dec-simple-test ✅ 50/50 frames decoded, status 0; byte-exact vs ffmpeg-libva-fourier ground truth
ffmpeg -hwaccel vulkan (h264) ❌ falls back to SW at semaphore-create
Future Chromium-VulkanVideoDecoder ❓ unknown — likely same SYNC_FD requirement
Brave 148 ❌ structurally walled (chromeos pipeline ImageProcessor — separate issue, see marfrit-packages context)

Probable fix

Add VK_EXTERNAL_SEMAPHORE_HANDLE_TYPE_SYNC_FD_BIT (and likely VK_EXTERNAL_FENCE_HANDLE_TYPE_SYNC_FD_BIT) to the supported handle types panvk reports in:

  • panvk_GetPhysicalDeviceExternalSemaphoreProperties (currently advertises OPAQUE_FD only?)
  • panvk_GetPhysicalDeviceExternalFenceProperties
  • corresponding pan_kmod_* sync-import/export plumbing (kernel-side sync_file fd round-trip via DRM_IOCTL_SYNCOBJ_*)

Most panvk paths already use drm-syncobj internally, so the kernel-side support is there — the gap is at the Vulkan API surface where panvk doesn't claim SYNC_FD as a supported handle type.

Scope note

This is panvk-wide, not video-specific. Any Vulkan consumer wanting cross-API drm-prime sync on panvk hits the same wall. Filing here under marfrit/fourier because the immediate motivating consumer is the fourier HW-decode stack (ffmpeg-vulkan-h264 on Bifrost SBCs).

References

  • Campaign close: ~/src/panvk-bifrost-video/phase0_evidence/phase5_post_review_2026-05-21.tgz (the source delta that proves the decode path correct)
  • Working consumer recipe: Vulkan-Video-Samples/build/vk_video_decoder/test/vulkan-video-dec-simple-test (byte-exact validation 2026-05-21)
  • Memory: [[panvk-bifrost-video-state]] covers the full campaign

🤖 Filed by Claude Code on behalf of mfritsche

## Summary `mesa-panvk-bifrost-video` (panvk-bifrost-video campaign, sibling of mesa-panvk-bifrost-r4) ships VK_KHR_video_decode_h264 byte-exact-correct on Mali Bifrost SBCs (verified 2026-05-21: 48/48 BBB display frames byte-identical to ffmpeg+libva-v4l2-request-fourier on the same hantro VPU). But **ffmpeg's own Vulkan hwaccel falls back to software decode** when wired against this driver — the decoded image can't cross the API boundary to ffmpeg's downstream code. ## Repro on ohm (PineTab2, RK3566, Mali-G52 r1 MC1) ``` sudo pacman -S mesa-panvk-bifrost-video export PAN_I_WANT_A_BROKEN_VULKAN_DRIVER=1 export VK_ICD_FILENAMES=/usr/lib/panvk-bifrost-video/icd.json export MESA_VK_VERSION_OVERRIDE=1.3 ffmpeg -hwaccel vulkan -i /tmp/bbb_1080p30.h264 -frames:v 5 -pix_fmt yuv420p -y /tmp/out.yuv ``` Observed: ``` Failed to create semaphore: VK_ERROR_INVALID_EXTERNAL_HANDLE [h264 @ ...] Failed setup for format vulkan: hwaccel initialisation returned error. ... (ffmpeg falls back to SW h264 decode silently) ``` ## Diagnosis The Vulkan device DOES advertise the external-sync extensions ffmpeg looks for: ``` VK_KHR_external_semaphore : extension revision 1 VK_KHR_external_semaphore_fd : extension revision 1 VK_KHR_external_memory : extension revision 1 VK_KHR_external_memory_fd : extension revision 1 VK_KHR_external_fence : extension revision 1 VK_KHR_external_fence_fd : extension revision 1 VK_EXT_external_memory_dma_buf : extension revision 1 ``` So the gap isn't extension availability. It's the **handle-type granularity inside the extensions**: ffmpeg's `hwcontext_vulkan.c` requests `VK_EXTERNAL_SEMAPHORE_HANDLE_TYPE_SYNC_FD_BIT` for the post-decode cross-API handoff (sync_file / drm-prime), and `vkGetPhysicalDeviceExternalSemaphoreProperties` on panvk-bifrost reports it unsupported. Hence `VK_ERROR_INVALID_EXTERNAL_HANDLE` at semaphore-create time. panvk likely exposes `VK_EXTERNAL_SEMAPHORE_HANDLE_TYPE_OPAQUE_FD_BIT` (the Mesa-side common minimum), but not `SYNC_FD`. The latter is what Linux drm-prime consumers want. ## What works vs what doesn't (same device, same driver) | Consumer | Result | |---------------------------------------------|--------| | Khronos `vulkan-video-dec-simple-test` | ✅ 50/50 frames decoded, status 0; byte-exact vs ffmpeg-libva-fourier ground truth | | `ffmpeg -hwaccel vulkan` (h264) | ❌ falls back to SW at semaphore-create | | Future Chromium-VulkanVideoDecoder | ❓ unknown — likely same SYNC_FD requirement | | Brave 148 | ❌ structurally walled (chromeos pipeline ImageProcessor — separate issue, see marfrit-packages context) | ## Probable fix Add `VK_EXTERNAL_SEMAPHORE_HANDLE_TYPE_SYNC_FD_BIT` (and likely `VK_EXTERNAL_FENCE_HANDLE_TYPE_SYNC_FD_BIT`) to the supported handle types panvk reports in: - `panvk_GetPhysicalDeviceExternalSemaphoreProperties` (currently advertises OPAQUE_FD only?) - `panvk_GetPhysicalDeviceExternalFenceProperties` - corresponding `pan_kmod_*` sync-import/export plumbing (kernel-side sync_file fd round-trip via DRM_IOCTL_SYNCOBJ_*) Most panvk paths already use drm-syncobj internally, so the kernel-side support is there — the gap is at the Vulkan API surface where panvk doesn't claim SYNC_FD as a supported handle type. ## Scope note This is panvk-wide, not video-specific. Any Vulkan consumer wanting cross-API drm-prime sync on panvk hits the same wall. Filing here under `marfrit/fourier` because the immediate motivating consumer is the fourier HW-decode stack (ffmpeg-vulkan-h264 on Bifrost SBCs). ## References - Campaign close: `~/src/panvk-bifrost-video/phase0_evidence/phase5_post_review_2026-05-21.tgz` (the source delta that proves the decode path correct) - Working consumer recipe: `Vulkan-Video-Samples/build/vk_video_decoder/test/vulkan-video-dec-simple-test` (byte-exact validation 2026-05-21) - Memory: `[[panvk-bifrost-video-state]]` covers the full campaign 🤖 Filed by Claude Code on behalf of mfritsche
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
marfrit (Markus Fritsche)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: marfrit/fourier#1
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?