Experimental RADV Vulkan Video Gets H.264 & H.265 Decode Working

Interest and support around Vulkan Video for adding GPU-accelerated video encode/decode to the Vulkan API has been (sadly) rather slow. But at least the Mesa Radeon Vulkan driver "RADV" has seen some new work around enabling H.264 and H.265 video decoding over Vulkan Video.

The initial Vulkan Video extensions were published in early 2021 and effectively remain in beta. There has been some additions over time like H.265 encode but so far there hasn't been any published extensions for AV1 or VP9 encode/decode, as one unfortunate limitation so far. NVIDIA has provided Vulkan beta drivers with Vulkan Video support while support elsewhere among desktop GPU drivers at least has been rather limited.

Going back toward the end of last year there was some RADV Vulkan Video enablement work happening but not a priority. Now as we approach the end of 2022, Red Hat's David Airlie has returned to working on RADV Vulkan Video support and has been working on it with FFmpeg developer Lynne.

The Khronos Group slide providing an overview of the Vulkan Video decode process.

With code currently staged via Airlie's personal Mesa Git branch, the RADV Vulkan Video support is updated against the current beta extensions and an initial implementation of H.264 and H.265 video decoding is in place. This Vulkan Video H264/H265 support is working for AMD Tonga through RDNA2 GPUs. Basic video conformance tests are passing but more complicated tests for this Vulkan Video support are currently failing.

Those interested in experimenting with the Vulkan Video support for Mesa's RADV driver in conjunction with FFmpeg can find all the build instructions via Airlie's blog.