Vulkan Video Finally Introduces AV1 Video Decoding Extension

[phoronix]

Phoronix: Vulkan Video Finally Introduces AV1 Video Decoding Extension

Introduced in April 2021 was the initial Vulkan Video support for a new video encode/decode API built around Vulkan. That initial Vulkan Video support was catered to H.264 and H.265 while finally with today's Vulkan 1.3.277 release there is a new extension introduced for AV1 video decoding...

https://www.phoronix.com/news/Vulkan-Video-AV1-Decode

[Gusar]

Nice! A lot of youtube is still VP9 though, so I wonder if Vulkan VP9 is also coming. Neither AV1 nor VP9 extensions mean much though until browsers adopt Vulkan Video.

[JEBjames]

@Michael

typo

"That initial Vulkan Video support was catered to H.264 and H.265" (double past tense) should be "catered"

extra words pasted at the end:

"Introduced in April 2021 was the initial Vulkan Video support for a new video encode/decode API built around Vulkan."

I think "built around Vulcan" should be removed (it's redundant). What else is a Vulcan API is going to be built around?

Enjoying the articles as always,

- J

[r1348]

@Michael

typo

"That initial Vulkan Video support was catered to H.264 and H.265" (double past tense) should be "catered"

Uh no, that's a past perfect tense, and it makes sense in this case since the old extension is no longer supported.

[rommyappus]

How does this interact with graphics cards and av1 decode acceleration? is it basically a standard way of doing the decode using software or the graphics card or does it also use the appropriate hardware acceleration support?

[shmerl]

Nice!

for the vast majority of users it's the AV1 decode they are after anyhow

May be for majority, but encoding is still quite important (video calls, OBS screen capture and etc.). So hopefully encoding will be forthcoming too.

[shmerl]

Nice! A lot of youtube is still VP9 though, so I wonder if Vulkan VP9 is also coming. Neither AV1 nor VP9 extensions mean much though until browsers adopt Vulkan Video.

Good point about VP9. I think Youtube simply never uses AV1 unless video resolution is high (like 4K may be?). So surely VP9 Vulkan decode would be good to have.

[shmerl]

How does this interact with graphics cards and av1 decode acceleration? is it basically a standard way of doing the decode using software or the graphics card or does it also use the appropriate hardware acceleration support?

It's Vulkan API for accessing whatever hardware video accelerator is exposed. I.e. it can be seen as an alternative to VAAPI. So it is only for hardware encoding / decoding in theory.

But may be someone can plug in a software backend for it similar to software Vulkan rendering for graphics, kind of like lavapipe?

Not sure what would be the benefit of that vs using a software decoder / encoder directly.

[usta]

Good point about VP9. I think Youtube simply never uses AV1 unless video resolution is high (like 4K may be?). So surely VP9 Vulkan decode would be good to have.

Nope in youtube's setting there is one option to select just stream Av1 if the video is SD , or use Av1 for all time. Looks like youtube is planing to move its SD content to av1 first [ https://www.youtube.com/account_playback ]

[shmerl]

Nope in youtube's setting there is one option to select just stream Av1 if the video is SD , or use Av1 for all time. Looks like youtube is planing to move its SD content to av1 first [ https://www.youtube.com/account_playback ]

There is a setting, but I've checked a bunch of videos with yt-dlp --list-formats <url> and most of them don't have AV1 option at all. Only high res ones do (even 1440p don't have AV1, so I'd guess it's limited to 4K now). May be they do plan to enable it everywhere, then good.