NVIDIA To Create Protocol For VDPAU

[phoronix]

Phoronix: NVIDIA To Create Protocol For VDPAU

After releasing a standalone VDPAU library, NVIDIA's Aaron Plattner shared an interesting tid-bit on the X.Org mailing list in response to questions raised by Red Hat's David Airlie. The Video Decode and Presentation API for Unix will have its own protocol, similar to that of XvMC and DRI...

http://www.phoronix.com/vr.php?view=NzU0Mw

[tmpdir]

I'm not at speed in the graphics department on linux, what benefits will give this?

[jscurtu]

I'm not at speed in the graphics department on linux, what benefits will give this?

Well this will bring high quality, Accelerated Video playback... :)
VDPAU can be described as the X Window System equivalent of Microsoft's DxVA (DirectX Video Acceleration) API for Windows.

I would like to see this as part of Gallium3D, that would be the best way so that all Cards that use a Gallium3D driver would automatic benefit and be able to have vpdau support!

[89c51]

how many video APIs do we actually have for Linux (Xorg to be more precise)?????

its this by NVIDIA, Xvsomething, Vaapi, i think AMD has its own that will (is??) ported to linux and probably some more


:confused::confused::confused::confused:

[MaestroMaus]

That's normal in OSS world. Get used to it... (I know it's hard somethimes)

[lithorus]

how many video APIs do we actually have for Linux (Xorg to be more precise)?????

its this by NVIDIA, Xvsomething, Vaapi, i think AMD has its own that will (is??) ported to linux and probably some more


:confused::confused::confused::confused:
Competition and different approaches are good for Linux and OSS in general. It's true that there are quite a good number of implementations around now that does the same thing. IMHO Nvidia has proven that VDPAU is rather mature and has proven this with so many projects adopting it. It would be interesting to see if this could become the OpenGL for video playback. Imagine if this could be ported/implemented on other platforms aswell.

[thefirstm]

Competition and different approaches are good for Linux and OSS in general. It's true that there are quite a good number of implementations around now that does the same thing. IMHO Nvidia has proven that VDPAU is rather mature and has proven this with so many projects adopting it. It would be interesting to see if this could become the OpenGL for video playback. Imagine if this could be ported/implemented on other platforms aswell.

I hope you are right about VDPAU becoming the OGL for video playback. I don't see any reason why it shouldn't be, since both ends of the standard are open and quite a few media players already support it.

[89c51]

Competition and different approaches are good for Linux and OSS in general. It's true that there are quite a good number of implementations around now that does the same thing.

i disagree

we dont need a billion APIs, libraries or apps that do the same thing bad

the user needs one that does what its supposed to do really well


this approach (multiple project aiming the same thing) is the Plague of
Open Source

[jscurtu]

how many video APIs do we actually have for Linux (Xorg to be more precise)?????

its this by NVIDIA, Xvsomething, Vaapi, i think AMD has its own that will (is??) ported to linux and probably some more


:confused::confused::confused::confused:

Sure there are a few, like Xv and others ... but there is nothing close to the acceleration and quality to nvidia's vpdau so that makes it for Linux very important..

AMD is developing XvBA, but it is uninteresting because it is only for their fglrx drivers and I think closed??.. >please correct me if im wrong<

[Kjella]

how many video APIs do we actually have for Linux (Xorg to be more precise)?????

It's complicated because the graphics hardware *can* do many of the parts of a rendering process.

Resize, color conversion, frame flips, OSD, compositing and decoding. Decoding also depends on which codec, and it can be either with fixed function hardware or on shaders. It can be full decoding or just GPU-assisted decoding. Many options lead to many APIs, or one very big API.

Anything other than plain X/DirectFB, and trust me you don't want that, is using the graphics hardware in some way. It's just not on a linear path from little to more, it's branching out in different directions. For example XvMC only matters if you play MPEG2 video, nothing else.

[nanonyme]

AMD is developing XvBA, but it is uninteresting because it is only for their fglrx drivers and I think closed??.. >please correct me if im wrong<Actually since both VDPAU and XvBA can work as VA API backends, all you'd in theory have to do is create appropriate wrappers (AMD would have to offer their own for their XvBA), add VA API support in video players (yes, you video player developers, stop caressing VDPAU and get to work to add the support for VA API ;) ) and we're done. ^^
After that it doesn't matter which video acceleration API drivers implement, it Just Works, given there's a wrapper from VA API to the API they use.

[Kano]

When XvBA suffers from the same problem (very bad quality because of missing vsync) as fglrx's xv then you can put it in the dustbin.

[greg]

Actually since both VDPAU and XvBA can work as VA API backends, all you'd in theory have to do is create appropriate wrappers (AMD would have to offer their own for their XvBA), add VA API support in video players (yes, you video player developers, stop caressing VDPAU and get to work to add the support for VA API ;) ) and we're done. ^^


OTOH, why not implement XvBA and VA-API backends for VDPAU? VDPAU certainly has the best application support.

[RealNC]

When XvBA suffers from the same problem (very bad quality because of missing vsync) as fglrx's xv then you can put it in the dustbin.

I'm not sure I understand. Does VDPAU and XvBA actually render anything? If it's anything like DXVA, you still need a renderer. For example, DXVA does the decoding, OpenGL or DX7-Overlay shows you the actual picture. So you would use a renderer that supports vsync in this case (like gl).

Is that not the case with VDPAU/XvBA? If not, then whoever designed the thing wasn't thinking, I'd say.

[bridgman]

All of the decode APIs offer separate functions for decode and render (aka presentation). Some APIs allow you to intercept the decoded result and feed to something like OpenGL, others do not.

[_txf_]

I'm not sure I understand. Does VDPAU and XvBA actually render anything? If it's anything like DXVA, you still need a renderer. For example, DXVA does the decoding, OpenGL or DX7-Overlay shows you the actual picture. So you would use a renderer that supports vsync in this case (like gl).

Is that not the case with VDPAU/XvBA? If not, then whoever designed the thing wasn't thinking, I'd say.

Actually, DXVA acceleration is tied to the renderer (DX9). DXVA2 introduced in vista is not but AFAIK you can only use EVR to render (due to drm and stuff I guess).

For VDPAU the situation is the same as DXVA. You need to use the vdpau renderer to display video, this creates problems for things like subtitles (mplayer works beautifully tho). I'm not sure if the renderer is required for technical reasons but everything else has to be done in VDPAU else you loose things like bitstream decoding (plus video data has to be copied back and forward from cpu to gpu instead of remaining in the gpu).

VDPAU does use vsync and it works fine when not using compositing. As soon as you use compositing you either turn on vsync in the compositor which ruins smoothness or you suffer video tearing.

[DeeZiD]

VDPAU does use vsync and it works fine when not using compositing. As soon as you use compositing you either turn on vsync in the compositor which ruins smoothness or you suffer video tearing.


Well, here it works tearing free with composite on and with fluent 1080p playback of course. :D


Dennis

[greg]

For VDPAU the situation is the same as DXVA. You need to use the vdpau renderer to display video, this creates problems for things like subtitles (mplayer works beautifully tho).

No, VDPAU allows the frames to be reused by OpenGL and it's possible to download them to the host system as well, for later display with the renderer of your choice.

By the way, what kind of problems with subtitles are you referring to?

(plus video data has to be copied back and forward from cpu to gpu instead of remaining in the gpu).

That's exactly what you want to avoid and why VDPAU offers a renderer (plus you get interesting post-processing options, like high-quality deinterlacing).

VDPAU does use vsync and it works fine when not using compositing. As soon as you use compositing you either turn on vsync in the compositor which ruins smoothness or you suffer video tearing.

Actually VDPAU now properly avoids tearing w/ compositing managers. At first it didn't, but this has been fixed a few driver revisions ago.

(Note: with "VDPAU" I usually mean NVidia's implementation for their GPUs)

[Jimmy]

Now if we could only get this into Flash since the world of web TV is bent on cramming it into flash (and that fucking Move Media Player plug-in... DAMN YOU ABC!!). I know, I'm a dreamer. :D

We have all these media sophisticated media players that can stream video in all their "phenomenal cosmic power (http://en.wikipedia.org/wiki/Genie_%28Aladdin%29)" and we get stuck with Flash.

[_txf_]

No, VDPAU allows the frames to be reused by OpenGL and it's possible to download them to the host system as well, for later display with the renderer of your choice.

By the way, what kind of problems with subtitles are you referring to?


In which case, the limitation then is with the media players and not with VDPAU itself? The vdpau renderer is nowhere near as flexible as the standard renderers in mplayer (I want to be able to add noise).

I believe in the early days ASS subtitles did not work as everything had to be piped into VDPAU in a particular format. I didn't try it out then so I'm just repeating what nvidia and other people have said. It works fine now though in mplayer. Haven't tried vlc (does it support vdpau?) or xine (xine is useless with subtitles anyway). I should have said "created".

Regarding tearing, I guess my problems are with kwin then. Tearing occurs a lot less frequently and visibly than it did, but it still happens. Usually just as a flicker but with lots of motion the line is very visible.

[tmpdir]

Well this will bring high quality, Accelerated Video playback... :)
VDPAU can be described as the X Window System equivalent of Microsoft's DxVA (DirectX Video Acceleration) API for Windows.

I would like to see this as part of Gallium3D, that would be the best way so that all Cards that use a Gallium3D driver would automatic benefit and be able to have vpdau support!

Thanks jscurtu, but I was (badly) refering to the "standalone VDPAU library" part.

[greg]

In which case, the limitation then is with the media players and not with VDPAU itself? The vdpau renderer is nowhere near as flexible as the standard renderers in mplayer (I want to be able to add noise).

Yes, but you can't blame the renderer/video output. Adding noise etc. is usually done with video filters, and traditionally these run on the CPU. For video decoding acceleration this means you'll have to do a costly gpu -> host -> gpu round trip with the decoded frames, and currently MPlayer doesn't support that. If you use software decoding, you can do as much video filtering as you want, even with the VDPAU renderer, of course. In fact the VDPAU renderer can *de*noise, and blur/sharpen in the postprocessing, unfortunately it can't add any noise. :)

I believe in the early days ASS subtitles did not work as everything had to be piped into VDPAU in a particular format. I didn't try it out then so I'm just repeating what nvidia and other people have said.

ASS/SSA subtitles have been supported pretty much from the beginning. OSD/subtitles use the blending functionality of VDPAU which makes it quite easy to display something on top of the video.

[_txf_]

Yes, but you can't blame the renderer/video output. Adding noise etc. is usually done with video filters, and traditionally these run on the CPU. For video decoding acceleration this means you'll have to do a costly gpu -> host -> gpu round trip with the decoded frames, and currently MPlayer doesn't support that. If you use software decoding, you can do as much video filtering as you want, even with the VDPAU renderer, of course. In fact the VDPAU renderer can *de*noise, and blur/sharpen in the postprocessing, unfortunately it can't add any noise. :)


True...It would indeed be a bad thing, however with compositing there seems to be a stage whereby things are going through the cpu anyway, but this can probably be attributed to x and not vdpau specifically. When watching videos 1080p with compositing and vdpau enabled I get something like 20~30% (mostly used by xorg) usage at lower p states 800Mhz~1.2Ghz without it I get less than 10%.

I wonder how video is done with compositors in windows and osx as they don't seem to waste cpu cycles with when videos are running.

[bridgman]

The Win and MacOS graphics stacks were designed around compositing from day one, so it's not hard to keep all the video buffers in VRAM.

In the X/DRI world compositors are optional extras, implemented using standard "application" APIs which don't know about buffers in VRAM.

Most of the work right now is building the lower level code for the "new stack", but I imagine compositor integration will be relatively high on the priority list after KMS/DRI2/Gallium become broadly available. In the meantime, Wayland will probably offer a good example of what can be done if the compositor is fully integrated into the stack.

[gbeauche]

Now if we could only get this into Flash since the world of web TV is bent on cramming it into flash (and that fucking Move Media Player plug-in... DAMN YOU ABC!!). I know, I'm a dreamer. :D

You are not a dreamer. This is working under Linux on AMD and NVIDIA platforms and possibly the Intel Moorestown platform. I am preparing the packages. Of course, this uses VA API. ;-)

Note: this only accelerates Flash HD (H.264) videos, at VLD level.

[greg]

This is working under Linux on AMD [...] platforms

Oh, come on. No it isn't working until AMD opens XvBA.

Of course, this uses VA API. ;-)

Just curious: I think it'd be nice to have *one* API for video decode acceleration instead of multiple ones on Unix and in my opinion, VDPAU is the better choice. First simply because it's a nice API and well-documented, second because there's a lot of application support already and third because it offers some functionality that is missing in VA-API.
Why are you advocating VA-API, what it makes it more suitable for this task in your opinion?

I'm getting the impression it's all just a NIH problem...

[gbeauche]

First simply because it's a nice API and well-documented, second because there's a lot of application support already and third because it offers some functionality that is missing in VA-API.

What functionality do you think is missing in VA API?

Why are you advocating VA-API, what it makes it more suitable for this task in your opinion?

You seem to confuse API and implementation. There are more than 6 VA drivers available now. VDPAU does not have that many implementations. Sure, there are probably more players supporting VDPAU but if they don't cover many user base because their GPU or VPU is not supported, that won't be very useful.

[greg]

What functionality do you think is missing in VA API?

Post-processing, especially deinterlacing better than bob; blending in VA-API is not very powerful.

You seem to confuse API and implementation. There are more than 6 VA drivers available now. VDPAU does not have that many implementations.

No, I'm not. :) I'm aware that only NVidia and S3 implement VDPAU at the moment, but where do you get these 6 VA-API implementations from? As far as I know, only Intel and S3 have native implementations.

Sure, there are probably more players supporting VDPAU but if they don't cover many user base because their GPU or VPU is not supported, that won't be very useful.

That's also why I think wrapping to VDPAU instead of VA-API might a better idea.

[gbeauche]

Post-processing, especially deinterlacing better than bob; blending in VA-API is not very powerful.

Deinterlacing is indeed missing from the API, but this can be added. Why is blending not very powerful? BTW, it is now even possible to use OpenGL, so I hope this one is powerful enough in blending for your taste. :)

No, I'm not. :) I'm aware that only NVidia and S3 implement VDPAU at the moment, but where do you get these 6 VA-API implementations from? As far as I know, only Intel and S3 have native implementations.

Actually, I even forgot to count a few other implementations but those were not announced yet. Native implementation or not does not really matter since the goal is to support the underlying chip. In no particular order: Intel chips (US15W and G45 for 3 drivers, more to come), S3 chips, NVIDIA chips, AMD chips. I think this covers most what users have.

[greg]

Deinterlacing is indeed missing from the API, but this can be added. Why is blending not very powerful?

It's only possible to use a fixed blending equation (or does it even support chroma-keying only? I'm not sure... would be great if the documentation sucked less... seriously). OpenGL of course is powerful indeed, but it also complicates things a lot.

Well, are you going to add deinterlacing to the API? Sure, this could be done with OpenGL as well, but then every application would probably reimplement this again.

Native implementation or not does not really matter since the goal is to support the underlying chip.

If that is what you care about I have to ask again why you settled for VA-API, since a API that is not used anywhere isn't helpful and application support for VA-API is barely existing. VDPAU pretty much is the only video decode API relevant in practice at the moment.

[gbeauche]

If that is what you care about I have to ask again why you settled for VA-API.

VDPAU did not exist at that time. I don't mind how many players are using this or that API, I care about existing capabilities and drivers for our solutions. Our solutions cover AMD, Intel, NVIDIA products and I wanted a single API. I picked what worked well first. And between VA API and XvBA, the choice was obvious (openess was one criterion, among others).

[Kano]

As it seems you are the one to blame for the libva, mplayer and gnash patches maybe you could tell me a few things:

a) Why did you --disable-i965-driver? I patched it back of course to try it. vainfo shows Mpeg2 support (using U 9.10 daily) but mplayer crashes using a Q45.

b) What's the logic of that -va vaapi option to mplayer? I understand that you use -vo vaapi[:gl][:reflect], but -va vaapi gives to me no new info, so what's the purpose? Also are there any other options, like deinterlacing or whatever which are not written somewhere?

c) I tested of course the mplayer rendering using the video vdpau wrapper, that worked fairly well. Will you add divx accelleration for dx 10.1 cards too?

d) Could you add a way to use internal ffmpeg for gnash and maybe the hwdecode-demos as I definitely do not intent to install patched ffmpeg libs globally?

e) Could you provide a working patch against latest vlc with internal ffmpeg?

[gbeauche]

Why did you --disable-i965-driver? I patched it back of course to try it. vainfo shows Mpeg2 support (using U 9.10 daily) but mplayer crashes using a Q45.

In the version I distribute? Building the G45 driver requires libdrm2.4, which I don't have on my dev systems.

In the upstream version? Actually, the i965/g45 driver should have had its own repository, and distributed packages. Unfortunately, fd.o admins work on a best effort basis and getting things setup is not a fast process. e.g. the list was created ~6 months after the request. So the driver was integrated in the main libVA tree. Packaging-wise, this driver should not be provided within libVA.

b) What's the logic of that -va vaapi option to mplayer? I understand that you use -vo vaapi[:gl][:reflect], but -va vaapi gives to me no new info, so what's the purpose?

It's used to select the Video Accelerator. I had versions with -va vdpau for example. Unfortunately, making the Video Accelerator choice based on the Video Output (-vo) didn't seem a good idea, I was told IIRC. Anyway, I am working on an alternative that will make VA API enabling much easier, so the conventional vo_vaapi could be dropped since it duplicates code from the vo_x11 code for example (the same applies to vdpau). In the end, the plan is to only keep -va vaapi.

Also are there any other options, like deinterlacing or whatever which are not written somewhere?

There are no other options yet. I have not implemented OSD yet because I initially did not know how to use it, neither with what kind of files I should use actually.

c) I tested of course the mplayer rendering using the video vdpau wrapper, that worked fairly well. Will you add divx accelleration for dx 10.1 cards too?

If I get one. :) I thought my GTX 280M module should be suitable (G92 core?), I have not checked for sure. I have not received the new NVIDIA shipment yet either.

d) Could you add a way to use internal ffmpeg for gnash and maybe the hwdecode-demos as I definitely do not intent to install patched ffmpeg libs globally?

There will be a maintenance problem. The better intermediate approach is probably to have some extra tarball (e.g. an ffmpeg-vaapi-20090715.tar.bz2 or whatever) that glues itself in the top_srcdir and that is used if available? Something à la GCC or binutils.

The "patched ffmpeg libs globally" shall not be a real problem because the goal is that it's no longer patched but fully integrated within the upstream FFmpeg sources. I'd prefer to focus on this approach.

e) Could you provide a working patch against latest vlc with internal ffmpeg?

VLC developers sent me the patches, but I hadn't had the time to check it out yet. :( That seemed to be generated from the current GIT tree though.

[Kano]

a) I only tested your version when you look at my script. You are correct that it needs just libdrm, it build on a slightly upgraded lenny too. I guess upstream is:

http://cgit.freedesktop.org/libva/

But there the debian packageing is broken, I would like to see a working one. Why are your patches kept only on your site and not in that git?

b) That's not really what i like, i need subtitles and such things. vdpau is much better than. I have got no problem to specify overrides in a config file, but you need to provide at least one solution to let it enabled all the time with fallback for unsupported codecs.

c) You need new hardware. G92 is too old, i know that because i have got a 8800 GTS 512 (G92) myself. I also want a new card.

d) I really like that idea, you could do it similar to the mplayer checkout script too - did you have it first or nvidia ;)

e) Well the only patch i found applied and i changed some things in the code, but as it needed external ffmpeg it was not successfull.

http://forum.videolan.org/viewtopic.php?f=13&p=216296

[gbeauche]

a) I only tested your version when you look at my script. You are correct that it needs just libdrm, it build on a slightly upgraded lenny too.

Yes, libdrm2.4 builds fine on Lenny but I wanted to provide basic and generic builds.

Why are your patches kept only on your site and not in that git?

If you look at the GIT changes, you will realize that the latest ones come from my tree. The VA compat code is not suitable for upstream libVA. The usual procedure is people rebuild their applications. Upstream projects rarely provide compatibility glues for older API. When something is obsoleted, it's gone. I have not posted other patches yet. VA/GLX extensions are being discussed.

I won't propose the libdrm patches because IMHO this currently is a hacky workaround for the latest fglrx driver changes. I have not investigated yet whether this was AMD's fault or linker/loader fault.

b) That's not really what i like, i need subtitles and such things. vdpau is much better than. I have got no problem to specify overrides in a config file, but you need to provide at least one solution to let it enabled all the time with fallback for unsupported codecs.

The fallback for unsupported codecs will automagically work with the new approach I have in mind and if I don't mess with my implementation. For subtitles, this will require dedicated code that would happen when I have testcases. Hey, I am not an mplayer user, so I don't know how to use it and/or how people usually use it. :)

[Kano]

Maybe you could fix the packaging at least in upstream git so a simple checkout and build is possible. Btw. vainfo reports:

libva: libva version 0.31.0-sds3
libva: va_getDriverName() returns 0
libva: Trying to open /usr/lib/va/drivers/i965_drv_video.so
libva: va_openDriver() returns 0
vainfo: VA API version: 0.31
vainfo: Driver version: i965 Driver 0.1
vainfo: Supported profile and entrypoints
VAProfileMPEG2Simple : VAEntrypointVLD
VAProfileMPEG2Main : VAEntrypointVLD

But package is called:

ii libva1 0.31.0-1+sds4