ATI R600g Gains Mip-Map, Face Culling Support

[pingufunkybeat]

Like Vincent said, it's not just a "driver", as a separate piece of code.

The open source drivers are distributed between the kernel (drm), the X driver and Mesa (which does OpenGL). Now all drivers are migrating over to the Gallium3d infrastructure.

Fglrx and the Nvidia blob do all of this inside the kernel blob, and they duplicate much of the code that sits in the X server and the linux kernel as a result.

Either you port fglrx to match the open stack (which is a lot of work), or you carefully separate out the parts that can be released into a separate branch, and then develop it with no community help whatsoever and maintain the whole thing on your own. Neither is a good proposition.

The most reasonable thing to do is to do the open drivers the Linux way -- by building on the OSS technology like Gallium3d and the kernel drm infrastructure, KMS, etc. This makes the fglrx code difficult to port directly, and the effort it would take is probably no less than writing the thing from scratch.

The important thing is opening the docs and providing the basic infrastructure and getting the community involved. I think that AMD are doing it the right way. The problem that few people in the community have both the know-how and the time to contribute is a real one, but a different issue altogether.

[Agdr]

@Agdr,
Do you have any idea about just how much these driver _stacks_ differ?

There is no need for code sharing because the know-how is already there and the 'two' drivers differ so much that any code effort of sharing could be spend coding the triple amount of usefulness.

If fglrx/Catalyst were to be opened, it could be used as a full standalone OpenGL stack, without using Mesa and DRM.

Alternatively, it could be turned into a Gallium driver: the Gallium driver interface is very similar to the Direct3D 10/11 one, so it should be relatively feasible to convert the Catalyst Direct3D 10/11 driver to become a Gallium driver. The kernel interface would need to be replaced with libdrm-radeon.

A third option would be to just integrate the VLIW shader optimizer into r600g. I'd say this is likely the most interesting (i.e. hard to rewrite well, and most self-contained) part of fglrx, and the one easiest to actually reuse in a Mesa/Gallium driver. It is also very AMD-specific, so it shouldn't provide a competitive advantage.

Even in this case, the generic parts of the 3D driver could be rather useful to have available when writing equivalent functionality for r600g, especially for advanced stuff like Evergreen tessellation support that the Mesa classic driver probably won't get soon.

The source code of their OpenGL 4 implementation, while probably not directly usable in Mesa, could still be insightful and speed up work on GL3 and GL4 for Mesa. nVidia already has their own, so it might be possible to open (unless deeply intertwined with 3rd-party licensed code).

[V!NCENT]

@Agrd,
gallium is not like Direct3D at al. Not even remotely close.

Gallium is not a driver, nor a 3D library. Gallium is a layer for modern GPU's. That layer is a sort of API. On top of that API functionality can be written, like OpenGL, OpenCL, DirectX, Glide, X.org, vector graphics acceleration, database crawlers, number crushers and whatever you can think of, which is why you _do NOT_ want fglrx take its place.
Below that layer is a driver that exposes that Gallium API and _that_ is where the floss drivers kick in.

[Agdr]

@Agrd,
gallium is not like Direct3D at al. Not even remotely close.

Below that layer is a driver that exposes that Gallium API and _that_ is where the floss drivers kick in.

The Gallium API is very close to the Direct3D 10/11 API, especially the user-mode "DDI" API that Windows device drivers implement.

If you actually look at it, you'll notice that Gallium functions essentially map 1:1 to Direct3D functions (e.g. pipe_context::create_rasterizer_state vs ID3D10Device::CreateRasterizerState).

Of course there are some differences, and some features that one supports and the other doesn't, but the general structure is identical (unlike OpenGL, which is a completely different API).

[bridgman]

All good questions, although I've answered them all a few times before. Maybe I need to write a book

So Bridgman, if your fglrx team would have the same documentation (not counting DRM) as AMD is going to give to the public, how much of a performance decreas would we be talking about? Zero?

Somewhere on the order of "a few percent", certainly less than 5%. The primary reason for the performance difference between proprietary and open driver stacks is that the proprietary drivers get maybe 50x the development resources because the work is shared across the entire PC market rather than being specific to one OS.

I'd like to thank you for engaging the community, but I find this reasoning quite puzzling. The only DRM that I can think of is the one related to the playback of BluRay and other HDCP protected media, and that surely doesn't impact the 3D driver. It seems you should be able to open source both fglrx and the Windows driver except for the video acceleration+DRM parts relatively easily.

On the DRM side, the problem is that we have to protect not only the bits of code which are actually doing DRM-ey things (which are as small as you expect) but also all of the code *below* them (ie between that code and the hardware) in order to protect against attackers interposing code between the DRM-specific bits and the hardware.

The 3D part of the driver uses all many of the same lower level bits (surface management, memory management, command submission etc..).

Besides, all video DRM seems completely cracked and BluRay releases seem widely available on public BitTorrent sites, so I don't think you could possibly make the situation worse for content producers.

The problem is that IHVs (HW vendors like us) are required to meet specific standards of robustness rather than "not making things any worse than they already are", ie our DRM-related obligations do not go down if other parts of the DRM stack are broken. For better or for worse, that's how content providers ensure that DRM does not collapse into rubble as soon as the first big cracks start to appear.

The argument of wanting to keep the driver secret to not give a free gift to competitors seems instead much more understandable. Is this the real reason? However, opening some things like your AMD IL -> r600+ ISA compiler/optimizer shouldn't give any competitive advantage to Intel/nVidia since they use completely different non-VLIW architectures. It would be great and very interesting to have a much more detailed and accurate rationale for why you could or cannot open each component of fglrx/Catalyst.

Yep, on the 3D side the reasons for not opening the code are more related to competition than to DRM. Two main issues :

1. We regard our shader compiler technology as part of the "secret sauce" which allows us to use a VLIW hardware approach which, in turn, gives us advantages in performance vs transistor count / die size / cost.

2. If you limit the discussion to our major competitor in the discrete GPU market, we both have roughly the same features and performance but I'm sure our code has clever ideas that our competitor doesn't have and that their code has clever ideas we don't have. If one competitor opens their code while the other does not, that tips the balance subtly (again, probably only a few percent but those few percent are worth a lot of $$).

And opening fglrx and your Windows driver would have several advantages:
1. Having bugs actually fixed by third parties, or at least more detailed bug reports
2. Rock solid Linux support and dominance of the Linux market (all Linux distributions and enthusiasts would be in the position to recommend buying ATI GPUs exclusively)
3. More mindshare among commercial game programmers, resulting in games that work better on your hardware. This would be due to the obvious advantage of being able to find out exactly what the driver is spending CPU time on and look intimately at what 3D calls are being translated to and how the hardware actually works.
4. Possibly getting commercial game developers to help you in tuning your drivers for their games (without needing cumbersome NDAs), resulting in better performance on your cards compared to competitors, especially in things like multi-GPU support

Yep, there are definitely benefits, but again the number of outside developers who can *and* are likely to work on the drivers is sufficiently small that it's *really* hard to make the costs and benefits work out. We have driver developers with source access working side-by-side with outside ISV developers already so I *think* the largest potential area of gain from opening the driver is already covered.

Opening code is about a thousand times as cumbersome as a source license agreement, and maybe 10 thousand times as cumbersome as an NDA

So, for parts of the driver where DRM, third party code, or competitive advantage concerns don't apply, I think you should consider opening the code (for both Linux and Windows).

Yep, and that's what we are doing. It's just that the "parts of the driver where DRM, third party code, or competitive advantage concerns don't apply" are much smaller than you might think. The driver stack was not written with an eye to opening it, and so opening almost any part of it requires some refactoring to move out the sensitive stuff which doesn't *have* to be in that part of the code but which was put there anyways because it made the code simpler/cleaner/faster.

I don't really see much chance of opening up the Windows parts of the code, but we would like to open up some more of the Linux-specific bits over time.

Try to imagine sifting through a few hundred thousand lines of fglrx code to determine if it contains any licensed code that mustn't be shared, uses any patents that aren't covered outside of fglrx or tells internals about the hardware which AMD wants to keep secret.

Yep, and we're talking about >>10 million lines of code here.

I think it should take much less time to review code than to rewrite it from scratch, obviously, especially for very non-trivial parts like how to maximize utilization of the r600+ VLIW ALUs.

Sure, but the whole point is that we are not rewriting it from scratch. The open source driver is not trying to duplicate fglrx, it's trying to be "<5% of the code size delivering 80-90% of the results", so the comparison is between inspecting and sanitizing the entire stack vs rewriting 5% of it.

As long as the open driver is separate from the primary driver, hardware will always be supported months late (e.g. no HD 5xxx 3D support almost a year after release!), and the driver won't be competitive for 3D gaming, which is definitely not a desirable outcome for users, and arguably not desirable for AMD either (unless other concerns are more important).

Nope, you need to consider the "catch-up" work that has been happening, with community and AMD developer resources working first on getting support in place for older GPUs and second on bringing the open source graphics architecture up to date (KMS, GEM/TTM, Gallium3D) before working on the newest GPUs. Even Evergreen support work started well after launch because the devs were finishing off support for previous generations.

Our goal is to be more or less caught up by the time the first Fusion parts start to ship, and from that point on I expect we should be able to start at least some of the open source support work before subsequent parts are launched.

Our launch-time focus will still be on the proprietary drivers because the ability to share common code across 100% of the PC market is too useful to ignore, but everyone needs to avoid falling into the trap of believing that just because we restarted open source support in 2007 and worked in part on hardware dating back to 2000 that means we'll be stuck with that initial backlog forever.

[Agdr]

All good questions, although I've answered them all a few times before. Maybe I need to write a book

Thanks for your answers. A detailed FAQ would probably be helpful, maybe with a sticky post here pointing to it, so you could then just point people to it, and chastise them for not having read it beforehand

Somewhere on the order of "a few percent", certainly less than 5%. The primary reason for the performance difference between proprietary and open driver stacks is that the proprietary drivers get maybe 50x the development resources because the work is shared across the entire PC market rather than being specific to one OS.

Yes, and that's why it would be nice to have all that work benefit the open driver at least partially, if it were somehow possible to arrange.

On the DRM side, the problem is that we have to protect not only the bits of code which are actually doing DRM-ey things (which are as small as you expect) but also all of the code *below* them (ie between that code and the hardware) in order to protect against attackers interposing code between the DRM-specific bits and the hardware.

The 3D part of the driver uses all many of the same lower level bits (surface management, memory management, command submission etc..).

OK, but what if you released both an open driver without the DRM, and a monolithic blob with DRM?

This would only be a problem if contracts specifically said that the source code for any part of the whole driver including DRM cannot be publicly released (which seems pretty draconian, but I guess possible).

Yep, on the 3D side the reasons for not opening the code are more related to competition than to DRM. Two main issues :

1. We regard our shader compiler technology as part of the "secret sauce" which allows us to use a VLIW hardware approach which, in turn, gives us advantages in performance vs transistor count / die size / cost.

I thought the idea of nVidia or Intel moving to a VLIW architecture and taking advantage of your ideas/code was considered unrealistic, but evidently you don't, and might be right at that.

2. If you limit the discussion to our major competitor in the discrete GPU market, we both have roughly the same features and performance but I'm sure our code has clever ideas that our competitor doesn't have and that their code has clever ideas we don't have. If one competitor opens their code while the other does not, that tips the balance subtly (again, probably only a few percent but those few percent are worth a lot of $$).

Yes, you would face the risk of an hypothetical slight loss of your competitive edge, which could be perhaps compensated by other advantages.

Another theoretical option would be to attempt to make an agreement with nVidia to both open your drivers, although I suppose that is very unrealistic.

Yep, there are definitely benefits, but again the number of outside developers who can *and* are likely to work on the drivers is sufficiently small that it's *really* hard to make the costs and benefits work out. We have driver developers with source access working side-by-side with outside ISV developers already so I *think* the largest potential area of gain from opening the driver is already covered.

Opening code is about a thousand times as cumbersome as a source license agreement, and maybe 10 thousand times as cumbersome as an NDA

It's more cumbersome for you, but much less cumbersome for third parties, which means you get more contributions and more developer mindshare. Of course whether this is significant is debatable.

There is also the Linux market advantage, where a top performing open driver could possibly lead to near total dominance of the Linux enthusiast market, and workstations users might be influenced too once RHEL and other enterprise distribution start advertising better or exclusive support for ATI and Intel cards as opposed to nVidia ones (and the Intel ones are of course useless for the workstation market).

Right now people seem to still buy nVidia cards for Linux use because the open drivers are not competitive and they either deem fglrx inferior to nvidia, or consider them equivalent and go nVidia for other reasons.
An open fglrx might change this, and make ATI cards a must, while with the current strategy, this won't happen at least for a few years, if at all (i.e. not until Mesa supports the latest OpenGL release and performance is almost that of fglrx).

Obviously it's a much smaller market than Windows, but being recommended by Linux users might somewhat affect the Windows market too, and the fact that Linux is likely more prevalent among developers and GPU compute users might increase its importance.

Anyway, I guess you already analyzed this, and decided the possible gains would be lower than the risk of reducing your competitive advantage and the cost of the effort.

I don't really see much chance of opening up the Windows parts of the code, but we would like to open up some more of the Linux-specific bits over time.

I think it would be nice to have the closed source 3D userspace optionally work with the open DRM driver, and an open X driver, enhancing them if necessary beforehand (possibly with Linux-specific code from fglrx if applicable).

This should be doable (except possibly for the video DRM stuff, which I think you could just not support in this mode of operation), and would eliminate a lot of the major issues with a closed 3D driver like trouble with newer kernels and X servers, system crashes, security issues, kernel taint, and make it very easy to switch between fglrx and the Mesa/Gallium stack, and even use them side by side.
It would be an unique advantage over nVidia and Nouveau and would also possibly allow you to eventually drop the kernel and X driver, and focus only on the proprietary OpenGL/OpenCL userspace.

[drag]

All good questions, although I've answered them all a few times before. Maybe I need to write a book

Wikis are good for this sort of thing.



@Agdr:

ATI survives based on the superiority and competitiveness of their proprietary driver. Nvidia in the past almost killed them off back in the day because Nvidia came out with a new driver architecture that netted them as little as a 10-15% boost in performance on benchmarks and nobody in ATI is going to forget that.

It's going to take a huge amount of effort to prove to ATI that spending resources on being open is the way to go. HUGE. This is something that I do not see happening any time this decade. Badgering poor Bridgeman is not going to make any difference.

Hell, I expect a complete re-architecture from bottom to top of the PC architecture and the elimination of GPU as a discrete processor long before ATI or Nvidia will be convinced that their source code 'secret sauce' is not nearly as precious as they think it is.

[nanonyme]

@Agrd,
gallium is not like Direct3D at al. Not even remotely close.

Gallium is not a driver, nor a 3D library. Gallium is a layer for modern GPU's. That layer is a sort of API. On top of that API functionality can be written, like OpenGL, OpenCL, DirectX, Glide, X.org, vector graphics acceleration, database crawlers, number crushers and whatever you can think of, which is why you _do NOT_ want fglrx take its place.
Below that layer is a driver that exposes that Gallium API and _that_ is where the floss drivers kick in.

So it sounds more like DirectX than Direct3D (except that DirectX still handles lots of other stuff too than Gallium3D). DirectCompute on DirectX vs OpenCL on Gallium3D, Direct3D on DirectX vs OpenGL on Gallium3D etc.

[marek]

So it sounds more like DirectX than Direct3D (except that DirectX still handles lots of other stuff too than Gallium3D). DirectCompute on DirectX vs OpenCL on Gallium3D, Direct3D on DirectX vs OpenGL on Gallium3D etc.

Wrong thinking. Direct3D in DirectX is like pipe_context in Gallium. DirectShow in DirectX is like pipe_video_context in Gallium. A 3D game engine atop DirectX is like OpenGL atop Gallium.

Indeed, Gallium looks the same as Direct3D 10/11. There is even a comparison between the two APIs:
http://cgit.freedesktop.org/mesa/mes...s/d3d11ddi.txt

[Otus]

it already can, but - shadows doesnt work, and screen rotatin on cube - all broken, but windows wobling

Ok, I meant (and should have written) how close is it to properly running Compiz?