Gallium3D's Softpipe Driver Now Runs Faster

[phoronix]

Phoronix: Gallium3D's Softpipe Driver Now Runs Faster

Keith Whitwell has gone ahead today and merged the softpipe-opt branch of Mesa into the master Mesa branch, which will eventually work its way into Mesa 7.7. The softpipe-opt branch brings performance optimizations to the "softpipe" driver of Gallium3D...

http://www.phoronix.com/vr.php?view=NzU1Mg

[cb88]

SMP support? that would be nice.

[patstew]

When is gallium going to replace the old mesa/3d drivers?

[nanonyme]

When is gallium going to replace the old mesa/3d drivers?In a distant future populated by robots arguing with each other which API is the best for accelerating holografic video output?

[crispy]

So we can get software accelerated compositing if you don't have a hardware to run it? :)

[whizse]

So we can get software accelerated compositing if you don't have a hardware to run it? :)

You have never tried software rendering in Mesa, have you? ;)

Joking aside, having a finely tuned software rendering option in Gallium would be very nice. Microsoft has shown that it doesn't have to be that slow: http://techreport.com/discussions.x/15968

[nanonyme]

Neat although actually beating a discrete card would have been *far* more impressive. ;)

[mirza]

Amiga demo scene veterans (like me) are amazed Compiz ever needed HW support. That kind of effects was cool back in 1989 on 7-10 MHz machines (albeit lower res then now). Thats why I am always so mad about graphic subsystem and its memory usage. In my mind, X should need this amount of memory: 1680*1050 (resolution I have), * 4 (RGBA) * 2 (double buffering) * 2 (lets have some comfort, memory is cheap) < 30 MB !!! My X is using right now 100 MB of virtual memory (Ubuntu) with Compiz _disabled_. I know, I know, it is storing bitmaps for applications. but it means X is storing such amount of bitmaps it could cover my desktop completely 10 times or more! How can that make any sense to anyone? Thats so wasteful. That, and Get Of My Lawn!;)

[nanonyme]

Hehe, you're probably right. ;) With good enough low-level coders it could use a lot less system resources than it does. (not that nearly anyone does that low-level coding anyway anymore)

[highlandsun]

Hehe, you're probably right. ;) With good enough low-level coders it could use a lot less system resources than it does. (not that nearly anyone does that low-level coding anyway anymore)

Yes, that's the biggest problem. Everyone thinks that optimized java/python is good. Blecch....

[nanonyme]

Yes, that's the biggest problem. Everyone thinks that optimized java/python is good. Blecch....It's not just that... Even if you put modern coders to write Assembly, you'd still probably have at least as bloated code if not more. Most of the current generation of coders simply didn't learn to take everything and a bit more out of their computers.

[fabiank22]

Hehe, you're probably right. ;) With good enough low-level coders it could use a lot less system resources than it does. (not that nearly anyone does that low-level coding anyway anymore)

Well, first of all developers are harder to come by that memory. Nearly everyone nowadays has 1GB or more, meaning nobody will mess with X-Server for months just to get 4 or 5MB saved space out of it.

Secondary there will always be the dispute low-level versus high-level, as we move to higher and higher levels of abstraction. I can remember an uncle of mine disputing C++ and object-oriented programming in general, because of the performance lost - he had learned to think about memory with the requirements of his old Apple in mind.

I'm one of a last few of students in germany that will still be learning C(starting in 2 weeks), basically because I also signed up for embedded development, but aside from that it's Java and C++, and in some cases even C#(which I'm also going to have to learn - unholy :\ ). I'm not even sure if there is an option to learn more assembler than the usual "hey, it speeds up C".

Today I see C++-Developers bitching about Python and Java,(although Java certainly gained performance with the 6 series, I've even seen 3D done right with it). Tommorrow I will see the Java people bitch about the performance loss of OS-independent virtual machine scripting langs in the style of python or something like that.

Look at what KDE is doing today opening everything on the Desktop for scripting, signaling and remote controlling. Sure it will eat performance like it's nobodys business, and the Atari Kids will have a smile, but in the end, when the real use of things like remote-control desktop enviroment(which is not just VPN/SSH!), social desktop or context-sensitive applications hits, you'll see that none of this could have been made with C without driving everybody nuts.

Sure, within the X-Server or the Kernel it makes no sense or isn't even possible to switch to a high-level lang, but even if everybody knew C, there's still a world of hurt with the specifics of driver development.

[fabiank22]

It's not just that... Even if you put modern coders to write Assembly, you'd still probably have at least as bloated code if not more. Most of the current generation of coders simply didn't learn to take everything and a bit more out of their computers.

While I agree to the statement in theory, think about it again. Why do you think so many people are using KDE/Gnome with Compiz, a piece of software written by people that have obviously never even heard the words "stable", "optimized" or "doesn't crash and burn the shit out of your system". Simple, because it had features not offered by other solutions. Whether these features are that good is not a developers problem. It only shows there's a market for it.

So let's do this one again from the KDE-angle, because I use that: You could a)optimize the shit out of 3.5, a series with code already called unflexible and unmaintainable by tons of developers and third-parties. You could only barely introduce new features due to lack of manpower and code flexibility. In the end you have a rock-solid system with a feature-set equal of Windows XP in 2009.

or b)start all over, modernize your code and feature scripting to modern langs(aka the ones people use), introduce tons of features, while having releases that eat Ram and performance like crazy.

In the case of KDE b) was really the only option.

[nanonyme]

Today I see C++-Developers bitching about Python and Java,(although Java certainly gained performance with the 6 series, I've even seen 3D done right with it). Tommorrow I will see the Java people bitch about the performance loss of OS-independent virtual machine scripting langs in the style of python or something like that.Eh, I've seen all of that for at least the past ten years... Don't think anything has changed. People have always complained about the slowness of interpreted languages like Java and Python and others. I've also seen just the same complaining about C++ being a ridiculous memory hog and nothing in those claims has changed either.
You're right when you say that developers are hard to come by if you mean software project designers by that. Coders are cheap and you find them under every rock and bridge.

[fabiank22]

You're right when you say that developers are hard to come by if you mean software project designers by that. Coders are cheap and you find them under every rock and bridge.

You're right. Let's change my statement to "developers that know what they are doing" or qualified developers(and by that I mean qualified enough to correctly optimize a project of the magnitude "X-Server"). Sure, you can get "Web-Designers" or PHP-People cheap even below the average wage of a kitchen help(at least in germany, where you make more money serving salad than badly managing servers/designing Web-Pages)

But even when people still had to go through assembler and compilers had about the intelligence of "Ugh, FORTRAN not compile, not know problem, ugh" things nevertheless weren't all good. We just changed from speed to feature, all the while gaining a lot of speed on hardware-basis. Just look at Linux back then: sure it was hand-optimized and fast and all that, but that didn't make people magically use it. No, features did. What we need isn't a Softpipe driver so optimized it could run Doom III(although I'm certain that would be possible in some way), we need features, features, features, and a little speed to keep up. Because (good) manpower isn't going to magically appear.

[bridgman]

This has been one of the big debates in software engineering for at least 40 years. When programming languages were arcane and close to the metal there was a self-limiting effect such that only highly technical and detail-oriented people could write code in the first place.

A lot of effort has been made to simplify programming tools and lower the entry threshold for programming, allowing many more people to become effective programmers. This works as long as the highly technical folks are willing to focus their efforts on design, review and mentoring the new people rather than writing code themselves - that way "leaf" code may become inefficient but the overall design and "core" code of the major software projects can stay clean.

There has been some progress in this area (Linus doesn't code much these days) but we are probably only 30% of the way there.

In the meantime fast hardware and cheap memory *do* cover a multitude of sins :D

[Svartalf]

You have never tried software rendering in Mesa, have you? ;)

I have with Heretic II to verify what the scene was supposed to look like when I had rendering errors with RagePRO, Rage128, and G200/G400 cards. "Painfully Ssssloooow" doesn't even begin to describe it.

Having said this, it's more due to trying to do precisely the OpenGL rendering pipeline in software, which is really something designed to have some semblance of a hardware assist. Having a softpipe that optimizes for a rendering pass and removes most of the un-used/redundant crap before it's attempted per frame would actually improve the speed quite a bit- much of the loss in speed is due to attempting to do the OpenGL 1.4 pipeline in it's entirety and trying to do the shader side without optimizations at all.


Joking aside, having a finely tuned software rendering option in Gallium would be very nice. Microsoft has shown that it doesn't have to be that slow: http://techreport.com/discussions.x/15968

Indeed. And it's useful for doing software Vertex shading for IGPs and it's possibly useful as a base for starting Larrabee driver work. This observation doesn't diminsh the need for Radeon support, but you need to make sure your assumptions are correct before applying them to a set of shader cores- as it can be as complicated and time consuming as the process they're going through right now with the softpipe.

[Svartalf]

In the meantime fast hardware and cheap memory *do* cover a multitude of sins :D

It does, indeed... ;)

[Svartalf]

Sure, within the X-Server or the Kernel it makes no sense or isn't even possible to switch to a high-level lang, but even if everybody knew C, there's still a world of hurt with the specifics of driver development.

No kidding. It's not easy, even with something as "simple" as a network card. Worse, we're talking about modern GPUs, which are sophisticated stream processing engines as much as anything else. A supercomputer within your computer, as it were. They're not easy to drive poorly, let alone efficiently.

[nanonyme]

No kidding. It's not easy, even with something as "simple" as a network card. Worse, we're talking about modern GPUs, which are sophisticated stream processing engines as much as anything else. A supercomputer within your computer, as it were. They're not easy to drive poorly, let alone efficiently.Yeah, you're right. The hardware we're running on top of is quite a lot more complicated than it used to be in the good old days when coders took everything out of the hardware. Probably unrealistic to expect as efficient usage of the hardware as before but hey, we're allowed to be nostalgic, aren't we?

[Kjella]

A lot of effort has been made to simplify programming tools and lower the entry threshold for programming, allowing many more people to become effective programmers. (...) There has been some progress in this area (Linus doesn't code much these days) but we are probably only 30% of the way there.

Well, it's not exactly people that passed "Visual Basic for dummies" that is passing code up to Linus... I'd say the greatest sin I see is insanity like "I don't want to use KDE or Gnome because they take up 20-50MB of memory" and instead you:

1) Reinvent the wheel
2) ...poorly
3) Creating your own thing means everyone else must understand it all the way from the ground up, following no common patterns.
4) Takes no advantage of shared libraries meaning you actually increase memory use if there's a hundred applications like yours.
5) Your "light-weight" toolkit ends up taking on more and more crap eventually becoming what you wanted to avoid, only worse.
6) Try optimizing out of trying to use shorts and ints and whatnot to save 2-4 bytes here and there in completely non-critical code
7) Because your documentation sucks and noone knows what it really does, people copy-paste snippets that they know works. Spaghetti...

What I have found is that there are very few cases where you're doing it "right" and still get unacceptable performance. Usually there's some other problem, for example that you put something stupid inside a loop or the wrong loop and the compiler didn't save your ass. Been there, done that - the difference between O(n) and O(n^2) is pretty high when n=50000. But I'd take 5 lines of library-calling code over 100 lines of DIY code any day.

This kind of you see on both kinds of the stupid scale - those that are just so clueless they don't know what the library can do for them, and those that know it can and reject it because it's not "good enough". If so, get a patch upstream to the library and if you think that's too hard, the red blinkenlights should be on already.

[mugginz]

Well, it's not exactly people that passed "Visual Basic for dummies" that is passing code up to Linus... I'd say the greatest sin I see is insanity like "I don't want to use KDE or Gnome because they take up 20-50MB of memory" and instead you:

1) Reinvent the wheel
2) ...poorly
3) Creating your own thing means everyone else must understand it all the way from the ground up, following no common patterns.
4) Takes no advantage of shared libraries meaning you actually increase memory use if there's a hundred applications like yours.
5) Your "light-weight" toolkit ends up taking on more and more crap eventually becoming what you wanted to avoid, only worse.
6) Try optimizing out of trying to use shorts and ints and whatnot to save 2-4 bytes here and there in completely non-critical code
7) Because your documentation sucks and noone knows what it really does, people copy-paste snippets that they know works. Spaghetti...

What I have found is that there are very few cases where you're doing it "right" and still get unacceptable performance. Usually there's some other problem, for example that you put something stupid inside a loop or the wrong loop and the compiler didn't save your ass. Been there, done that - the difference between O(n) and O(n^2) is pretty high when n=50000. But I'd take 5 lines of library-calling code over 100 lines of DIY code any day.

This kind of you see on both kinds of the stupid scale - those that are just so clueless they don't know what the library can do for them, and those that know it can and reject it because it's not "good enough". If so, get a patch upstream to the library and if you think that's too hard, the red blinkenlights should be on already.

+++

More people need to take this view I think.

A bit of code consolidation would be a brilliant thing to have happen.

There's so many projects out there re-inventing the wheel, and providing software that's almost useful, but is too broken to use on a daily basis.

[nanonyme]

Well, you might say most modern opensource projects (including Linux kernel) originate from reinventing the wheel. :) Reinventing the wheel is nice and fine if you're a brilliant software designer. Most just are not.

[fabiank22]

But I'd take 5 lines of library-calling code over 100 lines of DIY code any day.


What most people that try to critizise KDE/Gnome for their bloated libraries is that those libs are there and in that size the moment any application uses it. 20-40 Megs may be much if you're just using it to play KMahjong, but on my Setup I have 20-30 apps using that. If all those apps used their own code for file-viewing and saving configs and all that I wouldn't have any space left.

Everybody talking to me about how "lightweight" Wxwidgets or whatever toolkit of the month they have is only uses one viewpoint: "My application is the only one ever getting used".

That's why I like the way the LSB and QT and GTK have been going these last years: More and more functionality is going into the main libs and being standardized, thus getting more and more code shared. Sure it makes the main lib larger, but it shrinkens every other app using it.

And the way I understand it, Gallium isn't much different. That's why I'm preparing for a shitstorm on Phoronix, because initial releases of Gallium WILL be slower and buggier than doing everything seperate for every driver in Mesa. But it will be the right thing.

[Svartalf]

And the way I understand it, Gallium isn't much different. That's why I'm preparing for a shitstorm on Phoronix, because initial releases of Gallium WILL be slower and buggier than doing everything seperate for every driver in Mesa. But it will be the right thing.

Heh... And, it'll serve for more than Mesa. Besides, if you want Larrabee, or any other modern GPU design, you have to do something along those lines.

AMD has something VERY similar- just ask our resident AMD rep in the forums... :D

NVida has their own sort of answer- and it's mostly the same story.


It'll be a bit twitchy at first, much like Utah-GLX and then DRI that followed; but in the end, it'll be the right way and it'll all gel. In all truth, Gallium3D's something we've needed for a long time now.

[whizse]

And the way I understand it, Gallium isn't much different. That's why I'm preparing for a shitstorm on Phoronix, because initial releases of Gallium WILL be slower and buggier than doing everything seperate for every driver in Mesa. But it will be the right thing.

I guess those of us with both too much time on our hands and an extremely large amount of patience could start the testing and bug filing right now, with the softpipe.

[Death Knight]

Does any one know how could I use/test this Gallium3D's Softpipe Driver? I am compiling from git trunk but I cannot 'use' this driver yet. (Using radeon r600 dri2 kms)

[whizse]

Not sure about softpipe in general, but llvmpipe have instructions for building and running:
http://cgit.freedesktop.org/mesa/mesa/tree/src/gallium/drivers/llvmpipe/README