Raspberry Pi GPU Driver Turns Out To Be Crap

[makomk]

"Turns out to be crap"? It's more like it turned out to be exactly what the foundation said instead of being the unicorn people want to believe in. If you actually read and understand the announcement, it becomes quite clear what to expect from the source release. The vast majority of tech sites got it wrong.

The Raspberry Pi Foundation themselves have been doing their best to confuse and mislead people into thinking that their source code release is more than it actually is, though. For instance Liz had this to say in reply to Luc Verhaegen's entirely accurate comment:

No.

There’s some microcode in the Videocore – not something you should confuse with an ARM-side blob, which could actually prevent you from understanding or modifying anything that your computer does. That microcode is effectively firmware.

She's redefined the entire OpenGL driver - including the shader compiler! - to be "microcode" and is pretending it's equivalent to the trivial closed-source microcode in competing hardware which actually has fully open source OpenGL drivers.

[dcw803]

For instance Liz had this to say in reply to Luc Verhaegen's entirely accurate comment:


She's redefined the entire OpenGL driver - including the shader compiler! - to be "microcode" and is pretending it's equivalent to the trivial closed-source microcode in competing hardware which actually has fully open source OpenGL drivers.

But what if a company (eg Broadcom) design a GPU where the GPU does nearly all the work (including GL), doesn't this necessarily imply that the ARM-side code is a thin shim?
Eben Upton stated quite clearly that this was the case in one of his comments on theraspberrypi.org where he said:

We happen to have a GPU which exposes a comparatively high level (GL-like) interface, such that many of our userland functions are message passing shims... These are design decisions on the part of the respective GPU teams, which have wide-ranging implications for the software and hardware structure of the devices which use the resulting cores. The VideoCore driver isn’t structured this way to pull the wool over your eyes, it’s structured this way because of a genuine judgment that this is the best structure given the resources we have on the chip, which includes a vector DSP to which we can offload much of the low-level register access.

I'd really like some of the critics to engage with Eben's point, ideally without name calling. In this situation, it's clearly easier to release the ARM-side GPU driver, fewer IP issues for a start. This may explain why Broadcom were able to release this code at all, but that doesn't make it misleading to say that they've released the complete device driver!

If you think it'll help, you can define the Broadcom GPU as a). a bizarre design or b). not a GPU at all, as some commenters seem to imply. I'm not sure how that helps the discussion?
The GPU exists, the Raspberry Pi team choose to use it, now they've persuaded Broadcom to open up the admittedly thinner-than-you-might-expect GPU driver. Useful? yes! Perfect? no. Misleading? no!

- Just my opinion; your mileage may vary.

[KeyJ]

But what if a company (eg Broadcom) design a GPU where the GPU does nearly all the work (including GL), doesn't this necessarily imply that the ARM-side code is a thin shim?

That's right. However, one doesn't simply build an OpenGL ES 2.0 GPU in hardware. For example, a large and important part of an OpenGL (ES) 2.0+ driver is the shader compiler that turns C-like high level shader code ("GLSL" code) into GPU machine code. In theory, it might be possible to do that in hardware, but in practice, that wouldn't make any sense whatsoever. Nobody ever built a compiler for a C-like high level language into silicon, and nobody ever will. There are simply no benefits for such an approach. Zero. None. So it's perfectly safe to assume that Broadcom's shader compiler is a piece of software. Problem here: The ARM-side userland doesn't contain the shader compiler. The shims simply pass the GLSL code over to the GPU side and have it compiled there. So there is definitely some non-trivial amount of meaningful driver code running on what Broadcom calls the "GPU" (which is actually more like a DSP). So, no, we don't have a full open source driver.

Again, I'm not going to say that this is a bad design decision. You just shouldn't boast to have "fully functional, fully open source" drivers when the largest part of these drivers is a closed-source blob running on a DSP.

[glisse]

But what if a company (eg Broadcom) design a GPU where the GPU does nearly all the work (including GL), doesn't this necessarily imply that the ARM-side code is a thin shim?
Eben Upton stated quite clearly that this was the case in one of his comments on theraspberrypi.org where he said:



I'd really like some of the critics to engage with Eben's point, ideally without name calling. In this situation, it's clearly easier to release the ARM-side GPU driver, fewer IP issues for a start. This may explain why Broadcom were able to release this code at all, but that doesn't make it misleading to say that they've released the complete device driver!

If you think it'll help, you can define the Broadcom GPU as a). a bizarre design or b). not a GPU at all, as some commenters seem to imply. I'm not sure how that helps the discussion?
The GPU exists, the Raspberry Pi team choose to use it, now they've persuaded Broadcom to open up the admittedly thinner-than-you-might-expect GPU driver. Useful? yes! Perfect? no. Misleading? no!

- Just my opinion; your mileage may vary.

It's misleading ! Here is the architecture of the GPU (ascii art powered) :

OpenGL
|
Kernel driver shim rpc
|
------------------------------------- Frontier [ARM LINUX on ARM CPU] everything above is what have been opensourced
|
[SMALL CPU WITH PROPRIETARY OS RUNNING A DRIVER EXPOSING THE SHIM]
|
[driver running on small cpu that convert GL into GPU command stream]
|
[GPU hw]


The hw never talk or understand GL and no one in his right mind would try to design a GPU that understand GL directly. That's crazy. It's like trying to design a CPU than can directly execute C or PHP or high level language. You don't do that, nobody does that.

Eben is misleading people into believing that it's the hw design, it's not an hw design, there is a second cpu that is running the real driver and this real driver is exposing an rpc api to the linux driver.

Again i repeat myself the line at which point you know if it's useful is when you can answer yes to the following question :
Can i fix bug ?
Can i improve performance ?
Can i implement another API leveraging hw capacity ?

In this case, it's no to all of the above. You can't fix bug because bug are in the closed driver, you can't improve performance because all the logic is in the closed driver, you can implement newer api by leveraging hw, you would need to implement it on top of GL.

[uid313]

The hw never talk or understand GL and no one in his right mind would try to design a GPU that understand GL directly. That's crazy. It's like trying to design a CPU than can directly execute C or PHP or high level language. You don't do that, nobody does that.

Actually the ARM processor with the Jazelle instruction extension can run Java bytecode.

Actually, Sun Microsystems developed the MAJC microprocessor to run Java programs.

[glisse]

Actually the ARM processor with the Jazelle instruction extension can run Java bytecode.

Actually, Sun Microsystems developed the MAJC microprocessor to run Java programs.

Both of which use microcode ie low level representation of java, none of them can directly get a string of java code and execute it. I repeat my self for people that don't understand, nobody design hw that understand directly high level language, i repeat nobody does that.

[dcw803]

That's right. However, one doesn't simply build an OpenGL ES 2.0 GPU in hardware. For example, a large and important part of an OpenGL (ES) 2.0+ driver is the shader compiler that turns C-like high level shader code ("GLSL" code) into GPU machine code. In theory, it might be possible to do that in hardware, but in practice, that wouldn't make any sense whatsoever. Nobody ever built a compiler for a C-like high level language into silicon, and nobody ever will. There are simply no benefits for such an approach. Zero. None. So it's perfectly safe to assume that Broadcom's shader compiler is a piece of software. Problem here: The ARM-side userland doesn't contain the shader compiler. The shims simply pass the GLSL code over to the GPU side and have it compiled there. So there is definitely some non-trivial amount of meaningful driver code running on what Broadcom calls the "GPU" (which is actually more like a DSP). So, no, we don't have a full open source driver.

Again, I'm not going to say that this is a bad design decision. You just shouldn't boast to have "fully functional, fully open source" drivers when the largest part of these drivers is a closed-source blob running on a DSP.

Ok, you clearly know much more about the internal structure of OpenGL, shader compilers etc than I do, I'm sure that your analysis of the internal structure of the "Broadcom GPU" (let's call it the bGPU) is accurate. We both agree that a large amount of complex software that would normally run on the CPU-side runs instead on the bGPU. Whether you call the bGPU a DSP or a GPU or an aardvark, it's still a single component, designed with an interface which the CPU-side driver writer has no choice but to use.

Where we differ is that I can't agree to call the bGPU's firmware "driver code" as you do. I'd define "driver code" as code running on the CPU (i.e ARM-side) - is this an unreasonable definition? With this definition, no firmware (no matter how complex) in components outside the CPU counts as "driver code".

My core point remains - and I think you agree with it: the shim structure of the ARM-side driver is an inevitable consequence of the bGPU's design, as Eben said. 100% of the ARM-side code (what I call the complete "driver", but you don't) is now open sourced. From your "bGPU firmware is driver code" viewpoint, I can see why you think calling the ARM-side driver "fully functional" is misleading. But from my viewpoint, I hope you can see why I think that the open-sourced ARM-side driver code is not only "fully functional", but given the bGPU design it's the *only possible* ARM-side driver architecture.

Interesting discussion!

[KeyJ]

I'd define "driver code" as code running on the CPU (i.e ARM-side) - is this an unreasonable definition?

I'm not quite sure actually. I don't know if there is a formal "official" definition of what a hardware driver is. If you ask me, I'd say that a driver is the code that directly interacts with the actual fixed-function hardware, by doing things like setting up hardware-specific buffers and data structures, writing registers and handling interrupts. Broadcom's ARM-side drivers don't do that: They only communicate with another processor.

My core point remains - and I think you agree with it: the shim structure of the ARM-side driver is an inevitable consequence of the bGPU's design

I highly doubt that. If there is a limitation in Broadcom's design that makes direct interaction with the hardware from the ARM impossible, then I bet it's an artificial one (or one that results from oversight, like "we don't need to connect that port to the ARM, it's sufficient if our DSP/GPU can access it").

But OK, the BCM2835's silicon is as it is, and let's assume that there's really no way for the ARM to access the GPU's hardware registers. Still, I'm a bit put off by the bold claims to have a "fully functional open-source driver", because to me (as a graphics programmer) a graphics driver is something that *does* contain all the meaty stuff like the shader compiler. If they really wanted to keep their word, they'd have to release the source code for the GPU/DSP blob as well. They obviously won't do that, and I can understand that, but they should not act as if they had open-source OpenGL drivers then.

Again, let me stress that I'm very pleased that they released the sources for all the ARM code and that the announcement they had on their web-site was very accurate about what has been open sourced and what hasn't. It's just one single sentence, written in bold face, that oozes hypocrisy, as it promises more than what has been delivered.

[dcw803]

(dcw803: what's the definition of "driver code")
If you ask me, I'd say that a driver is the code that directly interacts with the actual fixed-function hardware, by doing things like setting up hardware-specific buffers and data structures, writing registers and handling interrupts. Broadcom's ARM-side drivers don't do that: They only communicate with another processor.

Ok, that's a perfectly fair definition which makes sense of your other views, I just can't agree with it. Logical implications are: 1). disk firmware is part of the driver, not part of the disk, 2). a hardware raid array's XOR-based raid5 algorithm is part of the driver if it runs in software (on the raid controller), but not if it's a circuit. 3). back in mini-computer days several minis (eg. VAX and Gould Powernode to name two) came with highly complex I/O subsystems - I believe the Gould came with a complete Z80-based I/O controller, running CP/M if I remember right. Would you define all the Z80 code (both CP/M and the controller software) as part of the Gould's disk driver? I suspect we may have to agree to differ on this subtle semantic point.

(dcw803: given the hardware design, the shim ARM-side driver design is inevitable)

I highly doubt that. If there is a limitation in Broadcom's design that makes direct interaction with the hardware from the ARM impossible, then I bet it's an artificial one (or one that results from oversight, like "we don't need to connect that port to the ARM, it's sufficient if our DSP/GPU can access it").

But OK, the BCM2835's silicon is as it is, and let's assume that there's really no way for the ARM to access the GPU's hardware registers. Still, I'm a bit put off by the bold claims to have a "fully functional open-source driver", because to me (as a graphics programmer) a graphics driver is something that *does* contain all the meaty stuff like the shader compiler. If they really wanted to keep their word, they'd have to release the source code for the GPU/DSP blob as well. They obviously won't do that, and I can understand that, but they should not act as if they had open-source OpenGL drivers then.

Again, let me stress that I'm very pleased that they released the sources for all the ARM code and that the announcement they had on their web-site was very accurate about what has been open sourced and what hasn't. It's just one single sentence, written in bold face, that oozes hypocrisy, as it promises more than what has been delivered.

Ok, I accept that my suggestion that the thin shim design was inevitable was a bit strong. Broadcom choose to embed all the complex software in the bGPU and thus reveal the complex GL-like interface to a surprised world. That, along with the hardware design, was THEIR CHOICE. So, not only is the "BCM2835's silicon is as it is", but the BCM2835's internal complex software is as it is too.

Broadcom could have chosen to expose both your proposed "inner proper GPU"'s low-level hardware-only interface and the existing bGPU GL-like interface, so that ARM-side developers would have the option of using the built-in GL (via thin shim), or porting ARM-side OpenGL to use the low-level interface. But a). they didn't and b). that would have meant two massively complex OpenGL subsystems in a very small box. My natural "ruthless simplification" instincts dislike (b) a lot.

Perhaps we can agree if I restate my core point as "the thin-shim design of the ARM-side driver code is the simplest solution given the bGPU design choices, both hardware and software"?

Finally, with you I'm very pleased that they have released all the ARM-side driver code. Despite it not being useful to everyone, several commentators have given specific clear scenarios where the open source ARM-side code is useful (eg. porting other OSes to the RPi). I still think terms like "oozes hypocrisy" and "keep their word" are a bit strong, but I can see you have a consistent view that bGPU-side software is part of the driver (I disagree), think that their announcement implied they were opensourcing all the OpenGL/shader compiler code too (I have no
strong opinion about that), and that even if it didn't, it should have, cos you need/expect it:-) (I disagree, but then I'm not a graphics programmer).

I suspect we're not going to get much greater agreement than that, but I'm happy to keep trying (in the spirit of "dcw803 is trying; very trying":-))

[cyring]

To be short: what other drivers can we use beside xf86-video-fbdev ?

( fyi: using ArchLinux ARM)