AMD's Background On The ROCm OpenCL Stack

[phoronix]

Phoronix: AMD's Background On The ROCm OpenCL Stack

A ROCm (Radeon Open Compute) developer at AMD has shared some of their background work on their OpenCL compiler stack, including the LLVM focus, as well as some of their current performance focuses for this open-source compute offering...

http://www.phoronix.com/scan.php?page=news_item&px=ROCm-OpenCL-Background

[Space Beer]

It would be great if AMD made their own distro (like Clear Linux) with complete ROCm stuff (kernel, drivers, tools, most-used software....)

[boxie]

It would be great if AMD made their own distro (like Clear Linux) with complete ROCm stuff (kernel, drivers, tools, most-used software....)

you might think that, but I would be more than happy for them to keep their limited resources aimed directly at getting their stacks upstream and up to scratch

[Ground0]

It would be great if AMD made their own distro (like Clear Linux) with complete ROCm stuff (kernel, drivers, tools, most-used software....)

I did not hope they do that. I hope they take a Distribution (One of them who they donate resp. they are a official sponsor ;-) ) add a repo with her special stuff who is not mainlined and push it this way. Another Distribution for special case is mostly shit, why have a modular distribution and don't use it, else do another distribution like 100'000 are available ?

[tiwake]

OpenCL: Torch-CL, Gromacs

Gromacs is the core of code that Folding@Home uses for simulating proteins. It would be nice if fahbench was added to the regular OpenCL benchmarking lineup. :3

[Michael]

Gromacs is the core of code that Folding@Home uses for simulating proteins. It would be nice if fahbench was added to the regular OpenCL benchmarking lineup. :3

I had problems with fahbench last time I tried, though off hand do not remember what the issue was.

[darkbasic]

Gromacs is the core of code that Folding@Home uses for simulating proteins. It would be nice if fahbench was added to the regular OpenCL benchmarking lineup. :3

+1 (stupid character limit)

[Meteorhead]

Could someone enlighten me: what in the name of God is up with all the driver stacks of AMD???

fglrx
AMDGPU-PRO
ROCm
AMDGPU
RadeonSI
RADV
AMDKFD

And also a definitive answer to where Mesa enters the picture would be nice.

I understand that fglry is the discontinued, closed-source driver of AMD that is pretty much abandoned. It is superseeded by AMDGPU-PRO that incorporate most of fglrx with some of the open-source stuff. AMDGPU and RadeonSI are both totally open-source drivers to different architectures? Do they overlap? RADV is an unofficial Vulkan implementation that is built atop... what exactly? Does it have it's own driver back-end? ROCm is the brave new world of compute driver stacks. Then how in the blazes did Vega appear with an AMDGPU-PRO driver (that SOLELY supports Vega and nothing more)??!! Sure ROCm support is inbound, but how come initial support comes to the hybrid stack before the newer stack? AMDKFD is a kernel module which ROCm depends on but cannot be mainlined until DC/DAL makes it into the kernel, which seems like an eternity, all because someone forgot to tell AMD folks that a proprietary HAL has nothing to do in the Linux kernel. And which of these shares anything with the Windows kernel? If there's actually a HAL in place in any of their drivers, how Windows does not recieve any of the ROCm technology and remain a second rate computing/development platform?

This all seems to me like a Brazilian soap opera where Julio Cezar Alvarez is trying with all his might to crush every effort of Antonio Miguel Diegos efforts in creating a working driver stack.

[haagch]

Ah, I was wondering about ethereum mining performance on ROCm. On clover it's relatively slow (~1/3 the hashrate it should get) and causes GPU faults and doesn't work while X is running at all. https://bugs.freedesktop.org/show_bug.cgi?id=101584

The OpenCL stack from amdgpu-pro doesn't even mine at all, while having similar problems; I only tested on the mainline kernel though, so on the proper hybrid driver it might fare better.

I hope ROCm receives enough attention so they can progress as quickly as e.g. Intel's beignet.

[bridgman]

Could someone enlighten me: what in the name of God is up with all the driver stacks of AMD???

fglrx
AMDGPU-PRO
ROCm
AMDGPU
RadeonSI
RADV
AMDKFD
...
I understand that fglry is the discontinued, closed-source driver of AMD that is pretty much abandoned. It is superseeded by AMDGPU-PRO that incorporate most of fglrx with some of the open-source stuff.

Your list includes both driver stacks and driver components; that is why it seems confusing. There are three "released" stacks (where we release matched sets of components together) and one we call "all-open" which is currently the set of open source components maintained in upstream projects.

Released stacks - Catalyst Linux (aka fglrx) / AMDGPU-PRO / ROCm -- with AMDGPU-PRO replacing Catalyst Linux as you say. Strictly speaking "fglrx" is the kernel driver of the Catalyst Linux stack but a lot of people used fglrx as a name for the full stack.

All-open stack - radeon or amdgpu kernel driver (newer HW uses amdgpu) plus libdrm plus Mesa plus an X driver (radeon/amdgpu/modesetting), plus OpenCL/Vulkan userspace drivers as we open source them and upstream the associated kernel code - assembled by distro packagers for each distro release

And also a definitive answer to where Mesa enters the picture would be nice.

The radeonsi and radv drivers plug into Mesa - the Mesa project includes code for things like OpenGL API support which is used by all HW platforms, in conjunction with lower level HW-specific drivers like radeonsi.

AMDGPU and RadeonSI are both totally open-source drivers to different architectures? Do they overlap?

RADV is an unofficial Vulkan implementation that is built atop... what exactly? Does it have it's own driver back-end?

amdgpu is a kernel driver; radeonsi is a userspace driver which plugs into Mesa. The two drivers (amdgpu and radeonsi) work together, although there are other smaller components connecting them including "winsys" and libdrm-amdgpu.

The radv driver also plugs into Mesa and runs over the amdgpu kernel driver.

ROCm is the brave new world of compute driver stacks. Then how in the blazes did Vega appear with an AMDGPU-PRO driver (that SOLELY supports Vega and nothing more)??!! Sure ROCm support is inbound, but how come initial support comes to the hybrid stack before the newer stack?

The two stacks came out a couple of days apart IIRC - June 27 for AMDGPU-PRO 17.20 and June 29 for ROCm 1.6. Just different release dates - the 17.20 stack was the one which had to align with Vega FE board launch.

AMDKFD is a kernel module which ROCm depends on but cannot be mainlined until DC/DAL makes it into the kernel, which seems like an eternity, all because someone forgot to tell AMD folks that a proprietary HAL has nothing to do in the Linux kernel.

Um... no. There is no connection whatsoever between amdkfd and DAL/DC other than the fact that they both have some connection to amdgpu. The DAL/DC code simply replaces basic display code written by our open source graphics team with more feature-rich code written by our display team.

The amdkfd code for dGPU could not be upstreamed initially because it allowed pinning from userspace (pinning on allocation to be precise) in order to align with existing programming models. We recently implemented a working "eviction" mechanism for amdkfd which allowed us to provide the illusion of pinning from userspace for amdkfd userspace code while ensuring that regular graphics userspace drivers & apps would not run out of physical memory as a consequence.

Now that the eviction code is in place we are going ahead with upstreaming the latest amdkfd code for dGPU. The amdkfd code for APU has been upstream for years.

And which of these shares anything with the Windows kernel? If there's actually a HAL in place in any of their drivers, how Windows does not recieve any of the ROCm technology and remain a second rate computing/development platform?

DAL/DC (aka "display") is the only kernel driver code shared with other OSes/platforms, although we do share userspace driver code between OSes/platforms in the case of the closed source OpenGL and Vulkan drivers. The DAL/DC code we upstreamed was actually written first for Linux.

The HSA/ROCm stack was initially developed for both Windows and Linux but there was essentially no perceived customer interest in the Windows implementation and as a result we focused primarily on Linux... IOW our customers seemed to feel Windows was a second-tier computing/development platform long before ROCm came along.

We will probably end up having to support the compiler/library code from the ROCm stack on Windows anyways but in most cases it seems to be a "checkbox" requirement rather than something that customers plan to use for large-scale computing. Microsoft has also recently announced their own Windows ML API, but at first glance it appears to be more focused on inference with already-trained models than on the more compute-intensive training work which will probably continue to focus on Linux.

This all seems to me like a Brazilian soap opera where Julio Cezar Alvarez is trying with all his might to crush every effort of Antonio Miguel Diegos efforts in creating a working driver stack.

I haven't had a chance to watch any Brazilian soap operas (although in fairness I don't think I have watched any non-Brazilian soap operas either) so I can't really comment, but if you think about the stacks in the following way things will probably make more sense:

AMDGPU-PRO stack - aimed at traditional CAD workstation market, replacing Catalyst Linux

ROCm stack - aimed at HPC, generally headless server systems

all-open stack - covers pretty much everything else including desktop/gaming etc..

The AMDGPU-PRO and ROCm stacks are growing together over time - both of them will continue to include some code which is considered non-upstreamable, eg the Kernel Compatibility Layer code which allows a single kernel driver source tree to build and run against a variety of kernel versions.