gpu guru guidance

[thehighhat]

I am looking for help with a setting up a computer that has both AMD and nVIDIA GPUs.


Lots of reading and tweaking xorg.conf and driver installations... all w/o success. Can any of you GPU gurus help?

Simple case:

Linux x86_64 kernel
nvidia GTX2xx GPU
ati 48xx GPU
mobo with many PCIe x16/x16 slots and both cards installed

nvidia cuda/opencl 2.3+ enabled drivers
amd stream SDK 2.0+ drivers


Neither AMD nor nvidia seem to be able to help with this.


Single driver installs are ok for each card.


Installing both nvidia and fglrx driver does not work, apparently due to libglx.so and libGL.so conflicts.


Thoughts?

[Kano]

That's no simple case. As soon as you install binary drivers you already lost the other card. I would NEVER mix em in one system, use 2. In theory you could create an extra dir with all needed libs for each card and setup LD_LIBRARY_PATH for every app depending on gpu, but usually that's not worth the effort.

[thehighhat]

That's no simple case. As soon as you install binary drivers you already lost the other card. I would NEVER mix em in one system, use 2. In theory you could create an extra dir with all needed libs for each card and setup LD_LIBRARY_PATH for every app depending on gpu, but usually that's not worth the effort.

true.. it's not simple, but the target case involves even more complexity.


yes, the nvidia driver installer has options for placing files in non-default locations (eg libglx.so in /usr/local/lib instead of /usr/lib/xorg/modules/extensions). and with the xorg.conf file modulepath option, this should be fine. however, xorg balks at this and i cannot get accelerated graphics (access to hw for opencl/cuda), even with LD_FLAGS. perhaps it is a misconfigured xorg.conf file.

Also, it is unclear that even if i could get this (non-default install locations) to work on one card... once i install the other driver, how will the applications properly resolve with libraries to use?

nvidia has libGL.so and libglx.so in /usr/local/nvidia/lib
ati has libglx.so in /usr/lib/xorg/modules/extensions

/etc/ld.conf has paths listed

/etc/X11/xorg.conf has modulepath listed



no go!


nvidia complains about missing libwfb.so even though it is clearly visible (actually, it complains about missing symbols when trying to start xserver)



thoughts?

[Kano]

If you need 2 cards, then use 2 from the same vendor and don't spend time for stupid hacks. You could install a 2nd xserver at a different postition too to run 2 Xservers, but do you REALLY think such a hacked system is good for testing anything? You could never use the normal way to update drivers - nvidia installer searches are files to replace and removes em, so you could basically write your own install script. Have fun doing so.

[thehighhat]

If you need 2 cards, then use 2 from the same vendor and don't spend time for stupid hacks. You could install a 2nd xserver at a different postition too to run 2 Xservers, but do you REALLY think such a hacked system is good for testing anything? You could never use the normal way to update drivers - nvidia installer searches are files to replace and removes em, so you could basically write your own install script. Have fun doing so.

it sounds like an awful thing to do, but there is a good reason.

first, multi-platform development can be facilitated by having all of the hardware in one machine.

second, building a release application that must detect available hardware and launch compute kernel that match.


believe me, i defintely dont want to have a uber-fragile monstronsity of a workstation...


but i have considered that it may be a case of me not having the know-how to make this happen.

[Kano]

Just get a 2nd board, much cheaper than investing time in that.

[thehighhat]

Just get a 2nd board, much cheaper than investing time in that.

true. true.

i have wasted way too much time chasing this.

reminds me of my gentoo days... ahahhaha...

thanks for the feedback. i needed to hear a voice of reason.

[rohcQaH]

the simplest way is to install a second userspace in a chroot. Kernel modules go into the (shared) kernel, one userspace driver in /, the other userspace driver in the chroot.
Sure, you'll need to maintain two installations, but you'd need to do so anyway when buying a second box.

If the kernel modules are compatible, that'll work. Installing the kernel module of the chroot-driver without messing up userspace is a bit more trouble, though.

it's all a bit hackish, but if you survived gentoo, you can do it