In reality, you can do it right now. The SH metaprogramming language will allow you this ability with the current NVidia, ATI, and Intel drivers. The problem with this being that: With the Intel drivers there's a dearth of the parts that will buy you something and the support for that functionality is still in it's infancy. With the currently shipped (not the about to ship drivers or the soon to happen open source driver...) ATI drivers, you might as well not bother because unless you've got a X1900XTX or comparable, you won't have enough speed to have bothered with in the first place- because the drivers are so damned slow at rendering things. Now, this is under Linux. The problem only really applies to Intel parts right at the moment under Windows- but Windows is hampered by other performance concerns (bad designs for supercomputing because it's more of a consumer desktop OS than a server/embedded OS with a slick user interface layered on top...).
Essentially these GPUs things are like CPUs, aren't they? To me this is something that simply cannot be done nearly as well under Windows due to the static closed-source nature of the OS.