Your best bet is to start reading existing code and hardware documentation and asking questions. That's pretty much how everyone gets in the driver programming.
Thx guys. I'll try reading some galium3d stuff.
I got some advices from bridgman on irc and I might try doing some hardware decoding via shaders implementing galium3d calls in ffmpeg. Then I don't have to implement all the non-parallizable code.