Well, I can't see how relying in the sheer absence of bugs for stability would be good practice under any circumstances. Preventing a problem is just as important as being able to deal with it once it arrives(cause it usually will).
An approach closer to a micro-kernel regarding the drivers would seem better in my opinion at least. And yes, I've gotten multiple kernel panics/crashes with at least nouveau lately ..

The micro-kernel part I'm talking about is kind of like Windows handles it. It would probably need more changes than just the kernel though, to do the graphics driver part as well though(being able to reload the driver without [basically] rebooting, since it seems Xorg will crash anyway, so all apps will too).