Yeah I know it's an evil thing to do, but I was tired and thought "what the heck, it's broken anyway...."
Of course, now I can look back through the install logs and see what was changed to make it work. Then find out how to put my system back in order again.
-- yes, that is a major drawback with the nvidia drivers. That's what they get for replacing bits of X!