Another issue is if you set the quality to 720p or higher, it downloads slow because if I understand correctly (I'm making an educated guess here), google isn't allocating as much bandwidth for the webm beta as their flash video.
Sounds like an issue which might be solved by buffering more of the file at once. Should be possible to sort out on the client end.
Maybe. I've tried just pausing the video and waiting and the buffering didn't speed up. Firefox still has some performance issues that are currently being tweaked, but they've made AMAZING progress. They're already much better than flash on my computer. The downloading is the only issue that I experienced when I last tested it myself (and some youtube bug where full screen didn't do full screen but "full browser"). For all I know though, they could be fixed because I haven't tested it in a month or so.
Maybe. I've tried just pausing the video and waiting and the buffering didn't speed up.
It might also be some idiotism in the player that it doesn't buffer the whole file if you keep it on pause... That's at least one thing Adobe has done Right; with Flash videos all you need is a bit of patience and then it doesn't matter how much bandwidth upstream gives since you have the whole file locally anyhow.