Well PA faced the similar problem with drivers that Network-Manager faced.
PA depends on correct drivers to function.
Traditionally with Linux if your audio sounded like shit then you had to mess around with asoundrc files and things like that.
The reason the audio sounded like shit was that the audio drivers sucked and were doing bad things. Messing around with buffer settings and other things in asoundrc was just people putting a lot of effort into working around driver bugs that never were getting fixed.
but this is retarded because we are not using Windows... we are using Linux. We have access to the source code and the drivers should be fixed rather then forcing users and applications to work around them.
This is one of the big issues with PA early on. It was running into all sorts of problems and instead of working around the bugs like everybody has been doing they fixed the drivers.
It takes the driver fixes a while to get back to the users...