I've been reading around here for quite a bit, trying to get my HD5550 to work under Linux with XBMC vaapi enabled. I tried a number of suggested combinations (different xorg and catalyst versions), but in the end I also always ended up with the garbled screens as seen here in the screenshots.

After some more searching on this forum, the conclusion seemed to be that XvBA support is "broken" for the Evergreen ATi GPU series (HD5xxx). Until now I guess .

I decided to install everything from scratch today, and was surprised to see catalyst 10.12 being released - eventhough the link still said 10.11.

Here is what I ended up with:
  • Ubuntu 10.10 AMD64 ("Maverick")
  • xvba-video_0.7.7.pre2-1 (AMD64)
  • libva 0.31.1-1+sds4 (AMD64)
  • XBMC PRE-10.5 r35635 (--enable-vaapi)
  • Ati Catalyst 10.12 (fglrx-8.801)

Result? (screenshots are from an antique CRT monitor)

No more distorted decoding. Very low CPU load (averages from 8 to 9%, with 10 to 11% peaks). Perfect image quality. I have not tried and tested lost of different videos, but so far the results seem great!

Curious what others will get for result!