There are lots of improvements which could happen with the current 5xx 3d implementation (you can read what nh_ and MostAwesomeDude are doing on #radeon or #radeonhd irc at radeonhd.org); lots of changes in just the last couple of days.
As to "how far can it go" it's probably approaching the point where it makes more sense to jump onto Gallium and build there, but that's not a hard, clear line -- just a point where you start to suspect that you're inventing something which Gallium already has.
I expect 5xx will go a lot further, certainly to GL 2.x, just not sure it makes sense to get there on the current code base. Once we get some 6xx 3d to the same level that 5xx is today (yesterday, I guess; it's further today

)
I expect we'll mostly jump onto memory manager and Gallium.