I also noticed that with Opengl applications and AIGLX I could play full screen (like 1280x960 or so) Return to Castle Wolfenstien over (my faster home network) 100Mb/s connection, even with ssh encryption. Like 20-40fps. Blender worked and all that. This was a while ago when AIGLX first came out. Over the same connection Gnome and KDE apps had more noticable lag.
If like your saying that X apps are barely using X aymore, or at least using it less and less, then it may be a good time to redesign X protocol to make it easier to compress and avoid the ping-pong type client-server behavior that causes lots of performance issues over higher latency connections.