Upgrading would not take much time since I have a ridicilously sophisticated configuration back-up and management system, but who knows what other snags I will hit. I already have to keep the kernel pinned to v3.4 otherwise nouveau regresses while hibernating. I bisected the commit, but since it was part of the huge nouveau rewrite (read: breakage everywhere for anything before nv50) it's hard to pin down.
XFCE takes 40MB (tops!) more than LXDE, and that is not much for a 512MB machine. Even the DE is 'slimmed', so conventional benchmarks probably don't apply to this specific situation. I.e. there is no gconfd or networkmanager running anywhere.
there is definitely a place for small lightweight linux distros which have simplified window managers and a set of core graphical libraries.
think tomtom, maemo, even android itself (though now Arm performance is very good and memory is cheap it's not so lightweight)!