I hope Michael checked for every compilation that the right flags were used. Some parts of pts did not care about CFLAGS exports and compiled unoptimized crap on amd platforms.
it would be more interessting compare that hand written asm with gcc generated code. Unless that is done there is no reason to turn off assembly just to create a testcase that is completely detached from reality.
Well, despite if -O2 beats -O3 in some tests or not, -O3 IS the optimization which is supposed to optimize the best so it's obviously the one to use in a benchmark (unless you are benchmarking across all -O levels). As for -O3 being buggy, it's not from my experience nor is it supposed to be anything but stable.
Optimizations that are not considered fully working are introduced as separate flags, not into one of the -O levels. If/when they are considered stable (as in actually improving code and not introducing bugs) they are often added to certain -O levels. Some optimizations like for instance -funroll-loops have been around for ages but are not part of any -O level simply because it's very difficult for the compiler to estimate unrolling and thus there can be great gains aswell as great regressions using this optimization. (Although it's turned on by default if you use PGO in which case the compiler has enough data gathered to guarantee making good judgements).
For the absolute best results though you'd most likely need to run something like Acovea (http://www.coyotegulch.com/products/...o5p4gcc40.html) which omits the -O levels and tests all the flag combinations, but it's not very practical.
I quite like the graphics horizontally, looks good.
But I'd had appreciated a better choice of colors. Something of a similar tint for the GCC stuff and separate tints for the others. I found myself scrolling up/down a couple of times to check which color is which compiler a couple of times.
While it certainly would also be interesting seeing how much better hand optimized assembly does against the code generated by these compilers it's not part of THIS benchmark. So yes, there's every reason to disable hand-optimized assembly here.
What sort of differences do you get with Mame and Handbrake (I guess you mean x264 in this case)?