GCC vs. LLVM Clang Is Mixed On The Ivy Bridge Extreme
Phoronix: GCC vs. LLVM Clang Is Mixed On The Ivy Bridge Extreme
Our latest Linux benchmarks from the Intel Core i7 4960X Ivy Bridge Extreme Edition processor are compiler tests on this $1000 USD processor. The last two stable releases of GCC and LLVM's Clang C/C++ compilers were compared: GCC 4.7.2, GCC 4.8.1, LLVM Clang 3.2, and LLVM Clang 3.3.
The Botan tests seem to be missing "-march=native".
What's really bad with the Botan tests is that they use -O2 which makes those tests worthless when it comes to comparing compiler vs compiler code performance. As I've explained to Michael over and over, there are no rules for which optimizations a compiler should add at -O2, meaning that if compiler A decides to enable more optimizations than compiler B at that level, A will win which in turn says nothing of how the compilers compare when set to their HIGHEST optimization level which is what you use when you want the FASTEST code, which again is what all tests except the 'build time' test here was measuring.
Originally Posted by s_j_newbury
Again, this is why you compare the compilers at their highest optimization level (-O3, and when it was used GCC won all performance tests) because at this level the compilers compete on fair ground, which is to create the fastest binary they can. Now -O2 can be interesting as sometimes -O2 beats -O3 due to optimization heuristics failing, but only in the context of actually having -O3 results to compare with.
I don't know why Michael persist at doing this, unless he is consciously trying to cook the results in order to get some pointless Clang/LLVM wins.
I dont think Michael is cooking the books.
Perhaps he went with o2 because most people use it as default?
He is comparing the optimization between two compilers. Again, -O2 on one compiler has nothing to do with -O2 on another other than name and overall goal, it's just a loose balance the respective compiler developers choose between optimization and compilation time for -O2 on THEIR compiler, thus it's absolutely pointless to compare those when measuring the best performance the compiler can get out of a certain piece of code (which is what all tests but one did here).
Originally Posted by steverweber
Meanwhile -O3 is standard highest optimization level between these compilers, here compilers say -'never mind compile time, enable all the appropriate optimizations to create the fastest code we can', which means that this option it the only one worth using when comparing code performance (again what is done here) UNLESS you compare both -O2 and -O3 to see if this particular test is one of the few where -O2 beats -O3.
So as it stands this test is pointless in terms of which compiler can generate the fastest code for Botan, as we won't know that until he does this test using -O3.
Unlike you I'm leaning more and more towards Michael's well known Clang/LLVM bias being the reason for throwing in tests using -O2, as I can't see any other logical explanation as to why he continues to do this.
Now the Botan tests are worthless from a compiler vs compiler performance standpoint, which is sad because it would be interesting to see where CLang/LLVM stands against GCC in those tests aswell.
Don't forget, for gcc there's also:
Disregard strict standards compliance. -Ofast enables all -O3 optimizations. It also enables optimizations that are not valid for all standard-compliant programs. It turns on -ffast-math and the Fortran-specific -fno-protect-parens and -fstack-arrays.
Where in the wild do we actually see O3 though?
Originally Posted by XorEaxEax
Practically all performance oriented software, like encoders, games, archivers/compressors, emulators, 3d renderers etc
Originally Posted by WorBlux