A Linux Compiler Deathmatch

[phoronix]

Phoronix: A Linux Compiler Deathmatch

Started by one of our readers more than a week ago was a compiler deathmatch for comparing the performance of GCC, LLVM Clang, PCC (the Portable C Compiler), TCC (Tiny C Compiler), and Intel's C Compiler under Arch Linux. This user did not stop there with compiling these different x86_64 code compilers, but he also went on to look at the compiler performance with different compiler flags, among other options. The results are definitely worth looking at and here are some more.

http://www.phoronix.com/vr.php?view=15657

[Thatguy]

Is anyone benchmarking the quality of the output in terms of execution and code quality and stability ?

More of a curiosity.

[korpenkraxar]

It would be interesting to see how the Intel compiler alternative compares these Free ones.

[ChrisXY]

If you want to test the speed of the resulting binaries, PLEASE test also what the compiler is able to do and not only what it does in default configuration. Please include maximum optimization in the benchmarks.

[FireBurn]

The Arch Linux benchmarks were a lot more interesting

If this was a deathmatch, who won?

[HokTar]

If it's not a too big problem, could you please run a set of FORTRAN compiler tests, too? Please include ifort and gfortran for sure!

The -O2 -march=native options seem to be the most interesting...

Thank you!

[chithanh]

Please add a comparison on ARM too. And consider benchmarking at different optimization levels (eg. -Os, -O2 and -O3 for gcc)

[Mo6eB]

Michael, unless you tell the compiler to optimise, it won't optimise. These results are completely useless. It's like testing how fast different graphics cards render with the Mesa software rasteriser.

The interesting results would be to see what effect different optimisation levels (-O2, -O3, -Os, plus compiler-specific features) have on the final executable. This will help individual developers and distro packagers to pick the best compiler options for specific programs. This would need about 5-6 different configurations per compiler and most of us don't have the time to do it at home, so we would be very grateful if you do a proper benchmark.

[mat69]

Completely useless test.
You can't compare compilers with their default settings as they have different optimizations by default.

So what should have been done is to compare the different optimization settings of the compilers.

[RealNC]

"All compilers were tested in their "out of the box" configuration without specifying any extra flags."

This is just brain damaged. What a waste of time and resources. Seriously, this is from the same guy who *wrote* the Phoronix benchmarking suite?

I honestly can't believe this. Seriously. Could Michael please explain what's the point in this?