Benchmarking LLVM & Clang Against GCC 4.5

[mlau]

I find it funny that the first benchmark result in these kinds of articles is always the 'time to compile'.

Sure it's nice if the compiler can build things faster, I personally care more about if the resulting binary performs better, and don't mind double compile times!

Well you only compile the source once, the developers do countless
times a day. (on the other hand: http://xkcd.com/303/ )

I remember compiling kernels on a 486SX which took 2 hours easy!

Yeah, those were the days ;-)

[oliver]

You would assume so, being a dev myself however, I compile several things 100s of times.

Luckly, most of the time you only have to rebuild 1 o and relink the final binary; so not an issue in that regard, also -O0, if decent fast would be fine for that situation as well, increasing optimizations can take longer and longer without much issue imo though

[mlau]

You would assume so, being a dev myself however, I compile several things 100s of times.

Luckly, most of the time you only have to rebuild 1 o and relink the final binary; so not an issue in that regard, also -O0, if decent fast would be fine for that situation as well, increasing optimizations can take longer and longer without much issue imo though

True, noone really rebuilds large codebases a few times a day, but
even a slight reduction in build time can be advantageous if you're
pressed for time.

LD can take _ages_ to link larger binaries (I've seen it take ~2 hours to link QtWebkit on MIPS and then bail out with a nonsensical "Bad value" error).

[Craig73]

Michael,

It might have been interesting to also include results for previous versions of Clang/LLVM-GCC... to see how they are improving. IE, they might not be as fast as GCC in some cases, but Clang 1.1 might be significantly better than Clang 1.0.

Also, could you leave the space for the bar there when it fails. (Perhaps a red bar just high enough to say "fail", so that it's easier to visually skim the results as the graphs would be consistent).

[dvohwinkel]

Not to bash Michael, benchmarking is hard...

Wouldn't it make sense to actually compile programs using the new power of the compiler? If he did a standard "make all" he didn't take advantage of any of the new optimizations available in gcc 4.5.0. I would love to see an article where we see what these new features/optmizations can do.

[toxygen]

Thanks for the article, however, I would welcome some comparison to industry standard compiler e.g. intel compiler or ibm xlc.
Just a thought...

[darthscsi]

Not to bash Michael, benchmarking is hard...

Wouldn't it make sense to actually compile programs using the new power of the compiler? If he did a standard "make all" he didn't take advantage of any of the new optimizations available in gcc 4.5.0. I would love to see an article where we see what these new features/optmizations can do.

Likewise none of the whole program optimizations were done on the llvm side. (At least I didn't see anything about compiling at O4 and using the llvm plugin for gold)

[Armorix]

True, noone really rebuilds large codebases a few times a day, but
even a slight reduction in build time can be advantageous if you're
pressed for time.

LD can take _ages_ to link larger binaries (I've seen it take ~2 hours to link QtWebkit on MIPS and then bail out with a nonsensical "Bad value" error).

At work, we build automatically our large project when people check in, and we also perform a clean build every time we push a set of changes to be tested by QA (a few times a day). Improving build times does improve our productivity.

I do agree, however, that what is more important is the performance of the end-product.