GCC performance testing request

[birdie]

Hello, phoronix guys!

Can you PLEASE test these versions of GCC: 4.2.4, 4.3.4 and 4.4.2 using whatever benchmarks you like (the more the better).

I suggest running at least these benchmarks:

Multimedia
1) LAME compression/decompression (mp3)
2) FLAC compression/decompression
3) VORBIS compression/decompression (ogg)
4) x264 compression/decompression (H264)

Web
1) Firefox page rendering time/miscellaneous JS benchmarks
2) Chrome
3) Aurora (rebuild Qt as it's based on Qt's WebKit component)

Compression
1) GZIP/BZIP2/7-ZIP compression (of big enough TAR archive)
2) If you manage to compile e.g. GCC 4.2.4 using all these compilers (which is difficult, since GCC rebuilds itself as a part of a compilation process), then please test compilation times of e.g. heavy C code (say entire kernel 2.6.30 tree) and some heavy C++ code (say Qt library)

Games
1) Quake 2 performance when running in software rendering mode
2) Quake 3 performance using software GL mode (say, running in VESA X.org server), MesaGL library should be rebuild using different compilers, of course

Rendering
1) Blender rendering time
2) PovRay rendering time

Please try to avoid testing IO bound applications since those results will be meaningless.

Thank you!

[birdie]

And if you have enough time, please, test GCC 3.4.6 and ICC 11 (in this case your CPU must be Intel Core 2 or higher, as Intel compiler is known to pessimize code for AMD CPUs).

[birdie]

If you haven't yet started testing, please, use the same CFLAGS/CXXFLAGS across all compiled applications.

I suggest using this set:

-O2 -march=native -pipe (for GCC >= 4.2)

and

-O2 -march=pentium4 -pipe (for GCC 3.4.6 if you are going to use it).

all other flags can be detrimental or bring unnecessary noise to results.

[birdie]

Are you going to run these tests?

[StringCheesian]

Probably he'd want to publish benchmarks like that in response to some news. Like when gcc 4.5 is released.

The next time something happens in the world of compilers, bring it to his attention and say it calls for benchmarks like the ones you want. It could work.

[Ant P.]

It'd be nice to see a benchmark with and without the -omg-rice CFLAGS, just to see how much difference it makes compared to Ubuntu's i686 binaries. Total size of /usr/{lib,bin} would be interesting to see too.

I've noticed a few people saying the new ones in GCC4.4 actually make a difference.

[svrocket]

if u really want to see something dramatic, run your own bake-off between gcc 2.95, 3.2, and 4.x.

yes, even gcc gets more strick, more bloated, and slower over time. All software suffers from Creeping Featureitus.

[birdie]

You could at least answer to this thread.

"No" will qualify just fine.

[bnolsen]

And if you have enough time, please, test GCC 3.4.6 and ICC 11 (in this case your CPU must be Intel Core 2 or higher, as Intel compiler is known to pessimize code for AMD CPUs).

Seems like you just gave every reasson needed to not include icc in this specific test since it then would fail as a good general compiler. Writing a sepaprate article about icc amd compatibility itself might be interesting.

[bnolsen]

If you haven't yet started testing, please, use the same CFLAGS/CXXFLAGS across all compiled applications.

I suggest using this set:

-O2 -march=native -pipe (for GCC >= 4.2)

and

-O2 -march=pentium4 -pipe (for GCC 3.4.6 if you are going to use it).

all other flags can be detrimental or bring unnecessary noise to results.

probably nocona for 64bit. At this point 32bit testing should be nothing more than a sideshow.