Page 1 of 5 123 ... LastLast
Results 1 to 10 of 49

Thread: CompilerDeathMatch: surprising results

  1. #1
    Join Date
    Jan 2010
    Location
    Ghent
    Posts
    208

    Question CompilerDeathMatch: surprising results

    Hi all

    I have played around a bit with comparing various compilers available to ARCH
    First off, I have been running the Free compilers for 64-bit:

    http://global.phoronix-test-suite.co...07-15267-29041

    TODO:
    - I will also add ICC and make an AUR for open64 and LCC (builds fail atm)
    - 32bit tests, include SolarisStudio and openWatcom

    The weird thing about these results is that it seems like TCC is beating all of the other compilers handsomely (I would have expected GCC to be the performance choice) and also PCC gives surprisingly good results.

    I exported
    CC=/path/to/compiler
    CXX=/path/to/compiler
    and did not add any extra flags (planning on trying -x c for PCC for GCC compatibility mode)

    I had to manually edit the xml file to set the compiler info for system-info correctly for each set up in the comparison (I noticed that too late, hence the "spamming" of global.phoronix.com)

    Is there something I have done wrong or are these results for real???

  2. #2
    Join Date
    Apr 2010
    Posts
    1,946

    Default

    Awesome idea!

    Indeed, from TCC homepage
    FAST! tcc generates x86 code. No byte code overhead. Compile, assemble and link several times faster than GCC.
    Compilation Speed:...
    TinyCC is about 9 times faster than GCC.
    I wonder if it is possible to compile the kernel with tcc and if it will bring any benefits.

  3. #3
    Join Date
    Jun 2010
    Posts
    10

    Default

    The really interesting part is how tcc-compiled executables seem to be faster than the gcc-compiled ones. The tcc documentation doesn't list any flags for optimisation, so my hunch is that tcc optimises more aggressively than gcc by default. Could you run the benchmark again, but set CFLAGS to -O2?

  4. #4
    Join Date
    Jan 2010
    Location
    Ghent
    Posts
    208

    Default

    Quote Originally Posted by crazycheese View Post
    Awesome idea!

    Indeed, from TCC homepage




    I wonder if it is possible to compile the kernel with tcc and if it will bring any benefits.
    Apparently one used to be able to compile linux with TCC, or even run it "just in time" directly from C-code.

    No idea whether this still works.

    It will be interesting to see if the trend stays the same for 32bit and how the compilers compare to Open64, LCC, openWatcom and non-free ones like ICC and SolarisStudio. If I am able to, I would also like to package and test KenCC. To really make a real-world comparison I guess different optimization flags, different hardware etc needs to be compared though...

  5. #5
    Join Date
    Jan 2010
    Location
    Ghent
    Posts
    208

    Default

    Quote Originally Posted by Mo6eB View Post
    The really interesting part is how tcc-compiled executables seem to be faster than the gcc-compiled ones. The tcc documentation doesn't list any flags for optimisation, so my hunch is that tcc optimises more aggressively than gcc by default. Could you run the benchmark again, but set CFLAGS to -O2?
    Sure I can try that too, although that will have to wait a bit I am afraid (I run the benchmarks over night). People are ofcoure welcome to repeat the experiments with other settings With more independent analyses, the comparisons will be more reliable.

  6. #6
    Join Date
    Aug 2007
    Location
    Europe
    Posts
    401

    Default

    Quote Originally Posted by staalmannen View Post
    I exported
    CC=/path/to/compiler
    CXX=/path/to/compiler
    and did not add any extra flags (planning on trying -x c for PCC for GCC compatibility mode)
    Excellent work!

    If you find the time, play around with "Acovea - Using Natural Selection to Investigate Software Complexities" (http://www.coyotegulch.com/products/acovea/) to see which compiler flags may be the best!

  7. #7
    Join Date
    Feb 2009
    Location
    France
    Posts
    307

    Smile

    Quote Originally Posted by staalmannen View Post
    TODO:
    - I will also add ICC and make an AUR for open64 and LCC (builds fail atm)
    Hey, good to see you want to test open64 but it's intended to be replaced by path64. I made an AUR package for path64 (https://aur.archlinux.org/packages.php?ID=41846). Hope you'll be able to test it If there is a problem with the package, please tell me

    Quote Originally Posted by staalmannen View Post
    Is there something I have done wrong or are these results for real???
    Check the level of optimization was the same across all the compilers, there is something really *wrong* here, but I'm not a compiler guy at all.

  8. #8
    Join Date
    Jan 2010
    Location
    Ghent
    Posts
    208

    Default

    Quote Originally Posted by MPF View Post
    Hey, good to see you want to test open64 but it's intended to be replaced by path64. I made an AUR package for path64 (https://aur.archlinux.org/packages.php?ID=41846). Hope you'll be able to test it If there is a problem with the package, please tell me


    Check the level of optimization was the same across all the compilers, there is something really *wrong* here, but I'm not a compiler guy at all.
    Thanks for the tip about Path64 - I was interested in trying the AMD branch as well. I will try them both I made an AUR from the binary open64.net distribution (https://aur.archlinux.org/packages.php?ID=45558).

    So for 64-bit the TODO now includes: ICC, Open64, Path64, and possibly LCC
    and after that I can try some CFLAGS, and for those that have it, alternative LD and AR etc...

    For 32-bit there are lots of things left to do, but also a number of extra compilers available that look interesting...

    The annoying thing is that it basically takes a whole night to run one of these tests on my laptop so the results will not come directly. I hope you guys are patient
    I will post updates here as they come.

  9. #9
    Join Date
    Jan 2010
    Location
    Ghent
    Posts
    208

    Default

    update: Included ICC

    http://global.phoronix-test-suite.co...3426-6741-3877

    did even worse than the rest, which really highlights that it will be interesting to compare when various optimizations are set.

    next up: open64 and then there will be aggressive optimizations set (-O3) and perhaps later more moderate ones.

  10. #10
    Join Date
    Jul 2008
    Location
    Greece
    Posts
    3,791

    Default

    You cannot do "benchmarks" without optimization, because it tells us absolutely nothing. All the results are simply wrong.

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •