Results 1 to 10 of 10

Thread: GCC 4.6, LLVM/Clang 3.0, Open64 Benchmarks

  1. #1
    Join Date
    Jan 2007
    Posts
    14,900

    Default GCC 4.6, LLVM/Clang 3.0, Open64 Benchmarks

    Phoronix: GCC 4.6, LLVM/Clang 3.0, Open64 Benchmarks

    For those interested by last week's articles on the AMD Bulldozer benchmarks of the GCC, Open64, and LLVM/Clang compilers and the FX-8150 compiler tuning, but curious to know how the new LLVM/Clang 3.0 and other compilers perform on other processors, here's some new test results. These tests are looking at GCC 4.6, LLVM/Clang 3.0-RC1, and AMD Open64 compilers on Intel Sandy Bridge and AMD Shanghai hardware.

    http://www.phoronix.com/vr.php?view=16665

  2. #2
    Join Date
    Sep 2011
    Posts
    62

    Default Ekopath

    What about that Ekopath compiler ? After all the fuss when it was open sourced I heard nothing about it. Maybe I missed something but still I'd like to see some results for that one too.

  3. #3
    Join Date
    Sep 2007
    Posts
    51

    Post SunStudio/Solaris studio comparison

    I would like to see comparison side-by-side with Solaris Studio (Formerly SunStudio).

  4. #4
    Join Date
    Jul 2009
    Posts
    221

    Default

    What I find most interesting is that Open64 performs inconsistently while consitently failing to compile on intel platforms.

    Still a big gap between llvm and gcc in one test too.

  5. #5
    Join Date
    Oct 2009
    Posts
    845

    Default

    I've often found the Phoronix test results very different from that of my own. The 7-zip test for instance, it yields very different results from what I get when I use the built in benchmark functionality so I figured I would do a real (as in non-synthetic) test using p7zip on my corei5 machine (nehalem):

    Code:
    GCC		-O1 -march=native		0m36.376s
    GCC		-O1 -march=native		0m36.369s		Profile guided optimization
    GCC4.7		-O1 -march=native		0m35.814s
    GCC4.7		-O1 -march=native		0m35.761s		Profile guided optimization
    Clang		-O1 -march=native		0m42.236s
    GCC		-O2 -march=native		0m35.734s
    GCC		-O2 -march=native		0m33.435s		Profile guided optimization
    GCC4.7		-O2 -march=native		0m35.663s
    GCC4.7		-O2 -march=native		0m33.831s		Profile guided optimization
    Clang		-O2 -march=native		0m39.658s
    GCC		-O3 -march=native		0m34.197s
    GCC		-O3 -march=native		0m33.428s		Profile guided optimization
    GCC4.7		-O3 -march=native		0m34.693s
    GCC4.7		-O3 -march=native		0m33.224s		Profile guided optimization
    Clang		-O3 -march=native		0m39.642s
    These are the results of compressing an .iso file (Arch linux netinstall to be exact) on a core i5, 64bit Arch Linux system using the following compilers:
    GCC 4.6.2-1, Clang 2.9-7, GCC 4.7 snapshot 20111105 (I built the LLVM 3.0 release aswell but it failed compiling stating it was unable to find libc++, I guess I'll wait for the full release to hit the Arch repos for that one)

    Anyway there wasn't much to compare against this time in Phoronix's test since for some reason there was no Clang/LLVM test and Open64 (4.2.4-2) wasn't able to build p7zip on my machine. But as has always been the case in my tests, GCC outperforms Clang/LLVM by a good margin.

  6. #6
    Join Date
    Oct 2009
    Posts
    845

    Default

    Oh and if anyone wants it, here's the (ugly, my bash-script-fu is very weak) bash script I threw together to do the test:

    Code:
    #!/bin/bash
    name=p7zip_
    version=9.20.1
    dir=$name$version
    results=../p7zip-results.txt
    makefile=makefile.linux_amd64
    arg="./bin/7za a -mx9 -m0=LZMA2:d64m -mmt dummy.7z archlinux-2011.08.19-netinstall-i686.iso"
    flags="-O3 -march=native"
    path64flags="-O3 -march=auto"
    
    # uncomment compilers to be tested
    gcc=1
    clang=1
    #gcc47=1
    #path64=1
    
    
    if [ ! -d $dir ]; then
    	echo "Downloading and extracting p7zip..."
    	wget -q http://downloads.sourceforge.net/project/p7zip/p7zip/${version}/p7zip_${version}_src_all.tar.bz2
    	bzip2 -q -d "$dir"_src_all.tar.bz2
    	tar xf "$dir"_src_all.tar
    fi
    
    cd $dir
    
    if [ ! -e archlinux-2011.08.19-netinstall-i686.iso ]; then
    	echo "Downloading iso..."
    	wget -q http://ftp.ds.hj.se/pub/os/linux/archlinux/iso/2011.08.19/archlinux-2011.08.19-netinstall-i686.iso
    fi
    
    
    if [ ! -z $gcc ]; then
    # GCC ----------------------------------------------------------------------------------------------------
    cp $makefile makefile.machine
    sed -i "s/OPTFLAGS=-O/OPTFLAGS=$flags/" makefile.machine
    make clean
    make -j5
    bench=`{ time ${arg}; } 2>&1 | grep real | awk '{print $2}'`
    echo -e "GCC\t\t$flags\t\t$bench" >> $results
    rm dummy.7z
    # pgo
    cp $makefile makefile.machine
    sed -i "s/OPTFLAGS=-O/OPTFLAGS=$flags -fprofile-generate/" makefile.machine
    make clean
    make -j5
    bench=`{ time ${arg}; } 2>&1 | grep real | awk '{print $2}'`
    rm dummy.7z
    cp $makefile makefile.machine
    sed -i "s/OPTFLAGS=-O/OPTFLAGS=$flags -fprofile-use -fprofile-correction/" makefile.machine
    make clean
    make -j5
    bench=`{ time ${arg}; } 2>&1 | grep real | awk '{print $2}'`
    echo -e "GCC\t\t$flags\t\t$bench\t\tProfile guided optimization" >> $results
    rm CPP/7zip/Bundles/Alone/*.gcda
    fi
    
    if [ ! -z $gcc47 ]; then
    # GCC 4.7 ----------------------------------------------------------------------------------------------------
    cp $makefile makefile.machine
    sed -i "s/OPTFLAGS=-O/OPTFLAGS=$flags/" makefile.machine
    sed -i 's/CXX=g++/CXX=g++-4.7/' makefile.machine
    sed -i 's/CC=gcc/CC=gcc-4.7/' makefile.machine
    make clean
    make -j5
    bench=`{ time ${arg}; } 2>&1 | grep real | awk '{print $2}'`
    echo -e "GCC4.7\t\t$flags\t\t$bench" >> $results
    rm dummy.7z
    # pgo
    cp $makefile makefile.machine
    sed -i "s/OPTFLAGS=-O/OPTFLAGS=$flags -fprofile-generate/" makefile.machine
    sed -i 's/CXX=g++/CXX=g++-4.7/' makefile.machine
    sed -i 's/CC=gcc/CC=gcc-4.7/' makefile.machine
    make clean
    make -j5
    bench=`{ time ${arg}; } 2>&1 | grep real | awk '{print $2}'`
    rm dummy.7z
    cp $makefile makefile.machine
    sed -i "s/OPTFLAGS=-O/OPTFLAGS=$flags -fprofile-use -fprofile-correction/" makefile.machine
    sed -i 's/CXX=g++/CXX=g++-4.7/' makefile.machine
    sed -i 's/CC=gcc/CC=gcc-4.7/' makefile.machine
    make clean
    make -j5
    bench=`{ time ${arg}; } 2>&1 | grep real | awk '{print $2}'`
    echo -e "GCC4.7\t\t$flags\t\t$bench\t\tProfile guided optimization" >> $results
    rm CPP/7zip/Bundles/Alone/*.gcda
    fi
    
    if [ ! -z $clang ]; then
    # CLANG/LLVM ---------------------------------------------------------------------------------------------
    cp $makefile makefile.machine
    sed -i "s/OPTFLAGS=-O/OPTFLAGS=$flags/" makefile.machine
    sed -i 's/CXX=g++/CXX=clang++/' makefile.machine
    sed -i 's/CC=gcc/CC=clang/' makefile.machine
    make clean
    make -j5
    bench=`{ time ${arg}; } 2>&1 | grep real | awk '{print $2}'`
    echo -e "Clang\t\t$flags\t\t$bench" >> $results
    rm dummy.7z
    fi
    
    if [ ! -z $path64 ]; then
    # PATH64 ---------------------------------------------------------------------------------------------
    cp $makefile makefile.machine
    sed -i "s/OPTFLAGS=-O/OPTFLAGS=$path64flags/" makefile.machine
    sed -i 's/CXX=g++/CXX=pathCC/' makefile.machine
    sed -i 's/CC=gcc/CC=pathcc/' makefile.machine
    make clean
    make -j5
    bench=`{ time ${arg}; } 2>&1 | grep real | awk '{print $2}'`
    echo -e "Path64\t\t$path64flags\t\t$bench" >> $results
    rm dummy.7z
    fi
    Put it in a directory and execute it, it will download p7zip and the iso used to compress the first time it's run and then perform the benchmarks and place the results in a textfile named p7zip-results.txt in the directory of the benchmark script. Just change the cflags in the script to those you want to test and also comment/uncomment the compilers you wish to test (note that open64 and path64 fails on p7zip on my machine). Also this assumes a 64bit system, if you want to benchmark on a 32-bit system you need to change the makefile variable to one of the 32-bit makefiles which accompany p7zip (weird buildsystem in p7zip, give me ./configure anyday!)

  7. #7

    Default

    Please use normal images in your posts!

  8. #8
    Join Date
    Oct 2011
    Posts
    2

    Default SmallPT benchmark is useless when comparing compilers

    SmallPT is not optimized at all in PTS, therefore when you compare to compilers you don't really compare the potential speed of their executables but something else which is not related to speed at all.

    CLANG/LLVM opts to generate slower code when in no optimization mode. It is slower than GCC when unoptimized but you can't learn anything about its code generation when optimizing for full speed.

    See for example the difference of optimized and unoptimized SmallPT. The difference is HUGE. The optimized version is more than 3 times faster.

    (My machine, Phenom II X6 @ 3.6 GHz)
    http://openbenchmarking.org/result/1...GR-SMALLPTWI19

    The smallpt test should be optimized in future versions of PTS

  9. #9
    Join Date
    Nov 2011
    Posts
    1

    Default

    The SmallPT test isn't fair, as Clang don't support OpenMP, so it's not using multiple threads, which makes the test pointless...

    And it's not made clear what optimization level was given to the compilers to build the tests? GCC's and LLVM's are different scales...

  10. #10

    Default

    Most important question ... How did you manage to install Open64 5.0 on an Ubuntu 11.10 distribution ??

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •