Results 1 to 4 of 4

Thread: Broken tests in PTS 3.0.0

  1. #1
    Join Date
    Jul 2007
    Posts
    11

    Default Broken tests in PTS 3.0.0

    I am attempting to use PTS 3.0.0 to benchmark a new server I got at work. This is a Dell R910 with 32 cores (64 with SMT which I have disabled for these tests), and 256GB of DDR3. I am running the universe-cli suite, as I do not intend to install a GUI on the system. I have run into several tests failing to run, and thought it might be good to report the errors here so that they may be fixed in a future version of PTS.

    First off, two tests which fail with >8CPUs, mencoder & ffmpeg. Neither of these tests like it when you tell them to use more than 8 threads. As I mentioned above, the system I'm testing happens to have 32 cores, so using $NUM_CPU_CORES causes the tests to fail.

    Next, compress-pbzip2 which as far as I can tell seems to work when I run the command line by hand, but tells me "The test did not produce a result" each time it runs. I think this is a timing issue (as in, the test isn't timed correctly, or assumes that the result will be greater than specific value and is throwing away the results because they're too fast?).

    I also had an issue with a couple of Compile Bench tests, Compile and Read Compiled Tree, though the Initial Create test worked fine. I think this is a parsing error, as the results I get from running the test in debug mode seem to indicate that both the compile & the read tree worked correctly:
    Code:
    run complete:
    ==========================================================================
    intial create total runs 10 avg 146.98 MB/s (user 0.59s sys 0.91s)
    no runs for create
    no runs for patch
    compile total runs 10 avg 792.47 MB/s (user 0.12s sys 0.74s)
    no runs for clean
    no runs for read tree
    read compiled tree total runs 3 avg 826.78 MB/s (user 0.38s sys 0.71s)
    no runs for delete tree
    delete compiled tree total runs 10 avg 0.95 seconds (user 0.25s sys 0.69s)
    no runs for stat tree
    no runs for stat compiled tree
    Next is mrbayes, which is another case of too many CPUs:
    Code:
          The number of chains must be at least as great
          as the number of processors (32)
    Finally, java-scimark2 FFT test failed as well. This appears to be another parsing error, as the test definitely produces results:
    Code:
    ========================================
    Java SciMark (Run 4 of 4)
    ========================================
    
    
    Test Run Command: cd /home/thalin/.phoronix-test-suite/installed-tests/pts/java-scimark2-1.1.1/ && ./java-scimark2 TEST_FFT 2>&1
    
    
    SciMark 2.0a
    
    Composite Score: 865.7334474774465
    FFT (1024): 538.1797821272396
    SOR (100x100):   963.7184161416388
    Monte Carlo : 459.6497601801464
    Sparse matmult (N=1000, nz=5000): 867.1650672662963
    LU (100x100): 1499.954211671911
    
    java.vendor: Sun Microsystems Inc.
    java.version: 1.6.0_20
    os.arch: amd64
    os.name: Linux
    os.version: 2.6.35-25-server
    
    Result Parsing Search Key: 
    FFT (1024): 
    
    
    Result Line: Composite Score: 865.7334474774465
    
    
    Test Result Value: 
    
    
    The test did not produce a result.
    I hope that these reports are useful, and if any further information is required I will be happy to provide it.

  2. #2
    Join Date
    Jul 2008
    Posts
    65

    Default

    Sounds like that would be a fun system to benchmark ...Too many CPUs...

  3. #3
    Join Date
    Jan 2011
    Posts
    7

    Default

    Sry for using this thread as a dump for other bugs

    There also seems to be a bug in the Bullet Physics benchmark. Using Fedora 14 the last test always fails, cause a new line is added using
    Code:
    echo "\n" >> $LOG_FILE
    . But to make echo interpret this as a new line, one has to use
    Code:
    echo -e "\n" >> $LOG_FILE
    instead.

    By the way, for compiling bullet with clang, one has to patch the source (http://http://code.google.com/p/bull.../detail?r=2222). Perhaps one could include this patch in the install.sh.

  4. #4
    Join Date
    Jan 2011
    Posts
    7

    Default

    Another bug.

    Nero2d fails to compile on Fedora 14 cause there are some hardcoded symlinks in the doc part of the package.

    i modified the corresponding install.sh to patch the makefile the way that the docs arent build

    Code:
    #!/bin/sh
    
    mkdir $HOME/nero2d_
    
    tar -zxvf nero2d-2.0.2.tar.gz
    
    patch -p0 <<'EOT'
    --- nero2d-2.0.2.orig/src/nexus/nexus.cpp	2009-04-03 09:42:29.000000000 -0400
    +++ nero2d-2.0.2/src/nexus/nexus.cpp	2009-06-13 21:52:42.589304348 -0400
    @@ -21,6 +21,8 @@
     #include "nexus.h"
     #include "engine.h"
     #include "base.h"
    +#include <cstring>
    +#include <cstdio>
     
     using namespace std;
     
    EOT
    
    
    patch -p0 <<'EOT'
    --- nero2d-2.0.2.orig/Makefile.in    2009-04-03 15:42:34.000000000 +0200
    +++ nero2d-2.0.2/Makefile.in  2011-03-02 19:23:12.179899327 +0100
    @@ -193,7 +193,7 @@
     # not a GNU package. You can remove this line, if
     # have all needed files, that a GNU package needs
     AUTOMAKE_OPTIONS = foreign 1.4
    -SUBDIRS = config benchmark doc examples src
    +SUBDIRS = config benchmark examples src
     all: config.h
            $(MAKE) $(AM_MAKEFLAGS) all-recursive
    
    EOT
    
    cd nero2d-2.0.2/
    ./configure --prefix=$HOME/nero2d_
    make -j $NUM_CPU_JOBS
    echo $? > ~/install-exit-status
    make install
    cd ..
    rm -rf nero2d-2.0.2/
    
    echo "#!/bin/sh
    ./nero2d_/bin/nero2d \$@ > \$LOG_FILE 2>&1
    echo \$? > ~/test-exit-status" > nero2d
    chmod +x nero2d

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •