Broken tests in PTS 3.0.0

[thalin]

I am attempting to use PTS 3.0.0 to benchmark a new server I got at work. This is a Dell R910 with 32 cores (64 with SMT which I have disabled for these tests), and 256GB of DDR3. I am running the universe-cli suite, as I do not intend to install a GUI on the system. I have run into several tests failing to run, and thought it might be good to report the errors here so that they may be fixed in a future version of PTS.

First off, two tests which fail with >8CPUs, mencoder & ffmpeg. Neither of these tests like it when you tell them to use more than 8 threads. As I mentioned above, the system I'm testing happens to have 32 cores, so using $NUM_CPU_CORES causes the tests to fail.

Next, compress-pbzip2 which as far as I can tell seems to work when I run the command line by hand, but tells me "The test did not produce a result" each time it runs. I think this is a timing issue (as in, the test isn't timed correctly, or assumes that the result will be greater than specific value and is throwing away the results because they're too fast?).

I also had an issue with a couple of Compile Bench tests, Compile and Read Compiled Tree, though the Initial Create test worked fine. I think this is a parsing error, as the results I get from running the test in debug mode seem to indicate that both the compile & the read tree worked correctly:

run complete:
==========================================================================
intial create total runs 10 avg 146.98 MB/s (user 0.59s sys 0.91s)
no runs for create
no runs for patch
compile total runs 10 avg 792.47 MB/s (user 0.12s sys 0.74s)
no runs for clean
no runs for read tree
read compiled tree total runs 3 avg 826.78 MB/s (user 0.38s sys 0.71s)
no runs for delete tree
delete compiled tree total runs 10 avg 0.95 seconds (user 0.25s sys 0.69s)
no runs for stat tree
no runs for stat compiled tree

Next is mrbayes, which is another case of too many CPUs:

      The number of chains must be at least as great
      as the number of processors (32)

Finally, java-scimark2 FFT test failed as well. This appears to be another parsing error, as the test definitely produces results:

========================================
Java SciMark (Run 4 of 4)
========================================


Test Run Command: cd /home/thalin/.phoronix-test-suite/installed-tests/pts/java-scimark2-1.1.1/ && ./java-scimark2 TEST_FFT 2>&1


SciMark 2.0a

Composite Score: 865.7334474774465
FFT (1024): 538.1797821272396
SOR (100x100):   963.7184161416388
Monte Carlo : 459.6497601801464
Sparse matmult (N=1000, nz=5000): 867.1650672662963
LU (100x100): 1499.954211671911

java.vendor: Sun Microsystems Inc.
java.version: 1.6.0_20
os.arch: amd64
os.name: Linux
os.version: 2.6.35-25-server

Result Parsing Search Key: 
FFT (1024): 


Result Line: Composite Score: 865.7334474774465


Test Result Value: 


The test did not produce a result.

I hope that these reports are useful, and if any further information is required I will be happy to provide it.

[skeetre]

Sounds like that would be a fun system to benchmark ...Too many CPUs...

[d1saster]

Sry for using this thread as a dump for other bugs

There also seems to be a bug in the Bullet Physics benchmark. Using Fedora 14 the last test always fails, cause a new line is added using

echo "\n" >> $LOG_FILE

. But to make echo interpret this as a new line, one has to use

echo -e "\n" >> $LOG_FILE

instead.

By the way, for compiling bullet with clang, one has to patch the source (http://http://code.google.com/p/bull.../detail?r=2222). Perhaps one could include this patch in the install.sh.

[d1saster]

Another bug.

Nero2d fails to compile on Fedora 14 cause there are some hardcoded symlinks in the doc part of the package.

i modified the corresponding install.sh to patch the makefile the way that the docs arent build

#!/bin/sh

mkdir $HOME/nero2d_

tar -zxvf nero2d-2.0.2.tar.gz

patch -p0 <<'EOT'
--- nero2d-2.0.2.orig/src/nexus/nexus.cpp	2009-04-03 09:42:29.000000000 -0400
+++ nero2d-2.0.2/src/nexus/nexus.cpp	2009-06-13 21:52:42.589304348 -0400
@@ -21,6 +21,8 @@
 #include "nexus.h"
 #include "engine.h"
 #include "base.h"
+#include <cstring>
+#include <cstdio>
 
 using namespace std;
 
EOT


patch -p0 <<'EOT'
--- nero2d-2.0.2.orig/Makefile.in    2009-04-03 15:42:34.000000000 +0200
+++ nero2d-2.0.2/Makefile.in  2011-03-02 19:23:12.179899327 +0100
@@ -193,7 +193,7 @@
 # not a GNU package. You can remove this line, if
 # have all needed files, that a GNU package needs
 AUTOMAKE_OPTIONS = foreign 1.4
-SUBDIRS = config benchmark doc examples src
+SUBDIRS = config benchmark examples src
 all: config.h
        $(MAKE) $(AM_MAKEFLAGS) all-recursive

EOT

cd nero2d-2.0.2/
./configure --prefix=$HOME/nero2d_
make -j $NUM_CPU_JOBS
echo $? > ~/install-exit-status
make install
cd ..
rm -rf nero2d-2.0.2/

echo "#!/bin/sh
./nero2d_/bin/nero2d \$@ > \$LOG_FILE 2>&1
echo \$? > ~/test-exit-status" > nero2d
chmod +x nero2d