Defining The Phoronix Test Suite
Phoronix: Defining The Phoronix Test Suite
It's going on four years that we have been providing benchmarks under Linux at Phoronix, and through this time the process has evolved as more applications have come about that gauge Linux system performance and meet our criteria for use in hardware reviews. While our internal process is continually refined, we have received requests for creating a Linux benchmarking manifest so that interested parties are able to easily reproduce our tests for comparative purposes and know everything that's involved. With that said, we are now working to establish the Phoronix Test Suite that will consist both of detailed documentation defining our hardware benchmarking protocols and free software that automates much of this work.
really, really great idea. i'm going to think about several components for such a benchmark, as soon as i've such results, i'm going to write them here
what licence will this be - gpl?
Originally Posted by hmmm
GPLv3 likely (chance it could be MIT/X11 but most likely GPL)
Why not add in benches of
Blender3D rendering, an alternative for the 3dsMax benches suites use (heck even use the project files from Elephants Dream)
GIMP running though a series of filters and scripts much like other bench suites use Photoshop
MySQL(or postgresql) benchmarks
x264 encoding benches (perhaps utilizing various filters in an app like avidemux2)
These apps would be a far more accurate representation of performance then any lame RAMspeed or hdparm test.
As well they should be compiled in 32-bit and 64-bit with all applicable flags.
(Oh gee look at that, all free software and all crossplatform)
I would stay away from propriatary apps like Quakewars. Keep the suite free of non OSS. Use one of the many free fps's instead so everybody can use it and it can be included with linux distributions.
Last edited by deanjo; 02-13-2008 at 02:12 AM.
Way to go, this is a great idea. I'm sure this will go far to create a standard benchmarking tool for Linux.
Are there any free games in there? OpenArena, Cube2, Nexuiz (shameless plug)? What about e.g. oggenc?
oggenc is in there.
SavageX: Last I checked out the Nexuiz benchmarking capabilities, I wasn't too impressed to be honest. Have they improved as of late? Last time I looked into it was about a year ago. If you're able to shed anymore light on it I'd be more than willing to investigate it further.
As basically all Quake-engine based games it has the nice timedemo command. Try e.g. "timedemo demos/demo1". I think it should even work directly at game startup: "./nexuiz-linux-whatever-binary +timedemo demos/demo1"
Originally Posted by Michael
Or you can create a benchmark config file (e.g. "benchmark.cfg" in the Nexuiz "data" directory):
and do a "./nexuiz-linux-bla-bla +exec benchmark.cfg" to automate testing.
Each run will print to console (ingame and to stdout) statistics like min/avg/max framerates.
We'll release a new version of Nexuiz hopefully later this month (see http://emptyset.endoftheinternet.org...nexuiz/builds/ for SVN builds) which will introduce (optional) rendering features like reflective water with refraction - this will make sure even fast cards are kept busy ;-)
(Plus we finally have a much nicer menu ;-) )