Auto-Determining Relevant Linux Benchmarks To Run

[phoronix]

Phoronix: Auto-Determining Relevant Linux Benchmarks To Run

As part of a larger undertaking with OpenBenchmarking.org, the Phoronix Test Suite is now capable of intelligently and automatically determining the most relevant Linux benchmarks for you to run based upon your system's installed hardware and software along with what sub-system you're wishing to stress...

http://www.phoronix.com/vr.php?view=MTIwMzE

[allquixotic]

This is awesome for making PTS more accessible to end-users and people who don't know what benchmarks are out there and which ones are good to be used.

Now if the time investment required to do a benchmark could be reduced somehow, that would make this truly mainstream.

Not saying it has to be lightning quick, but at least faster than it is now. Things like:

• Making the largest of the benchmark downloads smaller by removing unneeded data or using better compression (the former is likely to yield the most impressive results)
• Providing pre-compiled binaries or using distro packages, where appropriate, to avoid compiling native code from source, due to the large amount of time needed

Admittedly there are certain tests where it is interesting to do compilation and download large amounts of data. But, there are also tests where running the same compiled binary (with known performance characteristics and limitations and functionality) could be usefully compared on different hardware / drivers.

The distro package integration would at least be nice, because the download sizes of heavily compressed, pre-compiled distro packages are likely much less due to the way distro packages are nicely separated into bite-size chunks (as a silly example, you can download a particular 3d renderer and a trivial example "game" or demo for it rather than downloading a full-blown game with all of its data). This also lets you see how well the compiled and shipped distro packages themselves run "out of the box" without any custom compiling from PTS. This is more of a software test than a hardware test, but people into software are interested in that.

[Michael]

Admittedly there are certain tests where it is interesting to do compilation and download large amounts of data. But, there are also tests where running the same compiled binary (with known performance characteristics and limitations and functionality) could be usefully compared on different hardware / drivers.

The distro package integration would at least be nice, because the download sizes of heavily compressed, pre-compiled distro packages are likely much less due to the way distro packages are nicely separated into bite-size chunks (as a silly example, you can download a particular 3d renderer and a trivial example "game" or demo for it rather than downloading a full-blown game with all of its data). This also lets you see how well the compiled and shipped distro packages themselves run "out of the box" without any custom compiling from PTS. This is more of a software test than a hardware test, but people into software are interested in that.

Anyone can trivially create their own test profiles on OpenBenchmarking.org by modifying the existing test profiles to just fetch the distribution packages rather than dealing with the source packages, anyone is open to make their own test profiles.