HPC Challenge, New OpenCL Linux Benchmarks

There's several new and updated benchmarking applications for measuring the performance of Linux system's ranging from small embedded environments to GPUs to super-computers.

Yesterday I announced the release of Phoronix Test Suite 4.8.4 as the latest stable release of our flagship benchmarking platform. All users are encouraged to upgrade to this latest release as there's many improvements to be found and can provide immediate benefits to our community users and customers.

Going in hand with the Phoronix Test Suite 4.8.4 update are several new test profiles this week on OpenBenchmarking.org. With the exception of HPC Challenge, all the new/updated test profiles to be mentioned should work fine with existing PTS 3.x/4.x releases, but for maximum feature exposure users are encouraged to be running the latest PTS 4.8+ release. For those wanting these new test profiles about to be mentioned, simply download the latest Phoronix Test Suite release but if you're a current user and have run PTS in the past 2~3 days -- the usual cache-to-live time for the OpenBenchmarking.org index/repository data -- first run phoronix-test-suite openbenchmarking-refresh.

The first big new test profile to talk about is the arrival of HPC Challenge! HPC Challenge is the very popular super-computing / cluster test that's also used by the Green/Top 500 lists. While we've always been aware of HPCC, it wasn't quick to land in the Phoronix Test Suite since it can't be automated too well. Published HPCC results are allowed to use customized input data files and use different math and MPI libraries plus various compiler tuning options. Each organization we've spoken with about HPC Challenge has their own preference on how they think HPCC should be tuned for best results.

At the end of the day though, thanks to commercial involvement from a leading organization, I have been spending much time recently on coming up with a HPC Challenge for the Phoronix Test Suite. As with all other tests, the HPCC test profile is fully-automated and we're going with a best-attempt approach to come up with an automated configuration and input file. As many of the MPI / LA libraries are added to the build script as possible for the common Linux distributions (namely RHEL and Ubuntu Linux) and we're trying to generate a decent hpccinf.txt input file to be used by default based upon the number of threads and system memory.

For easing customization -- as with any test profile -- users can customize the test profile within ~/.phoronix-test-suite/test-profiles/ and the test profile has established some basic environment variables for easy configuration, e.g. $MPI_PATH, $MPI_INCLUDE, $MPI_CC, $MPI_LIBS, $MPI_NUM_THREADS, $LA_PATH, $LA_INCLUDE, and $LA_LIBS. Those environment variables are beyond the usual environment variables honored by PTS like CFLAGS/CXXFLAGS, etc.

With these variables and acknowledging that serious HPCC users like to tweak the heck out of their configurations for trying to get the best performance results, this test profile makes use of some new Phoronix Test Suite 4.8.4 features for better data recording and reporting. Modern Phoronix Test Suite releases have always reported compiler build information, CFLAGS/CXXFLAGS, and other important system compiler information but now with HPCC it takes use of new pts-core features to report MPI/BLAS library information as footnotes on graphs, extra precision in test results, and other features. Those wanting to try out this automated configuration of HPC Challenge should upgrade to Phoronix Test Suite 4.8.4 and simply run phoronix-test-suite benchmark hpcc.

More information on the HPCC benchmark is available from its test profile page. For those unfamiliar with the PTS test writing process, you can see the HPC Challenge example source via OpenBenchmarking.org Innhold. The HPC Challenge test profile is actively being worked on and further improved, but I'm mentioning it now if anyone has any feedback or suggestions to provide, etc. Look for it to still improve a lot in the coming days.

Besides HPC Challenge in the Phoronix Test Suite, a test profile I did this week in an overall OpenCL push not associated with any specific customer needs (it's my birthday today and I just enjoy working on Linux benchmarks and playing with hardware, so here's a present to you all, but there's also more coming later today...) is introducing OpenDwarfs, an OpenCL computational benchmark consisting of several independent CL test-cases. To run this simply execute phoronix-test-suite benchmark opendwarfs. See the aforelinked test profile page for more information and reference results.

Next up, JuliaGPU, MandelbulbGPU, SmallPTGPU, and MandelGPU have all seen updates to these OpenCL compute tests. The updates to these tests are mostly build changes so they should work easier "out of the box" on more AMD / NVIDIA Linux systems. Phoronix Test Suite 4.8.4 also has expanded external dependency support for handling the OpenCL headers on more distributions.

Last but not least in this latest OpenCL benchmarking push is the updating of Rodinia to handle OpenCL tests. Rodinia has been in the Phoronix Test Suite for months but only with the MIP code-paths. After making some changes to the test profile this week, Rodinia can now be used for carrying out a number of OpenCL GPU benchmarks.

While not too much to share today, I'm also in the process of updating NPB to support OpenCL benchmarking workloads and separately bringing updates to Open Porous Media.

So head on over to Phoronix-Test-Suite.com and fetch the 4.8.4-Sokndal release to take advantage of new benchmarking features and then go browse OpenBenchmarking.org to find interesting test profiles and/or reference benchmarks for yielding an interesting benchmark experience.

While the Phoronix Test Suite / OpenBenchmarking.org already has hundreds of interesting test profiles, new ones can be easily added and patches to our open-source software and new test profiles are gladly accepted. There's also PTS Commercial for organizations interested in custom engineering services, contracting the creation of new test profiles, or other purposes. Your support is gladly appreciated. For those that appreciate all the work done by Phoronix.com, you can always help out on an individual level by subscribing to Phoronix Premium or sending over a PayPal contribution are most greatly appreciated.