As outlined in last night's
Phoronix Test Suite 5.4-Lipki M5 release, the official release of this huge update to our open-source (GPL) benchmarking software is due out in the next few weeks. The major focus of the 5.4 cycle has been on finishing the overhaul of the next-generation version of Phoromatic that's open-source and built into the Phoronix Test Suite for serving centralized test management and test orchestration needs for clusters of systems running the Phoronix Test Suite for routine (or in some cases, continuous) benchmarking.
Phoromatic has long been our remote management system for the
Phoronix Test Suite. Phoromatic allows the automatic (hence the name Phoromatic) scheduling of tests, remote installation of new tests, and the management of multiple test systems all through an intuitive, easy-to-use web interface. Tests can be scheduled to automatically run on a routine basis across multiple test systems. The test results are then available from this central, secure location. The original implementation of Phoromatic that launched around 2010 was based at Phoromatic.com and available for licensing to organizations wishing to run it behind-the-firewall. With Phoronix Test Suite 5.4, the basic open-source port of the next-gen Phoromatic became available and built into the phoronix-test-suite source tree. With Phoronix Test Suite 5.4, the new Phoromatic has surpassed feature parity to the prior implementations and is ready for enterprise/professional use while being freely available.
Launching the new Phoromatic is just a matter of installing the Phoronix Test Suite 5.4 client on the centralized server you wish to use for managing all of the tests. From there you can launch the server using
phoronix-test-suite start-phoromatic-server or there's also a
phoromatic-server.service for systemd to launch the server that way. From there the server will start-up (you can also configure the port numbers and such too, see the documentation for more details) and you can access the interface from a web browser as shown in this article.
It's easy to make accounts and get going. The only new requirements on the dependency end are SQLite support and optionally Avahi Tools if you wish to have zero-conf networking so the server/clients can auto-discover each other on your network.
It's very quick and easy to then allow any Phoronix Test Suite 5.4 system on your LAN to then communicate with the Phoromatic Server... There's the setup steps via the web UI or from
the Phoromatic PDF documentation.
With any Phoronix Test Suite system(s) you connect to your Phoromatic account, you're then able to view all of the system information from this centralized web interface, issue new test commands, etc. It's great if you're using the Phoronix Test Suite within organizations for carrying out regression testing, performance tracking of new software, etc.
The Phoromatic Server is extremely extensible to accommodate your testing needs. New features are constantly being added and will be up through the Phoronix Test Suite 5.4.0 release in the next few weeks... I also already have a lot of improvements in mind for the Phoronix Test Suite 5.6 cycle with its release in Q1'2015.
With test schedules you can make the connected Phoronix Test Suite systems run benchmarks always at a given time on given days per week with whatever "context" you'd like (i.e. whenever a benchmark is run, pass it an arbitrary script so it can be setup into your desired test state prior to or after running the tests). Instead of running a time-based test schedule, there's event driven testing too where by having any external script ping a defined URL and pass it an arbitrary event ID (e.g. Git hash, SVN revision, etc), you can have new tests commence whenever that happens and use that event ID to obtain the desired snapshot of code, etc.
The Phoromatic Server test scheduling is extremely flexible and more features are forthcoming. In the next few days will also be the ability to have non-scheduled/one-off testing. There's also Wake-On-LAN and other features still being tackled.
Whenever any tests are completed, they're uploaded directly to the Phoromatic Server so from there you can view the results, analyze the results, merge them against other earlier results or other systems, etc. It's similar functionality to what you find at
OpenBenchmarking.org.
All of this can be done behind corporate firewalls without any Internet connection. There's also support for serving as a cache server to distribute the OpenBenchmarking.org test/suite files, files needed for download by test profiles, etc. With Avahi support installed, it can all be done seamlessly on the client and server ends with very little setup needed.
There's also support for multiple users associated to one account for allowing a team/lab to collaborate over a given cluster of Phoronix Test Suite systems, etc.
This is just the quick rundown of functionality with Phoromatic on Phoronix Test Suite 5.4. See
the Phoromatic PDF documentation for more details or fetch the latest code for yourself via
Phoronix-Test-Suite on GitHub. Stay up to date with the latest Git as there's new activity going into it daily until the official 5.4.0 release (and beyond). For organizations in need of commercial support, custom engineering, or would like to simply sponsor the open-source development of PTS/OpenBenchmarking.org/Phoromatic,
contact us. End-users or enthusiasts can help out too by simply being a
Phoronix Premium subscriber or
making a PayPal tip.
2015 will yield even more interesting features for open-source Linux benchmarking and with the 5.4 changes made, the new OpenBenchmarking.org test farm for publicly tracking Mesa, the Linux kernel, and other projects will finally be able to get underway again. Your feedback and suggestions are welcome via our GitHub, forums, or by contacting us. Thanks!
