What Phoromatic Does For Managing The Benchmarking Lab

In the comments to yesterday's article about The Power Usage Of A 50+ Linux System Benchmarking Farm, it seems there's some confusion in not knowing what Phoromatic does for managing the benchmarking test farm.

Multiple comments in that forum thread and elsewhere suggested that the systems be powered off when not testing. That in fact is one of the many Phoromatic features. When the options are enabled, Phoromatic will automatically power-off systems when there's no tests scheduled to be run but will dynamically send WoL packets to wake-up the relevant systems whenever there is new tests to be run -- whether it be a scheduled test run or some spontaneous benchmarking or if needing to carry out any system maintenance as set via Phoromatic.

Taking a step back, as explained at Phoromatic.com, "Phoromatic is the remote management system of the Phoronix Test Suite. Phoromatic allows the automatic (hence the name Phoro-matic) scheduling of tests, remote installation of new tests, and the management of multiple test systems all through an intuitive, easy-to-use web interface. Tests can be scheduled to automatically run on a routine basis across multiple test systems. The test results are then available from this centralized, web-based location. Any test available via OpenBenchmarking.org can be run via Phoromatic."

For those that haven't tried the latest-generation Phoromatic server/client that's open-source and built into the latest versions of the Phoronix Test Suite, among the features with PTS 5.8 that can be tapped all from the centralized web interface are:

- The mentioned item of automatically powering on/off systems when needed. That is assuming the system's support WoL, otherwise the option can be disabled via Phoromatic or on a per-system basis for NICs / software not properly supporting Wake-On-LAN. That's the case for over a dozen of the 50+ systems is that they have issues waking up automatically even when configured for the said support from their UEFI/BIOS, so they're effectively black-listed on a per-system basis.

- Creating groups to easily manage multiple test systems devoted to the same task, e.g. benchmarking the Linux kernel or benchmarking a compiler.

- Running benchmarks on a scheduled basis across a select set of system(s) or group(s). These schedules could be on a timed-basis or whenever triggered via an external script with pinging a URL, e.g. to integrate with an existing build system, setting up hooks to automatically ping that URL whenever a new Git commit/revision is made, or other event takes place.

- The ability to run pre/post test install/run hooks for setting the system into the desired state, such as rebooting the system after fetching a specific kernel version.

- Scheduling one-off benchmark runs via the "benchmark" page.

- Easily building your own test suites (a collection of test profiles) from the centralized web UI.

- Monitoring all of the system hardware/software sensors of the connected systems in real-time.

- Viewing and merging all of the results collected on your network by the connected systems with functionality similar to OpenBenchmarking.org.

- Support for multiple accounts with varying levels of rights from just being able to view results to having administration control over systems.

- The ability to export results to PDF, CSV, and other formats.

- The ability to trigger arbitrary scripts for updating systems and/or the Phoronix Test Suite client across all systems.

- Easily view tables of all software/hardware information on associated systems.

- Serving as a download and OpenBenchmarking.org cache for distributing the needed test files on a LAN for faster speeds or if running on an isolated network without Internet connectivity.

- And many other features!

Give this functionality a shot by running phoronix-test-suite start-phoromatic-server on one of the systems and then follow the directions, which basically come down to enabling the phoromatic-client systemd/Upstart service on the test systems or manually running the client via phoronix-test-suite phoromatic.connect. The latest code is hosted on GitHub.

Commercial support, custom engineering, and other enterprise functionality is offered by contacting PTS Commercial.