As I've said since the beginning, what's taken so long -- for me at least -- in finding these regressions is that some manual intervention is required (switching the battery / AC power state by pulling the plug) and that testing was limited to mobile systems for easy monitoring of the power consumption. For several weeks, I've been looking for a USB-based power meter or UPS power supply that could monitor the AC power consumption connected to a desktop power supply. Ideally, I've been looking for such a solution that costs less than $100 USD.
This mission has now been accomplished with coming across the Watts Up Power Meter Pro, per a reader recommendation. The "Watts up? PRO" has a USB interface, provides an integrated LCD for also monitoring the power consumption and other data that can be toggled via two buttons, and accepts any US power connection. This power meter also supports logging the power data to the device itself and various other options from their software. The company only makes Windows software, but from their support page they reference a community-created open-source utility. For a little more than 1,000 lines of C code is this open-source utility that works under Linux with the "Watts Up" brand power adapters. It supports dumping various power metrics over USB, including the real-time Wattage.
The cost of the Watts Up Pro is $130 USD from the manufacturer web-page but can be found for as little as $110 from other Internet web-stores. I decided to buy one of these units after seeing the features, open-source Linux support (albeit spawned by the community), and the acceptable price.
The Watts Up Pro arrived today and soon as connecting it to Linux and compiling the small C program, the device was immediately working under Linux. Perfect. A few minutes later, the Phodevi power sensor module for the Phoronix Test Suite was extended to detect this "wattsup" Linux program and to automatically read the real-time Wattage information from it when requested. This power meter is working great and it's now supported under the Phoronix Test Suite in the latest Git code. The official support will land in Phoronix Test Suite 3.2.1 or 3.4.0. Here's an example on OpenBenchmarking.org.
With this being supported by the Phoronix Test Suite's Phodevi (Phoronix Device Interface) library, it can be used in the same way as the Linux battery monitoring support and can be tapped in the same way by all areas of the Phoronix Test Suite in a uniform manner. This means that the Linux kernel power regressions can be fully automated (assuming the powerful workstations exhibit this regressions) and can use this AC power meter to look at other interesting areas such as comparing the power consumption when using open vs. closed-source graphics drivers with desktop GPUs, how different operating systems compare power-wise, etc.
So far the Watts Up Pro is working great under Linux with the Phoronix Test Suite. My only complaint or feature request would just revolve around their Linux support as ideally I'd like to see an upstream Linux kernel module for the Watts Up devices that would expose the various power attributes via sysfs nodes. That way it'd be an out-of-the-box experience and not have to worry about fetching/building this small power utility. It would also be easier for the Phoronix Test Suite and other scripts to then just check these sysfs entries, similar to how many notebook batteries have their power information exposed. It should be relatively quick and easy to write a kernel driver for these devices by simply porting this simple utility's code, but alas I lack the time to do so myself. Look for interesting Linux power test results soon.