Autonomously Generating An Ideal Kernel Configuration

While most Linux users are fine with just using the kernel supplied by their distribution vendor, there are some enthusiasts and professional users who end up tweaking their kernel configuration extensively for their needs, particularly if they are within a corporate environment where the very best performance and reliability is demanded for a particular workload.

Unfortunately, with there being hundreds of different Linux kernel configuration options, this is often a deterrent for any hobbyist to sit down and invest a great deal of time in testing out the different combinations of kernel options to generate the best performance for one's needs. Even for the experienced professionals the process of tuning the kernel configuration for a specific workload can require significant man hours and manual testing.

But how would it be if you could find the optimal kernel configuration for your specific workload without any real human involvement? Well, it's possible to quite easily generate an ideal Linux kernel configuration in an autonomous manner.

Last October I shared how it was possible to autonomously find performance regressions in the Linux kernel. This was done by writing a module for the Phoronix Test Suite that hooked our testing framework into Git's bisect command to be able to autonomously traverse a Git repository and running some automated tests each step of the way looking for the lone commit that caused the targeted regression. This was a success and in a fully automated manner we were able to track down the individual commit that caused a severe performance drop in the disk performance for EXT4. This Phoronix Test Suite module could also be hooked in to find a regression in any other Git-managed code-base by simply adjusting a few test/code-specific bits.

Of course, this bisecting module isn't even needed if you use Phoromatic where you can automatically have the targeted code based on a daily basis or even a per-commit basis, such as we have done with benchmarking the Linux kernel on a daily basis and even benchmarking Ubuntu daily in looking for performance changes. However, inspired to see what else could be delivered after writing the autonomous kernel regression finder, an autonomous kernel configuration optimizer has been conceived.

With this new module that I have been writing for Phoronix Test Suite 2.6 "Lyngen" over the weekend, it becomes possible to find the ideal kernel configuration options for your specific workload by what boils down to an automated feedback optimization loop. It's just a matter of supplying the module with the Linux kernel source code version you wish to use, what kernel configuration options you wish to test and with what values to use, and then the specific performance test(s) you want to cater your kernel towards. Following that, this Phoronix Test Suite module will go through the process of autonomously building the kernel in each configuration (currently this module is just supported on Ubuntu), rebooting each time, and then carrying out the tests.

Once the automated kernel testing has finished, all of the results will be analyzed -- again, in a completely autonomous manner -- and a report will be generated showing the impact that each kernel configuration option has on your specific workload of tests. It's really meant to be that easy!

The code and related documentation will be published shortly for Phoronix Test Suite 2.6 along with a few example kernel configurations that have been auto-generated showing some performance improvements that were automatically detected. In the mean time though, if you have any feedback on the concept or any other ideas, feel free to voice them in the forums or by contacting us. This module could also be adapted to test out different combinations of options for other pieces of the OS stack, like graphics driver options too.

Phoronix Test Suite 2.6 (codenamed "Lyngen") is expected to be officially released in May. Among Phoromatic Tracker improvements, this self-optimizing kernel configuration tester, and other improvements, there should also be better Windows 7 support.