Automating kernel configuration makes sense if it is done right and not just based on benchmarks. Would make sense if the generator program would take into account mainly the hardware present as well as architecture-specific tweaks plus any user specified settings
and i really forgot to mention multislot/multilib.
You can have two (or more) versions of same thing installed and working.
There are almost always more than one version available by default (and more in overlays).
I'm sure I could come up with something to make it work like that to optimize it even further in terms of what modules are needed based upon the current hardware, but I probably wouldn't end up investing that much time into this module unless it becomes financed by a PTS Commercial client.
If you didn't make modules but instead built in it would be easier to build and test, plus the kernels (which are around 2MB each) could be kept and reused for further tests without recompiling them
This is the best resource around users looking for some mostly reliable information on kernel configuration, along with config files with sane defaults. Much thanks to him for his efforts.
As far as the idea of building a kernel based upon a benchmark - this sounds like a bad idea. Most kernel options aren't exactly "tunables" that you select in the interest of performance. You mostly enable support for the the features you need and the hardware you have. A benchmark isn't going to indicate that.
Some of the stuff he has turned on by default can be turned off, but I stopped making my .config files from the default settings and started using his as a basis. It is nice to have most of the junk I do not need turned off by default.
Gentoo Linux users have been by modifying .config file settings by hand to get improved kernel performance for a long time. I do not think automating it will do much for Linux performance, although this will probably save time for Linux users that were already doing this sort of thing.
Gentoo users are the best!!11! Nobody is performancyer! Also, Gentoo users are *not* presumptuous and ignorant! That is because Gentoo invented the source-based distro and even if there were others, Gentoo would have the best implementation, not only the best publicity! And binary distros can never be as efficient anyway. Because compilation is done in the background anyway so it uses no CPU, no memory and doesn't kill the climate!!!!!!
For combinatorial problems like this with a large search space (which is likely if you want to optimize a decent number of parameters), people have successfully applied techniques like genetic algorithms (http://en.wikipedia.org/wiki/Genetic_algorithm).
Of course, you may not find the globally optimal solution that brute-force can. However, you'll get closer in what is hopefully far less time.
ACOVEA takes such an approach to optimize compiler flags with GCC.