Gosh, Our Test Farm Already Finds Big Regression
Phoronix: Gosh, Our Test Farm Already Finds Big Regression
Back in October we made it possible to autonomously find performance regressions in a project's code-base by leveraging the Phoronix Test Suite atop Git's bisect command. A new Phoronix Test Suite module was created that would automatically run a binary search on a defined code-base and keep running the specified test(s) on each revision and then traverse to the next using Git bisect until the lone commit causing the defined regression was located...
Here are some suggestins:
1) It would be nice to have tests on both gcc and icc (and LLVM?). Gcc-generated code quality messure over time, on real world SW, is missing.
2) Tests for different versions of gcc, inluding ancient (3.x).
3) Tests for default compiler settings vs. CPU type specific options.