Autonomously Finding Performance Regressions In The Linux Kernel

[sreyan]

Autonomous (btw, why not automatic) regression testing is super useful.
How about extending it for regression testing wine with a set of windows programs? This could at least test for crashes. You could add automatic screen shots to get some of the functionality testing done too.

http://test.winehq.org/data/

[Michael]

This is awesome! Can anyone just run this, or do I need to buy something?

It's all open-source, but this module is just not something you can "just run". First you need to create a build script for whatever you wish to test, then module-setup bisect option to set all of the options, etc. Documentation for it will come once everything has been stabilized and new features added in. Of course, normal phoronix-test-suite is extremely easy to use and free.

[mtippett]

Autonomous (btw, why not automatic) regression testing is super useful.
How about extending it for regression testing wine with a set of windows programs? This could at least test for crashes. You could add automatic screen shots to get some of the functionality testing done too.

Ordered list, setup, test, fulcrum.

The screenshot testing is a considerably more difficult, and really needs application support to do it nicely.

[znmeb]

It's all open-source, but this module is just not something you can "just run". First you need to create a build script for whatever you wish to test, then module-setup bisect option to set all of the options, etc. Documentation for it will come once everything has been stabilized and new features added in. Of course, normal phoronix-test-suite is extremely easy to use and free.

Thanks! I've spent the last week or so doing Iozone testing on openSUSE 11.2 RC1 and I've automated as much of it as I can. But your suite is better than Iozone. ;-)

[Michael]

Thanks! I've spent the last week or so doing Iozone testing on openSUSE 11.2 RC1 and I've automated as much of it as I can. But your suite is better than Iozone. ;-)

Too bad you didn't discover:

phoronix-test-suite benchmark iozone

A week ago

[znmeb]

Too bad you didn't discover:

phoronix-test-suite benchmark iozone

A week ago

I actually did discover Phoronix a few months ago, tweeted and blogged about how wonderful it was, then promptly forgot about it and moved on to something completely different - social media analytics research. ;-) It's only been since openSUSE 11.2 Milestone 8 that I've started to care about Linux I/O performance again. ;-)

[mtippett]

I actually did discover Phoronix a few months ago, tweeted and blogged about how wonderful it was, then promptly forgot about it and moved on to something completely different - social media analytics research. ;-) It's only been since openSUSE 11.2 Milestone 8 that I've started to care about Linux I/O performance again. ;-)

As an OT diversion, do you have any references to your social media analytics research?

[znmeb]

As an OT diversion, do you have any references to your social media analytics research?

http://borasky-research.net/smart-at-znmeb

[kraftman]

@Michael

Why there are ia32-libs "needed" in PTS while I'm using 64bit OS (Kubuntu 9.10)?

When I wanted to install pgbench test, PTS wanted my password to install ia32-libs. I typed it and then, there was such error:

sudo: apt-get -y --ignore-missing install ia32-libs: command not found

however test is installed and working, but every time I want to run it, it displays this:

The following dependencies will be installed:
- ia32-libs

This process may take several minutes.
sudo: apt-get -y --ignore-missing install ia32-libs: command not found

[Michael]

@kraftman:

Some of the tests (particularly some of the games) require some of the ia32-libs, so I just added that check on there if running 64-bit to always fetch those 32-bit libs. However, for Ubuntu 9.10 and others where it is no longer present, I will make a workaround.