The Huge Disaster Within The Linux 2.6.35 Kernel

[MisterIO]

I'll just add that it should include positive regressions as well; I know I would like to have an email saying my patch increased XYZ by 15%

There are no positive regressions. By definition a regression means that the code got worse: http://www.thefreedictionary.com/regress

I read about positive regressions cited in an article here at phoronix, but it was probably just a little mistake or just simply BS.

[jalyst]

woah, this sounds kinda scary, hope it all gets corrected quickly

[mtippett]

There are no positive regressions. By definition a regression means that the code got worse: http://www.thefreedictionary.com/regress

I read about positive regressions cited in an article here at phoronix, but it was probably just a little mistake or just simply BS.

Let's not assume that all performance increases are good, great and what you want.. You accidentally go from Write-Through to Write-Back operations. You suddenly have increase in performance. Is this extra performance what you want? In a lot of cases no.

A better way is to consider a regression as an unexpected change in behaviour. Until you investigate and confirm that the change in behaviour has is not net negative, you have to assume that since it was unexpected that the net benefit is negative.

The key point is that you should always know how you are changing the system. If you don't know how you are changing the system, then an increase or decrease in performance is more likely to be bad than good.

[sabriah]

Michael, now after the bug was fixed...any improvement noted by Phoromatic yet?

When I looked at http://www.phoromatic.com/kernel-tracker.php a few minutes ago there was no improvement yet, and the latest date was 2010-05-30, with 5% threshold. Phoromatic chugs along, relentlessly.

[sabriah]

Let's not assume that all performance increases are good, great and what you want.. You accidentally go from Write-Through to Write-Back operations. You suddenly have increase in performance. Is this extra performance what you want? In a lot of cases no.

A better way is to consider a regression as an unexpected change in behaviour. Until you investigate and confirm that the change in behaviour has is not net negative, you have to assume that since it was unexpected that the net benefit is negative.

The key point is that you should always know how you are changing the system. If you don't know how you are changing the system, then an increase or decrease in performance is more likely to be bad than good.

Coincidentally, Slashdot has an article "When Mistakes Improve Performance" http://hardware.slashdot.org/story/1...ance?art_pos=3

Still, you point is valid

[kraftman]

A better way is to consider a regression as an unexpected change in behaviour.

Last time you didn't want to agree with this when there was an intentional change in Ext4 file system, so the change in behavior was also expected - better integrity in cost of performance (Linux devs were consious of this don't you think?). However, I can understand you're working on PTS.

Well that's why Michael and I created Phoromatic and PTS.

If there are kernel developers interested and motivated enough to work with us to tune the testing and the systems, we can push the kernel tracker way further than it is at the moment.

'Sane heads' probably aren't interested in meaningless Apache benchmark or in benchmarking different kernels with different fs modes. There are other tools which are actively used like sysbench, LTP and others.

I find pretty much the same here. There have been many times where I have found reporting an issue with the kernel to a distro that has kernel devels in it will result in quicker resolution.

That's for sure. Btw. I always get much more help (and much sooner) from the Arch Linux community then from Ubuntu.

And some of us appreciate that. It seems however, at least judging from a lot of the comments in this thread, if something bad is mentioned against the "church" all of a sudden you become public enemy #1 in their eyes.

PTS used properly is a nice tool and those who are complaining aren't usually complaining about PTS, but about stupid titles or meaningless or stupid tests (like Apache or when someone says there's a regression, but different modes were used). If something bad is mentioned against the "Phoronix church" all of a sudden you become public enemy #1 in their eyes.

[kraftman]

If there are kernel developers interested and motivated enough to work with us to tune the testing and the systems, we can push the kernel tracker way further than it is at the moment.

However, something like this with cooperation with the Linux devs would be just great and I can't imagine why there's no such cooperation yet. But, without consulting them it's just a FUD sometimes.

[Linuxhippy]

[QUOTE=phoronix;130708]Phoronix: The Huge Disaster Within The Linux 2.6.35 Kernel

Yeah, really - a huge desaster.
A not-even-rc kernel shows performance regressions.

Sorry, the BP accident is a desaster, this is just a regression

- Clemens

[mtippett]

Last time you didn't want to agree with this when there was an intentional change in Ext4 file system, so the change in behavior was also expected - better integrity in cost of performance (Linux devs were consious of this don't you think?). However, I can understand you're working on PTS.

I believe that my position is that the change was singular minded, not focused on secondary impacts. The cost here was silently making the system carry less integrity - silently.

'Sane heads' probably aren't interested in meaningless Apache benchmark or in benchmarking different kernels with different fs modes. There are other tools which are actively used like sysbench, LTP and others.

It's a canary. A canary has *NOTHING* to do with coal mining, but yet they used to be used since they would be a litmus test of the quality of the air - the death of a canary didn't differentiate between carbon monoxide or methane. It just reacted to the change in environment (and died).

LTP, sysbench, etc allow you to confirm which subsystem has changed and quantify the performance. You have to run a bucket load of tests (the entire LTP suite, etc) to see the system.

Generally, throughout the industry I have seen *lots* of people who do *really* well at the component level. The system level is a lot harder. So if you find a sensitive proxy, there is no problem in using it.

Again, the tests like apachebench are *NOT* intended to test apache. Since it's running on the same system, it measures the systems ability to do higly concurrent IO and memory access.

PTS used properly is a nice tool and those who are complaining aren't usually complaining about PTS, but about stupid titles or meaningless or stupid tests (like Apache or when someone says there's a regression, but different modes were used). If something bad is mentioned against the "Phoronix church" all of a sudden you become public enemy #1 in their eyes.

Hardly. The issue is with singular focused comments. If anything, these forums here are ripe and useful areas that at I learn about the real requirements and real interests of the community. Of course, if *ANYONE* wants to come up with a system-characterization test that can act as a canary, then I don't see Michael stopping it coming in.

[V!NCENT]

Btw. some people who're talking about Linux development model have no idea about it. There are no more "stable" releases (you can call, and they're called stable at kernel.org etc, but it's a different thing).

So you go ahead and down the latest git and find out that uhm... your server is generating 50% less pages ":>".

All this bitching about Linux development model which scares some people is plain stupid and it's very succesfull.

Right... My name is kraftman, the avarage computer user reads Phoronix (and is somehow interested in Linux he.. he he...), reads only that something terrible happened in the Linux kernel and suddenly runs to a store to buy Microsoft Windows...

Totally