The Huge Disaster Within The Linux 2.6.35 Kernel
Phoronix: The Huge Disaster Within The Linux 2.6.35 Kernel
For the past six months we have been monitoring the performance of the very latest Linux kernel code on a daily basis across multiple systems. We have spotted a few regressions -- both positive and negative -- on occasion using our automated daily testing of the Linux kernel, but nothing like what we have encountered the past few days: the Linux 2.6.35 kernel performance has fallen hard. In fact, the performance has fallen very hard in a number of tests and right now, we would consider it a disaster. While the 2.6.35 code has not even seen its first release candidate yet, there are some massive performance drops in a variety of different tests that have yet to be corrected and nothing like we have encountered with previous kernel release cycles especially for a regression that has lived now for about one week.
very curious what caused it- anybody know specifically?
yeah, I reported that. It's quite bad indeed. I first thought it was something related to udev and kernel changes but after upgrading to a newer build of udev (in Fedora 14/rawhide) it still remained.
Originally Posted by s4e8
It would be nice to know if these tests were done to see if udev or other processes were pegging the cpu.
Replying to myself is fun. They will revert the change.
Originally Posted by spstarr
Yep. This is a true disasticle alright.
It would be cool to monitor memory usage also, ever since 2.6.33 my memory usage increased by 300% compared to 2.6.22 http://bbs.archlinux.org/viewtopic.php?id=93523. I can't bisect it either since early builds of 2.6.33-rc1 panic on me .
After reading that conversation, it sounds like this issue is being addressed.
Originally Posted by s4e8
Just a few observations...
- The gist of the information in the article was relevant. A large kernel performance regression is a notable issue to be dealt with.
- The kernels in question were pre RC kernel snapshots pulled from GIT, and thus under heavy development.
- Some of the terminology in the article can be seen as inflamatory.
- BTRFS is also not production ready. Filesystems inparticular will often suffer steep regressions as necessary data protection mechanisms are implemented.
- Before Phoronix, I'm not aware of anybody doing real-world, comprehensive benchmarking against Linux that would quickly identify performance regressions such as this in a relatively consise manner.
- The author does not appear to have filed a bug or pinged the kernel mailing list to bring up the issue.
- A few of the Phoronix readers who happened on this article have researched the issue themselves and found that the kernel devs are already aware of the issue and are working to identify and correct it.
On one hand, I appreciate the fact that Michael took the time to write this article. The information is ultimately useful, and things like this are good to be informed about. On the other hand, the target audience eludes me. Was it intended for less technical people? My fear would be that such folks might read it and think, "Oh no, Linux performs like crap!" In fact, this is not really as pertinent as some of the hyperbolic language makes it out to be -- it is, as I mentioned, a pre-RC kernel under heavy development. If not for Linux kernel n00bs, is it for technical folks who can reproduce it and fix the issue? In the latter case, why not provide more information about the test stack (kernel configs, dmesg output, list of running processes, information on the base OS, etc)? More technical information would have been useful.
In summary, my personal thanks to Michael for a) the research and b) for the article. However, I also feel some of the criticism in this thread are valid. Maybe you should choose your words a bit more carefully in the future?
Did I ever post my code for determining whether it was CPU or I/O bound? I'm in the middle of another project at the moment, but meanwhile, here's the algorithm:
Tags for this Thread