OpenSuSE, Ubuntu, Fedora, Mandriva Benchmarks

[phoronix]

Phoronix: OpenSuSE, Ubuntu, Fedora, Mandriva Benchmarks

With it being a while since we last compared many Linux distributions when it comes to their measurable desktop performance, we decided to run a new round of tests atop four of the most popular Linux distributions: OpenSuSE, Ubuntu, Fedora, and Mandriva. To see where these Linux distributions are at, we used their latest development releases and then performed all package updates as of 2009-07-15. Following that, we ran an arsenal of tests using the Phoronix Test Suite. Here are the results.

http://www.phoronix.com/vr.php?view=14045

[Jimmy]

commmon Xorg... get 7.5 out in time for openSUSE 11.2! Oh and fix XkbSetDetectableAutoRepeat while you're at it. :D

Aside from that, having tried out openSUSE Factory, I mostly like where it's at (a few bugs to shake out still).

What's up with Fedora? I hope they can blame that on debug builds or something. Oh and Ubuntu can suck it and I'm sure Mandriva is cheating somehow on a few tests. (Calm down I'm just poking fun :D)

[hax0r]

Throw in archlinux next time :).

[MùPùF]

Throw in archlinux next time :).

I'm not sure Mickael would like to re-install ArchLinux everytime he does a test.

Even for me, as an arch user, it is not really straight forward and fast to install it. It usually takes me 2h (I have a really good internet provider) to get something working nicely. Ubuntu would have taken 30 minutes.

[chithanh]

It would have been good to include Gentoo in this comparison. I am curious why it was omitted. A test from the ISC (http://ftp.isc.org/isc/pubs/tn/ISC-TN-2008-1.html) in 2006 showed it to outperform other operating systems and distros when it comes to DNS performance. And installing Gentoo can be done in less than 30 minutes (http://www.youtube.com/watch?v=6XbEJoR-ypo), too. :)

[DDD78]

Please explain the Dbench result.

160MB/sec for Mandriva with an old WD1600JS-00M (7200RPM)?

[Obscene_CNN]

It would have been good to include Gentoo in this comparison. I am curious why it was omitted. A test from the ISC (http://ftp.isc.org/isc/pubs/tn/ISC-TN-2008-1.html) in 2006 showed it to outperform other operating systems and distros when it comes to DNS performance. And installing Gentoo can be done in less than 30 minutes (http://www.youtube.com/watch?v=6XbEJoR-ypo), too. :)

I too would like to see Gentoo benchmarked. However for Gentoo to even remotely achieve its potential a proper install must be done which recompiles everything with the CFLAGS properly defined for the processor. This takes more than 30 minutes.

I do know my Gentoo laptop is quite a bit quicker than desktops running Ubuntu with similar processors running 400MHz faster.

[Kano]

Some distros run background tasks like indexing. Those slow down the system. I am sure Kanotix Excalibur would be also fast on your PC as KDE 3.5 is faster than 4 anyway.

[bjrosen]

If you think SeLinux is the problem with Fedora why don't you turn it off? Turning off SeLinux is the first thing I do when I install Fedora, I suspect that's what most Fedora users do because SeLinux is useless outside of the enterprise and it causes som any problems.

[Michael]

If you think SeLinux is the problem with Fedora why don't you turn it off? Turning off SeLinux is the first thing I do when I install Fedora, I suspect that's what most Fedora users do because SeLinux is useless outside of the enterprise and it causes som any problems.

"All four distributions were left with their stock settings to represent an "out of the box" experience across all of them."

[mmmbop]

Fedora Rawhide clearly has all the debugging options on.

On my Fedora 11 system, memory usage more than doubles if I use the rawhide kernel.

IMHO it makes for a pointless benchmark.

[deanjo]

Fedora Rawhide clearly has all the debugging options on.

On my Fedora 11 system, memory usage more than doubles if I use the rawhide kernel.

IMHO it makes for a pointless benchmark.


Bingo, *_DEBUG=y is used everywhere in the rawhide kernel.

[bjrosen]

"All four distributions were left with their stock settings to represent an "out of the box" experience across all of them."

The whole point of using Linux vs that other OS is that it's infinitely tweakable. It would be really helpful to both Linux users and to Linux distro developers to know the reason for the performance differences that you find so that they can make the appropriate adjustments.

The distros that you tested were running the same revision components so you would have expected approximately the same performance. When you find a major discrepancy it's worth investigating. You had a couple of places where Fedora was underperforming by a huge amount. If it's something as simple as SeLinux (which can be disabled trivially, there is a menu item on the Administration menu, just flip it off and reboot) all Fedora users would want to know. Also this might convince RedHat to stop enabling it by default.

[mmmbop]

The problem is so clearly and obviously that rawhide has all the debug options on that it really must be repeated.

Rawhide is not a 'stock' Fedora at all. It is only useful for answering the questions: "Does this new code work?" and "The new code didn't work. What went wrong?"

It is not an indicator of future Fedora performance. It does not tell you how fast Fedora is. Its speed does not matter. Its speed is meaningless. How many ways can I say the same thing?

Even benchmarks of rawhide vs rawhide aren't useful. All that matters performance-wise is the optimized code that comes out when you tell the compiler to optimize things. Benchmark that.

It's like seeing how fast an athlete can run a marathon and also require that they go grocery shopping at every store along the way. They're not going to set any records that day.

[DanL]

Even for me, as an (Arch Linux) user, it is not really straight forward and fast to install it.
http://chakra-project.org/about-faq.html

[frantaylor]

This really is a useless article. Nobody is interested in benchmarking debug code.

I for one would like to a comparison of the RELEASED versions of OS's, and how things change over time. Even an older release like Fedora 10 will run faster now than it did when released, due to updates.

It would be interesting to benchmark the freshly installed OS right out of the can, before any updates are applied, and then benchmark again after bringing it up to date.

[ChuckDriver]

Does anybody know how RHEL does on the server oriented benchmarks? I'd hope better than Fedora, regardless of whether SELinux is enabled or not.

[energyman]

I'm not sure Mickael would like to re-install ArchLinux everytime he does a test.

Even for me, as an arch user, it is not really straight forward and fast to install it. It usually takes me 2h (I have a really good internet provider) to get something working nicely. Ubuntu would have taken 30 minutes.

two hours? You can get gentoo up and running in the same time....

[energyman]

Fedora Rawhide clearly has all the debugging options on.

On my Fedora 11 system, memory usage more than doubles if I use the rawhide kernel.

IMHO it makes for a pointless benchmark.

and the other ones do not?

rawhide lost badly against the other 'testing' versions, that should give you something to think about.

[energyman]

This really is a useless article. Nobody is interested in benchmarking debug code.

I for one would like to a comparison of the RELEASED versions of OS's, and how things change over time. Even an older release like Fedora 10 will run faster now than it did when released, due to updates.

It would be interesting to benchmark the freshly installed OS right out of the can, before any updates are applied, and then benchmark again after bringing it up to date.

already done:
http://www.phoronix.com/scan.php?page=article&item=fedora_9_11_perf&num=2

please enjoy the fact that rawhide was not so bad compared to the released versions, which makes it bad results comparted with osuse, brownone and mandrivel pretty damning.

[frantaylor]

already done:
http://www.phoronix.com/scan.php?page=article&item=fedora_9_11_perf&num=2



Almost. The problem is that even the releases are moving targets. New kernels come out all the time, and I wonder how the original kernel compares with the latest one.

Sometimes a bad benchmark result is due to a bug, and bugs sometimes get fixed. Sometimes a good benchmark result is all for naught, if an update to fix a security hole has the side effect of reducing throughput or adding delay.

Also, take a look at the apache benchmarks in the benchmark article you linked to. Rawhide's apache performance stinks there, too.

[energyman]

and some filesystems are optimized for benchmarks (ext3/4) and some for data safety. So what?

[deanjo]

BTW, small typo in the article Michael. It's openSUSE not openSuSE. I know, old habits die hard.;)

[shikamaru]

Please explain the Dbench result.

160MB/sec for Mandriva with an old WD1600JS-00M (7200RPM)?

and why it wasn't tested with ext4 filesystem too ;)

[deanjo]

and why it wasn't tested with ext4 filesystem too ;)

From the article:

All four distributions were left with their stock settings to represent an "out of the box" experience across all of them.Mandriva Cooker defaults to EXT3.

[sylware]

Indeed where is it?
ACCEPT_KEYWORDS="~x86-64" gcc 4.4.x(-O42) vanilla-sources-2.6.3x.x powa :p

From there I have an idea: use the instrumentation frameworks to benchmark finer. I'm thinking about the latency framework.

[energyman]

sylware - your Accep_Keywords are bogus.

[sandeen]

and the other ones do not?

rawhide lost badly against the other 'testing' versions, that should give you something to think about.

Unless you compare at least the debug options turned on in each, your statement doesn't mean much. Maybe they are similar, maybe not. But without a little useful analysis we don't know.

[energyman]

remember the test were different fedora versions were compared? rawhide wasn't slower than the others. In fact, in a lot of tests several 'real' fedora releases lost badly.

If rawhide is a 'faster' release - and still sucks compared to the other distributions than you must come to the conclusion that Fedora is dead slow.