Ubuntu 9.04 vs. Mac OS X 10.5.6 Benchmarks

[phoronix]

Phoronix: Ubuntu 9.04 vs. Mac OS X 10.5.6 Benchmarks

Last year we provided benchmarks looking at Ubuntu versus Mac OS X when it came to using the latest releases for both software platforms at the time. Both operating systems had performed competitively -- in some tests, the Apple OS wound up on top while in other areas Canonical had the advantage. Since that article back in November, Apple has released a minor update to Leopard (v10.5.6) and Canonical last month released Ubuntu 9.04. We have already looked at the performance of Ubuntu's Jaunty Jackalope, and even found it to perform with old hardware, but how does it now compete with Mac OS X? We have more benchmarks this morning to continue this performance investigation.

http://www.phoronix.com/vr.php?view=13822

[rohcQaH]

You just compared the performance of Linux vs. OS X on a computer designed for OS X. Now try the same on a computer designed for linux :D

[susikala]

You just compared the performance of Linux vs. OS X on a computer designed for OS X. Now try the same on a computer designed for linux :D

Doesn't that computer have a stock x86 CPU?

---

Also, from Phoronix's site description: 'Phoronix is the leading technology website for GNU/Linux and Solaris hardware news'.

I don't see any reason to post benchmarks of proprietary garbage here, honestly... there are enough Mac/Windows sites for that.

[bugmenot]

You just compared the performance of Linux vs. OS X on a computer designed for OS X. Now try the same on a computer designed for linux :D

Here is a benchmark:
http://grigio.org/ideneb_hackintosh_10_5_6_vs_ubuntu_linux_9_04_a6_b enchmarks

The video driver is all for the 3D performance.

Linux may be better for pure scientific algorithm tasks but it is full of hacks and regressions and if Compiz is enabled is even worse.
Mac rocks on Desktop and Netbook :P

[kraftman]

Also, from Phoronix's site description: 'Phoronix is the leading technology website for GNU/Linux and Solaris hardware news'.

Yeah, but they write Macos, before Solaris in articles and sometimes you can notice its advertisements. If Apple can why not advertising Microsoft too?

I don't see any reason to post benchmarks of proprietary garbage here, honestly... there are enough Mac/Windows sites for that.I do and I like it, but such benchmarks are only meaningful for users which don't tune their systems.

Overall, Mac OS X 10.5.6 already outpaces Ubuntu 9.04 when it comes to many respects of the desktop and server performance (http://www.phoronix.com/forums/../scan.php?page=article&item=ubuntu_jaunty_osx&num=12#).Yes, but in your benchmark :). I can't imagine people replacing Linux, Solaris or FreeBSD to Macos. In real world when comes to servers Macos just doesn't exist. I hope we'll see some benchmarks with Fedora 11 and 2.6.29.

[mendieta]

Michael, I know you mentioned this in the article, but the Intel drivers still have many issues in 9.04, of course it would be beat easily - This article is a reflection of that. It is not representative of ubuntu 9.04 vs OS X 10.5.6

A more fair comparison would be a desktop with a graphics card that is well supported, maybe at this point some NVIDIA card, or one of the ATI's that are acutally running with fglrx in 9.04 ;-)

[deanjo]

You just compared the performance of Linux vs. OS X on a computer designed for OS X. Now try the same on a computer designed for linux :D

Well, given that almost every piece of PC hardware is "designed" for windows, happy hardware hunting.

The computers "design" is a generic x86 design, you can find the same basic machine from aopen (although speced the same it costs more then the mini)

Even if you were to use "hackintosh" methods your benchmark delta would be much the same.


Also, from Phoronix's site description: 'Phoronix is the leading technology website for GNU/Linux and Solaris hardware news'.

I don't see any reason to post benchmarks of proprietary garbage here, honestly... there are enough Mac/Windows sites for that.

It's always a great idea to test your product against the competitors. It can expose area's of weakness that needs improving.


Yes, but in your benchmark :). I can't imagine people replacing Linux, Solaris or FreeBSD to Macos. In real world when comes to servers Macos just doesn't exist.

Your right, there are not a lot of OS X servers, most xServes are used in a super computer or renderfarm fashion or serve OS X networks. That is xServes target audience.

[kraftman]

It's always a great idea to test your product against the competitors. It can expose area's of weakness that needs improving.


That's right, but Phoronix benchmarks don't help. Maybe just when comes to OS Intel drivers.

[spykes]

This is interesting as it shows weaknesses and regressions.
I hope to see the same bench/comparison after the release of Fedora 11 (because it's what I'll use).
Fedora is more "bleeding edge" than Ubuntu, so I hope to see fixes especially regarding the kernel area, for the SQLite issue.

[krazy]

Thanks for the benchmarks michael. Haven't seen any along these lines before. Interesting java results. It will be very interesting to see 2.6.30+ based distros vs snow leopard in a few months time!

[bugmenot]

i'd like to see this benchmark with a Macbook with an Nvidia card..

[Sewje]

This comparison is flawed, it's a know issue in Jaunty that the intel drivers have problems with the new Xorg implementation. Even ATIs driver is broken in Jaunty, the best system to test would be one with nVidia GFX cards.

[Danyul]

I understand you perform the tests for out of the box solutions. But, the MacOS kernel is obviously optimised for the hardware they put in there. Why not compile a custom linux kernel that cuts out all the crap that is there for the wide range of compatibility.

Also, the intel driver that shipped with 9.04 is the WORST graphics driver I have ever come across performance and stability wise (and I worked through all the ATI mess). My wife's laptop has intel and upon installing 9.04 totem would not work, the computer locked up frequently (once every 2 - 10 mins), and when the computer WAS running it performed like a 2 legged greyhound.

On my computer I cut the crap outa the kernel and I get substantial increases in performance. When I regressed my wife's computer back to the intel driver included in 8.10 the computer stopped locking up and worked better than it had in 8.10.

All I'm saying is, while I believe MacOS will still come out on top in a few areas, linux doesn't stand a chance when you put something that is not optimised up against something that is.

[yotambien]

I don't get this. So, since it is known that Ubuntu 9.04 has quite a problem with intel cards, the test is flawed, unfair and not representative?

On the contrary. This is the sort of representative shit you get with Ubuntu (and with whatever distro that plays bleeding edge), and the benchmark just shows it. Perhaps you would suggest to use an older version that doesn't have those performance regressions? Ah, but then you would point out that the GCC versions were different or some bull like that.

It is very simple, folks. You buy a Mac, you install Ubuntu on it--TODAY--and you check what runs better. What's your problem? Kernel optimisations, different gfx manufacturer, hardware targeted at linux...jeez, all this crying is making me sad.

[kraftman]

I don't get this. So, since it is known that Ubuntu 9.04 has quite a problem with intel cards, the test is flawed, unfair and not representative?

On the contrary. This is the sort of representative shit you get with Ubuntu (and with whatever distro that plays bleeding edge), and the benchmark just shows it. Perhaps you would suggest to use an older version that doesn't have those performance regressions? Ah, but then you would point out that the GCC versions were different or some bull like that.

It is very simple, folks. You buy a Mac, you install Ubuntu on it--TODAY--and you check what runs better. What's your problem? Kernel optimisations, different gfx manufacturer, hardware targeted at linux...jeez, all this crying is making me sad.

Actually Ubuntu isn't bleeding edge and those results have nothing to do with real life experience (except slow graphic performance). The point is those benchmarks don't show what runs better (except 2D and 3D). Don't read and you wouldn't cry ;)

P.S. Do you consider proper kernel config and benchmarks used in proper way don't have influence on results?

[mendieta]

I think Michael earned the respect of the community with all his hard work, including a wealth of info on hardware that runs in linux (and how it runs), and the test suite work.

Having said that, I think this article sounds (unintentionally) like FUD, the way it was posted, and it would be nice of Michael to update it a bit, maybe change the title (this is _not_ a representative comparison of the operating systems), etc.

Cheers, and peace out :D

[spykes]

And what about the X86-64 version ?
It was performing better with the 64 bits version on the previous bench.
I am no more using a 32 bits distro those days.

[kraftman]

@mendieta

Micheal benchmarked those systems in their default configurations. But FUD in my opinion is quite good term here. I wonder who's interested in sqlite, postgre, unixbench tests if their results are meaningless probably for everyone in this case (no influence on desktop experience and meaningless for people interested in real world performance)? As a desktop user I would be more interested in system start up time, working or not hibernation, number of available apps, instalation time etc.

[SyXbiT]

embarrassing for us linux users...
we can't claim better usability
can't claim more compatibility
and now, not performance
just that it's open source, and stable/secure

[joshuapurcell]

SyXbit: The comparison is not embarrassing in any way. Read the rest of this thread and come to an understanding of what is actually being compared, what open source is, and how it all comes down to how various projects each with different development timeframes are able to come together at a specific point and compete against an operating system that has a single development timeframe, a single architecture, and a single developer/company behind it. Even under these circumstances this Linux system does relatively well... if you start working on optimization (which has already been done on the Mac side) then you would get even better results than what is posted here. Nothing embarrassing about that.

Another thing regarding compatibility... without a doubt Linux is more compatible than any other operating system with the available hardware. Not sure what else to say about that.

[yotambien]

Actually Ubuntu isn't bleeding edge and those results have nothing to do with real life experience (except slow graphic performance). The point is those benchmarks don't show what runs better (except 2D and 3D). Don't read and you wouldn't cry ;)

P.S. Do you consider proper kernel config and benchmarks used in proper way don't have influence on results?

Several things here.

Ubuntu certainly is a distribution that follows closely the newest software developments. Not only because every six months is taken from Debian sid, but more importantly because the Ubuntu team has a preference for the untested and experimental options. Call it what you like, the point doesn't change.

The benchmarks. Maybe they are not real life situations, as in they're not measuring the cpu load while watching a youtube video and opening this particular pdf file I've been sent today; however, they put numbers to pretty reproducible computing tasks that stress different parts of the system. So, I don't spend the day signing 4096-bit RSA keys, but I surely know I prefer my computer to do it faster rather than slower.

As for the 'proper' kernel config...next time you take a look at a car magazine and see a comparison between two cars, don't forget to complain because they didn't tweak the suspensions or added nitrous oxide to the engine.


PS. In any case, no, I don't think that the kernel configuration has a critical effect in anything regarding performance.

[oyvind]

embarrassing for us linux users...
we can't claim better usability
can't claim more compatibility
and now, not performance
just that it's open source, and stable/secure

Well, I'm sad to say that Linux has regressed quite a bit stability-wise as well, because of the graphics mess. So then there's secure and open source left.

Don't come here and laugh about Windows BSODs, Linux crashes just as much. When KMS becomes widely available, we can have our own "BSOD": the PSOD (Penguin Screen Of Death).

[kraftman]

Several things here.

Ubuntu certainly is a distribution that follows closely the newest software developments. Not only because every six months is taken from Debian sid, but more importantly because the Ubuntu team has a preference for the untested and experimental options. Call it what you like, the point doesn't change.

Some packages, drivers are experimental, but entire distribution isn't using the newest software. Kernel isn't the newest. Call it what you like, but reality doesn't change.

The benchmarks. Maybe they are not real life situations, as in they're not measuring the cpu load while watching a youtube video and opening this particular pdf file I've been sent today; however, they put numbers to pretty reproducible computing tasks that stress different parts of the system. So, I don't spend the day signing 4096-bit RSA keys, but I surely know I prefer my computer to doing it faster rather than slower.The question is to do what is faster then slower?

As for the 'proper' kernel config...next time you take a look at a car magazine and see a comparison between two cars, don't forget to complain because they didn't tweak the suspensions or added nitrous oxide to the engine.Do the same. One car is a jeep and second car is a passenger car. Don't forget to complain, because passenger car is adapted for different tasks then the jeep.

PS. In any case, no, I don't think that the kernel configuration has a critical effect in anything regarding performance.It actually has. That's why there's server and desktop (however it has a different approach then Windows or OS X and when compared to them it's more server then desktop system) version of Ubuntu etc.

@joshuapurcell

if you start working on optimization (which has already been done on the Mac side) then you would get even better results than what is posted here. Nothing embarrassing about that.There's no optimization needed for Linux to perform ways better in those tests (oh, maybe it's actually performing very well). Numbers are just not enough in this case to make clear judgment. An only area where Linux is loosing is graphics (and marketshare ;))

[Kano]

Well to be fair, no Mac user would ever run those things which are benchmarked here. It does not matter if it is faster or slower, the target Mac user does not run phoronix-test-suite. U 9.04 has well known intel gfx problems, so it is hard to win anything gfx related.

[kraftman]

Well to be fair, no Mac user would ever run those things which are benchmarked here. It does not matter if it is faster or slower, the target Mac user does not run phoronix-test-suite. U 9.04 has well known intel gfx problems, so it is hard to win anything gfx related.

Exactly, but such numbers are nice advertisement :>

It's often the case that desktop systems will cheat at writes while servers don'tUbuntu desktop edition is in my opinion more server then desktop system (of course when compared to Windows and OS X) and maybe that's why it's 'loosing' in many tests here.

[yotambien]

Some packages, drivers are experimental, but entire distribution isn't using the newest software. Kernel isn't the newest. Call it what you like, but reality doesn't change.

Tell me which distributions, out of the main ones, update the whole stack every six months. And add to that the known preference of Ubuntu for new and flashy bits that break the system in many colourful ways.


The question is to do what is faster then slower?

I'm not sure I understand you here...but...yes?


Do the same. One car is a jeep and second car is a passenger car. Don't forget to complain, because passenger car is adapted for different tasks then the jeep.

I don't complain. The analogy doesn't hold. Two operative systems targeted at desktop users, one computer, one benchmark. What's wrong?


It actually has. That's why there's server and desktop (however it has a different approach then Windows or OS X and when compared to them it's more server then desktop system) version of Ubuntu etc.

You are playing here. You know perfectly well that they didn't benchmark Ubuntu server edition. And you know that the differences between the server and desktop versions lie more in the packages than in the kernel included. So, give me the numbers showing that changing the kernel configuration from the default one has a critical and positive impact on performance.

Well to be fair, no Mac user would ever run those things which are benchmarked here. It does not matter if it is faster or slower, the target Mac user does not run phoronix-test-suite.

Because of course, you know what Mac users run or not. The important point here is the difference between the benchmark applications used and what is actually being benchmarked. According to you, because Ubuntu user X doesn't play Tremulous, this benchmark doesn't mean anything to him/her.


U 9.04 has well known intel gfx problems, so it is hard to win anything gfx related.

Well, that's just too bad. Again, that it is a known fact doesn't change a thing.

Insert coin, please (in six months time).

[Odon]

A few observations:
1. I find it a bit odd that the x86_64 variant of Ubuntu had been left out from the benchmarks. The previous comparison (10.5.5 vs. 8.10) showed visible improvements in several non-GUI related tests

2. The set of tests is changing as the test suite is being developed. This makes it really difficult to make a comparison between the test rounds. It would be interesting to have a direct comparison between OS X 10.5.5 and 10.5.6 on the same hardware with the same set of benchmarks

3. I looked at the performance comparison between the Linux kernels so as to make some conclusions myself. What a surprise again another set of benchmarks

[thefirstm]

x86_64, please?

This would be a much more valid comparison. Most modern computers are 64-bit, and the popularity of 64-bit OSes is increasing. Very soon, MacOS as well will be able to do 64, so 64-bit would be a much better comparison in this case. Also, please lose the Intel gfx card. We all know that the Intel driver is undergoing a major restructuring right now. It would be a much more fair comparison to use an Nvidia graphics card, where Ubuntu would be able to perform much more strongly.

[deanjo]

Well to be fair, no Mac user would ever run those things which are benchmarked here. It does not matter if it is faster or slower, the target Mac user does not run phoronix-test-suite. U 9.04 has well known intel gfx problems, so it is hard to win anything gfx related.


I would have to disagree Kano. Content creation, sqlite performance, compression disk performance and somewhat lesser gaming are all key components in the average Mac user's desktop life. SQLite performance , compression, and disk performance are key systems that effect how well items such as Mail, Time Machine, Safari, Finder, Spotlight, iTunes, etc etc deliver the end user experience. Java does as well and always has been an archillies heal with OS X (thanks Sun). The only real tests that really do not apply to the average user are items such as compile time which is usually a one shot deal, compile once and forget about afterwards.

For all of you screaming "Use Nvidia or ATI and frglx" that actually kind of slaps the face of what linux is trying to achieve and as well Michael can't simply "slap in a ati or nvidia" card. It's a Mac Mini. It's a small form factor with IGP. So unless you want to donate cash to him so he can either afford a Mac with discreet graphics (or just one of the new ones with a nvidia chipset) your asking him to do the impossible. I might also add that out of the box Ubuntu does not have those blobs included. It would default to using the ati opensource or nvidia opensource drivers anyways.

This comparison is completely valid. Good, bad and ugly no matter what OS you like. I find it funny that love linux are basically saying that "it sucks without propriatary blobs".

As far as the "OS X is optimized for this machine" arguement goes, well here is some news for you. OS X 10.5 is compiled to the lowest capabilities of the first intel based machine that debuted in 2006 (lowest end intel proc is a Intel Core, not even a duo or Core 2). This means the kernel and supporting libraries was not compiled with all the advanced features that newer revisions of the intel chip came out with. Tweaking? Sure there is a lot of tweaking that could be done on OS X too from the base setup if the person wants to put the time and effor into it. OS X by default is running several processes such as Spotlight (OS X's indexing service) that slow the system down just for one example.

[drag]

Hrm.

I was never impressed with Ubuntu's technical prowless. They are good at providing a decent user experience, but otherwise they are lacking in lots of technical know-how.


I am much more interested in the results of Fedora 11 because that more accurately shows the state of the art. I am not expecting a huge performance difference between Ubuntu vs Fedora... it's just that Fedora is going to be in a better state of tune.

[drag]

Oh. And Microbencharks are worth only so much and can be very misleading.

For example, the SQLite benchmark can be easily fooled by the file system.

If SQLite is requesting a fsync() and the file system says it does a fsync()... but doesn't actually sync the contents of the file to the harddrive, then yes I can easily make something that will perform 1000x as fast as Linux does with Ext3.

[thefirstm]

And another thing. This benchmarking suite is ALPHA software. It could very well be screwing in the upwards direction. It seems very unrealistic that two different UNIX-like OSes running on the same hardware could have such a huge difference in the instructions-per-second benchmarks.

[deanjo]

Oh. And Microbencharks are worth only so much and can be very misleading.

For example, the SQLite benchmark can be easily fooled by the file system.

If SQLite is requesting a fsync() and the file system says it does a fsync()... but doesn't actually sync the contents of the file to the harddrive, then yes I can easily make something that will perform 1000x as fast as Linux does with Ext3.

Actually when it comes to SQLite in OS X, SQLite sends a F_FULLFSYNC request to the kernel to ensures that the bytes are actually written through to the drive platter. This causes the kernel to flush all buffers to the drives and causes the drives to flush their track caches. So if anything, OS X should be slower.

[deanjo]

And another thing. This benchmarking suite is ALPHA software. It could very well be screwing in the upwards direction. It seems very unrealistic that two different UNIX-like OSes running on the same hardware could have such a huge difference in the instructions-per-second benchmarks.

The suite itself does not determine the benchmark results. The suite is simply a front end for running those individual benchmarks and parsing the results they give.

[kwag]

To format the MAC Mini with Ubuntu 9.04, and then run the benchmarks again.
Now chart the results!
Then nobody can bitch about unfair test (hardware wise, that is).

Cheers

[Yfrwlf]

that it is a known fact doesn't change a thing.

It doesn't change the results of the test, no, but that doesn't change the fact that it's an important point, either. If OS X had some small change in a particular version that created a huge regression in performance, that wouldn't be any less relevant to note. What does that fact tell you? That the test is skewed in a sense, because it's not quite representative of the "normal performance" you'd find perhaps in any other version. So while it does mean that that specific version contains regressions which is important, it's not the best test to use to really gauge the "capability" if two competing operating systems in the long term.

Basically, be as wise to what is going on as possible so you can get a comparison that is the most "fair". All software needs critics.

But yes, try the same test using Nvidia or ATI graphics, or basically on a machine that has common hardware that isn't nitch to OS X, and that might be a much more fair test. Might be. The problem is, if OS X and Linux are never really geared to run on each other's hardware and code/drivers were never really optimized for it, there might not be any way to really have a "fair" test. Fair is always to some degree an illusion, but at least you can try your best to get close. ^^

[Yfrwlf]

To format the MAC Mini with Ubuntu 9.04, and then run the benchmarks again.
Now chart the results!
Then nobody can bitch about unfair test (hardware wise, that is).

Cheers

Yes you can. For one thing, there's never "totally fair", you can always just get "closer to fair", so there will *always* be a way to bitch. But in this case, if OS X has really good Intel video drivers because the developers actually spent a lot of time on them optimizing them for that OS, and Linux for whatever reason has crappy drivers (but may be increasing rapidly in quality and capability) that are behind that if OS X's Intel driver performance and features still, and then you compare both OSes and use graphics tests, obviously the OS X tests will win.

So now it comes down to should we run tests on Linux's known best hardware, or OS X's known best hardware, or try to find a happy middle ground somehow? That's pretty difficult to do. I mean, I don't even know if there are ANY *standard* Nvidia or ATI cards that work with OS X, I thought they were basically all "the Mac version". They shouldn't have any major differences, but who knows, the fact that they aren't mainstream cards means the Linux drivers might not have cared too much about them and thus they might suck.

But yeah, I'd be curious to see ATI and Nvidia tests as well as tests with Fedora 11 if some of the regressions have been fixed in it.

[damentz]

Doesn't that computer have a stock x86 CPU?

---

Also, from Phoronix's site description: 'Phoronix is the leading technology website for GNU/Linux and Solaris hardware news'.

I don't see any reason to post benchmarks of proprietary garbage here, honestly... there are enough Mac/Windows sites for that.

“If you know the enemy and know yourself, you need not fear the result of a hundred battles. If you know yourself but not your enemy, for every victory gained you will also suffer a defeat. If you know neither the enemy nor yourself, you will succumb in every battle.”

This is not war, but this quote is applicable to most every day life. You're dumb for excluding competition because they didn't tell you how they tied their shoe laces.

[deanjo]

Yes you can. For one thing, there's never "totally fair", you can always just get "closer to fair", so there will *always* be a way to bitch. But in this case, if OS X has really good Intel video drivers because the developers actually spent a lot of time on them optimizing them for that OS, and Linux for whatever reason has crappy drivers (but may be increasing rapidly in quality and capability) that are behind that if OS X's Intel driver performance and features still, and then you compare both OSes and use graphics tests, obviously the OS X tests will win.

So now it comes down to should we run tests on Linux's known best hardware, or OS X's known best hardware, or try to find a happy middle ground somehow? That's pretty difficult to do. I mean, I don't even know if there are ANY *standard* Nvidia or ATI cards that work with OS X, I thought they were basically all "the Mac version". They shouldn't have any major differences, but who knows, the fact that they aren't mainstream cards means the Linux drivers might not have cared too much about them and thus they might suck.

But yeah, I'd be curious to see ATI and Nvidia tests as well as tests with Fedora 11 if some of the regressions have been fixed in it.

OK first of all the "Mac Editions" of the hardware are EFI ready. Once the video is initialized it identical to the function of any PC card. Thus is the reason why you can use plain jane Nvidia driver when running windows on a mac. Any nvidia/ati card given that they have a large enough bios rom can be flashed to their mac counterparts and vice versa. Spec out a PC to the exact same hardware as a Mac, install the same version of linux on both and they will give you identical results. The only difference between a PC and a modern Mac hardware wise is the EFI capability. EFI on PC's are still generally only found on server/workstation class boards.

Sure you may get close to comparable performance using blobs on linux. Part of that is because you would be getting optimized GL stacks instead of dealing with a generic "one-size-fits-all" mesa and blobs bypass much of X. But then again you would be going outside the "purity" of linux. Open source video solutions, frankly, suck for the most part on anything requiring more demanding that benifits from a optimized stack. They are supposed to be a "just work out of the box" solution, they are not how ever a "work out of the box and totally kick ass in performace" solution. You want max performance, you go closed sourced blobs instead of a generic solution.

OS X is not worried about a "open" video solution for their products. They use blobs for the best performance because when you look at their primary competitor, windows, all they care is about performance as well, as does 90+ % of the world does when it comes to desktop usage.

Bottom line is that put pure OS X against a "pure" ubuntu on the same hardware OS X will come on top on most areas that the average desktop user will use. No matter what the reasons are, the results still stand valid. The hardware does not contain any "specially designed hardware just for Apple". These are off the shelf components.

[Hephasteus]

Actually when it comes to SQLite in OS X, SQLite sends a F_FULLFSYNC request to the kernel to ensures that the bytes are actually written through to the drive platter. This causes the kernel to flush all buffers to the drives and causes the drives to flush their track caches. So if anything, OS X should be slower.

SQL is threaded to high heaven. OS/X is a microkernel OS. It loves threads. It's built from threads. A microkernal OS will always destroy a giant kernel OS in SQL and graphics.
You'd need GNU Debian HURD to even play in the same ballpark and that's like barely existing. Linux and unix's will likely evolve into macrokernel like OS's just because IBM, nvidia, ati, AMD etc all need them to be like that so OpenCL doesn't turn into a turd for linux. IBM probably already has it that way for custom CELL implimentations that it doesn't share with the general public.

http://en.wikipedia.org/wiki/GNU_Hurd

But judging from the size of kernels in linux these days. They seem to be making them more and more monolithic for now. I think once the kernel reaches about 250mb compiled and has drivers built in for everything they will stop and turn around.

[kraftman]

Actually when it comes to SQLite in OS X, SQLite sends a F_FULLFSYNC request to the kernel to ensures that the bytes are actually written through to the drive platter. This causes the kernel to flush all buffers to the drives and causes the drives to flush their track caches. So if anything, OS X should be slower.

Some guy tried to proof OS X isn't eight times slower then Linux in PostgreSQL or MySQL using the same argument :>

http://archives.free.net.ph/message/20071128.081908.201f71b8.pl.html

http://www.anandtech.com/mac/showdoc.aspx?i=2520&p=7

[kraftman]

SQL is threaded to high heaven. OS/X is a microkernel OS. It loves threads. It's built from threads. A microkernal OS will always destroy a giant kernel OS in SQL and graphics.
You'd need GNU Debian HURD to even play in the same ballpark and that's like barely existing. Linux and unix's will likely evolve into macrokernel like OS's just because IBM, nvidia, ati, AMD etc all need them to be like that so OpenCL doesn't turn into a turd for linux.

You're totally wrong. OS X doesn't have a chance when comes to real world benchmarks in SQL databases (but it's still better then Windows there). Hurd is slow piece of crap and it will probably never reach Linux or any Unix like system performance.

[kraftman]

Tell me which distributions, out of the main ones, update the whole stack every six months. And add to that the known preference of Ubuntu for new and flashy bits that break the system in many colourful ways.

When comes to bleeding edge software I always have Fedora on mind :) Btw. I never suffered from brokenness in Ubuntu.

I'm not sure I understand you here...but...yes?You missed my point here, but maybe I was wrong. Read Deanjo response to Kano post.

I don't complain. The analogy doesn't hold. Two operative systems targeted at desktop users, one computer, one benchmark. What's wrong?The point is those two operating systems have different approach to desktop. Ubuntu is still closer to server then to desktop in my opinion, but maybe that's intended.

You are playing here. You know perfectly well that they didn't benchmark Ubuntu server edition. And you know that the differences between the server and desktop versions lie more in the packages than in the kernel included. So, give me the numbers showing that changing the kernel configuration from the default one has a critical and positive impact on performance.The disk I/O scheduller is different in server edition, default Hz value is also different. You would probably see better results, but still "worse" then OS X, because Linux isn't buffering disk writes while OS X is.

Because of course, you know what Mac users run or not. The important point here is the difference between the benchmark applications used and what is actually being benchmarked. According to you, because Ubuntu user X doesn't play Tremulous, this benchmark doesn't mean anything to him/her.I said except 2D and 3D :) Those tests are really fair and they show Intel drivers sucks a lot right now (maybe not in 2D when compared to other cards).

Well, that's just too bad. Again, that it is a known fact doesn't change a thing.

Insert coin, please (in six months time).My coin is real world benchmark, but I just want to keep on this topic.

[drag]

Actually when it comes to SQLite in OS X, SQLite sends a F_FULLFSYNC request to the kernel to ensures that the bytes are actually written through to the drive platter. This causes the kernel to flush all buffers to the drives and causes the drives to flush their track caches. So if anything, OS X should be slower.


Well... The kernel can't do that. The firmware on the harddrives decide what gets written to the platter. That sort of thing isn't going to be something the kernel has control over.

The best the OS can do is flush it's own buffers and then send a request to the drive to flush it's buffers and sometimes the firmware lies about it and other times it doesn't. It usually lies.

but that sort of behavior would affect both Linux and OS X equally so it wouldn't give a performance advantage to either OS over the other.

Otherwise that's a nice insight and I am still reading other replies in this thread.

I'm just nit picking.

[drag]

SQL is threaded to high heaven. OS/X is a microkernel OS. It loves threads. It's built from threads. A microkernal OS will always destroy a giant kernel OS in SQL and graphics.

Holy crap, NO.

First off.. The kernel OS X uses is NOT A MICROKERNEL.

The OS X kernel is called XNU. It's a so-called 'hybrid' kernel that uses code from a development kernel that died in 1995 combined with BSD stuff. The Mach kernel at different times in it's history was a microkernel and then not a microkernel. OS X does not use a Microkernel.

The Windows NT kernel was another one that was based on a Microkernel design, but is not a Microkernel. Early versions of NT were microkernels, but unfortunately for that design Microsoft could not figure out how to make it scale and the excess overhead caused by the message-passing design doomed it. So later versions of the NT kernel were monolythic.

If you want to you can call them 'hybrid kernels', but I think that is just a made-up term to make the OS kernel sound all microkernel-ish and cool while it is, in fact, a modular monolythic design.


You'd need GNU Debian HURD to even play in the same ballpark and that's like barely existing. Linux and unix's will likely evolve into macrokernel like OS's just because IBM, nvidia, ati, AMD etc all need them to be like that so OpenCL doesn't turn into a turd for linux. IBM probably already has it that way for custom CELL implimentations that it doesn't share with the general public.

Nope. Not going to happen. Microkernels were essentially a pipe dream and only one Microkernel-based OS actually made it into widespread use. That OS was QNX and was popular for embedded systems due to it's realtime-like nature.

But it wouldn't scale to anything big and nobody wanted to use it as a desktop or server platform.

[drag]

The reality of the matter is that with this particular computer OS X is able to offer superior performance over Linux in many of the benchmarks.

It's obvious that OS X developers put a lot of time and effort optimizing and tweaking OS X to work as fast as possible on this machine.

That is one of the nice things about having a sharp focus. OS X developers are able to concentrate on specific hardware configurations and on specific uses. While Linux developers are producing a much more general purpose OS and it is much more modular, portable, and diverse. There are lots of layers and complexity that are designed into the system to allow this.

There is a definate penalty for doing this. In the past Linux was almost always faster then OS X.. but nowadays not so much.. OS X has caught up apparently. It would be interesting to see benchmarks of higher level of complexity that create higher loads on a system and exhaust the file systme buffers and see if the Linux scedualling and scalability come into play.

Also it would be interesting to see the performance on higher end hardware.

-------------------------------


As far as proprietary drivers go... Nvidia uses the same general OpenGL and driver code for all the OSes that they produce the drivers for.

For the longest time Apples sucked for graphics performance because it was Apple developers who were writing the drivers... but apparently they've decided to switch to the approach that Linux users use... which is to shove a bunch of code originally designed and developed for Windows into their systems.

----------------------

Keep in mind that there are still other benefits from using the Linux approach to modular operating systems.

The package system is superior to what Apple uses... they are able to make things pretty usable and very easy, but technically the package repository approach is a superior one.

Also Linux has much much better hardware support.

A few examples of my machines...
A)
A Dell Mini-9 with 16GB of disk and 1GB of RAM. OS X can install on it, but the wireless network is flaky, the wired port is unusable, the sound is flaky, and it takes a lot of effort to get hardware working as well as it can. And on this class of hardware OS X runs poorly.

Meanwhile with Linux on that system I have a snappy system with somewhat lousy graphics performance.. but compiz still works and is quite usable with little effort.

Uses about 256MB of RAM ideling with a browser window or two open.

B)
This laptop. Running Fedora 11 beta. Core 2 duo with 4GB of RAM, 320 gig 7200rpm harddrive, intel graphics, etc.

Got at a fraction of the price that similar hardware would cost from Apple.

Fedora runs great. Suspend works. Wifi works. My wacom tablet is now completely plug-n-play with extended input support and everything.

On top of that I have KVM which I use with Virt-manager. I have XP, Vista, Windows 7, Ubuntu, Debian, FreeBSD, NetBSD, and even OSX installed in the VM. All of them work pretty well except for OSX..

I use rdesktop to access the XP VM for times when I need to use MS Office.

The thing is fabulous and is a hog. With XP and full blown Gnome running on it consumes almost a 1 and a half gigs of RAM ideling. This is in 'high productivity' mode for when I need to get work for work done.

C)
My Dell Inspiron 4100.

A fabulous machine, it's being slowly being turned into a lovely hack box.

Uses Debian Sid. One of hte best operating systems you could ever use for anything. Usuability sucks somewhat, but I dont' care... this thing is for fun. This I rescued from work. They were going to throw it away. To slow, ran like crap, crashed, hanged, refused to boot... they thought it was hardware issues... It was just Windows sucking so hard it removed all the oxygen from the room.

Pentium III 1.13ghz, 512MB of RAM, 40GB harddrive, some ancient radeon mobility with 16MB of vram. I discarded the DVD drive and jammed both modular bays with batteries with the most life left I could fine. Has about 4-5 hours battery life.

This thing is _FANTASTIC_. Using Debian Sid it is running better then it ever did in it's entire life. Seriously. Blows XP right out of the water.

Midori runs fast and is very responsive. Uses about half the RAM that Firefox does. I am using the LXDE desktop environment. I added on some Gnome features I 'require' like network-manager and gnome-power-manager, but it's still mostly LXDE.

LXDE is to XCFE what XCFE is to Vista.

With the couple gnome-add-ons I have the system uses a total of 50MB memory to run, before openning up any applications. Suspend works perfectly. Suspend-to-disk and Supsend-to-ram.

I am going to get a couple Alfa 500mw 802.11g USB adapters, make some homemade high-gain directional antennas.. get that working with my GPS devices and some mapping software and go have some fun wardriving. With that configuration I should be able to pick up a wifi access point for up to a mile away.

I can watch DVD rip movies on it. It's fast, it's repsonsive it sips only a small amount of power (For this era of laptop) and there is no way in hell you could ever possibly get OS X (Or Vista, or Windows 7) to run nearly as well.

[jybumaat]

I think the best thing to test these 2 operating systems is to install ubuntu 9.04 as a sole OS on a mac computer and the other one is a mac with OS X 10.5.6 installed natively.

Or we could use similar hardware configuration to the mac computer with an PC intel X86 computer. That will be very exciting to see...:D

[kraftman]

I think the best thing to test these 2 operating systems is to install ubuntu 9.04 as a sole OS on a mac computer and the other one is a mac with OS X 10.5.6 installed natively.

Or we could use similar hardware configuration to the mac computer with an PC intel X86 computer. That will be very exciting to see...:D

Hardware has probably nothing to those results. Using noatime, using EXT4 with delayed allocation will probably speed up things a lot. Sadly, Phoronix benchmarks only defaults when comes to operating systems and then some people believe OS X is faster in Postgre or Mysql then Linux, FreeBSD, Solaris...

P.S. Why there's so many OS X advertisements now? On the left, on the right on top... :>

[teekee]

Personally I find the benchmark incorrect because it compares two systems that operate in different modes. In my opinion (and please correct me if I'm wrong), Mac OS X, runs 64bit applications even though the kernel itself runs in 32bit mode:

Because device drivers in operating systems with monolithic kernels, and in many operating systems with hybrid kernels, execute within the operating system kernel, it is possible to run the kernel as a 32-bit process while still supporting 64-bit user processes. This provides the memory and performance benefits of 64-bit for users without breaking binary compatibility with existing 32-bit device drivers, at the cost of some additional overhead within the kernel. This is the mechanism by which Mac OS X enables 64-bit processes while still supporting 32-bit device drivers.

(from http://en.wikipedia.org/wiki/64-bit)

Mac OS X uses an extension of the Universal binary format to package 32- and 64-bit versions of application and library code into a single file; the most appropriate version is automatically selected at load time.
...
Mac OS X v10.4.7 and higher versions of Mac OS X v10.4 run 64-bit command-line tools using the POSIX and math libraries on 64-bit Intel-based machines, just as all versions of Mac OS X v10.4 and higher run them on 64-bit PowerPC machines. No other libraries or frameworks work with 64-bit applications in Mac OS X v10.4

(from http://en.wikipedia.org/wiki/X86-64)

So what happens in my opinion is that the tests are executed in 64bit mode, and, not surprisingly, the results of Ubuntu are worse in this case. What brought me to this idea is the extremely bad performance of Crafty, while there should be no difference in performance at all. What Crafty does is mainly evaluating moves, accessing transposition table and using threads to utilize more processors. So apart from the thread usage, which involves system calls (but Linux is generally fast when creating threads), the rest of the program avoids any system calls and should be OS independent. On the other hand, 64bit versus 32bit application performance is very noticeable because of more efficient chessboard representation in 64bit mode.

In my opinion, when comparing Mac OS X and Ubuntu 64bit the difference of these two systems will be marginal (of course apart from the graphics benchmarks and possibly that SQLite regression). I would even expect Ubuntu to be slightly better. I also think that ext4 should be used for comparison - first, it will probably become the default file system in 9.10, second, it would be more interesting to see the performance of "the state of the art" FS (well, almost, until replaced by something better like btrfs) instead of an old FS with known performance issues.

[deanjo]

Well... The kernel can't do that. The firmware on the harddrives decide what gets written to the platter. That sort of thing isn't going to be something the kernel has control over.

The best the OS can do is flush it's own buffers and then send a request to the drive to flush it's buffers and sometimes the firmware lies about it and other times it doesn't. It usually lies.

but that sort of behavior would affect both Linux and OS X equally so it wouldn't give a performance advantage to either OS over the other.

Otherwise that's a nice insight and I am still reading other replies in this thread.

I'm just nit picking.

Utimately yes, firmware does decide data's fate. If the firmware is giving false responses back then there is nothing a OS can really do to effect that. If that is the case though it should effect all OS's. As a side note there is something drastically wrong when it comes to SQLite performance in Ext3. Switch to XFS and you will get much faster results and Ext3 has been getting slower since around ~2.6.18. It's something I have noticed for a while now.

[kxmas]

Michael, I know you mentioned this in the article, but the Intel drivers still have many issues in 9.04, of course it would be beat easily - This article is a reflection of that. It is not representative of ubuntu 9.04 vs OS X 10.5.6

A more fair comparison would be a desktop with a graphics card that is well supported, maybe at this point some NVIDIA card, or one of the ATI's that are acutally running with fglrx in 9.04 ;-)

I find it even more interesting that he did use the intel graphics for the tests. It shows just how bad they are in linux.

[Apopas]

The majority of Linux users today use 64bit operating systems since there are serious perfomance boost issues and no compatibility problems anymore. So why at first place these benchmarks took place with a 32 bit version of Ubuntu I can hardly understand. 64 bit would make the difference in ffmpeg, ogg and lame encoding and I don't speak for tweaking and hacking just a very dekstop choice. Currently the benchmarks shows 17 vs 12 for MacOSX favour while with 64 bit the result would be 14 vs 15 for Ubuntu's favour pretty easily.
A third benchmark with the 64 bit version of Ubuntu is essential imho.

[Hephasteus]

Holy crap, NO.

First off.. The kernel OS X uses is NOT A MICROKERNEL.

The OS X kernel is called XNU. It's a so-called 'hybrid' kernel that uses code from a development kernel that died in 1995 combined with BSD stuff. The Mach kernel at different times in it's history was a microkernel and then not a microkernel. OS X does not use a Microkernel.

The Windows NT kernel was another one that was based on a Microkernel design, but is not a Microkernel. Early versions of NT were microkernels, but unfortunately for that design Microsoft could not figure out how to make it scale and the excess overhead caused by the message-passing design doomed it. So later versions of the NT kernel were monolythic.

If you want to you can call them 'hybrid kernels', but I think that is just a made-up term to make the OS kernel sound all microkernel-ish and cool while it is, in fact, a modular monolythic design.


Nope. Not going to happen. Microkernels were essentially a pipe dream and only one Microkernel-based OS actually made it into widespread use. That OS was QNX and was popular for embedded systems due to it's realtime-like nature.

But it wouldn't scale to anything big and nobody wanted to use it as a desktop or server platform.

Ya I agree with ya on the most of your points. But I'd argue that microkernels are all over the place. In fact. I'd call every single memory and thread manager of every single SQL database system today a microkernel implimentation hybridized to their monolithic OS.

[kraftman]

Utimately yes, firmware does decide data's fate. If the firmware is giving false responses back then there is nothing a OS can really do to effect that. If that is the case though it should effect all OS's. As a side note there is something drastically wrong when it comes to SQLite performance in Ext3. Switch to XFS and you will get much faster results and Ext3 has been getting slower since around ~2.6.18. It's something I have noticed for a while now.


I did some SQLite tests in KVM and Ext3 is really slow when compared to Ext4:

http://img75.imageshack.us/img75/6714/ext3defaultssqlite.png

http://img75.imageshack.us/img75/9333/ext4defaultssqlite.png

It probably means Ext4 and HFS+ are using cache and Ext3 isn't. Or Ext 3 just sux ;)

P.S. results are reproducible.

P.S.2 If TeeKee is right (and probably is) this Phoronix benchmark sux a little...

[L33F3R]

Does room for error exist? absolutely! I think more tests need to be done using mac systems for sure! Why not put a MAC server against a debian server and compare how several linux desktop distros performs under a mac system. Put both of them in X86_64 and build linux with the same optimizations you get out of Intel macs, ie. SSE instructions. The Linux kernel doesn't even do things to benefit from this to the best of my knowledge.

But as i see it now. We got our asses handed to us. Is it sad? u better bet it is. But we know we can improve. Excuses are excuses... The whole fedora is amazing thing is a pile because we all know the difference is nominal at best.

We lost... lets not act like 9 year olds and debate that we really didn't. Certainly we can be grown men (and women) and discuss how we can actually make the situation better.

[drag]

Ya I agree with ya on the most of your points. But I'd argue that microkernels are all over the place. In fact. I'd call every single memory and thread manager of every single SQL database system today a microkernel implimentation hybridized to their monolithic OS.


I don't know anything about that.. but I do know that having a threaded model and memory management isn't something that is unique to SQL databases. Pretty much every large multi-threaded application is going to have to management it's memory and threads and such.


Remember what makes a microkernel a Microkernel is that the actual kernel doesn't do anything more then message-passing.

Then various seperate processes 'orbit' that kernel and provides services that the OS can use. Like the 'Hurd' is a collection of programs that provide low-level facilities to a L4 kernel. So you'd have a program that provides access to the harddrive, then another program that provides file system access, then another that provides POSIX APIs, etc etc.

So all the kernel does is then pass messages from one service daemon to another. It has zero functionality beyond that.

And, perversely, Microkernels tend to be hugely complicated. They are usually quite a bit larger then a Monolythic kernel even though they have no functionality built-in besides message handling.

It's pretty obvious why they are not really that successfull if you can step back and look at what they really are.


----------------

Now a modern monolythic kernel, like the Linux kernel, is a big object-oriented, multithreaded monster. Each major feature has it's own thread and there are lot of different small 'kernel level program'-type things that provide services and features that are used by the rest of the kernel. The difference being is that there is no message passing going on and they all occupy the same address space so one can twiddle the other's bits and read each other's memory in a very efficient manner.

This is why proprietary software like Nvidia's drivers that while high performance and have lots of features tend to like to stuff huge amounts of code into the kernel tend to suck. Then the nvidia driver, at any time, can abritrarially access and overwrite any other part of the kernel. If nvidia drivers have a memory overflow or other hicccup it can easily blow away the memory containing... say.. your Ext3 support.

With normal applications each one occupies it's own Virtual Memory space. That is each application see it's own unique address space. To the application all they see is their own virtual 4GB of RAM that they can do with whatever they will. This is the 'virtual' part of Virtual Memory. Each application has it's own VM sandbox and it's very difficult for that application to break out of it's memory sandbox... it can't even see what is going on in the memory of other applications. Now with kernel modules in Linux there is no memory protection features like that and a kernel module can very easily access any other part of the kernel and view and edit any other part of the running kernel.

There really isn't anything that would stop it and is the major design deficiency of a Monolythic kernel.

This is why the video OSS driver model tries to shove as much as the video driver into userspace as possible... the kernel portion is kept as small as possible and the majority of the video processing happens via the DRI2 protocol in userspace.

[Hephasteus]

Thank you for sharing your OS understandings. This has been a very enlightening thread. It's good to see more and more daemons running on linux and I see more clearly that a herd of daemons might not be that great. It'll be interesting watching them hybridize the kernel as it moves to more advanced video handling. Hope they do a good job. But it's looking like a bumpy ride so far. Can't last forever though.

[Apopas]

Does room for error exist? absolutely! I think more tests need to be done using mac systems for sure! Why not put a MAC server against a debian server and compare how several linux desktop distros performs under a mac system. Put both of them in X86_64 and build linux with the same optimizations you get out of Intel macs, ie. SSE instructions. The Linux kernel doesn't even do things to benefit from this to the best of my knowledge.

But as i see it now. We got our asses handed to us. Is it sad? u better bet it is. But we know we can improve. Excuses are excuses... The whole fedora is amazing thing is a pile because we all know the difference is nominal at best.

We lost... lets not act like 9 year olds and debate that we really didn't. Certainly we can be grown men (and women) and discuss how we can actually make the situation better.
To make the situation better is to make a 32 bit OS to act as if it was 64 bit. You admit defeat only after a fair battle.

[L33F3R]

To make the situation better is to make a 32 bit OS to act as if it was 64 bit. You admit defeat only after a fair battle.

Well you could be a child and fight over details or you could move on and improve your product. This principal has been shown in Japanese business and has proven to be quite successful. You cant have a fair battle on two different platforms, one of which consists of in-house hardware; apple is going to take the advantage in any OS fight because of this.

That brings up a good plus for Linux, unlike mac it can use a large variety of hardware. Historically problems have erupted with hardware drivers but I have noticed that in recent time the driver situation has been getting alot better. Additionally I can build a $300 computer and play ETQW on high quality with linux; the mac mini is $600 and has moderate HDD/RAM at best.

Dont look at the situation from a linear perspective. I agree more tests need to be done but lets not forget theres room for improvement.

[Apopas]

You cant have a fair battle on two different platforms, one of which consists of in-house hardware; apple is going to take the advantage in any OS fight because of this.
Exactly that's my point. I believe even in in-house Apple's hardware Linux will have the best performance. Just use the newest of it and the newest is the 64 bit one. When we have it and it's easy to install it then why to stick on the old one? It is not a minor detail, it's the logical choice.

[Yfrwlf]

I did some SQLite tests in KVM and Ext3 is really slow when compared to Ext4:

http://img75.imageshack.us/img75/6714/ext3defaultssqlite.png

http://img75.imageshack.us/img75/9333/ext4defaultssqlite.png

It probably means Ext4 and HFS+ are using cache and Ext3 isn't. Or Ext 3 just sux ;)

P.S. results are reproducible.

P.S.2 If TeeKee is right (and probably is) this Phoronix benchmark sux a little...

If true that's a freakishly big difference in performance between ext3 and 4. From what I know I wouldn't be too surprised if HFS+ beat ext3. ext3 is stable but quite old and fairly slow.

So, definitely sounds like more tests should be done, yup.

Or maybe this was just a ploy to get more Linux users on Phoronix to try to one-up Mac. ;)

[Apopas]

I checked the previous results between Ubuntu 8.10 and MacOSX 10.5. Fortunately there was there a 64 bit benchmark. You can see that in a lot of tests were MacOSX beats Ubuntu 32, the true winner is the 64 flavour of Linux, like mp3 and ogg enconding and 7zip compression. Unfortunately no ext4 back then. Oh come on Michael, give us a real benchmark :D

[Michael]

Ubuntu 64-bit results are coming soon :)

[Apopas]

Ubuntu 64-bit results are coming soon :)

Dabadabadu :D

[Michael]

I should also note that tomorrow there are OpenSolaris vs. Linux kernel benchmarks (using Nexenta CP2) tomorrow with both Ubuntu 32-bit and 64-bit, but it's unrelated and on a different test system.

[kraftman]

Ubuntu 64-bit results are coming soon :)

Great, can we expect Ext 4 benchmarks too?

@Xfrwlf

If true that's a freakishly big difference in performance between ext3 and 4. From what I know I wouldn't be too surprised if HFS+ beat ext3. ext3 is stable but quite old and fairly slow.It's probably, because Ext 3 doesn't support delayed allocation. Strange thing is each of the first tests has worse results then the second and the third. It may be due to some scripts in Ubuntu which consume 100% CPU time while after booting. 'Top' shows python using 100% CPU for a while.

[bugmenot]

isn't easier to you just to test Fedora with 2.6.30 ext4 on mac mini?

[iznogood]

The test compared only 32 bit linux with an operating system (Mac OS) that has 64 bit "extensions". It would have been better to include 64 bit ubuntu as well like you did on last year's test.

Also you did not use ext4. I have been using it for some time (a month or so) and i can confirm that it makes a big difference on my machine (24'' iMac)

[drag]

If true that's a freakishly big difference in performance between ext3 and 4. From what I know I wouldn't be too surprised if HFS+ beat ext3. ext3 is stable but quite old and fairly slow.

So, definitely sounds like more tests should be done, yup.

Or maybe this was just a ploy to get more Linux users on Phoronix to try to one-up Mac. ;)



Alright... here is the skinny on Ext3 and SQLite performance.

As you probably know.. most modern systems use asyncronous write file systems. This means that when a program writes out a file that file is not immediately written to disk. So the OS uses file system cache to make the illusion that the data is written out, so your application can read/write to files without caring.

This is done to improve performance, improve multitasking performance, improve file system layout on the drive to avoid fragmentation, etc.

Ext3, with it's defaults, means that it can take up to 5 seconds for data to get commited to disk.

--------------------------

Alright.

So the upside is better performance and efficiency.

The downside is that if your system crashes, power goes out, or other things go bad then the system won't have time to commit the changes to disk and you will end up with corruption or missing data.

To combat this most modern file systems use a form of journaling. Journaling means that a record of what is getting writen to disk is kept so that during system recovery you can replay the journal and compinsate for any damage.

Previously you would have to run a fsck on every poor shutdown, which takes a long time.

-------------------


The next thing to understand is that another technique to combat changes of file corruption is that applications can request things like 'sync()' or 'fsync()'

This tells the file system to commit data to the disk immediately, as soon as possible.

Applications like Vim, OpenOffice.org, Emacs and other well designed applications use that when you save your work so that there is little chance of data corruption with your important files. Most stuff that happens is not that important so that they don't request fsync all the time.

Fsync is 'file sync' and only should affect one file and the other is sync, which is a system-wide commit and is rarely done due to the huge performance penalty. There is a command called 'sync' that you can use that will perform a 'sync()' call for you.

----------------------------


Ext3 is designed to work on commodity hardware effectively.

Most file systems like XFS or ReiserFS that are designed for high performance are expecting that you are using battery-backed harddrive cache and UPSes and all that stuff to protect your system.

Ext3 tries to be quite a bit more robust. Because of this Ext3 supports multiple journaling modes which are configurable using mount-time options.

The options are:

data=journal
------------

This is the most robust design. Both data and metadata* is journaled.

That means that all your data gets writen to the drive twice.. once for the journal and then once again when it actually gets commited to disk.

The downside to this mode is that it's only rarely used so it isn't as well tested as other modes.

*metadata is 'data about data'. This is the stuff that the file system keeps track of. Filenames, file locations, file permissions, timestamps, etc etc.

data=writeback
--------------

This is the fastest mode. This mode matches what other journaling FSs like NTFS, XFS, ReiserFS, and I expect HFS+ uses. With this mode only Metadata is journaled and data is writen willy-nilly.. at any time.

With this it's designed to simply protect the file system from corruption. Individual files may end up with missing or corrupt data during a failure, but the file system is kept track of.


data=ordered
-------------

This is the default mode for Ext3.

With this mode only the metadata is journaled, however data gets written FIRST. This means that any changes to the data on the harddrive has to be written out before the file system on the drive is updated to reflect changes.

This ensures that you won't have empty files or half-written or otherwise corrupted files on a failure. You may miss new data, but old data won't be corrupted. Only fully written files are recorded.



----------------------

Now data=ordered helps Ext3 be more secure and be more robust then typical journaling file systems.

However due to this design you have some negative performance side effects with SQLite.


Ok....

Remember back when Firefox was causing machines to 'freeze up' for short times and cause other bad performance problems on Linux? Well that was done because of Firefox using SQLite. This was due to a combination of the I/O schedualer and the 'data=ordered' feature.

So your system is heavily using cache to improve performance. So the kernel sets up a write queue to keep track of what data should be written to disk.. it'll look something like this:

write, write, write, metadata, metadata, write, write, etc etc.

While the I/O queue just stuck the 'fsync()' command on the end of the queue... and since 'fsync()' says "WRITE THIS DATA NOW" and 'data=ordered' says "write data before updating file system metadata"...

So this transformed 'fsync' into 'sync'. The Linux kernel would flip out and write all the data and metadata in order to do the fsync() correctly.

And since Firefox used SQLite and SQLite requested 'fsync()' very often this problem would happen all the time while using firefox.


------------------------------


To fix this newer versions of the I/O scedular in the Linux kernel created a second queue for high-priority writes. So that way fsync() won't get transformed into sync.

However the 'data=ordered' still causes latency issues and they can't figure out how to make the schedualer work correctly.

-------------------------------


So to fix that they have changed the default in Linux 2.6.30 from 'data=ordered' to 'data=writeback'.

So this way this solves the majority of the problem with Ext3.

However distributions probably won't use this.

Why?

Because it lowers the security and robustness of Ext3.

Because the file system on the harddrive is updated, possibly, before a file is written to disk you can end up with situations were you have files created with just random junk in it if the system fails. That random junk may be other people's important data. Important configuration files, passwords, bank statements, etc etc.

So on multiuser systems this can cause a security hole.

To solve this problem a new mode is being created called 'data=guarded'. This solves the security issues, but keeps most of the performance of 'data=writeback'.

[Hephasteus]

If you can get Enron accountants to write your journaling data structures can you be rich until your system crashes?

[Apopas]

Drag, your post is very well written and enlightening but it doesn't change Yfrwlf's statement. As far as I have seen ext3 even in its writeback form is slower than ext4 and, indeed, more tests should take place.

[drag]

Drag, your post is very well written and enlightening but it doesn't change Yfrwlf's statement. As far as I have seen ext3 even in its writeback form is slower than ext4 and, indeed, more tests should take place.

It wasn't really ment to countradict anything he was saying.

It's just one of the reasons why SQLite performance is lousy on a Linux desktop.

And it's should be that Ext3 is slower then Ext4. A lot of effort was put into making Ext4 fast... :)

As far as more benchmarks.. that should be easy. We have easy access to the Phoronix test suite. :)

[Rip-Rip]

There's a problem in these benchmarks.
Mac OSX is designed to run on SSE3 processor, so gcc is optimising the assembly code to work only on SSE3 processor. Especially, sse is used for math computation, which is about 3 times faster than x87 which is the default on i386 linux.
A fair benchmark could be Mac OSX vs x86_64 linux. Because gcc on x86_64 is optimising by default for SSE2 processor, and using SSE for math computation.

My 2 cents

[Apopas]

As far as more benchmarks.. that should be easy. We have easy access to the Phoronix test suite. :)

Haha true, true... soon when I'll get my newer puter I'll spent lots of hours to test my gentoo system :D

There's a problem in these benchmarks.
Mac OSX is designed to run on SSE3 processor, so gcc is optimising the assembly code to work only on SSE3 processor. Especially, sse is used for math computation, which is about 3 times faster than x87 which is the default on i386 linux.
A fair benchmark could be Mac OSX vs x86_64 linux. Because gcc on x86_64 is optimising by default for SSE2 processor, and using SSE for math computation.

My 2 cents
Don't worry, Michael promised that soon we will have a 64 bit benchamrk as well ;)

[kraftman]

Don't worry, Michael promised that soon we will have a 64 bit benchamrk as well ;)

I'm starting to worry now. What we saw was very good Macos advertisement. Ubuntu 32 bit and slow Ext 3 (when compared to Ext 4) vs Macos with 64 bit capabilities and fs which buffers writes... Many portals will copy this benchmark and many people will buy it. I sometimes wonder if apple keeps its dirty hands on Phoronix?

[curaga]

Based on the fact Michael had to buy his own Mac Mini, and if I remember right the cheapest one at that, I doubt Apple had anything to do with it.

Once we start seeing some Mac Pro benchmarks with dual quad Xeons, that's when we need to start worrying :p

[deanjo]

Based on the fact Michael had to buy his own Mac Mini, and if I remember right the cheapest one at that, I doubt Apple had anything to do with it.

Once we start seeing some Mac Pro benchmarks with dual quad Xeons, that's when we need to start worrying :p

There already is MacPro benches in PTS Global. Even some G5 stuff.

[Apopas]

It is true that this paragraph from the conclusion of that benchmark is extremely positive for Apple and totally negative for Ubuntu:
Overall, Mac OS X 10.5.6 already outpaces Ubuntu 9.04 when it comes to many respects of the desktop and server performance. With the introduction of Mac OS X 10.6.0 "Snow Leopard" in a few months, the Apple gains will likely widen considering the efforts they are putting forth on improving the performance via a smaller memory footprint, OpenCL, etc. The graphics performance in Ubuntu 9.10, which will be out after the release of Mac OS X 10.6, may improve with Intel working to fix its regressions and Gallium3D could be enabled in time. Beyond improving the graphics performance and potentially some minor performance improvements thanks to an updated Linux kernel (well, a big improvement in SQLite unless they regress again) and the newer GCC 4.4 series, we would not anticipate the Ubuntu 9.10 performance to be drastically different.
while the next days we saw that 9.10 alpha was significantly faster that 9.04.
Imho Michael should emphasize that this was the 32 bit version of Ubuntu and truth to be told there was not any difference between this one and the benchmark that took place 1 year ago. Back then MacOSX was even faster than 32 bit Ubuntu but slower than 64 bit. So he should have said "stay tuned, till 64 bit benchmarks are out to actually see if MacOSX took indeed the lead"!
Personally, with a clear install and ext4 I anticipate the gap between 64 bit Ubuntu and MacOSX to be even bigger than the previous one because the 32 bit gap now is lower than the previous one.
Maybe he was just lazy to try 64bit Ubuntu, but yes this unfair benchmark will work for Apple's favour finally :(

[L33F3R]

I'm starting to worry now. What we saw was very good Macos advertisement. Ubuntu 32 bit and slow Ext 3 (when compared to Ext 4) vs Macos with 64 bit capabilities and fs which buffers writes... Many portals will copy this benchmark and many people will buy it. I sometimes wonder if apple keeps its dirty hands on Phoronix?

lol first of all some serious fudging occured in the benchmarks, we all know this and you acknowledged that.

Whats the end result? perhaps something similar to this by apple:

http://www.microsoft.com/windowsserver/compare/default.mspx

Unfortunately, Anyone with half a brain can tell that this is a load of BS. The movers and shakers are knowledgeable enough to compare apples to apples.

I dont think apple is stupid enough to make the same mistake microshaft made here. I wonder how many millions Microsoft paid for this pointless ad campaign XD.

[kraftman]

lol first of all some serious fudging occured in the benchmarks, we all know this and you acknowledged that.

Whats the end result? perhaps something similar to this by apple:

http://www.microsoft.com/windowsserver/compare/default.mspx

Unfortunately, Anyone with half a brain can tell that this is a load of BS. The movers and shakers are knowledgeable enough to compare apples to apples.

I dont think apple is stupid enough to make the same mistake microshaft made here. I wonder how many millions Microsoft paid for this pointless ad campaign XD.

I don't know if apple is as stupid as ms, but I already saw many noobs which believe this FUD :/ If you type 9.04 vs macos in Google you'll see many sites which mindlessly copied this comparision... Michael can proof Ubuntu outperforms Macos, but it seems he's not interested.

[Apopas]

If I could afford at this time the money to buy a mac-mini... :(

[kraftman]

If I could afford at this time the money to buy a mac-mini... :(

It's not worthy IMHO. Only few people would see your benchmarks. Of course if you don't own such portal like Phoronix ;)

[Michael]

Many portals will copy this benchmark and many people will buy it. I sometimes wonder if apple keeps its dirty hands on Phoronix?

Hahaha, wow. Apple has never even contacted Phoronix or me for that matter.

[kraftman]

Hahaha, wow. Apple has never even contacted Phoronix or me for that matter.

Whatever, what about promised x86_64 benchmark? And Ext 4 as a bonus will be nice...

[Michael]

Whatever, what about promised x86_64 benchmark? And Ext 4 as a bonus will be nice...

They're in the publishing queue for tomorrow morning.

[kraftman]

They're in the publishing queue for tomorrow morning.

Great to hear that!

[Apopas]

Great news! :D

[Yfrwlf]

They're in the publishing queue for tomorrow morning.

Obviously you're not shocked, Linux users are quick to defend Linux, go figure. ;)

Of course Linux has it's failures, I'm a big critic myself but still love Linux. It's of course important to admit (hopefully temporary!) defeat where appropriate though. I think the main reason for the way Linux users are is because since it's a community OS, many take it personally when things aren't good enough. In a sense, they're right, it is a failure of the community as a whole.

Perhaps an update for any progress in fixing any bugs and regressions if any caused those tests is in order, as well as more benchmarks which may be fairer/balanced and not purely Mac-oriented. For instance, since Mac also can run I think both AMD and Nvidia graphics, etc, or basically since you can install OS X on a normal computer that may not be so niche in Mac's favour, perhaps that would be an interesting experiment, at the risk of then being "not fair" with respect to Mac. :)

[Apopas]

Obviously you're not shocked, Linux users are quick to defend Linux, go figure. ;)

Of course Linux has it's failures, I'm a big critic myself but still love Linux. It's of course important to admit (hopefully temporary!) defeat where appropriate though. I think the main reason for the way Linux users are is because since it's a community OS, many take it personally when things aren't good enough. In a sense, they're right, it is a failure of the community as a whole.
Hehehe exactly! No wonder why Linux has by far the most fanatic lovers of the OSes :p

[Yfrwlf]

Hehehe exactly! No wonder why Linux has by far the most fanatic lovers of the OSes :p

Yup. ^^

Linux really solves a lot of issues of freedom, too, that really irks for instance Windows users. However, there are features/freedoms on those "other" OSes (sorry, couldn't help myself) that Linux also needs to adopt that Linux users should have the luxury and freedom of enjoying as well. Speed is certainly one of those things, as is universal packaging standards, but I won't get into that here. ;)