A Big Operating System Benchmark Comparison

[L33F3R]

3. SERVICES/FEATURES

FreeBSD and NetBSD comes as a BARE MINIMAL systems where you can add everything by yourself, Mac OS X, Ubuntu and OpenSolaris comes with all features and services by default, so be sure to disable many features and compare Oranges to Oranges, not Apples to Oranges ...

or apples to microsofts

[vermaden]

@L33F3R

Heh, yeah

[Qaridarium]

Its my first post on these forums, so I would like to say 'Hi' to everyone.

I only hope that person that will be doing these tests will read this post, since this thread/comments grown very big.

1. HARDWARE

Many people scream to use ATI or nVidia cards, but AMD does not provide their drivers to NetBSD and FreeBSD, nVdidia only provides i386 driver for FreeBSD, but there is no amd64 version, and no version for NetBSD either.

The only graphics card, that will made all these tests equal for all systems is Intel based integrated card, and a little older one, not the newest one.

For laptop tests (if there will be any) best chioce will be X3100 or GMA 950.

For Desktop tests best would BE GMA 3100 for example.

2. VERSIONS

If you will not include FreeBSD 8.0 (will be released in middle September) tests will not be fair since its one of the biggest FreeBSD releases ever, or at least include tests for 8.0-RC* versions.

I would also include these systems:
NetBSD 5.0.1
Ubuntu 9.04
CentOS 5.3
Mac OS X 10.6
OpenSolaris 2009.06

3. SERVICES/FEATURES

FreeBSD and NetBSD comes as a BARE MINIMAL systems where you can add everything by yourself, Mac OS X, Ubuntu and OpenSolaris comes with all features and services by default, so be sure to disable many features and compare Oranges to Oranges, not Apples to Oranges ...

ok its simpel intel is bad becourse the cloused source driver on macos and windows is much much much faster than the opensource one!
there is no need for testing hartware if the resuld be cleare bevor the
testing!
only nvidia and amd have a solution to have the same driver base to windows,macos and linux.
and macos.... is SSE3 compilet 32bit ubuntu only 486/686 compilet
macos is sse3 and 64bit ubuntu is only sse2...
macos has diverend compiler to macos use the intel compiler much much much faster than the GCC compiler!
if you make a real benchmark of the OS and not the compiler ...
you need to recompile a ubuntu linux completly on intelcompiler with SSE3 only support.-
then no "intel" VGA-- the result will be clear...
linux win every test and macos lose everytest.

but if you test like michael"phoronix" test in the past...

intelcompiler VS GCC

32bit-SS3 vs 486

32bit-sse3 vs 64bit-sse2

clousedsource intel driver vs opensource linux driver...

all overall.. its a shame to test like this!

[AdrenalineJunky]

actually - that is the most logical way to perform this test - as shows how they compare based on configurations that the vast majority of people use.

recompiling everything would be pretty much pointless because relatively very few people would have thier system set up like that.

[vermaden]

only nvidia and amd have a solution to have the same driver base to windows,macos and linux.

So read this part again:

Many people scream to use ATI or nVidia cards, but AMD does not provide their drivers to NetBSD and FreeBSD, nVdidia only provides i386 driver for FreeBSD, but there is no amd64 version, and no version for NetBSD either.

There WILL be diffrences in drivers and so, but ALL tested systems have ACCELRATED 2D/3D drivers for Intel cards.

AMD does not offer ANY drivers for FreeBSD/NetBSD/OpenSolaris.

nVidia only offers limited i386 driver for FreeBSD and NOTHING for NetBSD.

The other way to be EQUAL for all systems will be using some old graphics card/untypical card where ALL systems will be forced to use 2D VESA driver to accomplish same slowdown everywhere ...

[VinzC]

gentoo is probably too much trouble to install, but would be nice to know if it's actually faster if well-maintained.

If one candidate fits best for benchmarking it's Gentoo, IMHO. It would reflect to what extent one could fit a Linux distribution to the target CPU to get the best performance and to what extent the performance gain is worth the pain.

I also think we need to see gentoo represented in this benchmark. You can argue that there is no such thing as a stock gentoo - but why not just make a basic install and some sane USE and CFLAGS and specify these in the benchmark alongside all the other distro specifications. It would be very interesting to see what difference gcc flags make.

I couldn't agree more on that. Gentoo has always been advertised as the fastest distribution due to the high control over compilation flags -- not that I agree with that assertion but I'd also like to see how much it's different. BTW Optimizing Gentoo is not trivial but I'd like to see what results the best, sane optimization gives.

I would suggest benchmarking a stable ARCH (x86_64) against a recent Intel CPU with GCC 4.3.2 and sse4.1, for instance. And for bleeding-edgers, a ~ARCH (~x86_64) with the latest GCC 4.4.x, to see what impact the recent optimizations has over the whole system.

But in general, I would expect Gentoo to have (sane) CFLAGS adapted to the target CPU only.

[deanjo]

macos has diverend compiler to macos use the intel compiler much much much faster than the GCC compiler!

What are you smoking? OS X's compiler is GCC. 10.6 will also feature clang with LLVM. No intel compiler at all. It should also be noted that SSE3 is really only useful on processors with long pipelines (read: P4).

Edit: I should clarify that a bit more. SSE3 typically only shows huge gains when used with a processor that has long pipelines.

[T_Beermonster]

I'd be interested to see Dragonfly BSD thrown into the mix. The HAMMER filesystem is supposedly now production ready and the kernel architecture has diverged from the other free BSDs.

[vermaden]

@T_Beermonster

HAMMER filesystem is not a ZFS rival or something like that, its developed to work in cluster environments.

Abou DragonflyBSD itself, it scales as FreeBSD 4.x (DragonflyBSD forked from FreeBSD 4.8) and works well only with single core CPUs, it does not use more then one core, its still big GIANT LOCK kernel.

Check these benchmarks below:
http://people.freebsd.org/~kris/scaling/dfly-mysql.png
http://people.freebsd.org/~kris/scaling/dfly-write.png

So adding DragonflyBSD to the test will not bring nothing new here, it is and will be slow, until they rewrite SMP subsystem as it has been done in FreeBSD 4.x --> 8.x

[DDevine]

@T_Beermonster

Check these benchmarks below:
http://people.freebsd.org/~kris/scaling/dfly-mysql.png
http://people.freebsd.org/~kris/scaling/dfly-write.png

Absolutely pwnt by FreeBSD!

So adding DragonflyBSD to the test will not bring nothing new here, it is and will be slow, until they rewrite SMP subsystem as it has been done in FreeBSD 4.x --> 8.x

True dat.

The DragonflyBSD had good intentions for forking but so far it hasn't gone too well. When they get SMP they might be back on the map.