Ubuntu pre-release doesn't have debug flags on, so the test is invalid.
Originally Posted by energyman
I'll agree on the full release stuff, and personally rarely touch FreeBSD these days, but do bear in mind that it's been a year and a half since FreeBSD was released. Phoronix would have done better to compare FreeBSD7 to Ubuntu 8.10 or even 8.04 given the pace of change in both OSs.
Two major problems with this test:
Originally Posted by phoronix
1) It used a pre-release OS (FreeBSD 8.0) which can have a significant impact on performance.
2) FreeBSD 7.2/8.0 both used Gcc 4.2.1 vs Ubuntu using 4.4.1, which can make as much as a 20% difference (4.4.1 faster) in performance according to the GCC developers:
You will notice that compile time and run time for heavily computational tasks are better in Ubuntu, which is as expected with the GCC change.
While it may be a fair argument, that FreeBSD 8.0 shouldn't include Gcc 4.2.1, it isn't a fair statement to make that this test proves FreeBSD is slower. Frankly, I don't know how well it would perform if the test was done on a level playing field (same hardware, same compiler) but testing with these kind of low level software differences (gcc) from a 3rd party isn't a fair test.
Last edited by risner; 09-28-2009 at 03:36 PM.
What is the link to your launchpad account? I would love to see what you have contributed.
Originally Posted by Apopas
I'd personally like to see the people arguing about FreeBSD being unfairly treated do a PTS test run a single system with a single test demonstrating the performance delta between FreeBSD "current" + FreeBSD "dev-next" + FreeBSD "dev-next-without other options".
That will stop a lot of the discussion and supposition and replace it with fact. My expectation is that the numbers won't have as large a spread as people are suggesting.
I'd be even happy with a test contrasting FreeBSD "dev-next" + FreeBSD "dev-next-without other options" to simplify life.
A regression is simple to define.
Originally Posted by kraftman
A regression is a change in behaviour that occurs between a configuration under test and a secondary related configuration under test. The regression can be positive or negative, but it is the change in behaviour that is important.
If there is a document somewhere about what has been turned on or off in development version and what will make it to the final production version, then I agree that it can be communicated. If one doesn't exist, then it is a fair comparison since removing it is a black art that isn't communicated to anyone.
"When comparing performance with other filesystems, remember that ext3/4 by default offers higher data integrity guarantees than most. So when comparing with a metadata-only journalling filesystem, use `mount -o data=writeback'. " http://lwn.net/Articles/203915/
Meaning the writeback option increases performance at the cost of data integrity.
Please stop comparing desktop operating systems with server operating systems whose defaults include debugging, performance monitoring, fault tracing, and conservative compiler optimisations.
For desktop OS comparisons, please consider rather benching PC-BSD in future. It would be more informative, and if PC-BSD also under performs, the PC-BSD project is more likely to improve the situation and benefit from the tests.
FreeBSD users and developers don't care that the system is slower than Ubuntu as a desktop out of the box. In fact, many of us could have told you the same without your tests.
Last edited by aragon; 09-28-2009 at 02:22 PM.
blabla, and there they are, the FreeBSD apologists. Always claiming FreeBSD is faster - and when someone shows them that they are plain old lying, they start complaining. Not fair! FreeBSD is a server OS! So why does it suck at server tasks like file system benchmarks! Not fair! it is an old one, the latest one is so much faster! Really? The latest linux is faster too. When fefe compared linux 2.6.0 against then current FreeBSD they complained that the next FreeBSD would so much better. The next FreeBSD was not better as shown in 2006. But hey, now the next FreeBSD would be better, right? No, it isn't. But that must be linux fault! FreeBSD is a server os! Not fair using benchmarks testing memory access, scheduling or file access.
Which of the phoronix tests is a desktop test? sqllite? lame? imagemagick?
Little hint: none of them.