Why Software Defaults Are Important & Benchmarked
Phoronix: Why Software Defaults Are Important & Benchmarked
Almost every time benchmarks are published on Phoronix, there's always at least a handful of people - or more - that will right away say the benchmarks are flawed, meaningless, or just plain wrong. Why? Because the software configuration is tested with its default (stock) settings. These users then go on to say that the defaults are not optimized for performance and that "everyone else knows better" to use a particular set of options, etc. But it's my firm belief that it's up to the upstream maintainer -- whether it be the project itself developing the software in question or the distribution vendor that's packaging and maintaining the given component -- to choose the most sane and reliable settings, and that's what most people use. In addition, with open-source software, there's endless possibilities for how a given piece of software can be tuned and tweaked. Here's some numbers confirming these beliefs of testing software at its defaults...
One thing that I do ask Michael is that when you bench openSUSE in your articles you list the exact kernel package installed. The installer for openSUSE will make a decision during install based on the test systems hardware configuration and it could install the "generic -default" kernel packages or the " -desktop" kernel packages.
It will be said on OpenBenchmarking.org always, now that all results are to be hosted there.
Originally Posted by deanjo
Maybe you could write an article about xorg.conf settings and how they affect performance, with actual data.
I think the black magic of xorg.conf could need that.
As a Gentoo user, I can see the advantage to doing benchmarks with the default options.
I use a AMD Athlon II X4, which doesn't get tested at all on Phoronix, so any benchmark that has Core i7 optimizations won't necessarily apply to me. I'd rather see what the base performance is, rather than have to figure out if something is fast merely because of i7 optimizations.
That works great in theory but in many situations a xorg.conf is still required.
Originally Posted by mtippett
Well, I'll be damned, I was just about to say exactly what you said. A guide to xorg.conf settings (or xorg.conf.d/*) for
Originally Posted by sabriah
intel, nvidia and ati xorg-drivers.
and perhaps also some generic settings. I've been a Linux user since 2002 and I still use the default, except EXA (is that even the good thing to use anymore) and SoftwarePointer "No".
Do we need to enable Composite in the extensions anymore? Etc etc etc.
Most documentation is from the days before Compiz reached a fairly stable point (I still don't think they're there) and Compositing was done via XGL and other weird stuff.
Pleaaaase, Master, teaaach us!
It's true; stock configurations are all that 90% of the users will see. Even 80-90% of Phoronix readers are newbies, as taken from the previous years' Phoronix Linux Graphics Survey. And Phoronix readers are more savvy than the general populace.
There should be a huge challenge afoot in the distribution marketplace to create optimized distros that, while able to run on old hardware, can adapt to new hardware and run fast there as well. I'm not just talking about SSE, but maybe, things like having certain programs (very performance sensitive programs) specially compiled on install? Not everything is performance sensitive, but certainly the 3d graphics related stuff is, and certainly the kernel is.
I've seen a few binary distros around that purport to focus on optimization, but I think I'd rather have the mainstream distros focus on optimization. The mainstream distros are where most of the casual users are going, and we can afford to give them a better out of the box experience.