An Open Letter To Tech Review Sites

[The_Monkey_King]

Pimping your own test suite!

On the other hand, why would others perform Linux testing since:
a) hardware features are not fully exploited in Linux
b) majority of "off the shelf" applications used in testing not ported to Linux
c) Individual builds of Linux distros (not to mention the variation inside a single build of just distro) is YMMV, while Windows ISOs have variations of less than 1%
d) they can just point to your site (yay! more unique visitors and page clicks may add up to better ad revenue pay-outs!)
e) all the above

Answer: E


We're hitting the wall here people. Like that 80-20 rule you hear about in business classes, Linux distros are hitting 80% of the features found in Microsoft and Apple. And it's this last 20% that really is the differentiator that sells Windows 80% of the time.

Knowing this, Microsoft uses money and clout (being the largest OS distribution in business and home markets) to influence hardware and software vendors to delay or reject the Linux environment from taking the other 20%. And by delay, I mean offering proprietary drivers and forcing open source to accept proprietary abstraction layers.

[maniacmusician]

Pimping your own test suite!

On the other hand, why would others perform Linux testing since:
a) hardware features are not fully exploited in Linux
b) majority of "off the shelf" applications used in testing not ported to Linux
c) Individual builds of Linux distros (not to mention the variation inside a single build of just distro) is YMMV, while Windows ISOs have variations of less than 1%
d) they can just point to your site (yay! more unique visitors and page clicks may add up to better ad revenue pay-outs!)
e) all the above

Answer: E


We're hitting the wall here people. Like that 80-20 rule you hear about in business classes, Linux distros are hitting 80% of the features found in Microsoft and Apple. And it's this last 20% that really is the differentiator that sells Windows 80% of the time.

Knowing this, Microsoft uses money and clout (being the largest OS distribution in business and home markets) to influence hardware and software vendors to delay or reject the Linux environment from taking the other 20%. And by delay, I mean offering proprietary drivers and forcing open source to accept proprietary abstraction layers.

+0.5. Well put, though it misses the point partially.

To answer "Why would others...?"; well, the awareness of Linux as a viable desktop operating system has grown quite a bit recently. Readers of tech review sites know that it exists, but some of them also have misconceptions of it. By performing Linux testing, these other review sites will a) generate more readership and content, b) stimulate discussion and experimentation among readers (many of whom enjoy modding or trying new things), c) force hardware manufacturers to start paying attention to Linux, d) attract both more users and developers to an open-source platform, etc etc. The list could go on for a while, and it contains benefits for both the tech review sites and the communities that participate in them.

Right now, it's easy for people to complain loudly about Linux; trolls and whiners attract crowds, especially when the target is a relative minority. If there were more objective and constructive discussions happening, people would start adapting to things like the variation between distributions.

If Phoronix had followed point D from your post, well, that really would have been selfish. But is it that hard to see that the course of action they're trying to take is one that ends up positively affecting an entire spectrum of online communities?

Also, through this method of appealing to more readers, generating more content, and increasing the awareness of this platform, there will be a cumulative effect that will end up trying to address the 80-20 problem you mentioned. It will also end up soliciting a concrete reaction from many important groups (both formal and informal), including corporations like Microsoft. Have you noticed their recent trend of reluctantly starting to work with open-source developers, and opening up a couple of small things? In my opinion, that was a direct response to the growing pressure from the industry in general to start figuring out how to interact with open-source.

There was a general realization that the proliferation of open-source ideology is actually growing over time instead of fading into obscurity like they had hoped. So can either start to figure out how to reap profits from it in a mutually symbiotic way, or they can decide to ignore it until it's potentially an even bigger movement. After seeing it grow over the past few years, the latter option probably doesn't look too appealing to them at the moment.

I do emphatically agree with your assessment of the 80-20 problem and how corporations are trying to manipulate the 20 to suppress Linux growth for as long as possible.

[nanonyme]

That is a pointless argument since if the vendor is going to ship it with *any* OS, they'll STILL have to do compatibility testing. That should be considered as a component of the fixed assembly cost.

Yes, compatibility testing might cost roughly the same but compatibility cost per sold computer in an identical bunch of computers varies a lot. Compatibility cost per sold Linux computer is way higher than compatibility testing per Windows computer. It depends on sales.
Edit: People care for comfort more than they care for the cost. They care more that the computer they buy works as is than that they could have bought it a bit cheaper (OEM Windows licenses for certified vendors cost next to air) but they would have had to actually work to get it set up.

[midol]

Who did this letter go to?

[MakotoTheKnight]

Benchmarking software is good to have, and from what I've seen of th PTS suite itself, it's a decent piece of software.

Don't mind me, but I made an unofficial torrent so that those with a poor connection to Softpedia don't have to wait insanely long. The link is here. I plan to seed for the next fifteen hours, so hopefully someone picks up on it.

Now, to burn a copy of this thing and see if Java x64 regressed...

[Xipeos]

And here is the digg.

The article seems to have more "success" on reddit, but people there say it's just a shameless self-promotion. I guess you're entitled to it, but when I look at hardware reviews, I mostly look for what hardware is "better" than the rest, and that can be seen regardless of platform (as long as the tests are consistent).
Asking people to do an extra bench with your platform is weird because:
a) most people will ignore the linux benchmarks since they care about windows only
b) linux people will gain no new information, since the PTS bench results should show a hierarchy consistent with the win bench results.
And so, PTS would be just a tool to promote phoronix and to some extent linux (which I support, but then diggers come here and ruin the forum ).

[blackiwid]

whats up here, I dont understand the people here, opensource/foss people have often a problem with each attempt to make money and that then it cannot have a good aim. thats maybe often true, but not always.

If phoronix would force the people, to mention them, in their reviews, i would agree that its only marketing. But to ask (not to force) somebody who takes a piece of hard work from you to mention your name in a publication where you use it is not to much.

Also its a good advertisement for linux. Maybe 10% of the Press use own Linuxbenchmarks at least on some reviews, they maybe dont care about this because they dont need it, maybe a few switch to phoronix-suite because they think its better than their stuff and they didnt know this software yet. From the other 90% maybe 80% dont care, 5% maybe really consider to use it for future tests, and other 5% maybe hear again from linux and that it is easy to use also for such tasks and they hear maybe similar in the future from other sides again and then they maybe use it someday, too.

But thats marketing. You address 100 people and maybe 98 from it dont care but the 2 who you get feedback are enough. I dont see a possible negative impact from this letter.

And if some reviewers switch to multiplattform benchmarks that could have really big effects, because the press is the "multiplicator" that means they address thausends of people if there are again 2% who was maybe interested in linux before and because of such a article they maybe try it out.

I make miself no illusions, Linux will not be marktleader in next 5 years. (If google has not outstandig success ) But a world with doubled linux-market-share would be great.
And on some point what is maybe nearer than some here think, there is a breakpoint. A point where 99% of the companys cannot continuing ignoring linux, or deal with them as 2nd class customers.

[curaga]

More widespread Linux testing would cover some rather interesting situations as well, such as where the win results are the opposite of the linux results.

I can see only good coming from this.

[Penti]

The most important thing they should test is compatibility. They should stress hardware makers too support those users. They have some sway with them. But just running benchmarks on Linux I don't see much point except when it's server benchmarks.

[Penti]

Yes, compatibility testing might cost roughly the same but compatibility cost per sold computer in an identical bunch of computers varies a lot. Compatibility cost per sold Linux computer is way higher than compatibility testing per Windows computer. It depends on sales.
Edit: People care for comfort more than they care for the cost. They care more that the computer they buy works as is than that they could have bought it a bit cheaper (OEM Windows licenses for certified vendors cost next to air) but they would have had to actually work to get it set up.

OEMs selling computers need to decide which distribution to support. For the most time that will just be RHEL WS and SLED on workstation computers. Supporting a community distribution would be next to impossible. They could only hope to do that informally. Even a dist like Ubuntu have not had success breaking into this market. There simply is no consumer linux. Server linux is there, workstation linux kinda. But a Dell with Ubuntu can't even be bought in most European countries. And it's not even properly packaged for those markets. Maybe someone should try breaking Linux into the small but growing HTPC market in order to contribute to better multimediasupport for linux where it's laking. Multimedia licensing are a PITA but now there's not even any good alternatives to the homebrew ffmpegbased stuff. Journalists ignore that stuff like VLC aren't really legal (even Sweden owns some of the software patents so it's more that you couldn't support it commercially) or wont really be commercial grade in all fields, or that you would never get the needed frameworks in place if that's all there is. Stuff like video editing aren't really there as a result meaning media savvy users just end up on a Mac or Windows computer in the end instead, even if they didn't have any big demands. Linux users want stuff that just works too, just see those supporting Intels efforts. Frankly Windows computers aren't perfect OOB, so I'm sure people would be willing to pay for a linux-computer that's pretty good OOB and contains many useful tools instead of spending hours fixing just basic stuff (like you also would on a Win computer). An NLE video editing suit could for example have a foss version just supporting Theora and some other intermediaries (or even provide free non-distributable licensed codecs but somebody must pay for it) and a payed version which can be OEMd which would include support for mpeg2, avc and all that's needed.

Any way lots could be done, it requires putting some manpower on it though as they do to some extent on platforms like Nokias Internet tablets. There's no harm commercializing Linux (and making money) as long as they support FOSS and contribute back. You won't get anything for free but you can get collaboration and openness. Any way they would need a dist that's acceptable for the users to get any buyers at all, they will need to be available and have some support for alternative distributions, no blobs only one dist will include for one customer.