Gentoo Linux vs. Linaro ARM Benchmarks

[ryao]

The people on #gentoo-embedded irc channel say that he is 'dagger'.

That solves that mystery. It would seem that our records are inconsistent. The official record says that his name is "Robert Piasek", but he is using "Robert Sanders" on freenode.

[RealNC]

I assumed that the phoronix test suite was a bunch of scripts that used the system libraries and binaries. Is that incorrect?

AFAICT, the scripts download the sources of each individual benchmark, build them, and then run them. So in that case, GCC 4.7 will probably always win, regardless of whether you're using Gentoo or Linaro.

[ssvb]

If you have GCC 4.7, it's not Gentoo Stable anymore.

I don't disagree* with anything else you said, since it's not the point. The point is that Gentoo Stable was not tested in this benchmark.

* Well, maybe with not installing GCC 4.7 in Linaro. The Phronix Test Suite is built from source, so why wouldn't GCC 4.7 affect the results? I say it would affect them greatly.

Whatever. In any case, "Gentoo Stable" phrase feels unnatural to me and I don't see why anyone would insist on using it. "Gentoo" itself is a highly configurable distro where you can easily mix and match various versions of the packages, use different versions of gcc or even use different non-gcc compilers for some selected packages. Also keep in mind that upstream FSF gcc-4.7.1 is even not a point-zero release, that's a compiler with a bunch of bugs ironed out already. I don't understand your prejudice against it.

The plain fact here is that Ubuntu clearly underperformed in this test. It's up to somebody to run the benchmark on a properly configured linaro/ubuntu system (using 'performance' frequency scaling governor) with a better compiler and share the results.

BTW, the first initial Ubuntu 11.10 phoronix benchmark on Pandaboard ES showed such abysmal performance that it even got me motivated to register in this forum (because somebody was wrong on the Internet! ). So I had posted my own results from a similar 1.2GHz dual-core Cortex-A9 board for comparison and it used stable Gentoo with gcc 4.5 at that time (versus newer ubuntu gcc 4.6). There were some Ubuntu performance improvements since that time, which were depicted on phoronix as kinda "Ubuntu being on the forefront of ARM development" and not just "sore losers are catching up" Seriously, linaro is doing a lot of gcc related ARM work which eventually gets accepted in upstream gcc and deserves credit for this. But I get a feeling that the end users are treated as guinea pigs and the effect of a lot of minor incremental improvements gets eclipsed by some stupid bugs which ruin the overall performance in Ubuntu distro.

[ryao]

AFAICT, the scripts download the sources of each individual benchmark, build them, and then run them. So in that case, GCC 4.7 will probably always win, regardless of whether you're using Gentoo or Linaro.

Does it build the library dependencies as well? If not, Gentoo would have an edge because its system libraries would still be built with the system compiler, while Linaro's would be whatever Linaro ships.

[ssvb]

Does it build the library dependencies as well? If not, Gentoo would have an edge because its system libraries would still be built with the system compiler, while Linaro's would be whatever Linaro ships.

There is little point speculating Gentoo can be just rebuilt using gcc 4.6 (maybe even in two configurations: with and without thumb2) and benchmarks repeated on ODROID-X. With ondemand governor to make it fair.

[RealNC]

Does it build the library dependencies as well? If not, Gentoo would have an edge because its system libraries would still be built with the system compiler, while Linaro's would be whatever Linaro ships.

Unless the benchmark is redone properly, we'll never know.

Note that all I'm saying here is that this particular benchmark is null and void. You can't draw any conclusions from it. It was pretty much a waste of time both for Michael, as well as for us.

[ssvb]

Note that all I'm saying here is that this particular benchmark is null and void. You can't draw any conclusions from it. It was pretty much a waste of time both for Michael, as well as for us.

Sorry for repeating, but the conclusion is that the performance of Ubuntu-based Linaro 12.08 with ondemand governor is far from perfect. The hardware can do much better than that. Any objections?

[crazycheese]

About Gentoo compilation time. When using Gentoo for desktop (kernel, xorg, kde, firefox, libreoffice and quite a pack of other software) , on a standard quad-core (phenom II, core i first generation), the complete tree reemerge takes about 14 hours. But you hardly ever do this, because since glibc and gcc are usually kept same, packages are emerged or updated, as well as their dependencies, but never the whole system. I found Gentoo to be far easier to maintain than Arch.

[RealNC]

Sorry for repeating, but the conclusion is that the performance of Ubuntu-based Linaro 12.08 with ondemand governor is far from perfect. The hardware can do much better than that. Any objections?

This was a Linaro vs Gentoo bench. And in that respect, it failed. The only thing you can do then, is to ignore the Gentoo results and look at the Linaro numbers. And in that case, you don't have anything to compare them against. So how can you draw any conclusions about the governor? It takes at least two in order to perform any kind of comparison.

[ssvb]

So how can you draw any conclusions about the governor?

I'm not drawing any conclusions about the governor. The conclusions are about the "Ubuntu-based Linaro 12.08" as a whole.

It takes at least two in order to perform any kind of comparison.

You have the results from some other linux distro (which happens to be not "Stable Gentoo", but who cares).