Debian Wheezy GNU/kFreeBSD: Slower Than Linux

[0xBADCODE]

This results in any commercial OS based ON BSD code to be faster, better and more efficient than ANY BSD.

Which in turn has three multiple consequences for BSD:
- to exist only and only in shadow
.

....and that's why Linux beats it. Collaboration and alliances are clearly outrunning competitions and wars.

In *BSD world everyone have to reinvent wheel again. Just because competitor wants to slow all "enemies" down and outrun them. So everyone reinvents the wheel. In Linux world, wheel invented once and then similar bunch of people improves it instead of parallel re-invention of the same thing. Seems to give quite an obvious results.

[stevenc]

Oh yeah, it's 1.5-2 times slower in quite many tests and by 4 whole times slower in openssl. "A little slower". Hmmph.

Though I'm fail to understand how exactly *BSD guys managed to get such a bad results on same hardware. They managed to both get crapwrecked CPU scheduler and poor syscall servicing times? Broken hardware initialization? Or how they could lose whole 4 times in openssl which is more or less computational test? It does not even requires servicing many syscalls!

The OpenSSL result is particularly interesting. But I couldn't reproduce it on my own (AMD Opteron) systems. Using latest Debian 1.0.1c-3 packages, I ran 'openssl speed' which includes an RSA 4096-bit signing benchmark. The other output from this command (compile options) is helpful to see if trying to compare benchmarks. A 2.6GHz Opteron 285 (twin, dual-core) system with Linux 2.6.32 scored 73.7 sign/s whereas a 2.2GHz Opteron 248 (single, dual-core) GNU/kFreeBSD system (9.0-1-amd64 kernel) scored 62.1 sign/s. I assume the benchmark runs on a single core and thus these results are almost precisely in-line with the relative clock speeds. Certainly no 4x performance reduction seen.

I don't know how the benchmark for PTS works, but I'm curious why the test result panel says "OpenSSL 1.0.0e" which is not the (recently) packaged Debian version.

Otherwise, I'd guess the cause of this is something hardware-specific, with this particular CPU not reaching full performance under this FreeBSD kernel version. I would check for anything related to CPU scaling, power draw (which is nice to see in many Phoronix benchmarks), or otherwise see if FreeBSD have already made any changes in HEAD / kfreebsd-10 relating to this hardware.

[allquixotic]

Fedora 17 on a ThinkPad T530 with a Core i7 3820QM (Ivy Bridge):

OpenSSL 1.0.0j-fips 10 May 2012
built on: Tue May 15 18:29:32 UTC 2012
options:bn(64,64) md2(int) rc4(16x,int) des(idx,cisc,16,int) aes(partial) blowfish(idx) 
compiler: gcc -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DKRB5_MIT -m64 -DL_ENDIAN -DTERMIO -Wall -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4  -m64 -mtune=generic -Wa,--noexecstack -DMD32_REG_T=int -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DWHIRLPOOL_ASM
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
md2               3008.33k     6125.53k     8254.33k     9062.40k     9318.06k
mdc2                 0.00         0.00         0.00         0.00         0.00 
md4              84434.65k   264374.44k   644257.82k   996092.25k  1190922.27k
md5              63187.66k   188001.96k   423254.82k   610036.74k   704076.37k
hmac(md5)        53563.74k   158385.32k   381761.33k   587953.83k   699139.25k
sha1             70823.08k   203082.40k   443047.08k   641391.96k   755614.74k
rmd160           43356.44k   106884.32k   195177.22k   248272.40k   268170.58k
rc4             413256.97k   705812.38k   816161.71k   857680.14k   867560.11k
des cbc          73307.18k    75891.22k    76088.23k    76249.43k    76673.28k
des ede3         28634.92k    29143.91k    29111.04k    29096.28k    29140.51k
idea cbc             0.00         0.00         0.00         0.00         0.00 
seed cbc         81854.90k    81229.66k    81740.80k    81892.94k    81685.16k
rc2 cbc          50009.57k    50417.49k    50973.62k    50972.67k    51217.81k
rc5-32/12 cbc        0.00         0.00         0.00         0.00         0.00 
blowfish cbc    131123.55k   138414.21k   140498.02k   140976.13k   141442.14k
cast cbc        115903.82k   122357.83k   123058.09k   124024.55k   124466.52k
aes-128 cbc     131762.18k   144214.26k   146054.66k   147490.13k   148666.99k
aes-192 cbc     111920.14k   119723.99k   122373.22k   123111.08k   123778.65k
aes-256 cbc      97196.49k   103389.61k   104866.25k   105222.14k   105835.71k
camellia-128 cbc   103898.70k   158217.12k   178736.81k   185166.26k   185305.77k
camellia-192 cbc    89495.65k   122895.72k   134847.02k   138753.37k   140587.32k
camellia-256 cbc    89182.87k   122451.71k   133507.50k   137959.17k   140557.19k
sha256           53759.63k   120741.07k   207696.30k   260242.09k   280949.98k
sha512           42240.75k   170258.99k   260882.09k   367734.44k   426915.53k
whirlpool        25523.32k    54972.21k    91044.18k   108745.39k   115726.38k
aes-128 ige     134028.59k   138739.73k   140249.64k   140259.33k   140195.16k
aes-192 ige     112835.38k   116868.29k   118129.44k   118033.41k   118226.94k
aes-256 ige      98245.07k   101325.87k   102308.39k   102273.02k   102643.84k
                  sign    verify    sign/s verify/s
rsa  512 bits 0.000047s 0.000004s  21073.7 241610.2
rsa 1024 bits 0.000233s 0.000013s   4291.7  79594.7
rsa 2048 bits 0.001526s 0.000044s    655.3  22677.0
rsa 4096 bits 0.010720s 0.000177s     93.3   5659.5
                  sign    verify    sign/s verify/s
dsa  512 bits 0.000052s 0.000050s  19249.6  20171.6
dsa 1024 bits 0.000133s 0.000147s   7533.8   6822.4
dsa 2048 bits 0.000441s 0.000515s   2268.3   1942.4

[kraftman]

If BSD is so much slower than Linux, how did that article a while back come about that showed BSD was faster at running native Linux games?

It was explained. BSD was running KDE with suspended compositions and was tested against Ubuntu which is using compiz and has this option disabled. So it was KDE that was faster than compiz in this tests.

[kraftman]

Linux has no real filesystem, has it?

I just setup my laptop with ZFSOnLinux as root fs and Gentoo last weekend. Works fine so far except of grub2 didn't like to boot the pool directly, so it needed an extra 100MB boot partition.
As soon as I decided on a GUI (DE) etc. I'll try to run some benchmarks...

Damn troll. It has many real file systems unlike others. Who cares about your configuration?

[kraftman]

The OpenSSL result is particularly interesting. But I couldn't reproduce it on my own (AMD Opteron) systems. Using latest Debian 1.0.1c-3 packages, I ran 'openssl speed' which includes an RSA 4096-bit signing benchmark. The other output from this command (compile options) is helpful to see if trying to compare benchmarks. A 2.6GHz Opteron 285 (twin, dual-core) system with Linux 2.6.32 scored 73.7 sign/s whereas a 2.2GHz Opteron 248 (single, dual-core) GNU/kFreeBSD system (9.0-1-amd64 kernel) scored 62.1 sign/s. I assume the benchmark runs on a single core and thus these results are almost precisely in-line with the relative clock speeds. Certainly no 4x performance reduction seen.

I don't know how the benchmark for PTS works, but I'm curious why the test result panel says "OpenSSL 1.0.0e" which is not the (recently) packaged Debian version.

Otherwise, I'd guess the cause of this is something hardware-specific, with this particular CPU not reaching full performance under this FreeBSD kernel version. I would check for anything related to CPU scaling, power draw (which is nice to see in many Phoronix benchmarks), or otherwise see if FreeBSD have already made any changes in HEAD / kfreebsd-10 relating to this hardware.

The case is not hardware, but BSD and... date specific. Aren't you aware you tested few year old Linux kernel while there's much newer one used in Phoronix comparison?

[stevenc]

The case is not hardware, but BSD and... date specific. Aren't you aware you tested few year old Linux kernel while there's much newer one used in Phoronix comparison?

I'm aware that I used a much older Linux kernel for my quick test, but I don't expect that would make much difference. I don't imagine Linux 2.6.32 -> 3.2.x would make it 4x faster and thus bring it in line with the Phoronix result.

If I get the opportunity, I may try to do this properly (Linux 3.2.x vs. GNU/kFreeBSD 9.0, and use the same machine for both tests).

I'm also curious how the Phoronix OpenSSL result compares with 'openssl speed', in case it is a problem in the test suite. Perhaps it used all 4 cores on Linux but on BSD may not be detecting the number of cores correctly (the numbers would make sense).

[XorEaxEax]

I'm aware that I used a much older Linux kernel for my quick test, but I don't expect that would make much difference. I don't imagine Linux 2.6.32 -> 3.2.x would make it 4x faster and thus bring it in line with the Phoronix result.

Well 2.6.32 was released in december 2009 so I'd say it's very old by kernel standards and there's every reason to believe there's been alot of optimizations since then. That said I think a 4 x difference in performance is suspiciously extreme (although it was only one test where it was this extreme iirc), and while I'm not surprised Linux performs better than FreeBSD/PC-BSD I'd expect it to be much closer.

[curaga]

I'm also curious how the Phoronix OpenSSL result compares with 'openssl speed', in case it is a problem in the test suite. Perhaps it used all 4 cores on Linux but on BSD may not be detecting the number of cores correctly (the numbers would make sense).

"openssl speed rsa4096" is a single-threaded test, and ~40 is a completely normal result. For reference my phenom does ~58.

[nslay]

I think the results are unusual, especially the OpenSSL results. You should check for interrupt storms (vmstat -i?) and check that the CPU is really running at full steam. I've seen weird quirks where booting FreeBSD from battery causes the processor to be stuck in a power-saving state and running at reduced clock speed (even if you plug it back in).

Also suspect is SCHED_ULE ... this new scheduler was designed to ensure interactive application responsiveness under load. You could try changing it to SCHED_4BSD, however, that's not how GENERIC is shipped.

EDIT: Check this out: SCHED_ULE should not be the default. However, I'm not convinced this should make such a big difference in these toy benchmarks.

Lastly, FreeBSD is stuck at gcc 4.2 while the newer Linux distributions are using later versions of gcc. Maybe these later versions are better at optimization? Shouldn't, for example, John the Ripper run about the same on FreeBSD? Why should the same blowfish implementation be slower on FreeBSD? That shouldn't involve FreeBSD at all. You can compile all ports in FreeBSD with the same version of gcc by first building the same version of gcc from ports and then exporting CC and CXX and then building benchmark applications from ports the usual way.

Lastly, missing from all benchmarks on phoronix is significance levels. Significance should be measured overall and pairwise. The former measures if the overall timing difference is significant while the latter measures significance in consistency. For example, one test may not be significant overall, but be significantly consistently better/faster/etc. Tests should probably also consider a "warm-up" period to discard fluke speed-ups or slow-downs (if not already).