Ubuntu 11.10: Xen vs. KVM vs. VirtualBox

[DaemonFC]

Their clauses generally just mandate no publishing of benchmarks, not that they can't be carried out for your own personal research.... Granted VMware doesn't seem too bad, they even use the Phoronix Test Suite and OpenBenchmarking.org when publishing some of their own results.

It still meets the definition of anti-competitive business practices.

You can benchmark a product, find out it totally sucks, and are effectively gagged from telling anyone else. This is also clearly an infringement of freedom of speech.

Edit: I guess those sites which point out that Internet Explorer is the slowest and least standards compliant major web browser are in danger of being sued?

[Sadako]

Were the virtio drivers used with qemu, particularly the storage/disk driver?

Also, were the paravirtualisation options enabled in the qemu guest kernel?

If not, they'd be very interesting to see too, and if so, a comparison without these would be just as useful.

[felipevsw]

I have a doubt on the configuration that, I believe, was not explained on the description of the test.

If these are enabled by default, excuse my lack of knowledge.

Were these configurations running those para-virtualized modules for ethernet and disk on KVM? I believe it`s the 'virtio' modules.

[ppanula]

Whats the point to Xen _HVM_ Linux, eg. emulated network and block io drivers, you bet its slower comparing to PV KVM, virtualbox, etc. drivers. Whole test is invalid if you compare in that way...

[liam]

http://www.spec.org/virt_sc2010/resu...2010_perf.html

The takeaway is kvmqemu ~ esx. Considering kvm's age, that is pretty damn good.
Though I've never used esx, and I only use kvm for running a test image or two, my understanding is that vmware's advantage has been its gui tools. Now, RHEV is supposed to include some nice browser based interface but I've never used it (and, IIRC, it requires INTERNET EXPLORER... though I think that won't be the case for much longer).
Still, I have to wonder why it is that kvm isn't making a bigger dent in the virt market than it is. I'm assuming the issue is marketing since I see so much written about microsoft/vmware/citrix.

[palmje]

KVM offered the fastest performance in all of the tests except for SQLite, where VirtualBox was the fastest but that is due to a bug.

-- Phoronix

Not true - Xen won one of the benchmarks as well: NAS Parrallel Benchmarks v3.3 - Test/Class: IS.C

Bare Metal: 141.82
Virtual Box: 78.24
Xen: 97.68
KVM: 87.87

But as others have said, it would be interesting to see the results of Xen without the HVM as well...

[Vadi]

Looks like I'll be content with Virtualbox' performance here.

[gilboa]

I second the virt-io question.
Beyond that, it would have been interesting to compare networking performance.

As general, these results more-or-less mirror the VBox vs. qemu-kvm benchmark I did in-house when we selected our VM solution (qemu-kvm).

- Gilboa

[pasik]

What's the point to test the ancients Xen HVM paths? PV would have been MUCH more interesting...

First page of the articly says:
"The only Xen issue encountered when testing it with an Ubuntu 11.10 guest and host was the need for manually loading the xen-blkfront driver for disk support."

So that looks like they did actually use Xen PVHVM drivers in the HVM guest, so optimized drivers (xen-blkfront and xen-netfront) were available.
Still it looks weird that Xen did so badly in that benchmark.. pretty much the opposite results were posted some time ago at XenSummit. Xen was faster than KVM in almost every benchmark..

It would be nice to get more information about the benchmark setup so I could try the benchmark myself.

[pasik]

Yup, I use Xen, I've never used HVM for a linux guest and I don't get the point of it.
You can also use LVM logical volumes on the host as disks for your guests, and I think it will change a lot of results

what disk backend did this benchmark use in dom0? file: ?