I wonder how reliable tests run inside a VM are, e.g. some of those tests might well be testing the speed of I/O emulation in Qemu instead of the speed of the kernel running on it...?
and how much are these benchmarks worth - running in a vm?
how much are these benchmarks worth - when one of them can't even do SMP?
PS on the article from Michael: It's indeed sad that the tests were carried out in a VM, that seems not too really realistic to me. At least I expect all sorts of falsifying effects from the VM/host machine. But with this hardware support projects like HURD, Plan0/Inferno might stay an experimental research project for geeks. But maybe some of the interesting info won there could be used in the more mainstream products.
Benchmarking OSes under virtual machines reminds me this ;)
I didn't mean to say tests inside a VM are completely useless, but they are meaningless without knowing what is being tested. Doing the same tests on the same machine on bare hardware might help, provided the machine is booted with only one core/thread active. And of course, as somebody already mentioned, more different tests would be useful too.