Imagine this instead:
Sure, developers could run PTS, but to be honest, most fs devels I know don't think the PTS filesystem tests are worth a warm pile of spit. And I say that as nicely as I can. FS developers certainly do run relevant benchmarks before pushing.The ext4, xfs, and btrfs results were much, much, slower than ext3, but that is due to the extra data integrity measures they employ, critical for databases. When we spoke with the developers of these filesystems, we learned that this is largely only needed when a volatile write cache is present in the storage, and when that is not the case (for example, with a battery backed raid cache, or reliable UPS), barriers may be safely disabled to enhance the speed. To verify this, we re-tested with this tuning, and here are the results:
I'm hoping to go through some of the PTS tests and make suggestions to make them more repeatable & relevant. But for example the "mp3 file encryption" tests which used to be run, as noted over and over, simply do not stress the filesystem.
Really, even the same is likely true for the apache static page serving test; in my testing, the only significant FS activity was writing errors to the error logs.
Another oddity is the iozone 4G read tests; the report is that ext4 is doing 220MB/s on a single sata drive, which is most likely faster than the drive can actually get data off the disk. It may be measuring in-cache performance to some degree. A baseline of the storage performance would go a long ways towards putting results in context. Clearing caches before the read portion would show what the filesystem itself can do. Doing a dataset significantly larger than the system's memory would make it more relevant.
You are right that the tests simply run & report, but there is some onus on the tests to be running things that matter, when the results are presented as they are.
I really would like to see Phoronix be a genuinely useful resource for developers & users alike, but I think there's a ways to go.