EXT4, Btrfs, NILFS2 Performance Benchmarks

[sandeen]

To some extent I disagree with this. The upstream developers, lead developers who push to the kernel and the distribution vendors all have a part to play in the cycle of getting things to production.

Phoronix is just reporting on the state of whatever made it into the kernel.

My view is that the question isn't so much what Phoronix has highlighted with 2.6.30 and why didn't he discuss with the upstream developers before publishing, but rather why didn't the maintainers have an awareness of the benefits and deficiencies prior to pushing up into the kernel.

For most people, they will review the general performance, select the filesystem that best performs for their intended use, and then tune only that platform.

I appreciate the position that the developers are in, but PTS is trivial enough for the developers to be self-aware before pushing downstream. It shouldn't take any real effort to test before pushing.

Regards,

Matthew

I get your point, but going just a -little- deeper to analyze the results (beyond "shocking!") would do a great service.

Imagine this instead:

The ext4, xfs, and btrfs results were much, much, slower than ext3, but that is due to the extra data integrity measures they employ, critical for databases. When we spoke with the developers of these filesystems, we learned that this is largely only needed when a volatile write cache is present in the storage, and when that is not the case (for example, with a battery backed raid cache, or reliable UPS), barriers may be safely disabled to enhance the speed. To verify this, we re-tested with this tuning, and here are the results:

Sure, developers could run PTS, but to be honest, most fs devels I know don't think the PTS filesystem tests are worth a warm pile of spit. And I say that as nicely as I can. FS developers certainly do run relevant benchmarks before pushing.

I'm hoping to go through some of the PTS tests and make suggestions to make them more repeatable & relevant. But for example the "mp3 file encryption" tests which used to be run, as noted over and over, simply do not stress the filesystem.

Really, even the same is likely true for the apache static page serving test; in my testing, the only significant FS activity was writing errors to the error logs.

Another oddity is the iozone 4G read tests; the report is that ext4 is doing 220MB/s on a single sata drive, which is most likely faster than the drive can actually get data off the disk. It may be measuring in-cache performance to some degree. A baseline of the storage performance would go a long ways towards putting results in context. Clearing caches before the read portion would show what the filesystem itself can do. Doing a dataset significantly larger than the system's memory would make it more relevant.

You are right that the tests simply run & report, but there is some onus on the tests to be running things that matter, when the results are presented as they are.

I really would like to see Phoronix be a genuinely useful resource for developers & users alike, but I think there's a ways to go.

-Eric

[mason]

when the database numbers are as far off as they are in this test it makes me suspect that the fsyncs are being ignored by some filesystems and not by others. none of these filesystems are so far apart in design (except for possibly nilfs) that a 20x-60x variation is reasonable.

I'm not sure about nilfs, but ext4, Btrfs and XFS issue cache flushes on sata by default. For ext3 you'll want to mount -o barrier=1.

The Btrfs numbers on sqlite should improve with the 2.6.31 kernels, it is one of the things that we are targeting right now.

[Louise]

Chris Mason have just been interviewed a week ago about btrfs:

http://linux.com/news/featured-blogs...stem-for-linux

Very interesting to hear what goes on behind the scene

[Michael]

I get your point, but going just a -little- deeper to analyze the results (beyond "shocking!") would do a great service.

The reason that is not done more is the time involved with already being seriously bottlenecked due to the very limited human resources at Phoronix... If you know a way to handle this without incurring additional costs, then it would be done, but I don't have the magic answer yet.

I'm hoping to go through some of the PTS tests and make suggestions to make them more repeatable & relevant. But for example the "mp3 file encryption" tests which used to be run, as noted over and over, simply do not stress the filesystem.

Some of that has to do with satisfying page view requests, but that test, for instance, is no longer used in Phoronix.com filesystem tests.

[RoboJ1M]

Do any of these tests perform delete operations?

As a Myth user, one serious problem with ext3 is how long it takes to delete very large files (5GB and larger).

Try XFS, as far as I know it's the deleting-large-files supremo.

Every create/delete is identical in speed, so it then loses for deleting lots of small files. But for mounting as /mythmedia it rocks.

J.

[wateenellende]

Actually, I'd be very interested to see how reiser4 compares to other "modern" filesystems. It would seem that it is maintained, although I don't know if it is included in standard distribution kernels, so it may demand more time, which is probably a scarse resource at phoronix. Anyway, recent sources are here: http://kernel.org/pub/linux/kernel/p...dward/reiser4/

All further issues aside, I think we can all agree that reiser4 had some interesting technological features, and therefore it would be interesting to see how it compares to the most notable competitors, for example ext4, btrfs and xfs.

Given all the fuzz, I'd like to know whether reiser4 is significantly worse, comparable, or significantly better than the others. What better way is there to find out than to simply look at benchmark results ?

[dlang]

Try XFS, as far as I know it's the deleting-large-files supremo.

Every create/delete is identical in speed, so it then loses for deleting lots of small files. But for mounting as /mythmedia it rocks.

J.

ext2/3 suffer dealing with large object. this can be large files, or directories with large numbers of files in them.

as an extreme example of this

create a directory
create 100,000 files in that directory

time how long it takes to do a ls of that directory
time how long it takes to do a rm * of that directory
time how long it takes to do a ls of this now empty directory

create a single file in this directory
time how long it takes to do a ls of that directory
time how long it takes to do a rm * (or rm filename)

now do the same thing with other filesystems.

I know that XFS is pretty good at this. I haven't tried ext4 yet, but I saw some things recently that make me think that it will do better, but still have problems.

you don't have to go all the way up to 100,000 files to see a problem, I first noticed it on mail spools with 10,000 files in them

[rektide]

the article mentions these tests were done bone stock... btrfs does consitency checking by default. there are nehalem optimizations, but otherwise they're a pretty big cpu hit. it'd be valuable to also show results with consistency checking turned off, for comparison sake.

[Kano]

@dlang

rm is VERY fast on ext4. i did not see so many differences when mainly reading, but delete is faster.