Real World Benchmarks Of The EXT4 File-System

[phoronix]

Phoronix: Real World Benchmarks Of The EXT4 File-System

With the EXT4 file-system being marked as stable in the forthcoming Linux 2.6.28 kernel, and some Linux distributions potentially switching to it as an interim step until the btrfs file-system is ready, we decided it was time to benchmark this journaled file-system for ourselves. We ran a number of disk-centric Linux benchmarks along with several of our real-world tests from the Phoronix Test Suite to gauge how well the EXT4 file-system performance will be noticed by desktop users and computer gamers. We have compared these EXT4 results to the EXT3, XFS, and ReiserFS file-systems.

http://www.phoronix.com/vr.php?view=13199

[Kazade]

I'll be honest, I'm a little confused about using games as a benchmark for a filesystem. Games load resources from the disk before the game play starts, everything from that point on is stored in either RAM or VRAM while the game is in play (unless of course you run out of memory). Only an insane game developer would read or write from the disk during gameplay because it would kill frame rate.

If you were timing the loading times (or game saves) fair enough, but using the frame rate as a bench mark seems pointless.

[StefanHamminga]

May I suggest a real world test that might show a difference?
Find a way to bench:

apt-get update && apt-get dist-upgrade

It seems ('feels') faster with EXT4, coming from EXT3 (running Intrepid on a HTPC).

[Solitary]

Time elapsed from power on to login screen would have been interesting.

[b0uncyfr0]

Yes im really interested in EXT4 but i need to know if theres an increase in boot speed. That all i mainly care for.

Also deleting large directories isnt a strongpoint of EXT4. Thats a pity. EXT3 seems slow as it is, i cant imagine it geting any slower.

Thanks for the review..

[ssam]

it would be nice to see benchmarks for boot time and program load time. these are far more disk bound than the games and encoding tasks.

also i'd love to see error bars on these benchmarks (sorry i am a physicist). if you just give a result with 4 significant figures, you are implying that the result would be exactly the same if you ran it multiple times. i suspect that some of these results vary by a few percent with each run. so if one beats te other by 0.5% it is statistically insignificant.

all that would be needed would be to run each test 3 times, and put the standard deviation of the 3 runs as the error bar.

then if the difference in height between 2 bars is smaller than the error bars, it can be seen that it is insignificant.

http://www.graphpad.com/articles/errorbars.htm <- some info

(also if would be great if it were clearer which plots have big is good, and which are small is good. maybe an arrow labelled 'faster' pointing up or down)

[Gentooer]

I second the suggestion for error bars. They would make the results much more meaningful. It would have also been nice to see Reiser4 instead of the outdated v3.

[pjlbyrne]

I would like to have seen timings for a large code build - that it the kind of activity which would really benefit from aggressive file fragmentation prevention (AFFP?).

[Michael]

it would be nice to see benchmarks for boot time and program load time. these are far more disk bound than the games and encoding tasks.

also i'd love to see error bars on these benchmarks (sorry i am a physicist). if you just give a result with 4 significant figures, you are implying that the result would be exactly the same if you ran it multiple times. i suspect that some of these results vary by a few percent with each run. so if one beats te other by 0.5% it is statistically insignificant.

all that would be needed would be to run each test 3 times, and put the standard deviation of the 3 runs as the error bar.

then if the difference in height between 2 bars is smaller than the error bars, it can be seen that it is insignificant.

http://www.graphpad.com/articles/errorbars.htm <- some info

(also if would be great if it were clearer which plots have big is good, and which are small is good. maybe an arrow labelled 'faster' pointing up or down)

Tests from the Phoronix test Suite are usually ran at least three times and then averaged all automatically.

Once I upload these results to Phoronix Global, if using Phoronix Test Suite 1.6 you can run: phoronix-test-suite analyze-all-runs and it will generate modified candlestick charts (essentially your error bars) on the results.

[[Knuckles]]

It would be interesting to see tests with many small files. I once used XFS for my root filesystem and performance was horrible to do things like unpack sources and remove directories with many files, my system would just freeze while XFS was thrashing away at the disk.

It's all nice and dandy with multi-gb files, when all those caches and delayed allocations do a great job, but I learned my lesson with XFS, and now use ext3 everywhere.

I also agree that boot time and application load times are important, although I guess both may be hard to measure with the PTS.