4-Disk Btrfs RAID Benchmarks On Linux 4.7

[phoronix]

Phoronix: 4-Disk Btrfs RAID Benchmarks On Linux 4.7

Going along with the recent Linux 4.7 file-system benchmarks, here are some tests of Btrfs' built-in RAID functionality when tested on the Linux 4.7 kernel across four SATA SSDs.

http://www.phoronix.com/vr.php?view=23385

[deimios]

Aren't Raid 0+1 configs the default for Raid 1 (mirroring) these days? I wonder if there is something stopping btrfs Raid1 from reading chunks from all disks at once Raid 0 style.
In theory the read speed of a 4 disk Raid 1 should be the same as a 4 disk Raid 0.

[DrYak]

RAID1 on BTRFS is rather different from RAID1 on mdadm and dm.

In mdadm, there are as many copies of the data as there are disks in te raid array.
So in a 4-disks array, there are 4 copies of the data and the total size you get is 1x the drive size.

In BTRFS, tere is currently always exactly 2 copies of the data spread on whatever 2 different drivers out of all.
So in a 4/disks array, there are 2 copies of the data and the total size zou get is 2x the drive size.
The only difference wit RAID10 is that the spread of the copies is more or less random.
- that make it possible to use it efficiently with uneven size of drive (say 1TB drive and 2x 512GB drives. You'll get correctly 1TB worth of 2x duplicated data0
- whereas RAID0 makes better organisation of the data so you can better leverage te RAID0 acceleration. (BTRFS RAID1 isn't as good as mdadm/DM's RAID at read load balancing, apparently)

RAID-1 x4 and RAID-0 x4 CANNOT be the same speed with BTRFS because currently BTRFS uses only exactly 2 copies with RAID-1 and as many as there are currently free devices with RAID-0 (so normally 4x, unless one of the drives got already full).

User selectrable number of copies in RAID-1 and width of stripe in RAID-0,5,6 is on their todo list.
(Though with the necessary rewrites, maybe there are going to be some delays until they have time to take care of this)

[duby229]

I was about to ask a question, but DrYak answered it before I had to ask. Thanks.

I'm sorry but RAID should not be used as a backup strategy, that's just not what it's for. Its real point is to keep the system functioning if a drive fails. Btrfs basically misses every single possible point.

[Stellarwind]

Aren't Raid 0+1 configs the default for Raid 1 (mirroring) these days? I wonder if there is something stopping btrfs Raid1 from reading chunks from all disks at once Raid 0 style.
In theory the read speed of a 4 disk Raid 1 should be the same as a 4 disk Raid 0.

Afaik btrfs does not support n-way mirror - 4 disk raid1 doesn't mean data is being mirrored on all 4 disks, more like raid10 with redundancy level of raid5.

[DrYak]

Forgot to add that the checksuming definitely adds an aditionnal layer of security, which is also good for backup storage. (similar to "SHA1SUMS" files and similar in classical backup solutions)

[DrYak]

Afaik btrfs does not support n-way mirror - 4 disk raid1 doesn't mean data is being mirrored on all 4 disks

As I've said above, according to docs, currently RAID1 on btrfs is limited to exactly 2 copies, no matter the number of devices.
Means that indeed the maximum number of drive an array can loose is always 1, exactly like an RAID5 array (but using copies instead of parities).
It is indeed more or less similar to RAID10, except there isn't a real RAID0 part like in a real RAID10 which is in charge of spreading the data in an organized fashion, which means you wont get the RAID0 acceleration like in an actual RAID10.
On the other hand, RAID1 write allocation is "2 copies on whichever 2 devices have currently the most free place". So an array with 1TB drive and 2x 512GB drives correctly gets TB worth of data, each with 2 copies (1 copy on the 1TB drive, and the other spread accross the 2 remaining drives).

[Zan Lynx]

I was about to ask a question, but DrYak answered it before I had to ask. Thanks.

I'm sorry but RAID should not be used as a backup strategy, that's just not what it's for. Its real point is to keep the system functioning if a drive fails. Btrfs basically misses every single possible point.

I am not sure where you are coming from with this. With any type except Single or RAID-0, BTRFS keeps two copies of the data, so that if one drive fails the system keeps functioning. A data backup is how any version of RAID does this.

If a drive fails and you still have more than 1 drive then you can run a balance operation and it will build a new copy of the lost data. Or you can immediately replace the drive and balance with the same effect.

If you have Linux and a USB hub this is really easy to experiment with. Grab several cheap USB storage sticks, plug them in and build a BTRFS array with them. Run tests. Pull one of the sticks out. You can play around with all of the failure and recovery that way.

[kreijack]

[...]

I'm sorry but RAID should not be used as a backup strategy, that's just not what it's for. Its real point is to keep the system functioning if a drive fails. Btrfs basically misses every single possible point.

I am not sure where you are coming from with this. With any type except Single or RAID-0, BTRFS keeps two copies of the data, so that if one drive fails the system keeps functioning. A data backup is how any version of RAID does this.

Backup is different from raid. Benefit of backup:
- multiple versions of the copy: a common strategy is to do a full-backup for week and an incremental backup per day. This means that you are able to go back with a "day granularity" for a week. Depending how many weekly backup you store you are able to to back in the past for several week with a "week granularity"
- typically a backup is append only, so you cannot delete a file: i.e. if you cancel a file from a raid, you lost it; a backup is secure until you reuse the support for another backup.
- a backup may (must!) be stored in a fire-prof enclosure, and in a different location

[Ardje]

I don't know whay, but people are talking about backups? raid==raid. It's a redundant storage of data.
Backup is a completely different concept and has absolutely nothing to do with filesystems.
Back to raid: btrfs raid is better raid than MD raid, especially with raid 1, because on btrfs you know which of the 2 copies of the file is the correct copy due to checksums on the data.
On md-raid, you can only check which of the disks has the highest serial number. There is no checksum. If one of the 2 disks fails on another position, you have a high chance of being stuck with 2 disks you cannot retrieve the data from because of 2 different unreadable parts (MD-raid fortunately is a lot smarter than that these days).
Btrfs would be the best raid if it supported raid 6 with good parameters on a lot of disks. It would have data checksums on each stripe, so it knows which of the stripes are bad. It can than recalculate the file using the ones that are good.

Things in btrfs are different to md-raid, because btrfs is a *filesystem* that can deliver redundancy on raw disks, and md-raid is just a block layer that delivers block layer redundancy on which you still have to put a filesystem.

That being said, I still don't trust my valuable data to btrfs. md-raid+ext4 is proven and fast, and md-raid can easily be exported using fcoe.
Having data checksums on files is very important though, but btrfs has screwed me too often to trust it.
But I will probably setup a new testbed (btrds on local disk and fcoe disk in raid 1 mode. If local disk dies, it should be able to continue on network in degraded mode).