ZFS On Linux 0.6.5 Adds 4.2 Kernel Support, New Zpool/Zdb Functionality

[phoronix]

Phoronix: ZFS On Linux 0.6.5 Adds 4.2 Kernel Support, New Zpool/Zdb Functionality

ZFS On Linux 0.6.5 was released this week with support for the new Linux 4.2 kernel and a variety of new features...

http://www.phoronix.com/scan.php?page=news_item&px=ZFS-On-Linux-0.6.5

[jhenke]

It would be very interesting to see some ZFS on Linux vs ZFS on FreeBSD benchmarks. Especially since ZFS is a standard option for FreeBSD.

[uid313]

Is Oracle doing anything for ZFS on Linux with Oracle Linux?

[Raven3x7]

Is Oracle doing anything for ZFS on Linux with Oracle Linux?

No. ZFSOnLinux isn't even based on the Oracle version but on OpenZFS. If they wanted to help ZFS gain widespread adoption on Linux they could just relicense it under a GPL compatible license.

[ryao]

IANAL, but actual lawyers have told me that the existing license poses no legal problem for ZoL adoption. A GPL compatible license is unnecessary for a port of a driver from another platform. Both the userbase and distribution support are growing at a brisk pace. ZFSOnLinux has a very bright future under the CDDL.

That said, a substantial portion of the code (probably 20% to 40%) is owned by other parties. Many like the CDDL and I imagine at least a few of them would decline a request to change the licensing. It would be a mistake to think that Oracle deciding to offer the original OpenSolaris code under a GPL compatible license would result in ZoL changing its license.

[Turion]

I wouldn't even dare touch it until it becomes part of the mainline kernel and considered officially supported. Talking about home use of course, storing my important data on it.
I would prefer btrfs over this for... say... my torrent partition. Currently at least. Should it ever make it into mainline and gain widespread adoption, it would be really cool!

[jacob]

It would be very interesting to see some ZFS on Linux vs ZFS on FreeBSD benchmarks. Especially since ZFS is a standard option for FreeBSD.

I second that. Especially on HDDs, not SSDs.

[Dorsai!]

I would prefer btrfs over this for... say... my torrent partition. Currently at least. Should it ever make it into mainline and gain widespread adoption, it would be really cool!

Torrents (like databases and VM images) are literally the worst case scenario for btrfs. Its CoW-everything strategy in combination with the way a torrent client randomly writes the data will lead to the data becoming extremely fragmented really fast. And this is not just a theory. I had a btrfs on a single HDD once and had to set the NO_COW attribute (which also removes btrfs's equivalent of a journal and thus consistency checks) for my torrent directory because I couldn't even watch a downloaded movie because of the I/O latency from fragmentation.

This is no problem on an SSD of course where fragmentation is not an issue.

ZFS follows a different strategy and CoWs only if there is an existing snapshot of the data (like LVM2). This has its own disadvantages compared to btrfs but it works acceptably well for the above workloads.

[ryao]

I wouldn't even dare touch it until it becomes part of the mainline kernel and considered officially supported. Talking about home use of course, storing my important data on it.
I would prefer btrfs over this for... say... my torrent partition. Currently at least. Should it ever make it into mainline and gain widespread adoption, it would be really cool!

Being in the mainline kernel does not imply widespread adoption. For example, JFS is in the mainline kernel, but have you heard of anyone using it?

That said, I think your perception of safety is somewhat backward. ZFS' data integrity capabilities far exceed that of any in-tree filesystem:

http://open-zfs.org/wiki/System_Administration#Data_Integrity

ZFS also has a better track record for data integrity than any mainline filesystem. The only regression in a tagged release that caused anything that could be called data loss involved the non-default ZoL-specific xattr=sa setting and was fixed last year. Few people were affected and the project has been fortunate enough to never have a regression in a tagged release cause the loss of file data. No mainline filesystem can make this claim.

That is not to say that it will never happen, but so far, the ZoL project has had a unique combination of obscene good luck and caring about data integrity that has been enough to prevent it. The project dodged a data integrity regression by delaying the 0.6.5 release until it was fixed because the Illumos community caught it, raised the alarm and we realized what it was minutes before the 0.6.5 release. The regression was so incredibly rare that it exceeded our present ability to detect in time, but the Illumos community catching it in production enabled us to delay release until it was fixed. This is a benefit that in-tree filesystems do not have.

That said, the project has put measures into place to detect entire classes of data integrity regressions that affected either HEAD or the pre-releases. The more recent regression is in a class that might be just beyond our ability to detect in advance, but that should not stop us from trying. One idea on my mind is to modify the project's buildbot to improve coverage of L2ARC. That will be discussed before the next release and if history is any indication, it should be acted upon before the next release.

In case it is not apparent from how I write about ZoL, I am one of the ZFSOnLinux project developers. I became involved with the project in 2011/2012 (evaluated it in December 2011 and began writing patches in early 2012), because I wanted a cheap and performant way of withstanding drive failures at my home, ZFSOnLInux did it when MD RAID did not and it just needed a few patches to fix some issues I encountered. Those issues are long fixed, but I enjoyed working on it so much that I kept working on it to make it even better.

[ryao]

Torrents (like databases and VM images) are literally the worst case scenario for btrfs. Its CoW-everything strategy in combination with the way a torrent client randomly writes the data will lead to the data becoming extremely fragmented really fast. And this is not just a theory. I had a btrfs on a single HDD once and had to set the NO_COW attribute (which also removes btrfs's equivalent of a journal and thus consistency checks) for my torrent directory because I couldn't even watch a downloaded movie because of the I/O latency from fragmentation.

This is no problem on an SSD of course where fragmentation is not an issue.

ZFS follows a different strategy and CoWs only if there is an existing snapshot of the data (like LVM2). This has its own disadvantages compared to btrfs but it works acceptably well for the above workloads.

This is incorrect. Everything in ZFS uses CoW. The reason that it has worked better for you is that zfetch (ZFS prefetch), ZIL (ZFS Intent Logging) and ARC (the Adaptive Replacement Cache) tend to mitigate the issues that you experienced on btrfs, although zfetch and ZIL need work to be better. zfetch tends to prefetch too aggressively on purely random workloads and generates many useless read IOs, which hurts IOPS in situations where the underlying hardware is IOPS/throughput bound. The Illumos community wrote a patch that is intended to fix that which should be in the next release:

https://github.com/zfsonlinux/zfs/pull/3765

ZIL's weakness is that it does not convert reads triggered by partial record modification into asynchronous ones when data needs to be read from disk on IO that is not O_SYNC. ARC's high hit rate tends to mitigate the issue, which is why it has gone unfixed for *years*. However, the betrfs' developers microbenchmark on 4-byte microwrites made it painfully obvious and it will likely be fixed in the future.

Lastly, the Open ZFS wiki has configuration tweaks for the workloads that you mentioned. I have been told that they helped production workloads that were previously struggling on ZFS with the default settings:

http://open-zfs.org/wiki/Performance_tuning#Bit_Torrent

http://open-zfs.org/wiki/Performance_tuning#Database_workloads

http://open-zfs.org/wiki/Performance_tuning#Virtual_machines