FreeBSD Lands Important ZFS Performance Fix For Some Going From ~60MB/s To ~600MB/s

Thanks to the BSD wizards at iXsystems, FreeBSD has received an important performance fix/optimization around their ZFS file-system code.

In optimizing their ZFS zlib zalloc/zfree callbacks, for some scenarios the write speed to ZVOLs while using GZIP compression has gone from around 60MB/s to around 600MB/s.

IXsystems' Alex Motin explained in the change:

The previous code came from OpenSolaris, which in my understanding require allocation size to be known to free memory. To store that size previous code allocated additional 8 byte header. But I have noticed that zlib with present settings allocates 64KB context buffers for each call, that could be efficiently cached by UMA, but addition of those 8 bytes makes them fall back to physical RAM allocations, that cause huge overhead and lock congestion on small blocks. Since FreeBSD's free() does not have the size argument, switching to it solves the problem, increasing write speed to ZVOLs with 4KB block size and GZIP compression on my 40-threads test system from ~60MB/s to ~600MB/s.