Modernized Zstd Merged Into Linux 5.16 For Much Greater Performance

While Zstd is used in various areas of the Linux kernel for data compression from transparent file-system compression with the likes of Btrfs to allowing kernel modules be compressed with this Zstandard algorithm, the in-kernel code had fallen years outdated. Finally with Linux 5.16 that Zstd in-kernel implementation is now being brought up to modern standards and delivering better performance too.

Merged on Saturday evening for Linux 5.16 is the overhauled Zstd code for the Linux kernel. This revised code has a new kernel-style wrapper API around Zstd that should also facilitate easier updates moving forward and to be automatically generated/derived from the upstream Zstd source code.

The existing kernel code is around four years old and in that time Zstd upstream has seen many bug fixes and performance optimizations. With this new code for Linux 5.16, Btrfs Zstd decompression can be ~15% faster, SquasFS Zstd decompression is around ~15% faster too, F2FS Zstd decompression can be upwards of 20% faster, zRAM decompression around 30% faster, kernel Zstd image decompression around 35% faster, and other wins.

There are even more performance optimizations for Zstd on the near-term horizon but first this milestone needed to be reached of getting the overhauled code merged to mainline. Zstd 1.5.1 should be coming soon to better unify things and deliver the latest improvements.

More details on this huge update for anyone relying on Zstd compression/decompression in the kernel can see this Git merge for all of the details. The pull request was submitted a few days ago but Linus Torvalds noted he intentionally wanted to hold off on merging it to allow time to review the code himself given the impact of it.

If you happen to be interested in Zstd CPU performance benchmarks, see this OpenBenchmarking.org page for the composite rankings of many different processors.