Tux3 Still Dreaming Of Design Improvements

[phoronix]

Phoronix: Tux3 Still Dreaming Of Design Improvements

The Tux3 Linux BTree-based file-system that isn't yet mainline in the Linux kernel is continuing to focus on new features and capabilities...

http://www.phoronix.com/vr.php?view=MTQyMjU

[uid313]

Read the link in the article

I can highly recommend reading the link in the article!

http://phunq.net/pipermail/tux3/2013-June/002042.html

It is very interesting.
Daniel Phillips was the guy who invented HTree, and who then went on and invented PHTree, and now he come up with Shardmap.

This is is a seriously smart guy developing Tux3!

Very interesting article.

[kaprikawn]

^
Agreed. When I read the article, my first thought was 'Do we really need another filesystem?'. But having read the mailing list post, I had to eat my words because the answer seemed to be yes.

I don't pretend to understand it all, but the message I got from it was that this is a significant step forward that can also be applied to other filesystems, such as EXT4 which is what I tend to use.

So while, at first glance, it seems to be duplicated effort, I can see that effort on one project can sometimes be applied elsewhere. Such is the beauty of OSS. It makes me think of a West Wing episode where they point out that many scientific discoveries are made without the people looking at that particular issue.

[kertoxol]

I don't pretend to understand it all, but the message I got from it was that this is a significant step forward that can also be applied to other filesystems, such as EXT4 which is what I tend to use.

This. Such improvements maybe could not fit for EXT4, but would be applied in EXT5, even with a mk.fs option (--enable-btree) just in cause developers won't accept these feature as main part of EXT5, but like a dialect/extended part of it.

[Maxjen]

I didn't understand most of it, but it sounds promising and it seems that there are plans to merge it into the kernel.

We expect a kernel implementation of Shardmap to land some time in the
next few months. In the mean time, we are now satisfied with this key
aspect of the Tux3 design and will turn our attention back to the
handful of remaining issues that need to be addressed before offering
Tux3 for mainline kernel merge.

I am interested in seeing how it compares to btrfs. It seems more inspired at the very least.

[rdnetto]

I'm interested in it as well - it has similar capabilities to btrfs in that it can hold multiple versions of files, but seems to have a greater focus on performance.
The wikipedia article discusses its differences to btrfs in a much more accessible way.

[movieman]

I'm not sure I quite get the data compression thing. Most of the big files on my system are video and other formats that are already compressed. Then, at the low level, the SSD performs compression to reduce the amount of data it has to write to flash. I'm sure there are specialised uses where it makes sense, but putting another layer of compression in the middle seems like a waste of CPU cycles for most people.

[DrYak]

Compression

Then, at the low level, the SSD performs compression to reduce the amount of data it has to write to flash.

Some SSD do compress, but most HDD don't. If your managing a big file server with several TB or more worth of space, you probably use slower HDD.
Same goes for a lot of other scenarios (onboard flash, USB sticks, etc.).
In fact, pretty much anything else except backup tapes and some SSDs lacks compression.
Compression is a big speed gain (saves bandwidth) in addition to saving some space.

Also compression isn't that much a significant waste of resources for the mighty multi-core multi-GHz modern CPUs.
Some compression scheme like LZ4 are designed to have an insignifiant CPU impact, while dealing good bandwith gains.
They probably even faster and, most importantly, much more versatile than the punny chip in a cheap SSD's controller.

With FS-level compression, you can tune the compression. For exemple you can use the much more efficient XZ (LZMA2) compression for seldom written/often read files.
(Compression is CPU intensive, but big gains in space and thus in bandwidth on slow media), and use ZLib and LZ4 for other files with different access pattern.

That would be simply not possible with the on-board compression featured with SSD and tapes.

Most of the big files on my system are video and other formats that are already compressed.

Your typical desktop usage is not the typical server usage.

Text format are very popular in a lot of other fields:
- HTML, PHP, etc. : If such often-read files are compressed, there is speed gain.
- life-science data format: most of them are huge plain-text files (DNA and protein sequences), XML files (with a few basae64 encoded data within) of flat-text data bases.
- data base...
- etc.

All these are situation where compression greatly improves the throughput.