EXT4 In Linux 3.5 Gets CRC32 Meta-Data

[F.Ultra]

There's a reason every FS and DB I've ever heard of that added checksums uses simple CRC hashing.

It's a simple algorithm that is well suited to the task.

You could replace it with SHA-1, but what would that give you? A much more complicated and CPU-taxing algorithm, and nothing else. Crypto hashes have to be a lot more complicated so that small changes to the data provide pseudo-random changes to the hash that can't be reverse-engineered. There's isn't a single good reason to want that overhead in an integrity check.

If you want to verify that the data hasn't been tampered with, you should just encrypt the whole FS.

SHA-1 and the various hash based algorithms will give you a much higher error detection rate than a simple crc-32, a crc-32 can only detect certain types of errors and certain degrees of errors within that (i.e it can for example catch burst errors but not if they are over X bits of length). They do introduce a major overhead though, however since this is for the meta-data only anyways, the performance overhead shouldn't be that much.

[uid313]

You could replace it with SHA-1, but what would that give you? A much more complicated and CPU-taxing algorithm, and nothing else. Crypto hashes have to be a lot more complicated so that small changes to the data provide pseudo-random changes to the hash that can't be reverse-engineered. There's isn't a single good reason to want that overhead in an integrity check.

These days you have hardware-accelerated cryptography, I know modern Intel processosrs support AES-NI (AES instruction set). Don't know if it do hardware-accelerated hashing though.

[jabl]

Modern x86 (and IIRC SPARC) processors have hw accelerated CRC32C. "Luckily", the exact same polynomial is used in the btrfs and ext4 crc checksum, among others, allowing those to benefit from the hw acceleration.

[curaga]

Via's cpus have supported hardware SHA-1/SHA-256 for years. Many ARM cpus also support it in hw.

[unknown2]

Does it checksum metadata only? Is there checksum support for file's data?

Also, CRC32 seems a little bit weak.
As far as i know, CRC32 is used by many hardware internally for error correction.
When data corrupted and undetected in hardware level (which means the hardware CRC32 test probably passed without error), i doubt how reliable that ext4's CRC32 can detect corruption.

At least an alternative implementation of MD5 should be provided, the overhead should be acceptable if those crypto library are written in SIMD instruction (MMX, SSE)