Page 2 of 2 FirstFirst 12
Results 11 to 15 of 15

Thread: EXT4 In Linux 3.5 Gets CRC32 Meta-Data

  1. #11
    Join Date
    Feb 2010
    Location
    Sweden
    Posts
    58

    Default

    Quote Originally Posted by smitty3268 View Post
    There's a reason every FS and DB I've ever heard of that added checksums uses simple CRC hashing.

    It's a simple algorithm that is well suited to the task.

    You could replace it with SHA-1, but what would that give you? A much more complicated and CPU-taxing algorithm, and nothing else. Crypto hashes have to be a lot more complicated so that small changes to the data provide pseudo-random changes to the hash that can't be reverse-engineered. There's isn't a single good reason to want that overhead in an integrity check.

    If you want to verify that the data hasn't been tampered with, you should just encrypt the whole FS.
    SHA-1 and the various hash based algorithms will give you a much higher error detection rate than a simple crc-32, a crc-32 can only detect certain types of errors and certain degrees of errors within that (i.e it can for example catch burst errors but not if they are over X bits of length). They do introduce a major overhead though, however since this is for the meta-data only anyways, the performance overhead shouldn't be that much.

  2. #12
    Join Date
    Dec 2011
    Posts
    2,194

    Default

    Quote Originally Posted by smitty3268 View Post
    You could replace it with SHA-1, but what would that give you? A much more complicated and CPU-taxing algorithm, and nothing else. Crypto hashes have to be a lot more complicated so that small changes to the data provide pseudo-random changes to the hash that can't be reverse-engineered. There's isn't a single good reason to want that overhead in an integrity check.
    These days you have hardware-accelerated cryptography, I know modern Intel processosrs support AES-NI (AES instruction set). Don't know if it do hardware-accelerated hashing though.

  3. #13
    Join Date
    Nov 2011
    Posts
    24

    Default

    Modern x86 (and IIRC SPARC) processors have hw accelerated CRC32C. "Luckily", the exact same polynomial is used in the btrfs and ext4 crc checksum, among others, allowing those to benefit from the hw acceleration.

  4. #14
    Join Date
    Feb 2008
    Location
    Linuxland
    Posts
    5,333

    Default

    Via's cpus have supported hardware SHA-1/SHA-256 for years. Many ARM cpus also support it in hw.

  5. #15
    Join Date
    Dec 2011
    Posts
    38

    Default

    Does it checksum metadata only? Is there checksum support for file's data?

    Also, CRC32 seems a little bit weak.
    As far as i know, CRC32 is used by many hardware internally for error correction.
    When data corrupted and undetected in hardware level (which means the hardware CRC32 test probably passed without error), i doubt how reliable that ext4's CRC32 can detect corruption.

    At least an alternative implementation of MD5 should be provided, the overhead should be acceptable if those crypto library are written in SIMD instruction (MMX, SSE)
    Last edited by unknown2; 07-21-2012 at 09:35 PM.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •