Page 1 of 3 123 LastLast
Results 1 to 10 of 22

Thread: Linux Kernels Can Now Be Compressed With LZ4

  1. #1
    Join Date
    Jan 2007
    Posts
    14,324

    Default Linux Kernels Can Now Be Compressed With LZ4

    Phoronix: Linux Kernels Can Now Be Compressed With LZ4

    The Linux 3.11 kernel will support kernel images compressed using the LZ4 compression algorithm...

    http://www.phoronix.com/vr.php?view=MTQwODY

  2. #2
    Join Date
    Mar 2008
    Posts
    205

    Default

    lz4 is pretty amazing in its speed. It should be quite useful for zram use as well if it hasn't been deployed for that already. I use it for cache compression and redundant data removal for some custom file formats.

  3. #3
    Join Date
    Dec 2011
    Posts
    2,003

    Default

    Maybe they can add it to some generic framework so its not only used for loading the kernel, but also accessible by file systems for compression such as on btrfs with transparent compression.

  4. #4
    Join Date
    Oct 2010
    Posts
    91

    Default

    Quote Originally Posted by uid313 View Post
    Maybe they can add it to some generic framework so its not only used for loading the kernel, but also accessible by file systems for compression such as on btrfs with transparent compression.
    i've been waiting for the lz4 stuff to be implemented in btrfs for ages. lz4 is definitely good, i was playing around with it a bit before, and you can tune it a bit more for more compression ratio or greater compression speed.

    the lz4hc stuff does pretty good compression, and decompression speed remains good, so good for kernels where you compress infrequently and decompress frequently.

  5. #5
    Join Date
    Dec 2011
    Posts
    2,003

    Default

    Quote Originally Posted by mercutio View Post
    i've been waiting for the lz4 stuff to be implemented in btrfs for ages. lz4 is definitely good, i was playing around with it a bit before, and you can tune it a bit more for more compression ratio or greater compression speed.

    the lz4hc stuff does pretty good compression, and decompression speed remains good, so good for kernels where you compress infrequently and decompress frequently.
    Linux has the Cryptography API which is available to all subsystems.
    Which makes the subsystem does not need to re-implement and re-invent it, they can just use the Crypto API.

    Does Linux have anything similar for compression?
    Would be great if all file systems that supported compression could use the same shared compression algorithms, and they were also available for initrd and other stuff.

  6. #6
    Join Date
    Feb 2008
    Location
    Linuxland
    Posts
    4,987

    Default

    Every time I hear about $NEW_FASTER_DECOMPRESSION, I think to myself why has nobody bothered to optimize zlib decompression speed. It compresses far better than the LZ*, and a mere 2x decompression speed improvement would put it on par with LZO.

    Then I recall the one time I searched for that, and found Intel has a closed-source optimized zlib library, well faster than the default one. http://www.intel.com/content/dam/www...sion-paper.pdf

    Intel's implementation claims 1.75x the decompression speed of the open-source zlib one.

  7. #7
    Join Date
    Mar 2008
    Posts
    205

    Default

    Quote Originally Posted by curaga View Post
    Intel's implementation claims 1.75x the decompression speed of the open-source zlib one.
    If you notice performance numbers printed are for ARM.

  8. #8
    Join Date
    Oct 2010
    Posts
    91

    Default

    Quote Originally Posted by curaga View Post
    Every time I hear about $NEW_FASTER_DECOMPRESSION, I think to myself why has nobody bothered to optimize zlib decompression speed. It compresses far better than the LZ*, and a mere 2x decompression speed improvement would put it on par with LZO.
    but lz4 is six times faster than zlib. and lz4hc is seven times faster than zlib with similar compression ratio to zlib -1.

    Then I recall the one time I searched for that, and found Intel has a closed-source optimized zlib library, well faster than the default one. http://www.intel.com/content/dam/www...sion-paper.pdf

    Intel's implementation claims 1.75x the decompression speed of the open-source zlib one.
    lz4 somehow is way faster than the others. the only advantage zlib has over lz4 is in compression ratios, and when people really care about that they tend to shift to lzma compression which gives even better ratios.

  9. #9
    Join Date
    Sep 2009
    Posts
    59

    Default Code duplication.

    Quote Originally Posted by uid313 View Post
    Linux has the Cryptography API which is available to all subsystems.
    Which makes the subsystem does not need to re-implement and re-invent it, they can just use the Crypto API.

    Does Linux have anything similar for compression?
    Would be great if all file systems that supported compression could use the same shared compression algorithms, and they were also available for initrd and other stuff.
    I was thinking exactly this. Cryptography may even be a good place to put it, but something similar should be made to reduce duplication when BTRFS, ZRam, ZSwap, init, network?, etc can all use these.

  10. #10
    Join Date
    Mar 2008
    Posts
    205

    Default

    Quote Originally Posted by snadrus View Post
    I was thinking exactly this. Cryptography may even be a good place to put it, but something similar should be made to reduce duplication when BTRFS, ZRam, ZSwap, init, network?, etc can all use these.
    lz4 or any compressor could be considered as well known encryption algorithms.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •