[HN Gopher] Lizard - efficient compression with fast decompression
       ___________________________________________________________________
        
       Lizard - efficient compression with fast decompression
        
       Author : pabs3
       Score  : 49 points
       Date   : 2022-05-25 04:33 UTC (18 hours ago)
        
 (HTM) web link (github.com)
 (TXT) w3m dump (github.com)
        
       | lifthrasiir wrote:
       | Note that a benchmark in the README refers to zstd 1.1.1 and
       | brotli 0.5.2, which are very old (the current versions are zstd
       | 1.5.2 and brotli 1.0.9). The same author maintains lzbench [1]
       | which is more current than the README. It shows that Lizard
       | heavily trades off compression speed and ratio for decompression
       | speed; zstd at the default setting (-3) is expected to be on par
       | with Lizard -45 in terms of ratio, but Lizard would be >10x
       | slower to compress while only ~3x faster to decompress.
       | 
       | [1] https://github.com/inikep/lzbench
        
       | deepsun wrote:
       | My take (I compare about ZSTD only, as that what we need to care
       | about): 1. Lizard's decompression speeds are much higher than
       | ZSTD for any level of compression. 2. Lizards's compression speed
       | is significantly (several times) lower than ZSTD for similar
       | compression ratios.
       | 
       | So it makes sense if we mostly don't care about CPU usage for
       | compression.
        
         | dataangel wrote:
         | if it's faster for both compression and decompression it just
         | seems better period? why do you say "if we mostly don't care
         | about CPU usage"? being faster means less CPU usage...
        
           | deepsun wrote:
           | I said it's significantly _slower_, not faster, for
           | compression.
        
             | metadat wrote:
             | Minor nit: I too was confused by the wording:
             | 
             | "compression speed is lower"
             | 
             | To me this reads as though it's _faster_. Maybe you meant
             | "compression speed is Slower"?
             | 
             | Disclaimer: I generally dislike being pedantic, especially
             | on HN. I hope this message doesn't come across that way.
        
               | darrenf wrote:
               | If compression _speed_ is lower, then it 's slower.
               | 
               | If compression _time_ is lower, then it 's faster.
        
           | kordlessagain wrote:
           | Faster/slower aside, it doesn't matter if it is slower to
           | compress, because you only do that once for most use cases
           | (thinking of downloadable repos or document stores).
           | Decompression happens much more frequently. So, one
           | compression algo that takes a lot of work (and time) to
           | compress but uncompresses with less work than anything else
           | is desirable because of the fact you only need compress once.
           | So, less compute is required.
           | 
           | There could be some use cases where this doesn't work out,
           | such as a compressed network connection, where everything has
           | to be compressed/decompressed in a 1:1 ratio.
        
         | CoolGuySteve wrote:
         | The thing I don't get about zstd is that their own github page
         | shows lz4 is faster at both compression and decompression at
         | the cost of some compression ratio:
         | https://github.com/facebook/zstd
         | 
         | But most people I work with will pick zstd every time even in
         | cases where decompression speed matters the most.
        
           | loxias wrote:
           | > But most people I work with will pick zstd every time even
           | in cases where decompression speed matters the most.
           | 
           | Zstd is _really good_. If you 're a math/compression theory
           | nerd it makes a lot of good choices. Also, if you're willing
           | to tune zstd instead of using the defaults, so far anyway, I
           | haven't found any application where the right answer _wasn
           | 't_ zstd.
           | 
           | At this point I don't think it's unreasonable to expect new
           | compression developments to to explicitly say "What is
           | improved over ZStandard".
        
             | ciupicri wrote:
             | Just to add to what you're saying, you can't tune zstd for
             | qcow images or for btrfs filesystems.
        
               | loxias wrote:
               | You're entirely right! And points out the blind spot that
               | I've used it much more as a library in tools I build/use
               | than integrated with other things. Never used brtfs, I'm
               | assuming it uses zstd internally for something? Cool.
               | 
               | Also very cool that qcow contains zstd support (didn't
               | know that either!).
        
           | adgjlsfhk1 wrote:
           | If you are bottle-necked by reading data (eg over networks or
           | slow storage), zstd will be faster to decompress since you
           | are transferring less data.
        
       ___________________________________________________________________
       (page generated 2022-05-25 23:02 UTC)