[HN Gopher] Anticipating the Future with LMDB (2016)
       ___________________________________________________________________
        
       Anticipating the Future with LMDB (2016)
        
       Author : crecker
       Score  : 7 points
       Date   : 2022-11-13 15:33 UTC (1 days ago)
        
 (HTM) web link (www.symas.com)
 (TXT) w3m dump (www.symas.com)
        
       | vlovich123 wrote:
       | My hunch is the analysis was fatally flawed.
       | 
       | > the cost of storage has been decreasing at 38%/year
       | 
       | This assumes that the growth rate in bytes used is staying
       | constant. I don't think that's accurate in the aggregate which is
       | why there's compression. Additionally, the space overhead isn't
       | just about space usage. Flash suffers horrible wear which means
       | that your space amplification is also write amplification. The
       | combination in a growth rate in the amount of data and a plateau
       | in write cycles means that without compression your cost of
       | storage is probably costing you much more than it should.
       | 
       | Additionally, it assumes the growth rate for storage won't hit
       | it's own plateau. Indeed, solid state prices seem to have leveled
       | out already only 6 years after this article because flash is
       | really hard to make cheaper (& even spinning disk has plateaued).
       | 
       | I think the analysis is right that you can't just assume you can
       | use as much CPU as you want, but I think it's wrong to eschew
       | compression, especially since the DB gets to choose where to
       | spend compression time on (& decompression is usually free
       | because you cache decompressed results so on average you're not
       | doing much).
        
         | cryptonector wrote:
         | My take is that a B-tree-like database should really have: a)
         | optional compression, b) variable-size blocks, c) a write-ahead
         | log (possibly indexed) to amortize the cost of B-tree
         | transactions. I.e., it should be like ZFS.
        
       ___________________________________________________________________
       (page generated 2022-11-14 23:02 UTC)