[HN Gopher] Tigerbeetle's Storage Fault Model
       ___________________________________________________________________
        
       Tigerbeetle's Storage Fault Model
        
       Author : LAC-Tech
       Score  : 60 points
       Date   : 2023-11-18 06:03 UTC (2 days ago)
        
 (HTM) web link (github.com)
 (TXT) w3m dump (github.com)
        
       | allknowingfrog wrote:
       | The "Storage Fault Model" section is about halfway down the page.
       | 
       | https://github.com/tigerbeetle/tigerbeetle/blob/main/docs/DE...
        
       | baq wrote:
       | > TigerBeetle detects and repairs disk corruption (3.45% per 32
       | months, per disk), detects and repairs misdirected writes where
       | the disk firmware writes to the wrong sector (0.042% per 17
       | months, per disk)
       | 
       | I'm kind of speechless actually, both at the fact that it can do
       | that and the fact that disk firmwares are actually so bad
        
         | morelisp wrote:
         | These numbers didn't ring true at all to me, even for spinning
         | disks. And indeed, I don't think the documentation presents
         | them with correct context - I can't figure out what's "per
         | disk".
         | 
         | > A total of 3.45% of 1.53 million disks developed latent
         | sector errors over a period of 32 months.
         | 
         | IOW, over 2-3 years 3.5% of disks will develop at least one
         | error. Also,
         | 
         | > For most disk models, more than 80% of disks with latent
         | sector errors have fewer than 50 errors.
        
         | throw0101b wrote:
         | > _I 'm kind of speechless actually, both at the fact that it
         | can do that and the fact that disk firmwares are actually so
         | bad_
         | 
         | The Sun/Solaris ZFS folks used to talk about this a lot in the
         | early days as a way of evangelizing the idea of checksums
         | covering everything in the file system. Bryan Cantril has given
         | a number of talks/rants on firmware, e.g., "Zebras All the Way
         | Down":
         | 
         | * https://www.youtube.com/watch?v=fE2KDzZaxvE
        
       | throw0101b wrote:
       | > _Disk performance and read and write latencies can sometimes be
       | volatile, causing latency spikes on the order of seconds._
       | 
       | See Brendan Gregg (and Bryan Cantrill) shouting at hard drives
       | for Sun Microsystem's Fish Works project (ZFS for data and DTrace
       | for instrumentation):
       | 
       | * https://www.youtube.com/watch?v=tDacjrSCeq4
       | 
       | Retrospectives on the video:
       | 
       | * https://www.youtube.com/watch?v=lMPozJFC8g0
       | 
       | * https://www.youtube.com/watch?v=_IYzD_NR0W4
        
       ___________________________________________________________________
       (page generated 2023-11-20 23:02 UTC)