[HN Gopher] Apple's big test of data integrity
       ___________________________________________________________________
        
       Apple's big test of data integrity
        
       Author : headalgorithm
       Score  : 30 points
       Date   : 2023-05-30 18:33 UTC (4 hours ago)
        
 (HTM) web link (eclecticlight.co)
 (TXT) w3m dump (eclecticlight.co)
        
       | sillywalk wrote:
       | I'm also curious if Apple SoCs' storage controllers have some
       | fancy error-checking, along with AES.
        
         | ryao wrote:
         | As far as I know, Apple has the NAND controller integrated into
         | the SoC. The NAND controller needs to have ECC to keep the
         | error rates on flash down to something acceptable.
        
           | sillywalk wrote:
           | That was sort of what I was wondering, if the NAND controller
           | has some extra Apple Stuff(tm) beyond standard ECC.
        
             | Eric_WVGG wrote:
             | I'm pretty sure that is the reason why support for Fusion
             | Drives was abruptly ganked and punted out by a year.
        
               | daniel-cussen wrote:
               | [dead]
        
       | jeffbee wrote:
       | How do they checksum 9GB of data and still boot promptly?
        
         | droffel wrote:
         | Merkle trees! You (simplified) hash the data at the leaves of
         | the tree, merge adjacent leaves (concatenation for example),
         | and hash the new leaves, until you're left with just the Merkle
         | Root, a single hash representing the entirety of your data.
         | Verifying the root is easier than loading the data itself into
         | memory and hashing it there since you can verify it piecemeal
         | and without loading it all into memory at once.
        
         | [deleted]
        
         | w-m wrote:
         | Don't these machines have read speeds of like 5000 MB/s? So
         | just read the 9 GB blockwise and apply a super simple hash.
         | Should be done in two seconds.
        
         | sillywalk wrote:
         | FTA: "This is the ingenious part: verification carries little
         | overhead, as it runs as a rolling process once the top-level
         | 'seal' has been verified. So verification is a continuous
         | process as the SSV is mounted and accessed."
        
       | curt15 wrote:
       | Are there any distros that offer similar integrity checks? I
       | understand Chromebooks provide similar assurances using dm-verity
       | but what about more general-purpose distros?
        
       | refulgentis wrote:
       | > So while none of us can rule out data corruption due to cosmic
       | rays and similar causes, the chance of that happening appears
       | extremely remote
       | 
       | I don't buy that.
       | 
       | - a process opaque to us.
       | 
       | - seems to be not failing at scale enough to make it to press.
       | 
       | - therefore the chance of [random data corruption] is remote
       | 
       | versus ex. [in 2016](https://physicsworld.com/a/cosmic-challenge-
       | protecting-super...), couple UK PhDs laboriously chart 55,000 bit
       | flip errors on a 100 node cluster.
        
       | tedunangst wrote:
       | This contrasts with the ZFS team who said they started seeing
       | disk errors as soon as they switched to ZFS.
        
       | uf00lme wrote:
       | Common in Mobile and some of the IoT world. Blackberry was doing
       | this ages ago, it was why you never wanted to have to power cycle
       | them. Took ages to boot
        
       | anyfoo wrote:
       | > During the boot process, unless boot security has been
       | downgraded from Full Security, the contents of that SSV are
       | verified against its tree of hashes. In the event that they don't
       | match perfectly: "the startup process halts and the user is
       | prompted to reinstall macOS".
       | 
       | This is written a bit confusingly, if you don't already knows how
       | it works (even though a later reply to a comment tries to clarify
       | it). Because it sounds like the disk is fully and instantly
       | validated at every boot, and "the startup process halts" if there
       | is a mismatch.
       | 
       | Instead, depending a little on the actual implementation, most
       | hashes will very likely only be validated when the corresponding
       | block is actually accessed (up the entire path to the root, where
       | the higher level hashes may be validated now or may have been
       | validated already; this again is an implementation detail).
       | 
       | The result is also, that some blocks may never be validated
       | during normal operation, simply if some files are never accessed.
       | Unless there is some explicit whole disk validation sometime,
       | e.g. during installation, though any corruption happening after
       | that last "full disk check" will still lay dormant until access
       | or the next full check.
       | 
       | But since this is a tree of hashes, it still provides all
       | security and integrity benefits to anything that does get
       | accessed. What never gets accessed does not matter per
       | definition.
        
       ___________________________________________________________________
       (page generated 2023-05-30 23:01 UTC)