https://eclecticlight.co/2023/05/30/apples-big-test-of-data-integrity/ Skip to content [eclecticlight] The Eclectic Light Company Macs, painting, and more Main navigation Menu * Downloads * M1 & M2 Macs * Mac Problems * Mac articles * Art * Macs * Painting hoakley May 30, 2023 Macs, Technology Apple's big test of data integrity Well over two years ago, when Apple released Big Sur, it started the largest test of data integrity ever undertaken on Macs, and quite possibly the largest on any personal computer, in the Signed System Volume (SSV). Over the period since 12 November 2020, every T2 and Apple silicon Mac that has booted Big Sur, Monterey or Ventura in default Full Security mode has verified every last bit of their 9 GB SSV. Apple describes this process in its Platform Security Guide. During macOS installation or update, a tree of SHA-256 hashes is built for the snapshot made of the System volume. During the boot process, unless boot security has been downgraded from Full Security, the contents of that SSV are verified against its tree of hashes. In the event that they don't match perfectly: "the startup process halts and the user is prompted to reinstall macOS". Because the SSV is a snapshot, APFS makes both its file system and all its data read-only to all other processes. However, errors in APFS could result in unintended changes to the snapshot, and it remains fully exposed to changes taking place below file system level, including 'bit rot'. Those have commonly been ascribed to failures in the storage medium, and factors such as cosmic rays that have been claimed to be responsible for data corruption. Thus every T2 and Apple silicon Mac booting Big Sur or later in Full Security mode has contributed a measure of data integrity. No one outside Apple knows exactly how many Macs have taken part in this large-scale test. Since the release of Big Sur, Apple has sold around 20 million Macs each year, all of which are running Big Sur or later. Many existing Intel Macs with T2 chips have also been upgraded to Big Sur or later. At the very least, macOS 11 or later must have been installed and run on tens of millions of Macs, possibly as many as 100 million in total. Even if most of those Macs aren't booted daily, but left to sleep every night, each of them will have booted once for each macOS update or upgrade since installing Big Sur or later. In total, those Macs must have completed hundreds or even thousands of millions of verifications of their SSV. Although I believe that I have heard of one user whose Mac didn't complete a macOS update successfully, and was prompted to reinstall macOS, such events appear to have been exceptional in Big Sur and later. It certainly doesn't appear to have occurred sufficiently frequently to become noticeable on popular internet discussion forums or support groups, although other problems updating macOS are often more widely reported. My own experience with a minimum of four different Macs is that I have never encountered a verification failure against the tree of hashes, nor has this ever occurred in any of the virtual machines in which I have macOS installed. While these are anecdotal and not statistical, they're worth putting into the context of comparable failures, such as those requiring replacement of the logic board. Since the release of the first M1 Macs, I have heard of several, but no more than a dozen, Apple silicon Macs that have undergone logic board replacement. In some of those cases, the repair may have been precautionary rather than the result of any discrete hardware failure. However, in my experience an Apple silicon Mac is more likely to require logic board replacement than it is to fail to verify its SSV against its tree of hashes. So while none of us can rule out data corruption due to cosmic rays and similar causes, the chance of that happening appears extremely remote, and probably the least of your concerns with modern Macs. If you know otherwise, please don't hesitate to let me know. Share this: * Twitter * Facebook * Reddit * Pinterest * Email * Print * Like this: Like Loading... Related Posted in Macs, Technology and tagged Apple silicon, Big Sur, integrity, M2, macOS 11, security, SSV, T2. Bookmark the permalink. 12Comments Add yours 1. 1 [065cdd748ba3] f d on May 30, 2023 at 7:21 am Reply > Although I believe that I have heard of one user whose Mac didn't complete a macOS update successfully, and was prompted to reinstall macOS, such events appear to have been exceptional in Big Sur and later. This -- or something similar -- happened to me during one of the Monterey updates on a pre-T2 Intel iMac. After the update, the machine wouldn't boot. I had to reinstall the OS from a USB stick. Data was all still there after the reinstall, so I guessed the problem was an integrity issue with the update. LikeLiked by 1 person + 2 [6986a746f627] hoakley on May 30, 2023 at 4:51 pm Reply I have deliberately not involved non-T2 Intel Macs, because, although they do apparently validate the signature on the SSV, they may not validate the whole hash tree in the same way. Apple unfortunately doesn't describe this in the Platform Security Guide. My understanding of what happens if validation fails is that the Mac should start up in Recovery, so I'm not sure that your event was a failure to validate - indeed, the whole update might simply have aborted without even trying to boot. Howard. LikeLike 2. 3 [bc6d0f237e71] Enzo Vincenzo on May 30, 2023 at 7:30 am Reply Dear Howard, thank you! At the beginning of Mac OS X I was lucky enough to come across David Pogue's books "The missing manual..." which introduced me to the secrets of OS X. Now, for some time now, my reference has been you and your WEB site and I don't know if there are others in which to deepen and understand what is at the foundation of the Mac. This one of mine does not want to be a sweetie! I thank you with my heart! Vincent LikeLiked by 1 person + 4 [6986a746f627] hoakley on May 30, 2023 at 4:51 pm Reply Thank you. Howard. LikeLike 3. 5 [2af76302b284] Duncan on May 30, 2023 at 12:46 pm Reply Thank you for putting this all in perspective. While I am one of the more vocal users calling for Apple to implement user-data integrity checks, the numbers you've put forth in this article imply that the potential risks are indeed quite minimal. I am curious about one detail, nonetheless. You wrote: "During the boot process, unless boot security has been downgraded from Full Security, the contents of that SSV are verified against its tree of hashes." Is that to say that during each system boot, every (digital) bit of the SSV is verified? How much extra time does that add to the boot process? LikeLiked by 1 person + 6 [6986a746f627] hoakley on May 30, 2023 at 4:56 pm Reply This is the ingenious part: verification carries little overhead, as it runs as a rolling process once the top-level 'seal' has been verified. So verification is a continuous process as the SSV is mounted and accessed. Howard. LikeLike 4. 7 [ded6beb516a9] markbot2zero on May 30, 2023 at 3:49 pm Reply Thank you, Howard. re: "Thus every T2 and Apple silicon Mac booting Big Sur or later in Full Security mode has contributed a measure of data integrity." Evidently non-T2 Intel Macs behave differently. If you have a moment, could you elaborate on the difference, or perhaps refer me to one of your articles where you already have done so? I couldn't tell from the "Platform Security Guide, and a search of articles. Comment: Inasmuch as macOS is protected against "bit rot" as you have explained above, the actual damage done by errant cosmic rays, unlikely as it is, will probably be to user files. (Assuming the rays act randomly and are not malevolently AI infused, e.g., through contact with ChatGPT.) In my experience, the vast majority of Mac storage space is taken up by movies, music, pictures and books. If any of these, against all odds, gets a bit tweaked, it probably won't even be noticeable. Finally, thank you for so generously providing your "intch" set of utilities*, so that anyone still concerned about file integrity can take precautions against its occurrence. -Mark * I'm referring here to Howard's Dintch, Fintch & cintch utilities. LikeLiked by 1 person + 8 [6986a746f627] hoakley on May 30, 2023 at 5:05 pm Reply "Evidently non-T2 Intel Macs behave differently" Well, they have to, because they don't have a secure enclave or SEP. Unfortunately, in the Platform Security Guide, Apple describes how T2 Intel and Apple silicon Macs handle this, but not non-T2 Intel Macs. I understand that non-T2 models do still validate the top-level hash, or 'seal', but how that works, and what happens if there's a failure of validation, I simply don't know. Howard. LikeLike 5. 9 [4bf1508fb4de] Ron Gomes on May 30, 2023 at 4:36 pm Reply "During the boot process, unless boot security has been downgraded from Full Security, the contents of that SSV are verified against its tree of hashes." This is news to me. I have to run with reduced security because Rogue Amoeba's audio requires it (oddly, even though their audio capture engine is implemented as a system extension and not a kernel extension). So you're saying that in this mode I'm not getting the verification advantage of the SSV? That the Mac might boot successfully even in the presence of system corruption? LikeLiked by 1 person + 10 [6986a746f627] hoakley on May 30, 2023 at 5:15 pm Reply "So you're saying that in this mode I'm not getting the verification advantage of the SSV?" No, I'm saying that in Full Security, the contents are fully verified. Below that are two levels of reduced security, and the rules for those are spelled out in detail in the Platform Security Guide, to which I refer you. I believe that in your case, your Mac should still perform full verification. What I needed to do here was to exclude both forms of reduced security because of their added complexity. This article is about the consequences of Full Security mode, not the differences between the three modes. Howard. LikeLike 6. 11 [09d3a2aad8d5] bdmarsh on May 30, 2023 at 5:31 pm Reply I haven't had this happen on our work computers (yet). But I did, over a year ago, have a client with a M1 Mac mini that failed to boot after an OS update that went wrong (Big Sur likely) - It wouldn't boot - would just shut down again and it couldn't properly go into the recovery mode to re-install nicely (not sure why, tried at least a couple of times before giving up). That was when I had to build my first external Apple Silicon boot disk on my own personal M1 Mac mini to then boot the clients Mac and do a copy of the data partition and re-install booted from the external. It's been long enough I don't remember specific details - just generally finding it weird. I was glad it wasn't a T2 based Mac or I wouldn't have been able to boot from external. LikeLiked by 1 person + 12 [6986a746f627] hoakley on May 30, 2023 at 5:40 pm Reply I has a similar experience with an early M1 and Big Sur: I put it into DFU mode and refreshed its firmware. It then booted perfectly. I suspect that some of the first releases of Big Sur were a bit like that! Howard. LikeLike Leave a Reply Cancel reply Enter your comment here... [ ] Fill in your details below or click an icon to log in: * * * Gravatar Email (required) (Address never made public) [ ] Name (required) [ ] Website [ ] WordPress.com Logo You are commenting using your WordPress.com account. ( Log Out / Change ) Facebook photo You are commenting using your Facebook account. ( Log Out / Change ) Cancel Connecting to %s [ ] Notify me of new comments via email. [ ] Notify me of new posts via email. [Post Comment] [ ] [ ] [ ] [ ] [ ] [ ] [ ] D[ ] This site uses Akismet to reduce spam. Learn how your comment data is processed. Quick Links * Downloads * Mac Troubleshooting Summary * M1 & M2 Macs * Mac problem-solving * Painting topics * Painting * Long Reads Search Search for: [ ] [Search] Monthly archives * May 2023 (77) * April 2023 (73) * March 2023 (76) * February 2023 (68) * January 2023 (74) * December 2022 (74) * November 2022 (72) * October 2022 (76) * September 2022 (72) * August 2022 (75) * July 2022 (76) * June 2022 (73) * May 2022 (76) * April 2022 (71) * March 2022 (77) * February 2022 (68) * January 2022 (77) * December 2021 (75) * November 2021 (72) * October 2021 (75) * September 2021 (76) * August 2021 (75) * July 2021 (75) * June 2021 (71) * May 2021 (80) * April 2021 (79) * March 2021 (77) * February 2021 (75) * January 2021 (75) * December 2020 (77) * November 2020 (84) * October 2020 (81) * September 2020 (79) * August 2020 (103) * July 2020 (81) * June 2020 (78) * May 2020 (78) * April 2020 (81) * March 2020 (86) * February 2020 (77) * January 2020 (86) * December 2019 (82) * November 2019 (74) * October 2019 (89) * September 2019 (80) * August 2019 (91) * July 2019 (95) * June 2019 (88) * May 2019 (91) * April 2019 (79) * March 2019 (78) * February 2019 (71) * January 2019 (69) * December 2018 (79) * November 2018 (71) * October 2018 (78) * September 2018 (76) * August 2018 (78) * July 2018 (76) * June 2018 (77) * May 2018 (71) * April 2018 (67) * March 2018 (73) * February 2018 (67) * January 2018 (83) * December 2017 (94) * November 2017 (73) * October 2017 (86) * September 2017 (92) * August 2017 (69) * July 2017 (81) * June 2017 (76) * May 2017 (90) * April 2017 (76) * March 2017 (79) * February 2017 (65) * January 2017 (76) * December 2016 (75) * November 2016 (68) * October 2016 (76) * September 2016 (78) * August 2016 (70) * July 2016 (74) * June 2016 (66) * May 2016 (71) * April 2016 (67) * March 2016 (71) * February 2016 (68) * January 2016 (90) * December 2015 (96) * November 2015 (103) * October 2015 (119) * September 2015 (115) * August 2015 (117) * July 2015 (117) * June 2015 (105) * May 2015 (111) * April 2015 (119) * March 2015 (69) * February 2015 (54) * January 2015 (39) Tags APFS Apple AppleScript Apple silicon backup Big Sur Blake bug Catalina Consolation Console diagnosis Disk Utility Dore El Capitan extended attributes Finder firmware Gatekeeper Gerome HFS+ High Sierra history of painting iCloud Impressionism iOS landscape LockRattler log logs M1 Mac Mac history macOS macOS 10.12 macOS 10.13 macOS 10.14 macOS 10.15 macOS 11 macOS 12 macOS 13 malware Mojave Monet Monterey Moreau MRT myth narrative OS X Ovid painting Pissarro Poussin privacy realism Renoir riddle Rubens Sargent scripting security Sierra SilentKnight SSD Swift symbolism Time Machine Turner update upgrade Ventura xattr Xcode XProtect Statistics * 14,698,670 hits Blog at WordPress.com. Footer navigation * About & Contact * Macs * Painting * Language * Tech * Life * General * Downloads * Mac problem-solving * Extended attributes (xattrs) * Painting topics * Hieronymus Bosch * English language * LockRattler: 10.12 Sierra * LockRattler: 10.13 High Sierra * LockRattler: 10.11 El Capitan * Updates: El Capitan * Updates: Sierra, High Sierra, Mojave, Catalina, Big Sur * LockRattler: 10.14 Mojave * SilentKnight, silnite, LockRattler, SystHist & Scrub * DelightEd & Podofyllin * xattred, Metamer, Sandstrip & xattr tools * 32-bitCheck & ArchiChect * XProCheck, T2M2, Ulbow, Consolation and log utilities * Cirrus & Bailiff * Taccy, Signet, Precize, Alifix, UTIutility, Sparsity, alisma * Revisionist & DeepTools * Text Utilities: Nalaprop, Dystextia and others * PDF * Keychains & Permissions * LockRattler: 10.15 Catalina * Updates * Spundle, Cormorant, Stibium, Dintch, Fintch and cintch * Long Reads * Mac Troubleshooting Summary * LockRattler: 11.0 Big Sur * M1 & M2 Macs * Mints: a multifunction utility * LockRattler: 12.x Monterey * VisualLookUpTest * Virtualisation on Apple silicon * LockRattler: 13.x Ventura * System Updates Secondary navigation * Search Post navigation Trojan Epics: 14 Odysseus and Polyphemus Ukrainian Painters: Viktor Zarubin Search for: [ ] [Search] Begin typing your search above and press return to search. Press Esc to cancel. * Follow Following + [croppe] The Eclectic Light Company Join 3,220 other followers [ ] Sign me up + Already have a WordPress.com account? Log in now. * + [croppe] The Eclectic Light Company + Customize + Follow Following + Sign up + Log in + Copy shortlink + Report this content + View post in Reader + Manage subscriptions + Collapse this bar %d bloggers like this: [b]