https://eclecticlight.co/2021/10/29/how-macos-is-more-reliable-and-doesnt-need-reinstalling/ Skip to content [eclecticlight] The Eclectic Light Company Macs, painting, and more Main navigation Menu * Downloads * M1 Macs * Mac Problems * Mac articles * Art * Macs * Painting hoakley October 29, 2021 Macs, Technology How macOS is more reliable, and doesn't need reinstalling [montyintsimple] One of the worst longstanding problems with macOS has been its unreliability, and by that I'm not referring to bugs, but the fact that until a year ago you never knew whether your Mac's system was intact or had become corrupted. That changed with the release of Big Sur, and continues to improve. This article explains what has happened and how it affects the way that you use macOS. This is particularly aimed at those who have been running older versions of macOS, and are finding themselves suddenly dealing with Monterey, for example on a new M1 Pro/Max MacBook Pro. A robust file system To understand what has happened, consider the changes made in Mac OS and macOS from Sierra. First, High Sierra introduced a new file system, APFS. Its predecessor HFS+ had served long and well, and is known to have several longstanding flaws. One of the most significant advances in APFS is to make changes to the file system more reliable using copy-on-write. Whereas HFS+ needs journalling and frequent maintenance, and even then is often left with minor errors in the file system, APFS is designed to be thoroughly robust. In Mojave, macOS has reached its peak booting from a single volume, with a layout broadly similar to many versions of Unix. mojavebootsimple1 Many of the system folders within this are protected by SIP, but all are within a common file system, are installed and updated together, and there's no checking of integrity. As with previous versions, one of the most popular solutions to a wide range of problems has been to re-install the system. This often works well, as there's precious little to prevent system files from becoming corrupted, and no means of telling whether they have been. The boot volume group Catalina brought the intermediate step in addressing those shortcomings, by splitting the boot volume into two, another new feature of APFS. One, which only changes in macOS updates and installs, became the System volume, and is mounted read-only; the other, the Data volume, then contains all the writeable files, including user Home folders, and those parts of the system which can change between updates. Other changes were taking place in our Macs. While it was just possible to install Mojave on and boot from HFS+ on a rotating hard disk, by Catalina you have to boot from APFS, and that in turn requires an SSD. It's still possible to boot Big Sur and even Monterey from a hard disk, but that's not through choice. Apart from speed, SSDs brought additional benefits, of increased reliability and their lack of ill-effects from fragmentation. Macs supplied with internal SSDs also gained Secure Boot, thanks to their T2 chips, and most recently with the integral Secure Boot of the M1 series chips. From here on, I'll refer to Macs with Secure Boot; Intel Macs without T2 chips have more limited benefits. The Sealed System Volume Big Sur completed the transformation. Catalina's separate System volume became a sealed snapshot, protecting the system files even more, and that's what Monterey uses too. montyintsimple In this simplified diagram of the boot volume group in Monterey, folders stored on the System volume are shown in red, and those on the Data volume in blue. This layout segregates the contents of the system into files which don't change, except in a macOS update, and everything else which does. My favourite example of this is what used to be the single top-level Applications folder, containing a mixture of bundled apps which are installed with macOS, and our own apps which we download from the App Store and Internet. In Monterey, what appears in the Finder to be a single folder with the same contents actually consists of two folders: one on the System volume contains the apps installed as part of macOS, the other on the Data volume contains those we install. Those two folders are firmlinked together and the Finder creates the illusion that it's just a single folder. Not only are these immutable system files stored on their own System volume, but your Mac no longer boots from that as a conventional volume: it's actually a snapshot, another modernising feature of APFS which has made this all possible. Perhaps the best way to see how this works is to consider what happens when macOS is installed or updated, in Big Sur or Monterey. montyinstall1 Early during installation, the Data volume is unmounted, ensuring that its contents are protected from any ill-effects of failed installation. The System volume is mounted for writing and SIP disabled, then the contents of the update are written to it, just as would happen in Mojave. With all the updated system files in place, a SHA-256 hash is then calculated for every file in the system, and stored in its file system metadata. Each group of hashes is then hashed again, and so on until there's one top-level hash, known as the Seal. If a single bit in any of those system files changes, that changes that file's hash, which no longer matches its set value. That error propagates right up the tree, and the Seal itself no longer matches that laid down by Apple for that version of macOS. When that happens, the Seal is broken, and that Mac can no longer boot from that copy of macOS, so enters Recovery mode for the user to re-install macOS. The Seal therefore guarantees the integrity of every single bit in every single file on the System volume. Rather than booting from that sealed System volume, the installer then makes a snapshot of the System volume, which contains all the hashes and the Seal itself. After enabling SIP, the System volume is unmounted, and macOS boots from that snapshot. Snapshots are immutable, in that even macOS can't change them, a protection far stronger than mounting the System volume read-only as in Catalina. The Data volume is mounted, and any changes made to it are completed for that boot. By the time we've got to Monterey, for models with Secure Boot: * Your Mac will only boot from a snapshot of the System volume which can't be changed in any way. * Your Mac will only boot from a sealed system, which is identical to the last bit with that prescribed by Apple. * Nothing apart from a macOS installer/updater can change the System volume. * For much or all of the macOS update/install process, the Data volume is unmounted and can't be affected by problems occurring during the update/install. * All this is happening on an SSD, with its high reliability. * The file system used is APFS, with its high robustness. This doesn't prevent users from doing radical things like building their own kernel, though. Secure Boot can be disabled, and M1 series Macs offer three different settings in their Startup Security Utility to cater for pretty well everyone. How, then, can you check whether your installation of macOS is factory-fresh, or has become damaged or corrupted? Simply restart your Mac: if the Seal is broken, you should very quickly know about it, at least on Macs with Secure Boot when that is fully enabled. Limits The Sealed System Volume (SSV) addresses the integrity of those system components on the System volume, but doesn't currently extend its cover to system components stored on the Data volume, nor to user files there. As Mac malware doesn't normally try to tamper with what's now on the System volume, you can still get a severe bout of malware without Secure Boot providing any protection. So too can key files stored on the Data volume become damaged or corrupted, or could be encrypted in a ransomware attack (should Mac ransomware appear again). Another serious problem are user-installed kernel extensions, which aren't part of the System, but which are given access to the innermost citadel of kernel space. That not only makes them a liability in terms of security, but makes it all too common for them to cause kernel panics. That's why developers are moving quickly away from kernel extensions to system extensions, which run in user space instead. Before an M1 series Mac can load a third-party kernel extension, it has to be set to run at reduced security and you must explicitly enable their use. Practical importance If your Mac, running Big Sur or Monterey with Secure Boot enabled, boots normally, then its System volume is in perfect condition. Whatever might be wrong with it, re-installing the System volume isn't likely to fix it. At the end of a macOS update or install, if your Mac boots normally with Secure Boot enabled, then its System volume is in perfect condition. If the install fails, your best course is to enter Recovery mode and there re-install macOS. If that's successful, you can have confidence in the integrity of the installed System again. Although it may still be possible to 'clone' a System volume, the best and most reliable way to install Big Sur or Monterey on any disk is to use Apple's full installer app, as that ensures that the boot volume group is created correctly, with firmlinks between System and Data volumes. Secure Boot and the Sealed System Volume protect the System, not Data, volume. Keeping macOS up to date is the single most important way to protect your Mac overall. Don't install third-party kernel extensions if you can avoid it. Look for updated software which replaces them with system extensions. Share this: * Twitter * Facebook * Reddit * Pinterest * Email * Print * Like this: Like Loading... Related Posted in Macs, Technology and tagged APFS, Big Sur, macOS, macOS 11, macOS 12, Monterey, reinstall, reliability, SSV. Bookmark the permalink. 28Comments Add yours 1. 1 [2af76302b284] Duncan on October 29, 2021 at 11:47 am Reply Howard, thank you for these ever-informing articles, and this particular explanation. I know I have been quite vocal here when APFS was released in regards to its data-integrity checking, or lack thereof for user files, but with ongoing new information and MacOS development I'm wondering if we might see that feature added at some point. In your article you wrote: "With all the updated system files in place, a SHA-256 hash is then calculated for every file in the system, and stored in its file system metadata. Each group of hashes is then hashed again, and so on until there's one top-level hash, known as the Seal." Do you now, with your work on the various *inch utilities as well as investigation into APFS's own System volume checks, think that Data volume file integrity _could_ be implemented by Apple at some point? (Earlier on it wasn't clear because of the paucity of information on APFS internals.) I know you can't speak definitively on Apple's intentions, but I'm curious if there are any technical barriers at this point. To me it seems like they have already solved this problem for the System volume, so it's just a matter of applying that same work to the Data volume. (And perhaps this functionality might be limited to Apple Silicon Macs, which could better handle the dynamic hash computations on their Icestorm cores.) LikeLiked by 1 person + 2 [6986a746f627] hoakley on October 29, 2021 at 7:30 pm Reply Thank you. Apple owns the APFS format - it's completely proprietary - so could readily add checksums or hashes. However, I don't see them coming for a while yet, if ever. It's the tradeoff between making the file system and storage medium as reliable as possible, so making them unnecessary, and the penalty of checking them on, say, 100 GB files, which aren't that rare now. Whether they'll be introduced first as a high-integrity option, I don't know. Howard. LikeLiked by 1 person 2. 3 [2efde0679b0c] DavidB on October 29, 2021 at 4:54 pm Reply Thank you for an informative and understandable article on what is a very technical subject. Well done. Came here for the Mac-techincal articles -- and stayed for the art. LikeLiked by 1 person + 4 [6986a746f627] hoakley on October 29, 2021 at 7:37 pm Reply Thank you. Howard. LikeLiked by 1 person 3. 5 [8eb3202e3a78] ericrfmwp on October 29, 2021 at 6:18 pm Reply Thanks again for another helpful article. I especially appreciate the graphics. I bristle at the fact that the file system/boot/ install/update graph has gotten so complicated (I come from a Unix background which started out with just "The Disk". Period). But it actually makes sense. I'm waiting on my M1 MacBook Pro to see how Monterey works. I'm still hanging on to Mojave on my 2019 iMac, but Adobe has raised my ire by requiring Catalina for its most recent Lightroom update. Never! I'll never upgrade to Catalina I say! So I *might* upgrade the iMac to Monterey some day. I'll wait a while for some Monterey updates, 'cause my iMac has lots of things connected to lots of other things, which is the sort of configuration that can expose bugs in the dark corners of the OD. OTOH, I'll be able to run Lightroom on my new MBP so I guess there's no hurry LikeLiked by 1 person + 6 [6986a746f627] hoakley on October 29, 2021 at 7:41 pm Reply Thank you. Yes, several major software vendors seem to have followed Apple's unwritten policy of supporting only the current and previous two major releases of macOS. I don't really understand why. Some features in a product depend on specific versions being available. It often isn't difficult to detect whether that's the case and offer those features, or disable them when macOS is older. I wouldn't recommend Catalina or Big Sur. If you're going through the pain of losing 32-bit apps etc., then it makes more sense now to get maximum benefit with Monterey. Howard. LikeLiked by 1 person o 7 [8eb3202e3a78] ericrfmwp on October 29, 2021 at 9:43 pm Reply Hi Howard... I don't think I have any 32-bit apps, so that shouldn't be an issue. But I'll have to double-check, of course. As for not supporting older os versions, I'm pretty sure a big reason is they don't want to bother testing on older versions, which takes time (and that pesky money they keep worrying about) and possibly more ("vintage") equipment to test with. I'm guessing they use Apple's policy as an excuse, or to relieve them of guilt. This coming from a 40-year software development veteran. Another piece of software with the same attitude is TurboTax. For 2021 it requires Catalina or newer. Oddly enough, it only requires Windows 8.1 or newer for those systems. I'll have my new MBP by tax time, so it won't really matter, but I might just change tax software out of spite LikeLiked by 1 person # 8 [6986a746f627] hoakley on October 29, 2021 at 10:19 pm Reply Thank you. It's commonplace now - another which follows this is Microsoft with Office365, for example. Oddly, smaller developers find the time and resources to retain customers running old macOS as much as they can. But those with greater resources seem unable to. Such a common theme. Howard. LikeLiked by 1 person # 9 [ec859d707747] Sab Gigo on October 29, 2021 at 11:32 pm Reply Why not try cloning a backup of your system to another volume (assuming you have sufficient resources, of course -- such as an external drive with available space), and then updating the clone -- sort of a risk-free tryout, if you will. If you are not satisfied, there is nothing lost, and you can easily resume your use of the original operating system. LikeLiked by 2 people @ 10 [8eb3202e3a78] ericrfmwp on October 30, 2021 at 12:42 am That would be the plan, yes, and I'll probably make *two* backups (plus Time Machine). Thanks! LikeLiked by 2 people @ 11 [6986a746f627] hoakley on October 30, 2021 at 6:02 am That's a good suggestion, but it's important to note that upgrading an external disk will also update that Mac's firmware to that supplied with Monterey, and there's no going back once that has been completed. If that were to make a Mojave system unstable, you'd be forced to upgrade. Howard. LikeLiked by 2 people @ 12 [ec859d707747] Sab Gigo on October 30, 2021 at 8:43 am Howard, you are right about the firmware being updated, of course! I'm interested to know what your awareness/experience is of problems with backward compatibility of updated Mac firmware being used with prior macOS versions. I suppose it would be safer to trial a Monterey virtual machine, which should leave the Mac firmware unaltered -- such a virtual machine is possible with Parallels, I believe, but the procedure might be a bit complicated... LikeLiked by 1 person @ 13 [6986a746f627] hoakley on October 30, 2021 at 9:00 am Thank you. Yes, a VM is always a good platform for testing. Parallel Desktop 17 makes it particularly easy to set up a Monterey VM on Intel or on Apple Silicon, so there shouldn't be any trouble there. I don't know about VMware, other than that it doesn't yet support virtualisation of macOS on M1 series Macs at all. Howard. LikeLike 4. 14 [23cacb2a2017] brookter on October 29, 2021 at 7:12 pm Reply Very informative and interesting, Howard -- thank you very much! I'm not very technical, so the details are beyond me, but you make the general process very clear. Cheers! David. LikeLiked by 1 person + 15 [6986a746f627] hoakley on October 29, 2021 at 7:41 pm Reply Thank you. Howard. LikeLiked by 1 person 5. 16 [ec859d707747] Sab Gigo on October 29, 2021 at 11:26 pm Reply Brilliant summary as usual. Thank you so much for your tireless devotion to the Mac... and ditto for your contributions regarding paintings. LikeLiked by 1 person + 17 [6986a746f627] hoakley on October 30, 2021 at 6:00 am Reply Thank you. Howard. LikeLike 6. 18 [022aad4b8464] Bill Stanford on October 29, 2021 at 11:42 pm Reply Howard, sorry, I've think there's a typo at a critical point above. Where you write " When that happens, the Seal is broken, and that Mac can longer boot from that copy of macOS", I'm sure you intended "no longer". Thanks for this article, and for all your work! (Your Venice piece here is fabulous!) Bill LikeLiked by 1 person + 19 [6986a746f627] hoakley on October 30, 2021 at 6:02 am Reply Thank you - corrected. Howard. LikeLike 7. 20 [5138c2c4dd02] SMC on October 30, 2021 at 1:03 am Reply Thank you for the instructive and interesting article. A typo: that Mac can longer boot LikeLiked by 1 person + 21 [6986a746f627] hoakley on October 30, 2021 at 6:04 am Reply Thank you - corrected now. Howard. LikeLike 8. 22 [46b723068ce3] CKing123 on October 30, 2021 at 3:48 am Reply Nice summary! Monterey also seems to make use of System and Data volume to quickly erase and reset Mac through the Erase Assistant (it would just delete the encryption key of Data volume, making it quick) I remember in one of your post that some things like Safari are stored LikeLiked by 1 person + 23 [6986a746f627] hoakley on October 30, 2021 at 6:04 am Reply Thank you. Yes, it does. Howard. LikeLike o 24 [daba689ce91c] Pico on October 30, 2021 at 5:17 pm Reply I've noticed that you use the term "Sealed" System Volume for SSV, while Apple uses "Signed" System Volume (https:/ /support.apple.com/guide/mac-help/ what-is-a-signed-system-volume-mchl0f9af76f/mac). I know the SSV is both signed and sealed, but this difference in terminology may cause confusion. LikeLiked by 1 person # 25 [daba689ce91c] Pico on October 30, 2021 at 5:18 pm Reply Oops, I didn't mean for that to be a reply to another comment. iPhone scrolling mistake. My bad. LikeLiked by 1 person # 26 [6986a746f627] hoakley on October 30, 2021 at 9:58 pm Reply Thank you. Yes, and Apple's terminology is confusing in itself, as if it hasn't really decided what to call them. In Disk Utility, which is the closest most users come, it refers to sealed but not signed volumes. When describing the process of sealing (not signing), Apple refers to the top-level hash as the Seal, and then states that it's compared against Apple's signature, which I presume is a SHA-256 value. When that Seal is broken, it's referred to as have a broken seal, not signature, and the process is known as breaking the seal. I've never heard of a hash being referred to as a signature in a crypto sense, either, and most users associate the term 'signature' with a process of signing rather than hashing, involving some sort of certificate and keys, not hashes. Finally, in another situation in which both signatures and hashes are involved, code-signing, the signature involves certificates, and cdhashes are considered and used quite separately, as on Apple Silicon Macs. So maybe using Apple's terminology requires an explanation that a Signed System Volume isn't signed in the way that most of us understand it, but actually sealed using SHA-256 hashes? Howard. LikeLike 9. 27 [46b723068ce3] chiragroop on October 30, 2021 at 3:59 am Reply Nice Article! Monterey seems to now be able to reset the Mac quickly by erasing the Data volume (or rather, just deleting the encryption key to make it feel instant). I remember one of your post where some things like Safari are stored in the Data volume. Does it retain Safari and other apps by storing them elsewhere? (You might be able to corrupt the copy of Safari and then reset it) Or are those also stored in System volume? LikeLiked by 1 person + 28 [6986a746f627] hoakley on October 30, 2021 at 6:05 am Reply Thank you. Yes, Safari is installed on the Data volume, so it can be updated outside full macOS updates. I'll be looking more at how this works in the next couple of weeks. Howard. LikeLike Leave a Reply Cancel reply Enter your comment here... [ ] Fill in your details below or click an icon to log in: * * * * Gravatar Email (required) (Address never made public) [ ] Name (required) [ ] Website [ ] WordPress.com Logo You are commenting using your WordPress.com account. ( Log Out / Change ) Google photo You are commenting using your Google account. ( Log Out / Change ) Twitter picture You are commenting using your Twitter account. ( Log Out / Change ) Facebook photo You are commenting using your Facebook account. ( Log Out / Change ) Cancel Connecting to %s [ ] Notify me of new comments via email. [ ] Notify me of new posts via email. [Post Comment] [ ] [ ] [ ] [ ] [ ] [ ] [ ] D[ ] This site uses Akismet to reduce spam. Learn how your comment data is processed. Quick Links * Downloads * Mac Troubleshooting Summary * M1 Macs * Mac problem-solving * Painting topics * Painting * Long Reads Search Search for: [ ] [Search] Monthly archives * October 2021 (73) * September 2021 (76) * August 2021 (75) * July 2021 (75) * June 2021 (71) * May 2021 (80) * April 2021 (79) * March 2021 (77) * February 2021 (75) * January 2021 (75) * December 2020 (77) * November 2020 (84) * October 2020 (81) * September 2020 (79) * August 2020 (103) * July 2020 (81) * June 2020 (78) * May 2020 (78) * April 2020 (81) * March 2020 (86) * February 2020 (77) * January 2020 (86) * December 2019 (82) * November 2019 (74) * October 2019 (89) * September 2019 (80) * August 2019 (91) * July 2019 (95) * June 2019 (88) * May 2019 (91) * April 2019 (79) * March 2019 (78) * February 2019 (71) * January 2019 (69) * December 2018 (79) * November 2018 (71) * October 2018 (78) * September 2018 (76) * August 2018 (78) * July 2018 (76) * June 2018 (77) * May 2018 (71) * April 2018 (67) * March 2018 (73) * February 2018 (67) * January 2018 (83) * December 2017 (94) * November 2017 (73) * October 2017 (86) * September 2017 (92) * August 2017 (69) * July 2017 (81) * June 2017 (76) * May 2017 (90) * April 2017 (76) * March 2017 (79) * February 2017 (65) * January 2017 (76) * December 2016 (75) * November 2016 (68) * October 2016 (76) * September 2016 (78) * August 2016 (70) * July 2016 (74) * June 2016 (66) * May 2016 (71) * April 2016 (67) * March 2016 (71) * February 2016 (68) * January 2016 (90) * December 2015 (96) * November 2015 (103) * October 2015 (119) * September 2015 (115) * August 2015 (117) * July 2015 (117) * June 2015 (105) * May 2015 (111) * April 2015 (119) * March 2015 (69) * February 2015 (54) * January 2015 (39) Tags Adobe APFS Apple AppleScript Apple silicon App Store backup Big Sur Blake Bonnard bug bugs Catalina Consolation Console diagnosis Disk Utility Dore El Capitan extended attributes Finder firmware Gatekeeper Gerome HFS+ High Sierra history history of painting iCloud Impressionism iOS landscape LockRattler log logs M1 Mac Mac history macOS macOS 10.12 macOS 10.13 macOS 10.14 macOS 10.15 macOS 11 malware Metamorphoses Mojave Monet Moreau MRT myth narrative OS X Ovid painting Pissarro Poussin privacy realism riddle Rubens Sargent scripting security Sierra Swift symbolism Time Machine Turner update upgrade van Gogh xattr Xcode XProtect Statistics * 10,047,681 hits Blog at WordPress.com. Footer navigation * About & Contact * Macs * Painting * Language * Tech * Life * General * Downloads * Mac problem-solving * Extended attributes (xattrs) * Painting topics * Hieronymus Bosch * English language * LockRattler: 10.12 Sierra * LockRattler: 10.13 High Sierra * LockRattler: 10.11 El Capitan * Updates: El Capitan * Updates: Sierra, High Sierra, Mojave, Catalina, Big Sur * LockRattler: 10.14 Mojave * SilentKnight, silnite, LockRattler, SystHist & Scrub * DelightEd & Podofyllin * xattred, Metamer, Sandstrip & xattr tools * 32-bitCheck & ArchiChect * T2M2, Ulbow, Consolation and log utilities * Cirrus & Bailiff * Taccy, Signet, Precize, Alifix, UTIutility, Sparsity, alisma * Revisionist & DeepTools * Text Utilities: Nalaprop, Dystextia and others * PDF * Keychains & Permissions * LockRattler: 10.15 Catalina * Updates * Spundle, Cormorant, Stibium, Dintch, Fintch and cintch * Long Reads * LockRattler: 11.0 Big Sur * Mac Troubleshooting Summary * M1 Macs * Mints: a multifunction utility * LockRattler: 12.x Monterey Secondary navigation * Search Post navigation Impressionist painting in Britain: 14 Wynford Dewhurst Landscape Composition: 9 One piazza, eight views Search for: [ ] [Search] Begin typing your search above and press return to search. Press Esc to cancel. Loading Comments... Write a Comment... [ ] Email (Required) [ ] Name (Required) [ ] Website [ ] [Post Comment] Send to Email Address [ ] Your Name [ ] Your Email Address [ ] [ ] loading [Send Email] Cancel Post was not sent - check your email addresses! Email check failed, please try again Sorry, your blog cannot share posts by email. %d bloggers like this: [b]