[HN Gopher] How We Ported Linux to the M1
       ___________________________________________________________________
        
       How We Ported Linux to the M1
        
       Author : easton
       Score  : 309 points
       Date   : 2021-01-21 15:33 UTC (7 hours ago)
        
 (HTM) web link (corellium.com)
 (TXT) w3m dump (corellium.com)
        
       | benkoller wrote:
       | I'd be thrilled to hear how the energy consumption under Linux
       | would compare to native Big Sur - obviously with the M1 Macbook
       | Pros in mind.
        
       | Pasorrijer wrote:
       | I love this for a few reasons. First off, we get to see a deeper
       | look into what Apple did, and I'm enjoying how they went outside
       | the established norms... Sometimes the Apple cart (pun not
       | intended) needs overturning.
       | 
       | Secondly, the fact that they are able to dig into this much depth
       | bodes well for competitors to reverse engineer similar
       | performance into other platforms
        
         | mhh__ wrote:
         | What norms are there that actually make any difference? The
         | cool stuff in M1 is the design work into the processing
         | hardware, aren't the peripherals a bit of a hodge-podge of old
         | this and non-standard that?
        
           | 95014_refugee wrote:
           | > aren't the peripherals a bit of a hodge-podge of old this
           | and non-standard that?
           | 
           | There are certainly "old" peripherals in the SoC (the UART
           | comes to mind). There's probably not a lot of norm-busting in
           | this,
           | 
           | There are certainly "non-standard" peripherals in the SoC (if
           | by "non-standard" you mean "nobody else has them"). As the
           | article notes, many if not most other folks "just" check off
           | a couple dozen boxes on the ARM PrimeCell Shopping List (plus
           | a few 3p vendors) and stack the resulting blobs together.
           | 
           | Going your own way and building major components yourself
           | (and all of the major components, including the fabric /
           | interconnects) is very definitely not the "norm".
        
         | jbob2000 wrote:
         | Computers used to be a "commodity of commodities". The
         | motherboard was made by one manufacturer, the memory was made
         | by another one, the CPU by a different one, etc. etc.
         | 
         | Unless a competitor controls and owns their supply chain like
         | Apple does, I don't see how they can compete; they need those
         | "established norms" otherwise they won't be able to find
         | commodities that work with their final product.
        
           | Blikkentrekker wrote:
           | Is that not still the case?
           | 
           | My machines aren't that old, and I'm fairly certain that is
           | the case in both of them. In fact, I'm thinking about
           | upgrading my working memory on one of them looking at the 50%
           | indicator at my screen right now for what is only having 12
           | tabs open in a web browser and not doing much other heavy
           | lifting.
        
           | spijdar wrote:
           | Computers being "commodities of commodities" is kind of a
           | recent phenomenon. For a while, the norm was to do exactly
           | what Apple is doing. It was much more common to design and
           | market your own custom processor or, at the least, custom
           | chipsets coupled with your own firmware/OS, like most unix
           | workstations, non-IBM-PCs like the Amiga, Atari, Macs...
           | 
           | It'll be interesting to see if any other players try to move
           | back to this style in the desktop/laptop PC space.
        
             | AnIdiotOnTheNet wrote:
             | Personally I think that could be a good thing if it allows
             | us to once again become experimental with personal
             | computing operating systems, an are that has stagnated
             | significantly in the past 2 decades due in part to the fact
             | that any new contender has to support a ludicrous amount of
             | hardware to be usable on a wide variety of PCs.
             | 
             | Then again it could all go horribly wrong and leave us with
             | nothing but highly locked-down terminals suitable only for
             | connection to our glorious cloud masters.
        
               | ryukafalz wrote:
               | The flip side to that is that OSes that are built for
               | specific hardware are only useful so long as that
               | hardware continues to be produced. I don't think we'd
               | have Linux as a useful contender today if it had been
               | locked to a specific company's hardware; there's a reason
               | we're not still using BeOS today for example (Haiku
               | aside).
               | 
               | I'm more excited about things like rump kernels for
               | hardware support across disparate OSes:
               | https://en.m.wikipedia.org/wiki/Rump_kernel
        
               | spijdar wrote:
               | > there's a reason we're not still using BeOS today for
               | example (Haiku aside).
               | 
               | You're blaming BeOS's failure on the BeBox? Every release
               | of BeOS that was marked as a "full release" (as opposed
               | to a developer preview or a "preview release") could run
               | on Intel hardware, and the BeBox itself was discontinued
               | long before then.
               | 
               | This is a tangent, but I don't understand people who
               | think BeOS was the heir-apparent to
               | consumer/home/personal OSes. It had super half baked
               | networking support and absolutely no concept of multi-
               | user or privilege separation. Similar home OSes didn't
               | _enforce_ privilege separation or multiple users, but at
               | least had the framework -- BeOS essentially just runs
               | everything as root.
               | 
               | It had some neat tricks, but was a pretty underdeveloped
               | OS in some key areas. The fact it never took off isn't
               | that surprising to me, and it's not _just_ because Be was
               | a small company or that it 's hard for an underdog to get
               | market share in a crowded space (both of which are also
               | true). It just wasn't really well suited for "general
               | purpose computing" in the same way that e.g. Linux,
               | WinNT, or Darwin are.
        
               | setpatchaddress wrote:
               | I know people who were very upset that Mac OS X required
               | login/password. The "personal" part of "personal
               | computer" was important to some people.
        
               | spijdar wrote:
               | AFAIK the original releases of OS X by default did not
               | require a login and simply booted to desktop, same as Win
               | 9x/XP.
               | 
               | The distinction is these systems, even if they would
               | never ask you for a password or allow you to forgo a
               | password altogether (like WinXP) still have the notion of
               | user accounts and privilege separation/process isolation.
               | It's a lot easier to fake them not being there then to
               | retrofit them.
        
               | ryukafalz wrote:
               | >You're blaming BeOS's failure on the BeBox? Every
               | release of BeOS that was marked as a "full release" (as
               | opposed to a developer preview or a "preview release")
               | could run on Intel hardware, and the BeBox itself was
               | discontinued long before then.
               | 
               | Okay, that's fair - BeOS was a bad example. Let's say
               | AmigaOS then, which largely seems to be limited to Amiga
               | hardware. (Though amazingly it still seems to be
               | maintained?)
        
               | spijdar wrote:
               | Depends on who you ask, I think. I'm not into Amiga stuff
               | or that knowledgeable, but I understand AROS is
               | essentially Amiga 3 implemented on x86/ARM/PPC/M68K.
               | However, it's not seen as being "Amiga" because it
               | doesn't primarily run on "real Amiga hardware"?
        
               | AnIdiotOnTheNet wrote:
               | > This is a tangent, but I don't understand people who
               | think BeOS was the heir-apparent to
               | consumer/home/personal OSes. It had super half baked
               | networking support and absolutely no concept of multi-
               | user or privilege separation.
               | 
               | What need does a personal computer OS have for multi-user
               | support? The median number of users on a personal
               | computer at any given time is 1. Even in the rare case
               | that two or more people want to use the same computer at
               | separate times and desire to protect eachother's files
               | from eachother, they can use encryption rather than fake
               | OS enforced security.
               | 
               | Privilege separation, in order to keep malicious
               | _applications_ from compromising the user 's files, has
               | become significantly more important but it wasn't that
               | important back then.
        
               | spijdar wrote:
               | Malicious applications and internet malware weren't
               | raging problems in the late 90s, but they were known and
               | predicted dangers.
               | 
               | You really don't need "user accounts" per se, but a way
               | to restrict process privileges, of which user accounts is
               | just a decent abstraction. On WinNT, user accounts and
               | groups are just abstractions for providing SIDs for your
               | access token -- there are SIDs like NT SYSTEM which don't
               | _have_ a correlated user account.
               | 
               | The important bit is you can create a process that can't
               | accidentally fandango on core and overwrite all the
               | system files. You have to elevate privileges -- even if
               | there's no prompt for the user, the application dev still
               | has to intentionally request admin rights to touch
               | sensitive files.
               | 
               | All that aside, though, it's a lot easier to fake "no
               | user accounts" like Windows did and just have no
               | password, than to _genuinely_ have no privilege
               | separation and try to retrofit it on later. If you 're
               | trying to market a brand new OS with new APIs that has no
               | compatibility with anything, even in the 90s, I'd expect
               | some kind of plan for at least _optional_ security.
               | Anything at all!
        
               | AnIdiotOnTheNet wrote:
               | > You really don't need "user accounts" per se, but a way
               | to restrict process privileges, of which user accounts is
               | just a decent abstraction
               | 
               | I contend that they really aren't.
               | 
               | > The important bit is you can create a process that
               | can't accidentally fandango on core and overwrite all the
               | system files.
               | 
               | And here's why: the system files are easily replicable,
               | there's a copy on every OS installation media. My
               | personal files exist in far fewer places and the OS does
               | absolutely nothing to stop a malicious application from
               | damaging them. See also: https://xkcd.com/1200/
               | 
               | We call them user accounts because they are designed
               | around isolating users from each other and the system,
               | but that isn't what we care about in a personal computing
               | context. It is possible to jerry-rig the kind of
               | permission system we actually want using the user account
               | system, but it has a lot of problems in practice because
               | it isn't designed for that. It is not a good abstraction.
        
               | spijdar wrote:
               | Yeah, I'll concede that "decent" is too much credit. Even
               | on "true multi user systems" the simplistic models *nix
               | don't really work, and you end up seeing people using
               | combinations of virtual machines and containers as ham-
               | fisted sandboxes. I was thinking of it more along the
               | lines that it's at least kind of understandable for the
               | average person. More so than some alternatives, at least.
               | 
               | Still, I think it was a big oversight to provide nothing
               | at all in BeOS. Maybe they could have come up with a
               | better solution than the backwards-compatible status quo.
               | 
               | Or maybe personal computers would have just been better
               | off in an alternate timeline where networking wasn't as
               | pervasive and we still got software on disks, and mailed
               | off for a hard copy of GNU...
        
               | spideymans wrote:
               | Also innovation in the PC space has pretty much been
               | limited by whatever Intel is willing to accommodate. I'm
               | happy we're beginning to move away from this.
        
             | spideymans wrote:
             | >It'll be interesting to see if any other players try to
             | move back to this style in the desktop/laptop PC space.
             | 
             | It's possible. I wouldn't necessarily bet on it, but it's
             | possible.
             | 
             | Qualcomm is just licensing ARM's standard core designs. If
             | I were a PC OEM, designing your own custom silicon around
             | the ARM core might be tempting, if it allows you to better
             | differentiate your product.
        
       | ValentineC wrote:
       | Previous discussion:
       | https://news.ycombinator.com/item?id=25845201
        
       | mbreese wrote:
       | Here is how they actually set the kernel for booting. It's hidden
       | behind a `curl <url> | sh` in their post:
       | #!/bin/sh         bputil -d | grep "CustomerKC" | grep -v
       | "absent"         KC=$?         if [ $KC -eq 1 ]         then
       | bputil -n -k -c -a -s           csrutil disable           csrutil
       | authenticated-root disable         fi         cd /Volumes/Preboot
       | curl https://$LONGURL/linux.macho > linux.macho         kmutil
       | configure-boot -c linux.macho -v /Volumes/Macintosh\ HD/
       | echo "Kernel installed. Please reboot";
       | 
       | So, it looks like they download their pre-compiled kernel onto a
       | Preboot volume. This is then set as the booting kernel? How long
       | until there is a grub-like option?
       | 
       | I should add -- I don't know what most of these commands are. It
       | would probably be helpful if they had spelled these out more, but
       | this is all a great place to start.
       | 
       | bputil - Utility to precisely modify the security settings on
       | Apple Silicon Macs.
       | 
       | csrutil - Configure System Integrity Protection (SIP)
       | 
       | kmutil - replaces kextload, kextunload, and other earlier tools
       | for loading and managing kexts
        
         | saagarjha wrote:
         | (All those commands have good man pages.)
        
           | garrepi wrote:
           | common BSD user vs. common Linux user
        
         | azinman2 wrote:
         | And in doing so it turns of SIP. Probably necessary to boot a
         | non-Apple kernel, but it means if you do want to switch back to
         | the Mac side you've got one of the main security mechanisms
         | turned off.
        
           | GeekyBear wrote:
           | Security settings are stored per partition, not per machine,
           | so you can have a Linux partition where SIP is disabled and a
           | macOS partition where it is not on the same machine.
        
         | easton wrote:
         | The plan, IIUC, is to use PongoOS[0] as a bootloader which will
         | then be able to load XNU/macOS or Linux. PongoOS already works
         | on M1, so once the Linux support is done there will be a GRUB
         | substitute.
         | 
         | 0: https://github.com/checkra1n/pongoOS
        
       | ndesaulniers wrote:
       | Cool, looks like they've been posted upstream for review.
       | 
       | https://lwn.net/Articles/843221/
        
       | uHuge wrote:
       | "One you see it print" typo.
        
       | jeromenerf wrote:
       | The original title is "How we ported ...", less of an
       | announcement, more of a story.
        
         | easton wrote:
         | The title fixer messed it up, I've fixed it.
        
       | ogre_codes wrote:
       | From the sounds of it, the corellium guys and Marcan are being a
       | bit more cooperative now. Also, it sounds like much of the
       | Corellium code is going to be pushed up to the main line so there
       | won't be 2 Linux M1 branches for long.
       | 
       | This hopefully means more of the efforts going into Marcan's
       | projects can be spent on higher lever things like getting the
       | GPU, Power management and other important bits working well.
        
         | solarkraft wrote:
         | > You should also consider supporting the work being done by
         | the folks over at Asahi Linux.
         | 
         | This seems like quite a nice gesture after the bit of drama
         | that apparently happened (and made corellium look bad, imo).
         | 
         | I hope that eventually they're not going to duplicates effort
         | as much as they likely are now, but then again it seems like
         | Marcan thinks that their approach (with "insider knowledge") is
         | an inherent risk to the project's legality.
         | 
         | With or without Corellium Asahi will eventually get there,
         | though, and they're pretty certainly the ones who will make the
         | GPU work (thanks to the amazing Alyssa Rosenzweig of Panfrost
         | fame).
        
           | ballenf wrote:
           | I think the risk is more reputational for orgs wanting to use
           | the Asahi project while not souring relations with Apple.
        
         | starkruzr wrote:
         | is there any actual hope for the GPU? is it based on any
         | architecture for which there already exist Linux drivers?
        
       | CyberRabbi wrote:
       | What are they doing for video?
        
         | stefan_ wrote:
         | It's just a static framebuffer that on the real MacOS is likely
         | used for nothing more than a boot logo. All the graphics are
         | software rendering.
        
         | tedd4u wrote:
         | This person (Panfrost driver developer) is working on reversing
         | M1 GPU
         | 
         | https://rosenzweig.io/blog/asahi-gpu-part-1.html
        
       | gigatexal wrote:
       | Is there a what's working and what's not working list?
        
         | notaplumber wrote:
         | Very few drivers have been written, beyond handling low-level
         | SoC stuff. The ones that have been are explicitly mentioned in
         | the linked article, stuff like USB which leverages existing
         | Linux code.
         | 
         | No keyboard, no trackpad, no GPU drivers (besides an
         | unaccelerated framebuffer, setup and left by iBoot). There's
         | some indication of Wi-Fi support, as it's a Broadcom FullMAC
         | device (brcmfmac on Linux). But yeah, I'd say unusable without
         | a USB hub and a myriad of external devices. Again, said as much
         | in the article.
        
           | jamesfmilne wrote:
           | Looks like PCIe and networking working now too.
           | 
           | https://twitter.com/cmwdotme/status/1352361874076164098?s=21
        
       | marmaduke wrote:
       | > we've been tracking the Apple mobile ecosystem since iPhone 6,
       | released in 2014 with two 64-bit cores
       | 
       | It sounds like they were well prepared. More and more I see
       | preparation as make or break for engineering success, not budget
       | or talent or tech, but silent persistence.
        
       | tyingq wrote:
       | It would be interesting if this project and the Asahi one would
       | post a "wish list" of actions, documentation, and so forth that
       | they wished they had from Apple. Even if Apple never responded,
       | it would help clarify what the challenges are, and what issues
       | might pop up for future variations of Apple silicon. Or maybe
       | some Apple devs would offer up some helpful pointers outside of
       | official sanction.
        
       | codys wrote:
       | I feel like we're missing some of the details/steps here, perhaps
       | because the folks at corellium have a bunch of existing
       | experience with apple SoCs and as a result had non-public
       | knowledge about the platform.
       | 
       | For example: determining that interrupts were being routed to
       | FIQ. It's unclear how one would have figured that was occurring.
       | Or that there was an special AIC (apple interrupt controller)
       | that needed talking to, and how to talk to it. It feels like
       | there's a lot of "reverse engineer some Apple kernel bits" that's
       | potentially being elided over.
       | 
       | Edit: or alternately, some debug interface that isn't being
       | mentioned (swd/jtag/itm).
        
         | breakfastduck wrote:
         | I think there's probably some sort of understanding between
         | them and Apple which allows them to do their thing as long as
         | they don't publish too much publicly.
         | 
         | EDIT: In fact, it might not even be that. It doesn't exactly
         | make commercial sense for a company whose entire existence is
         | based on their extensive IP knowledge of apple hardware to post
         | every detail publicly for others to copy. They are a for profit
         | organization after all.
        
         | notretarded wrote:
         | Experimentation
        
         | ValentineC wrote:
         | Corellium was founded by some veterans from the iOS jailbreak
         | community:
         | https://www.reddit.com/r/jailbreak/comments/8rmpqt/question_...
        
           | dapids wrote:
           | Yes, this is well known, what isn't and what OP is describing
           | is how they arrived to their understanding of the solution.
        
         | johnmaguire2013 wrote:
         | I think this was one of the goals of Asahi also - to document
         | the process of porting Linux to the new arch.
        
         | sneak wrote:
         | > _To actually connect the USB port inside the M1 to the USB
         | type-C connectors on the back of the Mac Mini, we had to
         | interact with a chip on I2C (which means GPIO and I2C drivers)
         | which has customized firmware. We 've seen the protocol for
         | these while building our virtualized models; nothing is a big
         | surprise if you have a bird's eye view of the system._
         | 
         | You're not kidding. This is more of a flex than an explanation.
        
         | pantalaimon wrote:
         | They already ported Linux to Apple A10 in iPhone before, the M1
         | shares a lot with those chips.
         | 
         | https://projectsandcastle.org/history
        
         | azinman2 wrote:
         | I had the same thought. How does one even know where to start
         | when reverse engineering?
        
           | aeyes wrote:
           | You start with a problem, then you think what system might be
           | interacting with that particular subsystem. Then you start to
           | poke around in there trying to intercept, dump code, add
           | debugging and so on.
        
         | Reason077 wrote:
         | Presumably, they used the macOS/iOS kernel source code as a
         | reference? I'm no expert on kernels, but a quick grep reveals
         | plenty of references to "AIC" and "FIQ", in osfmk/arm64 for
         | example.
         | 
         | https://github.com/apple/darwin-xnu
        
       | liquidify wrote:
       | Good job guys. Awesome read.
       | 
       | Pretty recently read that Linus was wanting to run Linux on new
       | Macbook Air with M1, but he though it would be too much of a pain
       | in the ass. I imagine we will see it soon.
        
       ___________________________________________________________________
       (page generated 2021-01-21 23:01 UTC)