[HN Gopher] Ask HN: Where can I find a primer on how computers b...
___________________________________________________________________
Ask HN: Where can I find a primer on how computers boot?
As a developer, I recently encountered challenges with GRUB and
discovered I lacked knowledge about my computer's boot process. I
realized terms like EFI partition, MBR, GRUB, and Bootloader were
unfamiliar to me and many of my colleagues. I'm seeking
introductory and easy-to-understand resources to learn about these
concepts. Any recommendations would be appreciated!
Author : mog_dev
Score : 386 points
Date : 2023-03-20 09:15 UTC (13 hours ago)
| tlb wrote:
| When you turn on the power, a capacitor keeps the 6502's reset
| line low for several milliseconds. This causes the processor to
| fetch the instruction pointer from $FFFE, and execution proceeds
| from there.
|
| Modern PCs are a little more complicated. As a users, it seems to
| me like 20% of that complexity is necessary, such as to configure
| the DRAM (which definitely has to be done before running any user
| code.) The rest is all accidental complexity and compatibility
| with various legacy mistakes.
| oneplane wrote:
| There are some general/generic things that are the same on most
| platforms, but even just within Intel-based desktop computers
| there is a huge variety.
|
| Most links here are very useful describing booting from a
| firmware perspective, CPU perspective, kernel perspective etc.
| but it really depends on what level of detail and level of
| knowledge you want to dig in to.
|
| In general, this is what happens when you plug in the power and
| push the power on button: 1. When power is
| applied, the hardware (so no software or firmware!) makes sure
| that your power supply is healthy and uses stand-by power (which
| is always on) to check if you are pushing the button to turn the
| system on 2. Once you push the button, the circuit
| uses its stand-by power to turn on main power, which in turn does
| some hardware voltage and timing checks of its own, while also
| 'holding' the main processor and some other components in an
| endless reset state 3. If everything checks out, the
| clock runs and the power is good, it releases the infinite reset
| and the processor is allowed to run
|
| Those three steps are very generic, and highly dependant on
| standards like ATX, x86, and whatever component and mainboard
| vendors tack on to it. Diving deeper into this would direct you
| into the area of digital electronics, power management, timers,
| microcontrollers etc. On one hand it might not matter what the
| electrical details are, on the other hand, it is very much a part
| of 'booting' a computer. Primers on how this works haven't been
| shared so far, but I would dig some up.
|
| Onwards: 4. Only at this time does software or
| firmware even come in to play. When a CPU wakes up and comes out
| of reset, there is one pre-defined thing it was designed to do,
| specific to the how the CPU was built; usually it is going to
| have a special location in memory where it always begins to look
| for things to do: the reset vector. Decades ago, this was very
| simple, it would just expect you to connect a memory device like
| a BIOS memory chip to the CPU on specific pins, so that when the
| CPU comes out of reset it always starts at the top of the memory
| contained within that chip and just does what it tells the CPU to
| do. In a way, this could be referred to as IPL: initial program
| load, something that mainframes used to do (and in a way still
| do). You have to tell the processor what to do, otherwise if you
| give it power, it just sits there doing nothing. But you can't
| tell it what to do if there is no program running to tell it to
| do anything, hence the reset vector, a hard-wired first-step that
| it always performs. All you need to do is make sure that you have
| a chip put at the location it expects which contains some CPU
| instructions. 5. Once the CPU has started executing
| whatever it found at the reset vector, it is actually doing
| stuff. In most cases, it starts running BIOS code. Various
| references to information as to how this works are specified in
| the comments, but also on Intel's website, on the UEFI site,
| Coreboot site etc. If you were to dive much deeper into this, you
| would get into Firmware Support Packages, BringUp code, Boot
| ROMs, Management Engines, SMM etc. Keep in mind that all of this
| is specific to x86 PCs, if you were to check out x86 servers, ARM
| systems, PowerPC systems, RISC-V systems, they all do similar
| things, but slightly different. 6. At some point during
| this booting phase, the code the CPU found in the firmware chip
| (be it BIOS or UEFI) will have configured the DRAM so it can use
| the actual memory in the system, and perhaps started up the PCIe
| bus and some other systems like USB, PS/2 etc. This is an
| important moment because until now, the facilities you can use
| are extremely limited, there was no memory, no display, no
| keyboard etc. So after some of those were initialised, the
| primitive code gets switched out for some more advanced code
| which performs tasks you might be familiar with: POST codes,
| devices start showing some activiteit and the system starts
| powering up things like video output and keyboard input. At this
| point the system is also 'advanced' to the point where it can
| load your settings, such as your preferred boot device.
| 7. This is the phase where it gets closer to actually using the
| computer, and most of the events that happened until now might
| collectively take 1 second to complete. A lot of stuff happens in
| very little time! Next, once the system has started itself to the
| point where it can communicate with you, and with peripherals
| like storage devices, it can start thinking about what to do
| next. In the firmware settings you can often specify the boot
| behaviour, for example you might want to have the system boot
| from a specific disk or file on a disk, or you might want to boot
| from a network device, or maybe you want to boot and get dropped
| into a shell. All of those things are essentially the same from
| the firmware's perspective.
|
| The first two of four steps described are often referred to
| 'early firmware' because the things it can do are very limited
| and serve mostly to prepare the system for more advanced
| firmware. Some of the steps are 'secret' and are delivered by the
| CPU manufacturer (like Intel). As a firmware developer you would
| get a bunch of binary data from them, and some instructions where
| you need to let it take control of the entire system for a bit,
| and hope that it gives control back to you. The x86 FSP and
| Coreboot teams deal a lot with this 'first steps to make the
| computer work' stuff, and are good starting points if you wanted
| to dive into that. Other resources are limited, companies like
| Phoenix and Inside and AMI keep their software secret and even if
| they would tell you about it, you'd have to pay a lot of money
| and sign an NDA. Other resources like the EDK and UEFI spec have
| been shared in other comments. 8. Let's assume
| you want to run an operating system; you'd often find that the
| system has a bunch of files on a disk, using a filesystem. BIOS
| and UEFI are generally not very good at reading filesystems. The
| BIOS didn't really read much at all, and UEFI mostly does FAT32,
| with optional drivers for other filesystems. If you're on EXT4
| (Linux) or NTFS (Windows), that's no good, because you can't
| really assume that your firmware can read that at all. What's
| more: the operating systems are likely using kernels that need
| advanced features to run, and a few of those need to be setup
| beforehand. This is where you'd use a boot loader: a program that
| is designed to bridge the gap between a firmware that can't do
| enough, and a kernel that wants too much. 9. A
| bootloader is essentially just a program, so it can be loaded
| into memory, and started. This is a task the firmware can
| perform: BIOS would use a very rigid approach: read the first
| sector of the disk you picked, and hope it contains a boot
| loader. Not very flexible, and also very limited (one bootloader
| per disk for example). UEFI is a bit smarter: as long as you have
| a disk with a partition table of the GPT variant, and it contains
| a partition of the ESP type with a FAT32 filesystem, it can read
| directories and files all day long. So if you tell UEFI "run from
| disk 1 using file /efi/boot/banana.efi", it will go to the disk,
| go through the efi and boot directories, and load and execute the
| banana.efi program. It doesn't even really 'boot' anything, since
| an EFI program can be anything (it doesn't have to be a
| bootloader). 10. At this stage the bootloader of
| choice has started, and the firmware can mostly be ignored. The
| bootloader is likely smart enough to load its own settings, for
| example what operating systems it knows about and on what disks
| they can be found. It also prepares things for the operating
| system, like what devices are known ahead of time, and where in
| the computers memory the kernel is going to be loaded.
|
| The three steps above are mainly about control handoff and boot
| discovery. There are numerous standards and references on this,
| like the multiboot specification, but most of those (including
| GRUB) have been commented already. This is also the phase in
| which the computer does more or less what you see and use on a
| daily basis. Since there are many open source elements available,
| it is also much more open to exploration, and less 'hidden away'
| in proprietary secret documents.
|
| Nearly all of this follows a relatively simple pattern: start
| small and primitive, do a dedicated task, start the next phase
| which is slightly bigger and slightly more advanced. Almost all
| steps in this pattern were built from manual processes; the
| farther back you go in time with older and more primitive
| computers, the more steps you'd actually be doing yourself. This
| can be pretty helpful to figure out why those processes or phases
| exist in the first place. A "manual" startup on a very old
| machine: https://youtu.be/PwftXqJu8hs?t=244 (edit: I made a
| mistake, this is not the bootup, but does include the line
| "powered on, because there is nothing to boot up" -- the machine
| essentially powers up and does nothing, no reset vector, heck,
| not even an actual reset! I'll try to find the reference to the
| actual bootup)
| vsupalov wrote:
| I wondered about an adjacent topic recently ("what IS the kernel
| actually") and found out that a great way to understand it, would
| be to start with the boot process. Once I felt less lost, I tried
| to capture my learnings [1]
|
| The text is not edited yet, but as I had the benefit of learning
| about it with fresh eyes, it should be very approachable (and
| hopefully accurate enough)
|
| [1] https://vsupalov.com/demystifying-kernel/
| MuffinFlavored wrote:
| > It's firmware time. Your machine has a motherboard, there is
| a chip on it which has had BIOS (Basic Input/Output System)
| flashed on it.
|
| I wonder if it's worth it to call out the "processor microcode"
| firmware? Any cool startup things happen at that low level as
| well?
| vsupalov wrote:
| Way out of my depth! However, the stuff I saw the folks from
| Oxide [1] talk about comes to mind. Hardware stuff with Rust.
| If you want to look at a specific project, Hubris [2] is a
| kernel for "deeply-embedded" systems.
|
| [1] https://oxide.computer/blog/building-big-systems-with-
| remote...
|
| [2] https://github.com/oxidecomputer/hubris
| bcantrill wrote:
| Hubris is a part of booting on our systems in that it
| executes on both our root of trust and on our service
| processor -- but both of these execute long before the host
| CPU. To understand how we boot our host CPU (AMD Milan) on
| an Oxide compute sled, see my OSFC 2022 talk[0][1], as well
| as the episode of our Oxide and Friends podcast on holistic
| boot.[2]
|
| [0] https://speakerdeck.com/bcantrill/i-have-come-to-bury-
| the-bi...
|
| [1] https://www.osfc.io/2022/talks/i-have-come-to-bury-the-
| bios-...
|
| [2] https://share.transistor.fm/s/24beb248
| pjc50 wrote:
| The microcode is mostly irrelevant, and is also opaque. It
| might be doing something but Intel won't give you more
| details beyond the programmer's reference model.
|
| "Microcode" generally looks more like expanding a CISC
| instruction into other instructions, rather than something
| that looks like a program.
|
| Extremely low level paper: https://www.researchgate.net/publi
| cation/295010710_Booting_a... ; good to confirm that Intel
| systems still boot at 0xFFFFFF0 in real (16-bit!) mode. It
| also points out that microcode updates are applied _after_
| execution has started.
|
| There is also the fun of "cache as RAM"; it's usually quite a
| time consuming operation to get the DRAM controller up and
| running and "trained" to the particular signal properties of
| the motherboard, so the early boot phase has no RAM.
| blep-arsh wrote:
| There's indeed a lot of firmware (not just microcode). A
| bunch of code is signature-checked and executed from the boot
| ROM and firmware flash in order to bring up the CPU and its
| various auxiliary cores (starting with the security
| processor/management engine nowadays), the PCH and the DRAM
| controller - all before the CPU can start executing any
| bootloader code from RAM.
| junon wrote:
| https://osdev.org has a lot of resources about this.
|
| Your computer and all of its hardware are built on standards. For
| the most part, they are adhered to by hardware and BIOS
| manufacturers. GRUB _et al_ are just abstracting those away
| because they 're obtuse and not very ergonomic to work with from
| day to day.
| klodolph wrote:
| I also recommend the OSDev Wiki. It also includes information
| you'd need to write your own bootloader, if that's what you
| want to do.
| rnd0 wrote:
| Isn't a lot of the material there deliberately obscufated
| and/or wrong but not corrected for the sake of making people
| figure it out themselves?
| peter_d_sherman wrote:
| The following is a pretty decent historical page about the
| pre-(U)EFI MBR (Master Boot Record) boot process:
|
| https://thestarman.pcministry.com/asm/mbr/STDMBR.htm
|
| Note that EFI/UEFI -- occurred much later in time than MBR...
|
| The MBR boot process is also called "Legacy Boot" -- and is
| emulated on (U)EFI -- although it may not show as an option on
| some (U)EFI BIOS'es if the option is turned off...
|
| Related: https://en.wikipedia.org/wiki/Master_boot_record
|
| You might also wish to check out some emulators, most notably
| Bochs (https://bochs.sourceforge.io/) and QEMU
| (https://www.qemu.org/) because they simulate the boot process,
| and if you're in their debuggers, you should be able to inspect
| that process step by step -- but also more generally emulators
| for other machines/platforms/architectures
| (https://en.wikipedia.org/wiki/List_of_computer_system_emulat...)
| because in general, most of those emulators should realistically
| simulate the given machine/platform/architecture's boot
| process...
|
| The basic theory of booting is that when a system starts, it
| contains a little bit of persistent memory (BIOS ROM, EPROM?)
| that contains a little bit of code, which is just enough code to
| load the data of the first block/sector of the hard disk (or
| other persistent storage boot device) to a specific address in
| memory as code -- and jump to it.
|
| This data on the first block/sector -- is a small bit of machine
| code -- which although fairly stupid -- knows enough about the
| system to load the next N blocks/sectors of the hard disk (or
| other storage device) -- again into memory at a specific address
| -- and then jump (transfer control) to it.
|
| This pattern may repeat several times, for example, GRUB's first
| bootsector then loads an intermediate length program (a
| "chainloader") from the next N contiguous blocks/sectors of the
| storage device, this chainloader knows more about the hardware
| and filesystems than the initial boot block did, and then it
| proceeds to do a yet longer/more complex load of the main
| Operating System into memory.
|
| During the final load, the main Operating System might be on
| discontiguous blocks/sectors and those blocks/sectors may be part
| of a filesystem. But that doesn't have to be the case.
|
| But whatever the case, the final load (or perhaps "bootstrap
| phase") is usually more complex than reading continuous
| blocks/sectors into memory and subsequently jumping to its start
| address in memory -- but not always...
|
| Another way to think about it is that a 512 byte program (the
| bootsector) is loaded that then loads a 32K (let's say) program
| (the chainloader on N contiguous blocks/sectors) which then loads
| the multi-megabyte (or multi-gigabyte!) OS from multiple files
| from multiple file systems from multiple discontinuous
| blocks/sectors on the physical device (let's say, for example...)
|
| Simple Program (loads) -> More Complex Program (loads) -> Most
| Complex Program (OS)...
|
| Anyway, there's your basic theory...
|
| Good luck!
| fuzzfactor wrote:
| I also recommend the Starman for good documentation over the
| decades.
|
| Basically you're supposed to know BIOS/MBR booting quite well
| in order to make the best progress on UEFI/GPT.
|
| Originally upon powerup the BIOS would run from ROM and then
| boot control to a program known as the Master Boot Record which
| would reside on the first 440 bytes of sector 0 of the HDD. The
| partition table took up the final bytes of sector 0. The MBR
| checked which of the 4 possible primary partitions was flagged
| as active, and then booted control to the Volume Boot Record
| (Boot Sector) residing at the first sector of that particular
| partition which would then (usually, but specifically according
| to BOOT.INI on Windows [BCD today] or GRUB configs for Linux)
| load the OS contained within that partition. Usually by reading
| files from the filesystem beyond that point, but capable of
| being chained directly to other executable sectors.
|
| Starman can be a very good resource on underlying structures:
|
| https://thestarman.pcministry.com/asm/mbr/
|
| He's got lots more pages like this, and references too.
|
| Now, straight to modern UEFI:
|
| The MBR code itself is best not there on sector 0, just an
| imitation partition table in the final few bytes still exists
| which is considered "protective" and keeps BIOS PC's from
| recognizing the HDD since they can not handle partitions layed
| out by the GPT regime.
|
| Sector 1 designates GPT drive parameters, and sector 2 begins
| the defining data for each partition, which by default extends
| to sector 33 so you have room to define many more than 4
| partitions, and any of them can be booted to from a single EFI
| folder on a hidden ESP partition, depending mainly on the
| config files in the EFI folder. There are backups of these
| defining sectors (not the EFI folder itself) in the final area
| of the HDD.
|
| In the ESP partition which is often expected by MB UEFI to
| carry a specific ESP GUID (but often works just fine after
| changing to a "regular" GUID so you can unhide it for
| maintenance), the UEFI expects to find it formatted as FAT32,
| and there behold a recognizable EFI/BOOT folder containing the
| elusive BOOTX64.EFI file. That's the only thing normally found
| in the EFI/BOOT folder except for systems having bootability by
| the rare remaining 32-bit UEFI firmwares, then you might see
| BOOTIA32.EFI. Lots of times it will do just fine having an EFI
| folder on an NTFS partition too.
|
| UEFI 64-bit firmware just runs whatever file is currently named
| BOOTX64.EFI in the BOOT folder. Or falls back to secondary
| alternatives if not present.
|
| When you install an OS like Windows it adds an EFI/MICROSOFT
| folder, but each time you add a different version of Windows to
| a different partition, you do not add additional Microsoft
| folders to EFI, instead you add additional boot entries to the
| default boot menu already present in the Microsoft BCD file
| there. The OS installation procedure overwrites the existing
| BOOTX64.EFI file with one directing the boot process to
| continue from the Microsoft folder after that point, according
| to the NT6 boot menu you have in place at the time.
|
| Installing Linux creates an EFI/ubuntu folder for example,
| containing most necessary GRUB elements, while overwriting
| BOOTX64.EFI to direct the boot process to the ubuntu folder and
| use the GRUB bootmenu accessible from there. Each distinct
| distro will add its own folder like EFI/debian/, EFI/fedora,
| with the most recently added claiming the most coveted
| BOOTX64.EFI prize.
|
| Ideally, you install all the Windows versions you want into
| their designated NTFS volumes, which naturally builds up your
| NT6 bootmenu as you go. You will best do some BCDEDITing as you
| go just to keep the Descriptions in the bootmenu straight as to
| which entry goes to which Windows version on which partition.
|
| Ideally again, your final Linux install will be a distro that
| automatically detects all other Linux installations residing on
| their EXT-formatted partitions, while also correctly detecting
| & adding a bootentry for Windows, so GRUB can take you to your
| already-established NT6 bootmenu when you want.
|
| In the EFI/Microsoft folder you will find an os-specific file
| in EFI/Microsoft/BOOT named BOOTMGFW.EFI. Not to be confused
| with the basic EFI/BOOT/BOOTX64.EFI.
|
| These are EFI executables, either one will boot the PC from
| UEFI directly if you drop down to the UEFI Shell preboot using
| your MainBoard tools. Navigate to fsx:EFI/Microsoft/BOOT and
| run bootmgfw.efi to boot Windows (even without an EFI/BOOT
| folder present), or navigate to the main EFI/BOOT folder and
| run the latest BOOTX64.EFI that your final OS install has
| overwritten for you accordingly. Where "fsx:" is the filesystem
| which contains the EFI folder as detected by the UEFI firmware,
| often fs0. You can even find grubx64.efi files for Linux.
|
| Not only OS installs, but things like GRUB updates and BCDBOOT
| commands can overwrite BOOTX64.EFI.
|
| Finally, once different OS's are properly installed in a
| traditional self-contained way on each of their own partitions,
| you should be able to craft any number of alternative EFI
| folders or contents such that any or all of the OS choices are
| presented to you on a bootmenu upon powerup, booting to your
| established choice of default (or fallback) in case of
| unattended powerup. GRUB can still boot Windows but Windows NT6
| bootloader can no longer boot Linux when using GPT.
|
| But you can even still make a boot floppy you can use to load
| OS's present on the HDD without having any boot files on the
| HDD at all.
|
| UEFI can also sometimes do well when it finds a good
| recognizable EFI folder on an MBR-layout HDD.
| jaclaz wrote:
| >The MBR checked which of the 4 possible primary partitions
| was flagged as active, and then booted control to the Volume
| Boot Record (Boot Sector) residing at the first sector of
| that particular partition which would then (usually, but
| specifically according to BOOT.INI on Windows [BCD today] or
| GRUB configs for Linux) load the OS contained within that
| partition. Usually by reading files from the filesystem
| beyond that point, but capable of being chained directly to
| other executable sectors.
|
| You forgot the Windows boot manager, on Windows NT the chain
| is:
|
| Up to XP:
|
| BIOS->MBR->PBR of active partition->NTLDR
| (+NTDETECT.COM)->BOOT.INI->Choices->OS
|
| Vista and later:
|
| BIOS->MBR->PBR of active
| partition->BOOTMGR->\boot\BCD->Choices->OS
|
| The MBR only passes control to the PBR and the PBR only loads
| the Windows boot manager, it is this latter that does all the
| actual OS loading and that reads BOOT.INI and \boot\BCD
| settings and user choices.
| fuzzfactor wrote:
| Good to see your message jaclaz.
|
| You're another one like Starman who can fill in details
| like no-one else, in case people are not aware. Please
| kindly post a link to the message board where your most
| valuable information still can be found!
|
| What I'm seeing now is when Windows 8+ is installed to an
| MBR partition, and boots normally from a plain NT6 BOOT
| folder using BOOT\BCD in legacy CSM mode no differently
| than Vista (whether the boot folder is on a separate hidden
| volume or the same partition as Windows itself), the same
| Windows files on that partition can also boot using UEFI if
| available, from an alternative EFI\BOOT folder (plus
| accompanying EFI\Microsoft\BOOT\BCD) on the same or similar
| media.
|
| In the first case with BIOS it ends up running winload.exe,
| but under UEFI the firmware runs winload.efi instead, since
| both the "DOS" executable and the EFI executable are both
| present in the Windows\System32 folder, you have your
| choice here too.
| jaclaz wrote:
| Thanks for the comparison, though of course the Starman's
| site is a much better source than my scattered posts,
| mainly on:
|
| https://msfn.org/board/
|
| and:
|
| http://reboot.pro/
|
| (on this latter under the nick of Wonko the Sane)
|
| About your Windows 8+ report, yes, what you describe is
| perfectly normal, what many people tend to forget is that
| UEFI does not necessarily means GPT, a UEFI can access
| "MBR style disks" just fine (the opposite, BIOS
| acccessing a GPT disk needs special provisions, so-called
| hybrid MBR's and "special" code in them).
|
| You evidenced however yet another aspect of more recent
| NT Windows booting, until XP NTLDR was both the boot
| manager and the OS loader, now BOOTMGR (and bootmgr.efi)
| is just the boot manager and it chainloads a separate OS
| loader, the WINLOAD.EXE (or winload.efi).
| sebow wrote:
| This thread almost has all the good resources out there, however
| a good book I haven't seen mentioned is "Hands-on Booting"
| [ISBN-10: 1484258894]. It does not go into deep technicalities
| but if you want a good overall picture(how the microsoft
| bootloader works and how especially grub works) it's definitely
| beginner-friendly in my opinion.
| saidinesh5 wrote:
| Arch wiki is a really good place i learned this stuff from:
|
| https://wiki.archlinux.org/title/Arch_boot_process
|
| P.S I use Arch btw. ;)
| kfajdsl wrote:
| 2nd this, this is where I learned how computers boot up.
| iillexial wrote:
| I use Arch too.
| themaninthedark wrote:
| How do you know if a person uses Linux? >They will tell you.
|
| I use Mint :p
| IE6 wrote:
| I use ChromeOS >:)
| lom wrote:
| +1, this is also where I learned most of my knowledge
| Nextgrid wrote:
| OSDev might be a good resource:
| https://wiki.osdev.org/Expanded_Main_Page
| jiveturkey wrote:
| Plenty of good links here already. I'll just add that booting is
| enormously complex, and starts well before any of the dozen
| comments so far. I skimmed many of them (not all) and they all
| tell pretty much the same story whether on real hardware or
| virtual.
|
| But modern real hardware first boots a Management Engine (or BMC,
| or T2, or similar), which prepares things and starts subsystems
| like hardware controllers, before releasing the CPU where the
| "OS" finally boots, blissfully unaware that it is running on an
| abstraction.
|
| Watch https://www.youtube.com/watch?v=36myc8wQhLo from about
| 10:00 to about 20:00 for a decent super high level overview.
|
| These two are also good for a deeper understanding.
|
| https://i.blackhat.com/USA-19/Thursday/us-19-Davidov-Inside-...
|
| https://i.blackhat.com/USA-19/Thursday/us-19-Krstic-Behind-T...
|
| They both cover UEFI, even though the focus is on T2.
|
| I know that's well beyond what you were asking for, but the boot
| process goes well beyond GRUB.
| jeffbee wrote:
| So much this. Every part of a modern computer contains unseen,
| non-architectural processors of various kinds. When the main
| architectural CPU executes the first instruction, that is the
| last in a sequence of miracles. The only way to be familiar
| with the whole process is to build a computer from scratch.
| rustybolt wrote:
| OS dev forums and wiki have extensive information on, well,
| almost everything computer-related.
|
| https://wiki.osdev.org/Expanded_Main_Page
| viraptor wrote:
| Once you're done with the easy-to-understand and are hungry for
| more, check out https://wiki.osdev.org/Expanded_Main_Page
|
| You'll learn everything you wanted about booting and much more.
| swalsh wrote:
| Installing Arch is a great way to learn. As you go step by step
| you'll learn the parts of the OS, and kind kind of grok why
| they're there (and when you'll inevitably skip a step, you'll
| learn more). It's also a great system to have total control over
| your OS. And since you built it yourself you can fix it yourself.
| joebergeron wrote:
| I wrote an article on writing a tiny pseudo-bootloader a while
| back[0] that got a bit of traction on HN at the time; you may
| enjoy it as an accesible and fun stepping stone into the space.
| There's an accompanying GH repo[1] with all the resources you
| need to run it yourself :)
|
| [0]: https://www.joe-
| bergeron.com/posts/Writing%20a%20Tiny%20x86%...
|
| [1]: https://github.com/Jophish/tiny-bootstrap
| mronetwo wrote:
| https://handmade.network/p/308/lil-uefi/ - this might be of
| interest to you
| ignoramous wrote:
| The gist is (which doesn't hold universally true), when the CPU
| is powered up, it initializes its internal RAM, loads in a pre-
| boot ROM, executes it. pre-boot ROM's job is to initialize
| whatever it has to, to bring in the _initial bootloader_ which
| initializes RAM, finds a suitable bootloader program and loads it
| in. This program, also called _second-stage bootloader_ , is an
| Operating System in its own right. It can understand filesystems,
| explore attached hardware, speak multiple network protocols,
| initialize devices as needed. The _second-stage bootloader_
| (there may also be a _third-stage_ ) usually then boots into a
| Kernel. The Linux Kernel invokes the init process and things
| proceed into the userspace from there.
|
| At each of those stages, the bootloader images, the kernel image,
| the device trees, and the disk / filesystem are verified
| ("measured") for integrity eventually anchored on a "hardware
| root-of-trust".
|
| If you're a visual learner:
|
| How TI Beagleboard boots (33m):
| https://www.youtube.com/watch?v=DV5S_ZSdK0s /
| https://ghostarchive.org/varchive/DV5S_ZSdK0s
|
| How ARM Cortex M boots (9m):
| https://www.youtube.com/watch?v=3brOzLJmeek /
| https://ghostarchive.org/varchive/3brOzLJmeek
|
| Bootloader 101 (embedded devices; 38m):
| https://www.youtube.com/watch?v=UvFG76qM6co /
| https://ghostarchive.org/varchive/UvFG76qM6co
|
| Measured Boot (39m): https://www.youtube.com/watch?v=EzSkU3Oecuw
| / https://ghostarchive.org/varchive/EzSkU3Oecuw
|
| Coreboot (38m): https://www.youtube.com/watch?v=iffTJ1vPCSo /
| https://ghostarchive.org/varchive/iffTJ1vPCSo
|
| U-Boot (1h 25m): https://www.youtube.com/watch?v=INWghYZH3hI
|
| UEFI for U-Boot (38m):
| https://www.youtube.com/watch?v=VnsF3uRZzNk /
| https://ghostarchive.org/varchive/VnsF3uRZzNk
|
| Device Trees (43m): https://www.youtube.com/watch?v=Nz6aBffv-Ek /
| https://ghostarchive.org/varchive/Nz6aBffv-Ek
|
| ARM Trust Zone (46m): https://www.youtube.com/watch?v=q32BEMMxmfw
| hoherd wrote:
| Here's a write up I found recently.
|
| > This is a discussion of what happens when a CPU chip starts. It
| may be thought of as what happens when a whole computer starts,
| since the CPU is the center of the computer and the place where
| the action begins.
|
| https://lateblt.tripod.com/bit68.txt
| mallikarjun7 wrote:
| https://www.youtube.com/watch?v=1rnA6wpF0o4&list=PLHh55M_Kq4...
| scott_s wrote:
| If you're interested in a code-walkthrough of the Linux boot
| process in 2004: https://www.scott-a-s.com/the-linux-boot-
| process-of-2004/
| vishnugupta wrote:
| Perhaps it's easier to pick an OS and understand its boot
| process?
|
| And since no one's mentioned let me say that TLDP is a
| comprehensive, understandable, and the authoritative source on
| all things Linux.
|
| That said, here's two documents from TLDP.
|
| 1. [Boot process, Init and shutdown] https://tldp.org/LDP/intro-
| linux/html/sect_04_02.html
|
| 2. [The boot process in closer look]
| https://tldp.org/LDP/sag/html/boot-process.html
| BirAdam wrote:
| Decent explanation here, if you scroll a bit:
| https://www.abortretry.fail/p/an-introduction-to-linux-part-...
| tyingq wrote:
| It might be easier to study something other than an x86 PC, at
| least to start with. There's so much legacy in x86 that's
| distracting.
|
| This post[1] has an animated image that's pretty helpful:
| https://embeddedinn.xyz/images/posts/rvLinuxQemuBoot/bootAni...
|
| It's of course specific to one platform, but the visualization of
| building up a structures in memory with the right offsets/order
| and jumping in is pretty typical.
|
| [1] https://embeddedinn.xyz/articles/tutorial/RISCV-
| Uncovering-t...
| photochemsyn wrote:
| This one looks pretty good, for all the terms you list. It's
| refreshingly free of opinionating and just covers the
| technicalities and the history in a straightforward manner, and
| has links to good wiki overviews of EFI etc, compares the BIOS vs
| UEFI boot processes, and so on:
|
| https://en.wikipedia.org/wiki/GNU_GRUB
|
| Incidentally I discovered you can learn a fair amount about the
| boot process by making bad edits to your /etc/fstab file on a
| Linux system :) e.g.
|
| https://unix.stackexchange.com/questions/107828/how-is-etc-f...
|
| Also this central page for different bootloaders for a variety of
| systems, is kind of interesting, there's many different
| approaches.
|
| https://en.wikipedia.org/wiki/Comparison_of_bootloaders
| never_inline wrote:
| Does anyone have a list of these HN ask-resource threads?
| mariojv wrote:
| Someone else mentioned the Arch wiki under "Arch boot process"
| specifically. Since you mentioned GRUB, though, also check out
| the Arch wiki's article on GRUB:
| https://wiki.archlinux.org/title/GRUB
|
| I found it invaluable when debugging some issues a few years
| back. Try unpacking the initramfs and reading the scripts
| contained in there, too:
| https://wiki.archlinux.org/title/Arch_boot_process#initramfs
| ChrisMarshallNY wrote:
| I think Eclectic Light had something pretty comprehensive on Mac
| booting.
|
| https://eclecticlight.co
| mindcrime wrote:
| In addition to everything else mentioned here, you might find
| some useful material in the Grub documentation:
|
| https://www.gnu.org/software/grub/manual/grub/grub.html
|
| and/or in the Linux From Scratch book:
|
| https://www.linuxfromscratch.org/lfs/read.html
| mackid wrote:
| A good summary of UEFI booting:
|
| https://depletionmode.com/uefi-boot.html
| quux wrote:
| Andreas Kling did an awesome video about the Serenity OS boot
| process here: https://youtu.be/NpcGMuI7hxk
| uhtred wrote:
| How Linux Works is pretty decent book.
| https://nostarch.com/howlinuxworks3
| zcw100 wrote:
| First, start by mentally preparing yourself that this mess is the
| foundation of our technologically based society.
| Eyas wrote:
| Most of the links here help with your immediate questions, but if
| starting with base concepts and first principles is appealing to
| you, I have to recommend:
|
| - the MIT Operating Systems Engineering OCW course [^1]
|
| - specifically, Lecture #2 [^2] which describes the bootloader
| for 'xv6', a reimplementation of Unix v6, which does a great job
| showing what a solid OS with all the basics look like
|
| [^1]: https://ocw.mit.edu/courses/6-828-operating-system-
| engineeri...
|
| [^2]: https://ocw.mit.edu/courses/6-828-operating-system-
| engineeri...
| ddoolin wrote:
| This is cool, although I wish (and maybe I'm missing them) that
| there were lecture videos or recordings to go along with this.
| It seems like good material with not a lot of explanation.
|
| E: Sibling comment posted updated course with videos:
| https://pdos.csail.mit.edu/6.S081/2020/schedule.html
| kersplody wrote:
| [^1]: http://3zanders.co.uk/2017/10/13/writing-a-bootloader/
| [^2]: https://wiki.osdev.org/Main_Page
| rustypotato wrote:
| I'd also recommend this. There's even more information on a
| newer version of this course's site, 6.S081 [^1].
|
| [^1]: https://pdos.csail.mit.edu/6.S081/2020/schedule.html
| markus_zhang wrote:
| Ah I like that "for amusement" video.
| firstfewshells wrote:
| chatGPT
| temptemptemp111 wrote:
| [dead]
| hayst4ck wrote:
| searching on GitHub specifically would probably have the best
| stuff (I didn't look at these very hard):
|
| https://github.com/nu11secur1ty/All-Stages-of-Linux-Booting-...
|
| https://github.com/0xAX/linux-insides/blob/master/Booting/li...
| anenefan wrote:
| I would say start at wikipedia looking up terms and consider a
| couple of weeks to grasp the simple notions of what's involved.
| Start with MBR, limits on old drives, dig into various tricks to
| get around limitations ... not really relevant present day but
| it's good insight.
|
| Problems with bootlaoders are generally specific to the system
| it's on and how the drive(s) is / are set up.
|
| https://www.linux.org/threads/linux-bootloaders.8943/
|
| https://www.tecmint.com/best-linux-boot-loaders/
|
| I think even old versions of Scott Mueller's "Upgrading and
| Repairing Pcs" might be of use to you. Digital versions could be
| found at archive.org.
| fuzzfactor wrote:
| Nothing beats experimentation when you really need to find out
| for yourself and documentation is not ideal to begin with.
| westurner wrote:
| Booting process of Linux:
| https://en.wikipedia.org/wiki/Booting_process_of_Linux
|
| Booting process of Windows NT since Vista:
| https://en.wikipedia.org/wiki/Booting_process_of_Windows_NT_...
|
| UEFI > Secure Booting, Boot Stages:
| https://en.wikipedia.org/wiki/UEFI#Boot_stages
|
| The _EFI system partition_ is conventionally /boot/efi on a
| Linux system; and there's a _signed_ "shim loader" that GRUB
| launches, which JMP- launches the kernel+initrd after loading the
| initrd into RAM (a "RAM drive") and mounting it as the initial
| root filesystem /, which is pivot_root'd away from after the copy
| of /sbin/init (systemd) mounts the actual root fs and launches
| all the services according to the Systemd unit files in order
| according to a topological sort given their dependency edges:
| https://en.wikipedia.org/wiki/EFI_system_partition
|
| Runlevels: https://en.wikipedia.org/wiki/Runlevel
|
| runlevel 5 is runlevel 3 (multi-user with networking) + GUI. On a
| gnome system, GDM is the GUI process that is launched. GDM
| launches the user's Gnome session upon successful login.
| `systemctl restart gdm` restarts the GDM Gnome Display Manager
| "greeter" login screen, which runs basically runs ~startx after
| `bash --login`. Systemd maps the numbered runlevels to groups of
| unit files to launch: telinit 6 # reboot
| telinit 3 # kill -15 GDM and all logged in *GUI* sessions
|
| You can pass a runlevel number as a kernel parameter by editing
| the GRUB menu item by pressing 'e' if there's not a GRUB password
| set; just the number '3' will cause the machine to skip starting
| the login greeter (which may be what's necessary to troubleshoot
| GPU issues). The word 'rescue' as a kernel parameter launches
| single-user mode, and may be what is necessary to rescue a system
| failing to boot. You may be able to `telinit 5` from the rescue
| runlevel, or it may be best to reboot.
| fuzzfactor wrote:
| >The EFI system partition is conventionally /boot/efi on a
| Linux system;
|
| Still often _mounted_ as /boot/efi but physically residing on
| the root of the hidden ESP volume as EFI/boot & EFI/ubuntu, for
| distros like ubuntu.
|
| So when mounted you have something like
| /boot/efi/boot/bootx64.efi and boot/efi/ubuntu/grub etc. if
| visible and accessible.
| markus_zhang wrote:
| As a side note, I found the concept of BIOS/UEFI/Firmware virus
| fascinating. It might be useful to read a few analysis about
| them. Of course I understand little about the topic as I don't
| work in low level.
| low_common wrote:
| Ignore all these replies and just use ChatGPT-4. Super helpful at
| explaining things.
| marl0 wrote:
| This is the way.
| agingllama wrote:
| I wonder if learning how a more simple microcontroller boots
| might help you.
|
| I had the same questions as you around a year ago. As other
| commenters mentioned, modern hardware is incredibly complex and
| needs to go through many, many steps before it can boot a kernel.
| Modern CPUs have a bunch of legacy requirements that still exist
| to keep backward compatibility with older hardware.
|
| One good example of these legacy requirements is the A20 line on
| x86 [1]. A20 can be considered a boolean flag that determines
| whether the CPU can access more than 1MB of RAM. The A20 flag was
| introduced in the Intel 286 because its predecessor, the 8086 had
| a limit of 1MB of RAM. The 286 needed to access 16MB of RAM, so
| Intel introduced the A20 flag that determines whether the CPU can
| access memory past 1MB. The A20 line still exists in x86_64 CPUs
| today, so it needs to be enabled before the entire RAM address
| space is used.
|
| There are a TON of legacy requirements like the A20 line on
| modern CPUs (especially x86_64). It can make it very difficult to
| figure out what is going on in the boot process. For me, I found
| it much easier to start learning how simple pieces of hardware
| works before moving on to x86. The boot process for Raspberry
| Pi's RP2040 microcontroller is explained in the datasheet [2] and
| I found it to be a great resource to figure out which hardware
| needs to be initialized and what that actually means.
|
| Just like x86, microcontrollers have many tasks they need to
| complete before loading the main program. But these steps are
| much less ambiguous and are described very well in the RP2040
| datasheet. It lists every step that it goes through before the
| main program starts (including stuff like initializing the clocks
| to specified speeds). After I felt like I thoroughly understood
| how the RP2040 boots, it became much easier to understand why the
| bootloader needs to complete certain tasks.
|
| Not sure if you are familiar with assembly, but you can also
| checkout part of the RP2040's bootloader here [3]. There are a
| few different bootloaders for the RP2040, the one I linked is the
| one referenced in the datasheet. Its purpose is to load the main
| program from a specified location in the flash memory chip.
|
| Remember that the boot process on x86 processors has been
| changing for around 40 years, so expect it to take a while before
| you feel more comfortable with the terms used in bootloading and
| hardware. I've been learning this stuff for around a year and
| still feel like I only understand a fraction of the x86 boot
| process. But I find it so fascinating that I can't help but want
| to learn more of its complexity.
|
| I love talking about hardware and the boot process, so feel free
| to let me know if you have any more questions! :)
|
| [1] https://en.wikipedia.org/wiki/A20_line
|
| [2]
| https://datasheets.raspberrypi.com/rp2040/rp2040-datasheet.p...
|
| [3] https://github.com/rp-
| rs/rp2040-boot2/blob/main/src/boot2_w2...
| cpach wrote:
| For posterity, some *BSD links that could be useful to gain
| understanding:
|
| https://docs.freebsd.org/en/books/handbook/boot/
|
| https://man.openbsd.org/boot.8
|
| https://man.netbsd.org/boot.8
| rurban wrote:
| Quora is also very useful, with lots of good answers (and cat
| pics): https://www.quora.com/What-happens-during-the-booting-of-
| a-c...
| mardiyah wrote:
| Every Linux has
|
| Dig in on each of their home/official site e.g:
| https://www.gnu.org/software/grub/manual/grub/grub.pdf
|
| most say Gentoo or Funtoo (its improvement) excel in docs. among
| them
| easterncalculus wrote:
| I honestly had a very good exchange about this exact topic with
| ChatGPT not too long ago. It was able to draw connections between
| the different stages that blog posts/wikis about each often do
| not. Specifically I was asking about the Linux boot process and
| it was able to go into enough detail for me to Google for the
| rest. Hope that helps.
| soheil wrote:
| I'd be very interested to know what Ask HN posts will actually
| be answered by HN better than by ChatGPT. Maybe those will be
| things that we excel at.
| gjsman-1000 wrote:
| Also, heads up: GPT-4 is night and day smarter than regular
| ChatGPT and totally worth the subscription price if using it
| for general research.
| oldnewthing wrote:
| The 80386 System Software Writer's Guide.
|
| http://www.bitsavers.org/components/intel/80386/231499-001_8...
| 0xcde4c3db wrote:
| Jonathan de Boyne Pollard wrote some reasonably concise
| descriptions of this and related issues. His overall "FGA" index
| is at [1], and you might be more specifically interested in [2]
| and [3].
|
| [1] http://jdebp.uk./FGA/
|
| [2] http://jdebp.uk./FGA/pcat-boot-process.html
|
| [3] http://jdebp.uk./FGA/efi-boot-process.html
| thomastjeffery wrote:
| Most of the complexity is from when UEFI and GPT were introduced.
| Let me define all the terms for you, then the process that ties
| them together:
|
| A bootloader is what loads your OS. Alternatively, it can load
| another bootloader: for example, GRUB chainloading the windows
| bootloader. Even if a bootloader can support both BIOS and UEFI,
| it must install a matching configuration, because the UEFI boot
| process is structured very differently from the BIOS boot
| process. The bootloader also sets Linux kernel flags.
|
| UEFI replaces BIOS. A UEFI will probably emulate the BIOS boot
| methodology with "legacy mode". These are basically the OS that
| loads your bootloader. One of the two lives on your motherboard
| and configures your hardware.
|
| GPT replaces MBR. They are both partitioning schemes: they
| organize the areas in a storage media where a filesystem can
| exist. MBR has an overcomplicated partitioning scheme, and a few
| reserved sectors in the front that tell the BIOS where to find a
| bootloader. GPT has a simple partitioning scheme, and a special
| partition for the bootloader. BIOS does not support GPT. UEFI
| probably supports MBR with its legacy mode.
|
| That special GPT bootloader partition is called the "EFI System
| Partition" (ESP). A partition is "flagged" as ESP in the GPT
| partition table itself. It is expected that an ESP partition will
| be formatted with the fat32 filesystem.
|
| When using BIOS, Windows is infamous for overwriting the MBR to
| point to its bootloader.
|
| When using UEFI, Windows install media autogenerates the ESP
| partition alongside multiple backup partitions: if there is an
| existing ESP partition _on any present internal storage media_ ,
| the windows installer will simply use that one, even if there
| isn't enough space. If there are multiple ESP partitions present,
| you cannot tell the windows installer which one to install to.
|
| Since the MBR does most of the work, a BIOS only needs to decide
| which disk to boot. Usually it saves the order of disks that it
| will try. Some older BIOSes don't support booting USB media, but
| you can potentially work around this by loading another
| bootloader from a floppy/CD and chainloading.
|
| UEFI saves a list of boot entries. It can autogenerate them by
| scanning storage media, like USB install disks. An OS can save or
| edit these entries directly, so long as it has been booted with
| permission (some laptop motherboards overcomplicate this). The
| Linux kernel can be compiled with a minimal EFIstub bootloader.
| In this case, the kernel flags are saved in the UEFI boot entry
| on the motherboard.
|
| The boot sequence looks like this:
|
| 1. The BIOS or UEFI initializes hardware, and lets the user
| interrupt to edit BIOS/UEFI settings.
|
| 2a. UEFI has a saved list of boot entries. The default entry is
| loaded, or one is chosen from the list in the UEFI settings. The
| boot entry loads a bootloader, a Linux kernel EFIstub, or some
| arbitrary UEFI program like memtest86.
|
| 2b. BIOS must search all present disks for a bootloader. This is
| expected to be found in the first few sectors of the MBR
| partition table. The saved order of potentially bootable storage
| media is followed: the first bootloader found is loaded.
|
| 3. The bootloader does what it is configured to do. GRUB will
| usually present a list of entries configured by the OS it was
| installed with. The bootloader loads an OS.
|
| There are some more complexities:
|
| Libreboot/Coreboot is both a firmware (like UEFI) _and_ a
| bootloader. It 's made to skip as much hardware initialization as
| possible, and go straight to the OS _fast_.
|
| Apple implements their own proprietary "EFI", which is like UEFI,
| but less compatible, and has a very minimal pre-OS UI. It (at my
| most recent attempts nearly a decade ago) refuses to boot USB
| media. The bootloader rEFInd is pretty necessary if you want
| multiple OS installs.
|
| The world of ARM is different: no UEFI or BIOS. Usually whatever
| bootloader Android uses, but maybe [libre/core]boot.
| bpfrh wrote:
| I liked this: https://0xax.gitbooks.io/linux-
| insides/content/index.html
|
| It explains the boot process from the cpu initialization to a the
| end of the linux kernel initialization.
| rounakdatta wrote:
| Related: Also look at https://github.com/alex/what-happens-when
| freakcage wrote:
| https://en.wikipedia.org/wiki/Modern_Operating_Systems Modern
| Operating Systems by Andrew S. Tenenbaum, this book explains how
| Operating Systems works in general. Including boot.
| squegles wrote:
| +1 on this book. Fantastic resource.
| mallikarjun18 wrote:
| https://www.youtube.com/watch?v=1rnA6wpF0o4&list=PLHh55M_Kq4...
| zekica wrote:
| Here are a couple of resources I think can help you:
|
| - old x86 PCs boot in BIOS mode:
| https://opensource.com/article/17/2/linux-boot-and-startup
|
| With BIOS, your PC initializes hardware and contains firmware
| drivers for reading disk drives. BIOS loads the first sector
| (MBR) from the selected boot drive to a fixed memory location and
| jumps to it. Everything else is up to the bootloader (BIOS still
| provides methods for reading disk, interacting with the
| keyboard/moues and text or (S)VGA video output). GRUB2 is saved
| to the disk in two stages: Stage 1 is <512bytes and is only used
| to load stage 2. Stage 2 is commonly saved in the empty space
| before the first partition. Grub stage 2 then loads it's own
| drivers for filesystems, searches partitions and
| kernels+ramdisks... Once you select or manually type the path to
| a kernel and init ram disk, GRUB loads it into memory and jumps
| to it (still in 16bit real mode).
|
| - new x86 PCs and some ARM use UEFI:
| https://hackaday.com/2021/11/30/whats-the-deal-with-uefi/
|
| UEFI works similarly, but is cross-platform (can work on x86,
| x86_64, itanium, 32bit and 64bit ARM, possibly on RISC-V). UEFI
| doesn't look for boot drives but for EFI applications
| (bootloaders) on a special EFI System Partition (uses FAT32
| filesystem per spec, but can be any filesystem your UEFI firmware
| has a driver for). The firmware then loads the configured UEFI
| application and runs it (it also provides services for input
| devices, disks, networking, display). GRUB does the same as
| above.
|
| The only difference is if you are using a distribution that has a
| signed bootloader (using Microsoft's UEFI CA) probably also check
| whether the kernel and 3rd party modules are signed using
| distribution's or your (owner's) signing keys.
|
| - other ARM devices and most MIPS, RISC-V commonly use U-Boot
| with Device Tree (as without UEFI or ACPI, there has to be a way
| to inform the kernel of basic non-PNP hardware and how to
| communicate with it).
| Dalewyn wrote:
| As an aside, GRUB here for general purposes is synonymous with
| other bootloaders such as NTLDR, Windows Boot Manager, lilo,
| and so on.
| monocasa wrote:
| > other ARM devices and most MIPS, RISC-V commonly use U-Boot
| with Device Tree (as without UEFI or ACPI, there has to be a
| way to inform the kernel of basic non-PNP hardware and how to
| communicate with it).
|
| Additionally, UEFI on ARM (and probably RISC-V) will commonly
| give you a device tree too. The two aren't mutually exclusive
| and modern ARM SoCs more or less require kernel space to have
| more information than UEFI is capable of conveying.
| danieldk wrote:
| Apple Silicon Macs:
|
| https://github.com/AsahiLinux/docs/wiki/Introduction-to-Appl...
|
| https://github.com/AsahiLinux/docs/wiki/SW%3ABoot
|
| https://support.apple.com/guide/security/boot-process-secac7...
| bewuethr wrote:
| Didn't see these two mentioned yet:
|
| UNIX and Linux System Administration Handbook, 5th Edition,
| Chapter 2: Booting and System Management Daemons
|
| "Code" by Charles Petzold, 2nd Edition, Chapter Twenty-Six: The
| Operating System
|
| The first one is totally fine to read stand-alone. "Code" makes
| more sense to read cover to cover, though the chapter on its own
| isn't totally useless.
| joemi wrote:
| "Code" was such a great read even though I already knew a lot
| of what's in it. I highly recommend it.
| oumua_don17 wrote:
| https://539kernel.com/book/Chapter_1:_Let's_Start_with_the_B...
___________________________________________________________________
(page generated 2023-03-20 23:01 UTC)