https://mjg59.dreamwidth.org/66109.html
Account name: [ ] Password [ ] [Log in]
(OpenID?) (Forgot it?) [ ] Remember Me
You're viewing [personal profile] mjg59's journal
Create a Dreamwidth Account Learn More
[ ] [Interest ] [Go]
Reload page in style: site light
Matthew Garrett
Booting modern Intel CPUs
Booting modern Intel CPUs
Apr. 16th, 2023 05:12 pm
[personal profile] mjg59
CPUs can't do anything without being told what to do, which leaves
the obvious problem of how do you tell a CPU to do something in the
first place. On many CPUs this is handled in the form of a reset
vector - an address the CPU is hardcoded to start reading
instructions from when power is applied. The address the reset vector
points to will typically be some form of ROM or flash that can be
read by the CPU even if no other hardware has been configured yet.
This allows the system vendor to ship code that will be executed
immediately after poweron, configuring the rest of the hardware and
eventually getting the system into a state where it can run
user-supplied code.
The specific nature of the reset vector on x86 systems has varied
over time, but it's effectively always been 16 bytes below the top of
the address space - so, 0xffff0 on the 20-bit 8086, 0xfffff0 on the
24-bit 80286, and 0xfffffff0 on the 32-bit 80386. Convention on x86
systems is to have RAM starting at address 0, so the top of address
space could be used to house the reset vector with as low a
probability of conflicting with RAM as possible.
The most notable thing about x86 here, though, is that when it starts
running code from the reset vector, it's still in real mode. x86 real
mode is a holdover from a much earlier era of computing. Rather than
addresses being absolute (ie, if you refer to a 32-bit address, you
store the entire address in a 32-bit or larger register), they are
16-bit offsets that are added to the value stored in a "segment
register". Different segment registers existed for code, data, and
stack, so a 16-bit address could refer to different actual addresses
depending on how it was being interpreted - jumping to a 16 bit
address would result in that address being added to the code segment
register, while reading from a 16 bit address would result in that
address being added to the data segment register, and so on. This is
all in order to retain compatibility with older chips, to the extent
that even 64-bit x86 starts in real mode with segments and everything
(and, also, still starts executing at 0xfffffff0 rather than
0xfffffffffffffff0 - 64-bit mode doesn't support real mode, so
there's no way to express a 64-bit physical address using the segment
registers, so we still start just below 4GB even though we have
massively more address space available).
Anyway. Everyone knows all this. For modern UEFI systems, the
firmware that's launched from the reset vector then reprograms the
CPU into a sensible mode (ie, one without all this segmentation
bullshit), does things like configure the memory controller so you
can actually access RAM (a process which involves using CPU cache as
RAM, because programming a memory controller is sufficiently hard
that you need to store more state than you can fit in registers
alone, which means you need RAM, but you don't have RAM until the
memory controller is working, but thankfully the CPU comes with
several megabytes of RAM on its own in the form of cache, so phew).
It's kind of ugly, but that's a consequence of a bunch of
well-understood legacy decisions.
Except. This is not how modern Intel x86 boots. It's far stranger
than that. Oh, yes, this is what it looks like is happening, but
there's a bunch of stuff going on behind the scenes. Let's talk about
boot security. The idea of any form of verified boot (such as UEFI
Secure Boot) is that a signature on the next component of the boot
chain is validated before that component is executed. But what
verifies the first component in the boot chain? You can't simply ask
the BIOS to verify itself - if an attacker can replace the BIOS, they
can replace it with one that simply lies about having done so.
Intel's solution to this is called Boot Guard.
But before we get to Boot Guard, we need to ensure the CPU is running
in as bug-free a state as possible. So, when the CPU starts up, it
examines the system flash and looks for a header that points at CPU
microcode updates. Intel CPUs ship with built-in microcode, but it's
frequently old and buggy and it's up to the system firmware to
include a copy that's new enough that it's actually expected to work
reliably. The microcode image is pulled out of flash, a signature is
verified, and the new microcode starts running. This is true in both
the Boot Guard and the non-Boot Guard scenarios. But for Boot Guard,
before jumping to the reset vector, the microcode on the CPU reads an
Authenticated Code Module (ACM) out of flash and verifies its
signature against a hardcoded Intel key. If that checks out, it
starts executing the ACM. Now, bear in mind that the CPU can't just
verify the ACM and then execute it directly from flash - if it did,
the flash could detect this, hand over a legitimate ACM for the
verification, and then feed the CPU different instructions when it
reads them again to execute them (a Time of Check vs Time of Use, or
TOCTOU, vulnerability). So the ACM has to be copied onto the CPU
before it's verified and executed, which means we need RAM, which
means the CPU already needs to know how to configure its cache to be
used as RAM.
Anyway. We now have an ACM loaded and verified, and it can safely be
executed. The ACM does various things, but the most important from
the Boot Guard perspective is that it reads a set of write-once fuses
in the motherboard chipset that represent the SHA256 of a public key.
It then reads the initial block of the firmware (the Initial Boot
Block, or IBB) into RAM (or, well, cache, as previously described)
and parses it. There's a block that contains a public key - it hashes
that key and verifies that it matches the SHA256 from the fuses. It
then uses that key to validate a signature on the IBB. If it all
checks out, it executes the IBB and everything starts looking like
the nice simple model we had before.
Except, well, doesn't this seem like an awfully complicated bunch of
code to implement in real mode? And yes, doing all of this modern
crypto with only 16-bit registers does sound like a pain. So, it
doesn't. All of this is happening in a perfectly sensible 32 bit
mode, and the CPU actually switches back to the awful segmented
configuration afterwards so it's still compatible with an 80386 from
1986. The "good" news is that at least firmware can detect that the
CPU has already configured the cache as RAM and can skip doing that
itself.
I'm skipping over some steps here - the ACM actually does other stuff
around measuring the firmware into the TPM and doing various bits of
TXT setup for people who want DRTM in their lives, but the short
version is that the CPU bootstraps itself into a state where it works
like a modern CPU and then deliberately turns a bunch of the sensible
functionality off again before it starts executing firmware. I'm also
missing out the fact that this entire process only kicks off after
the Management Engine says it can, which means we're waiting for an
entirely independent x86 to boot an entire OS before our CPU even
starts pretending to execute the system firmware.
Of course, as mentioned before, on modern systems the firmware will
then reprogram the CPU into something actually sensible so OS
developers no longer need to care about this[1][2], which means we've
bounced between multiple states for no reason other than the
possibility that someone wants to run legacy BIOS and then boot DOS
on a CPU with like 5 orders of magnitude more transistors than the
8086.
tl;dr why can't my x86 wake up with the [S:gin:S] protected mode
already inside it
[1] Ha uh except that on ACPI resume we're going to skip most of the
firmware setup code so we still need to handle the CPU being in
fucking 16-bit mode because suspend/resume is basically an extremely
long reboot cycle
[2] Oh yeah also you probably have multiple cores on your CPU and
well bad news about the state most of the cores are in when the OS
boots because the firmware never started them up so they're going to
come up in 16-bit real mode even if your boot CPU is already in
64-bit protected mode, unless you were using TXT in which case you
have a different sort of nightmare that if we're going to try to map
it onto real world nightmare concepts is one that involves a lot of
teeth. Or, well, that used to be the case, but ACPI 6.4 (released in
2021) provides a mechanism for the OS to ask the firmware to wake the
CPU up for it so this is invisible to the OS, but you're still
relying on the firmware to actually do the heavy lifting here
Tags:
* advogato,
* fedora
* Previous Entry
* Add Memory
* Share This Entry
* Next Entry
---------------------------------------------------------------------
* 3 comments
* Reply
---------------------------------------------------------------------
Flat | Top-Level Comments Only
no subject
Date: 2023-04-17 12:49 am (UTC)
sweh: (Default)
From: [personal profile] sweh
Ah for the good 'ol days of the 6502 where reset was indirected via
$FFFC :-)
* Link
* Reply
Intel TXT
Date: 2023-04-17 11:55 am (UTC)
From: (Anonymous)
You mention Intel TXT. I would really like to read a blogpost where
you go into that.
* Link
* Reply
no subject
Date: 2023-04-17 04:51 pm (UTC)
onanyzmus: (Default)
From: [personal profile] onanyzmus
Do you plan to write about booting AMD processors too?
* Link
* Reply
* Previous Entry
* Add Memory
* Share This Entry
* Next Entry
* 3 comments
* Reply
Flat | Top-Level Comments Only
Profile
Matthew Garrett
About Matthew
Power management, mobile and firmware developer on Linux. Security
developer at Aurora. Ex-biologist. [personal profile] mjg59 on
Twitter. Content here should not be interpreted as the opinion of my
employer. Also on Mastodon.
Page Summary
* [personal profile] sweh - (no subject)
* (Anonymous) - Intel TXT
* [personal profile] onanyzmus - (no subject)
Active Entries
* 1: Booting modern Intel CPUs
* 2: Asking ChatGPT to write my security-sensitive code for me
* 3: Lenovo shipping new laptops that only boot Windows by default
* 4: Pluton is not (currently) a threat to software freedom
* 5: More in the series of bizarre UEFI bugs
* 6: Build security with the assumption it will be used against
your friends
* 7: We need better support for SSH host certificates
* 8: Poking a mobile hotspot
* 9: Blocking free API access to Twitter doesn't stop abuse
* 10: Further adventures in Apple PKCS#11 land
Expand Cut Tags
No cut tags
Top of page