[HN Gopher] Writing a BIOS bootloader for 64-bit mode from scratch
___________________________________________________________________
Writing a BIOS bootloader for 64-bit mode from scratch
Author : D4ckard
Score : 150 points
Date : 2024-07-14 08:46 UTC (14 hours ago)
(HTM) web link (thasso.xyz)
(TXT) w3m dump (thasso.xyz)
| AstralStorm wrote:
| How old is UEFI now? Pity nobody deprecated BIOS alongside long
| mode.
| deaddodo wrote:
| BIOS _is_ deprecated. All of its functionality on new
| motherboards is basically emulated via the UEFI; and it 's
| certainly not being extended upon.
|
| Deprecated doesn't mean deleted, it just means "no longer
| updated/developed with a _goal_ towards removal ".
| livrem wrote:
| This killed FreeDOS (and presumably all the other *DOS as
| well) on modern hardware unfortunately. It was fun as long as
| it lasted. I do not know what the next-best single-user,
| single-process, non-bloated OS would be to run on modern
| hardware that still has some reasonably modern software and
| can be used for distraction-free (hobby) development the way
| FreeDOS could.
| trueismywork wrote:
| Linux in single user mode
| mschuster91 wrote:
| That's still multi-process though, there's an awful lot
| of background tasks running in pretty much every non-
| fossil kernel version, not to mention userspace daemons
| (udev, dbus, dhcp) without which most normal userspace
| stuff doesn't even work.
| Brian_K_White wrote:
| None of that exists in single user. When you say
| init=/bin/foo, then that's it, the only process is
| /bin/foo.
| ryandrake wrote:
| /bin/foo is the initial process. It can fork and/or exec
| other processes, right?
| Brian_K_White wrote:
| Sure the _facility_ to fork still exists. So what?
| Observing that the kernel still provides fork() is like
| observing that the cpu still provides JMP.
|
| It won't fork random processes you don't explicitly tell
| it to. I thought it was obvious that if you don't want
| unsolicited processes, then don't specify /bin/init as
| /bin/foo. The practical example is /bin/sh, but it could
| be any other executable.
|
| Up to you to specify a binary that does what you want,
| and doesn't require a bunch of other processes like gdbus
| to function itself.
|
| init=/bin/sh is more or less like ms-dos loading
| command.com
| gtirloni wrote:
| It's obvious but many people here seem to be confusing
| the Linux kernel with kernel+systemd and complaining
| Linux has many processes like it's not customizable.
| jasaldivara wrote:
| > I do not know what the next-best single-user, single-
| process, non-bloated OS would be to run on modern hardware
| that still has some reasonably modern software and can be
| used for distraction-free (hobby) development the way
| FreeDOS could.
|
| Not sure why would you want a single-process OS on modern
| hardware, but there are some alternatives that run much
| less things on the background than regular Linux: Haiku,
| FreeBSD, NetBSD, OpenBSD, or some lightweight non-glibc,
| non-systemd Linux-based like Adelie or Alpine.
| rwmj wrote:
| What's the reason why FreeDOS can't use the CSM (the BIOS
| compatibility mode of UEFI)?
| EvanAnderson wrote:
| AFAIK it can. I believe some UEFI implementations don't
| have CSM.
| 5- wrote:
| note that you can switch to long mode directly, without going
| into protected mode first, with way less code:
|
| https://wiki.osdev.org/Entering_Long_Mode_Directly
|
| i've had a bootloader for a small 64-bit kernel based on this
| that fit comfortably into the bootsector, including loading the
| kernel from disk and setting up vesa modes, no stage2 required.
| D4ckard wrote:
| Yes, you can do that too
| dataflow wrote:
| > i've had a bootloader for a small 64-bit kernel based on this
| that fit comfortably into the bootsector, including loading the
| kernel from disk and setting up vesa modes, no stage2 required.
|
| How in the world do you fit all that in 512 bytes? I'm guessing
| you don't have a real-world filesystem (that allows the kernel
| to be anywhere on the disk just as a normal file)? Because just
| dealing with file fragmentation should bump you way over 512
| bytes I would imagine.
| 5- wrote:
| yes, the kernel was in a known location on disk (directly
| after the bootsector).
|
| the whole boot disk image was generated during build, which
| is common for small systems.
| rep_lodsb wrote:
| I've been thinking about how to structure a filesystem such
| that code complexity - in the boot loader and elsewhere - can
| be minimized. You'd want something similar to NTFS, except
| that it should be possible to refer to a file (MFT entry)
| directly by starting sector number. So they would either need
| to always remain at a fixed position, or have a link pointing
| back to any reference so it can be updated.
|
| Somewhere in the first sector (aligned to a 64 bit boundary),
| there would be a small structure that just contains a unique
| signature (not ASCII but some random value; 64 bits seems
| like more than enough as opposed to a GUID), and a pointer to
| the "FS info block". Leaving all remaining space in the
| sector available for boot code or possibly another
| "overlayed" filesystem.
|
| That info block in turn would point to the MFT entry for the
| stage 2 boot code. An MFT entry would contain at a minimum an
| easy to locate list of (start sector,count) extents. Maybe
| there could be a flag that tells the OS that a certain file
| should never be fragmented?
|
| File names would be entirely optional and for human
| consumption; for direct links within a filesystem, sector
| numbers would be used, and some kind of unique numeric
| identifier for anything "higher-level".
|
| I'm genuinely wondering if some expert here sees any problem
| with this scheme, other than it not conforming to how
| mainstream OSes do things?
| RiverCrochet wrote:
| > File names would be entirely optional and for human
| consumption; for direct links within a filesystem, sector
| numbers would be used, and some kind of unique numeric
| identifier for anything "higher-level".
|
| Each file has index number ("Inode"?) in the MFT. The first
| 24 (0-23) are reserved, and 11 of them contain metadata
| about the volume . https://flatcap.github.io/linux-
| ntfs/ntfs/files/ - somewhere in the Windows API is
| something that allows opening a file by "Inode" number.
| This link and info may be really old so it could be more of
| the reserved inodes are used now.
|
| So, if 23 isn't being used yet, you could use that to put
| your 2BL - create a file there and name it $2BL or
| something. Would be funny to see what future Windows update
| that does use it does to it, if that ever happens (and of
| course maybe it is used).
|
| > Maybe there could be a flag that tells the OS that a
| certain file should never be fragmented?
|
| Haven't looked but I recall from an old book I read that
| small files are stored right in the MFT and I think
| existing data + the cluster size is the limit there.
| blankx32 wrote:
| https://wiki.osdev.org/A20_Line
| ruslan wrote:
| Does this boot procedure work with EFI/UEFI ? If so, does UEFI
| supervisor emulate swithing real/protected/long modes or does it
| go in real hardware ?
| khaledh wrote:
| No. UEFI firmware creates a completely different environment
| for a UEFI bootloader than the legacy BIOS environment (real-
| address mode). The UEFI firmware enters 64-bit long mode
| directly on modern systems, and sets up a flat memory model
| GDT, as well as identity-mapped paging.
|
| I've written about creating a UEFI bootloader (for my hobby OS)
| here: https://0xc0ffee.netlify.app/osdev/05-bootloader-p1.html
| surajrmal wrote:
| I thought many UEFI implementations support legacy bios mode
| as well. Or well they used to.
| the_panopticon wrote:
| There is still support for CSM in the open source https://g
| ithub.com/tianocore/tianocore.github.io/wiki/Compat... and
| even nice projects like
| https://github.com/coreboot/seabiosto fabricate the CSM16
| binary, but many vendors have stopped validating this path,
| including production of legacy BIOS option roms for
| adapters (net, gfx, etc)
| https://www.phoronix.com/news/Intel-Legacy-BIOS-EOL-2020. I
| still believe CSP's maintain some of this support in their
| hypervisors' guest firmware for legacy OS binaries/ISO boot
| support? Also since Windows requires UEFI Secure boot
| enabled by default and CSM has to be disabled for the UEFI
| secure boot path, this is another reason legacy BIOS boot
| isn't exercised so much these days. We could have added
| legacy oroms hashes to UEFI Secure boot implementations
| https://patents.google.com/patent/US8694761B2/en, too, but
| again folks pushed back in their zeal to remove legacy BIOS
| overall. We didn't add the CSM spec https://www.intel.com/c
| ontent/dam/www/public/us/en/documents... to the PI
| https://uefi.org/specs/PI/1.8A/ since folks were hoping
| UEFI would remove the need for CSM. I still remember being
| challenged in the early 2000's by a long-time BIOS mgr at
| Intel "Is removing legacy a good idea with EFI? You know,
| we're really at legacy."
| amelius wrote:
| Is this any simpler on ARM?
| rwmj wrote:
| Only in the sense that every board vendor does their own random
| thing, which makes it simpler for the board vendors and
| horribly complicated for everyone else.
| surajrmal wrote:
| Yes. Bootloaders are still complex, but there is less legacy
| setup that is required. That said, if you're targeting UEFI
| instead of BIOS, it's a great deal simpler on x86 as well.
| gtirloni wrote:
| Not sure, I wouldn't count on it. Currently deep in RISC-V and
| it seems there's hope.
| ThinkBeat wrote:
| All to me entirely unnecessary steps required to get the CPU into
| the correct mode is astounding.
|
| They all seem to be steps needed for backwards compatibility.
|
| Could Intel just provide a flag, command, to start in the right
| mode from the beginning.
|
| Or just remove all the backwards compatibility.
|
| I think I remember doing some research and ARM64 has some of the
| same issues.
|
| Are there any CPUs that are designed from scratch as 64 bit it
| will not have any need for backwards compatibility and would
| enter the required state by default?
|
| I guess sthat was the goal / design of Itanium?
|
| are made to start in the desired 64 bit state from th
| LiamPowell wrote:
| UEFI exists. You just put a Windows-like binary in a folder on
| a partition and it runs in a hosted environment in 64-bit mode.
| And of course there's countless bootloaders that can take care
| of all this for you too.
| rep_lodsb wrote:
| And then you're free from dealing with the somewhat
| convoluted processor init stuff, but instead depend on the
| Windows PE format, FAT filesystem, and an overcomplicated
| API.
|
| Seems like a bad tradeoff, and part of a slippery slope
| towards a completely locked down system, where writing your
| own code and getting it to run on the 'bare metal' is flat
| out impossible.
| immibis wrote:
| What's wrong with depending on the Windows PE format, FAT
| filesystem and UEFI? You're always going to have _some_
| dependencies. FAT32 is better than having the first sector
| load some magic reserved sectors. Windows PE is better than
| a fixed memory address.
| rep_lodsb wrote:
| It's adding pointless complexity, and baking assumptions
| about how an OS should work into the firmware. Loading a
| sector at a fixed memory address and jumping to it (with
| some function provided so that your code can go on to
| load other sectors) is both easier to understand, and
| doesn't require you to use some multi-megabyte toolchain.
| immibis wrote:
| Wouldn't it be better to use a header to specify to load
| multiple sectors at multiple memory addresses?
| leeter wrote:
| This is fine if you're only running on a single core, however
| if you're a multiprocessor OS you still need to deal with
| legacy when bringing up the other cores. Intel and AMD should
| consider a mode that disables that and brings up the other
| cores using a 64bit SIPI. While I applaud Intel on the X86S
| idea... I think there is room for bits of that without
| throwing out all the backwards compat. An X86SC which drops
| real mode and only supports 16bit 'real mode' in
| virtualization.
|
| Yes, I see the argument that if you go to that point you
| might as well just use emulation. However running mixed
| 32bit/16bit code (Windows 98/95) becomes problematic just
| because of performance reasons. DosBox does well, but good
| luck supporting games for the Pentium 3 era that still used
| 16bit libraries because of course they did. (16bit
| Installshield was so common going into 64bit that MS just has
| code to replace it so 32bit applications can still be
| installed despite having a 16bit installer)
| LiamPowell wrote:
| I vaguely recall there being some multicore stuff in UEFI,
| but it's been years since I looked at it.
| nullindividual wrote:
| This is what Intel's proposed X86S [0] is designed for.
|
| > X86S is a legacy-reduced-OS ISA that removes outdated
| execution modes and operating system ISA.
|
| > The presence of the X86S ISA is enumerated by a single, main
| CPUID feature LEGACY_REDUCED_ISA in CPUID 7.1.ECX[2] which
| implies all the ISA removals described in this document. A new,
| 64-bit "start-up" interprocessor interrupt (SIPI) has a
| separate CPUID feature flag.
|
| [0] https://cdrdv2.intel.com/v1/dl/getContent/776648 [pdf
| warning]
| hyperman1 wrote:
| The 80286 has the Machine Status Word (MSW), a 16 bit register.
| The 80386 expands this to CR0, a 32 bits register. Then 64 bit
| long mode adds the EFER MSR and expands CR0 to 64 bits. But even
| today only 11 bits of CR0 are in use and EFER has 8 active bits.
| I wonder why intel/AMD did not simply use the free bits of the
| existing register, and made that decision twice?
|
| https://wiki.osdev.org/CPU_Registers_x86-64#CR0.
| rcxdude wrote:
| Probably for more robust backwards compatibility with software
| that might assume a given value for or write to the reserved
| bits. The assignment of bits to registers like this in the
| hardware is pretty arbitrary, there's not really any cost to
| using the higher bits
| monocasa wrote:
| Particularly AMD made the 64 bit extension without any real
| input from Intel and didn't want to use any bits that would
| later conflict with a bit Intel might use in CR0. So a brand
| new register was in order.
| rep_lodsb wrote:
| The flag register layout is another case of extreme backwards
| compatibility - its lower bits have the same definitions they
| had on the 8-bit 8080, even the same fixed values:
|
| Sign : Zero : always '0' : AuxCarry : always '0' : Parity :
| always '1' : Carry
|
| (the parity flag came all the way from the 8008 / Datapoint
| 2200[1], and is the inverted XOR of the result's lower 8
| bits; aux carry is the carry out of bit 3, used for BCD
| arithmetic)
|
| Flag bit 15 has also stayed reserved, except at one time it
| was used by the NEC Vxx chips for their 8080 compatibility
| mode. That feature had to be first unlocked by executing a
| special instruction, because there is code out there that
| loads the entire (16 bit) flag register with 0000 or FFFF.
| With the mode bit unlocked, that would inadvertently switch
| the CPU to running a completely different set of opcodes!
|
| [1] https://www.righto.com/2023/08/datapoint-to-8086.html
| rep_lodsb wrote:
| The most unnecessarily complicated thing in this article to me is
| the Makefile and linker script. NASM supports generating flat
| binary output, but apparently using it would be too "hacky"?
| darby_nine wrote:
| I find linker scripts much easier to read and reason about than
| flat nasm but that's just me. Especially with multiple source
| files.
| cf100clunk wrote:
| A laudable project. UEFI proponents here wondering why the person
| bothered to create a new bootloader approach might be missing the
| point of why people undertake such tasks as this. As the writer
| ends:
|
| > Cool if you actually came along this far.
|
| Cool indeed.
___________________________________________________________________
(page generated 2024-07-14 23:00 UTC)