[HN Gopher] Netboot Windows 11 with iSCSI and iPXE
___________________________________________________________________
Netboot Windows 11 with iSCSI and iPXE
Author : terinjokes
Score : 153 points
Date : 2025-02-28 11:47 UTC (11 hours ago)
(HTM) web link (terinstock.com)
(TXT) w3m dump (terinstock.com)
| KaiserPro wrote:
| Hahaha, this is batshit, and I am here for it.
|
| I knew it was possible, as I'd seen some thin terminals doing
| something similar (using ephemeral filesystems though) However
| I've only ever done something like this for linux, rather than
| windows.
|
| Excellent work, I salute your skill.
| toredash wrote:
| Next challenge: Wifi
| MortyWaves wrote:
| > You probably want to remove the disks anyways so Windows
| doesn't install its boot manager to an EFI System Partition on a
| local disk. To boot with iPXE later, you'll want Windows to
| create an EFI System Partition on the iSCSI disk. -
|
| Oh the frustrations I've had with Windows and more than one disk
| during install. For _months_ I had Windows installed on SSD,
| except the bootloader was on a HDD. I didn't even realise till I
| removed the HDD and got the familiar no boot drive found.
|
| Turns out if a drive has Windows on it, and then you decide to
| install Windows on a new drive, it will detect that bootloader on
| the old drive and use that. Instead of putting it on the drive
| you just asked it to install Windows to.
|
| Lesson learned here is always unplug any other drives when
| installing Windows!
| delta_p_delta_x wrote:
| > always unplug any other drives when installing Windows!
|
| Many motherboards and enterprise notebooks have functionality
| to disable SATA/NVMe ports, so there's increasingly less need
| to faff with physically unplugging discs.
| smittywerben wrote:
| It's easy when SATA drives are stacked to unplug the ones you
| want.
|
| It's not easy when M.2 requires removing a heatsink to press
| the PCI-e unlock to extract the 2 ton GPU to access the
| primary M.2 slot underneath to use your screwdriver on the
| screw the size of a PopRock clinging on trying to not shake
| because it's holding onto the screwdriver with the magnetic
| force equivalent of a fridge magnet.
|
| Swapping the actual nvme drive out is easy. But then getting
| that screw back in is even worse than removing it. Sometimes
| you get lucky and the motherboard is improving the
| hotswapping capability with the newest yearly
| reimplementation of a screw with a string on it so if your
| screwdriver has trouble holding a fridge magnet against the
| gravity of the earth then the string keeps the screw from
| phase warping. My favorite has been the rubber peg designs
| but we're 10 years out from combining the peg with the string
| tech.
|
| That's when reading the manual to figure out the port numbers
| to disable in the BIOS makes sense. It just seems like missed
| opportunity they didn't make M.2 external facing.
| delta_p_delta_x wrote:
| I should've clarified: they include functionality to
| disable these ports directly in the UEFI firmware. No need
| to open up the enclosure at all.
| fuzzfactor wrote:
| Too bad my Lenovo lets me disable the SATA but not NvMe
| :(
|
| Plus I remember doing this for years under bios until
| some version of NT6 came along and Windows could access
| them even when disabled in BIOS on some mainboards.
| smittywerben wrote:
| Whenever Windows self-repairs the bootloader it makes me want
| to learn DOS.
| fuzzfactor wrote:
| Learn the modern equivalent, the CMD console.
|
| You really want the Administrative Command Prompt for best
| functionality.
|
| However, remember if using Powershell instead it's still
| having trouble with the curly brackets so in every case they
| need to be quoted in commands, such as this example:
|
| Bcdedit /set "{default}" Description "Win11 23H2 on Dell
| 512gb SSD part 2"
|
| Which gives a much more useful description than the plain
| "Windows 11" that exists already on installations otherwise.
| Even though these "descriptions" never display until there
| are more than one choice on the bootmenu.
|
| Concentrate on the commands, BCDEDIT, BCDBOOT, and
| familiarize with DISKPART which opens its own sub-shell to
| handle drives. When you are done with Diskpart, you have to
| type "exit" to go back to the regular command line,
|
| Inbuilt Help docs for the regular command prompt are obtained
| the same old DOS way, for example "bcdedit /?".
|
| As an example, Bcdedit is too complex to learn from the Help
| messages, and there are hidden sub-help "chapters" which add
| to the confusion, so learn this one from the internet and its
| expert examples.
|
| OTOH Bcdboot is straightforward, but either way, to access a
| particular (naturally hidden) EFI folder (other than the one
| you are booted from at the time) on a chosen HDD or SSD you
| would first have to make the volume visible by giving it an
| alphabetic designator using Diskpart, the ASSIGN command.
|
| Under Diskpart, ASSIGN, "GPT ATTRIBUTES", SETID, ATTRIBUTES
| are 4 good ones to know even if you are not going to use
| Diskpart for partitioning.
|
| While in the Diskpart sub-shell, the inbuilt Help docs are
| accessed their own way, type "help assign" for instance to
| get Assign syntax examples.
|
| OTOH with the Attributes command, "Help Attributes" only
| tells you that you need to use "Help Attributes Volume" or
| "Help Attributes Disk" instead to get the examples you have
| in mind.
|
| A lot of this stuff I know already but I still use the help
| docs a lot to copy & paste the syntax examples from the CMD
| window into Notepad for editing before copying back to
| execute.
| NikkiA wrote:
| I type `bcdedit /set "{bootmgr}" path
| \EFI\refind\refind_x64.efi` so many times a year that it's
| probably getting close to time to buy a macro keyboard so I
| can set it up on a macro.
| fuzzfactor wrote:
| With the latest W11 you get even more :\
|
| When you install it to a system with other partitions or drives
| that are accessible, it silently autoencrypts them all using
| Bitlocker which is now on by default.
|
| It's not too bad unencrypting your C: volume afterward with a
| few clicks, but if you don't have a Microsoft Account then you
| won't have a readily available key (to type in) or easy
| decryption option for the other volumes. It can be a lot easier
| to reformat and recover those non-C: volumes from backup than
| bother to decrypt, you've got to virtually overwrite the entire
| drives anyway :(
|
| Simply installing the latest Windows is definitely more likely
| to be the direct cause for loss of more gigabytes than most
| malware I've seen.
|
| It didn't used to be this way.
| kentonv wrote:
| LOL they accidentally(?) created ransomware. "We've encrypted
| all your files. Want them back? You must sign up for a
| Microsoft account."
| kennysoona wrote:
| > When you install it to a system with other partitions or
| drives that are accessible, it silently autoencrypts them all
| using Bitlocker
|
| Only NTFS/FAT drives, right?
| fuzzfactor wrote:
| As far as I know.
|
| I assign the standard Linux GUIDs on non-Windows volumes to
| be sure, as appropriate even though the regular "Basic
| Volume" GUID is often found since it works "for all".
|
| The standard Linux Partition_type_GUIDs as shown in
| Wikipedia (please refer to it) after expanding ("show") the
| condensed table:
|
| 4F68BCE3-E8CD-4DB1-96E7-FBCAF984B709 for the OS root
|
| 0FC63DAF-8483-4772-8E79-3D69D8477DE4 for data
|
| 8484680C-9521-48C6-9C11-B0720656F69E for usr
|
| 0657FD6D-A4AB-43C4-84E5-0933C84B4F4F for swap
|
| These can be SET ID from Windows, plus you can set GPT
| Attributes to hidden and/or readonly temporarily because it
| is getting more difficult to trust Windows so that's why
| I'm experimenting so much.
|
| Also you can safely change the ESP volume GUID from
| standard to basic, and Windows will most likely still boot
| normally on many PCs, you can then assign a volume letter
| to the ESP and after another reboot have full admin access
| to the ESP and its black pearl, the EFI folder within, from
| the GUI.
|
| Standard ESP C12A7328-F81F-11D2-BA4B-00A0C93EC93B
|
| Basic Data EBD0A0A2-B9E5-4433-87C0-68B6B72699C7
|
| Check my other extensive comment coming soon. :)
| terinjokes wrote:
| These are the GUIDs from the [Discoverable Partitions
| Specification][dps] should you want to learn more about
| it (or the GUIDs for non-amd64 systems).
|
| [dps]: https://uapi-
| group.org/specifications/specs/discoverable_par...
| fuzzfactor wrote:
| Thanks for that :)
|
| I wondered what those were about, it's good to see this
| kind of progress toward having things just work.
| exe34 wrote:
| that's hilarious! I'm on nixos and the only time I had to
| mess with the partitions was when the old ssd died about 5
| years ago and I got a new one.
| kentonv wrote:
| Hey Terin! Nice post!
|
| I also netboot Windows this way! To run a 20 machines in my house
| off the same base disk image, which we use for LAN parties. I
| have code and an extensive guide on GitHub:
|
| https://github.com/kentonv/lanparty
|
| It looks like you actually figured out something I failed at,
| though: installing Windows directly over iSCSI from the start. I
| instead installed to a local device, and then transferred the
| disk image to the server. I knew that building a WinPE
| environment with the right network drivers would probably help
| here, but I got frustrated trying to use the WinPE tools, which
| seemed to require learning a lot of obscure CLI commands
| (ironically, being Windows...).
|
| You observed some slowness using Windows over the network. I did
| too, when I was doing it with 1G LAN, but I've found on 10G it
| pretty much feels the same as local.
|
| BTW, a frustrating thing: The Windows 10->11 updater also
| seemingly fails to include network drivers and so you can't just
| upgrade over iSCSI. I'm still stuck on Windows 10 so I'm going to
| have to reinstall everything from scratch sometime this year.
| Maybe I'll follow your guide to use WinPE this time.
| amazingamazing wrote:
| I recently saw this and want to do something similar with just
| 4 machines to play relatively simple games (e.g. age of
| empires). I might follow what you have here directly, but am
| also considering just using a single machine with 4 vms.
|
| The big constraint for me is I don't want to buy a bunch of
| hardware.
| terinjokes wrote:
| Hey Kenton!
|
| I figured you had done something similar with the LAN Party
| House. If I hadn't figured it out I was going to ask/look for
| your setup.
|
| > You observed some slowness using Windows over the network.
|
| Mini-ITX makes it a bit difficult to upgrade to 10GbE (only one
| PCIe slot!), and the slowness isn't bad enough in-game to deal
| with upgrading it just yet.
|
| > BTW, a frustrating thing: The Windows 10->11 updater also
| seemingly fails to include network drivers and so you can't
| just upgrade over iSCSI.
|
| I've read (and also observed, now) that if you install directly
| on iSCSI Windows doesn't make the recovery partition. This
| evidently also breaks 10->11 upgrades.
| kentonv wrote:
| Oh, maybe the lack of recovery partition is my real problem,
| because I definitely deleted mine. The installer isn't really
| clear why it's failing, it just does its thing for a bit and
| then reboots into Windows 10 and says it didn't work, no
| explanation.
| fuzzfactor wrote:
| Things like this iPXE have a similarity to multibooting to
| where you may be a lot better off NOT having a (single)
| dedicated Recovery partition at all, but you need full
| responsiveness to Windows Update.
|
| And please no hibernation for multibooting. POWERCFG /H
| OFF.
|
| For recovery manipulation you need to utilize the REAGENTC
| command, and take advantage of the existing default
| C:\Recovery folder. Normally this is not considered the
| best location for "recovery" since if C: gets borked so
| does default recovery console.
|
| And this really gives an example of essential use of
| inbuilt Helps.
|
| On a mainstream system all you have is a single (hidden)
| ESP volume containing its EFI folder:
|
| In admin CMD, type Bcdedit /Enum All to show all your boot
| entries.
|
| If "recovery" is even set up, there will be a Boot Loader
| entry where the Device is a Ramdisk. If so there will also
| be a volume letter in square brackets if the target volume
| is unhidden at the time.
|
| But for those of you with no recovery set up, you get to do
| it yourself as you see fit.
|
| Type Reagentc /info, it will confirm recovery is disabled.
|
| Reagentc /? gives you the next clue, the example
| /setreimage.
|
| Copy the displayed REAGENTC.EXE /setreimage /? from the
| display down to the active command line (or just type it
| in). You get two syntax examples, it's the second one. But
| you probably don't even have an R: volume and you want to
| use C: anyway to make Windows more self-sufficient from its
| own partition.
|
| edit it so:
|
| REAGENTC.EXE /setreimage /path c:\Recovery\WindowsRE
| /target C:\Windows
|
| After this operation is successful, there is still no
| apparent change under Reagentc /info, recovery is still
| disabled. And your BCD remains unchanged with no ramdisk
| object yet.
|
| "Reagentc /enable" will then autoadd the ramdisk recovery
| bootentry to the BCD, directing the recovery routine (when
| triggered) to point to your C: volume which will always
| exist accessibly, plus most likely have enough free drive
| space to accommodate the larger updates to the recovery WIM
| files themselves, which are recently giving users show-
| stoppers when their dedicated factory recovery partitions
| were not big enough for that.
|
| Now for the good news.
|
| Once you find a ramdisk recovery bootentry in your BCD, an
| easy baby-step toward multibooting is a piece of cake.
| Everybody already has a Windows boot menu that you are
| using every time you boot, it is simply not displayed if
| there is only one entry, like most factory set ups.
|
| So add the Recovery Console to the menu your dang self.
|
| It can come in really handy.
|
| From the Bcdedit output text, copy the shown GUID for the
| main ramdisk recovery object, so you can paste afterward
| and don't have to type it all into the following command:
|
| bcdedit /displayorder {your-guid-here} /addlast
|
| Then if you don't want to maintain the default 30 second
| timeout before progressing to the default OS on the
| bootmenu, set it for something like 5 seconds:
|
| bcdedit /timeout 5
|
| That's it.
|
| Now if you don't have a touchscreen, you probably would be
| better off with the "legacy" text menu, for that you would
| need to change the bootmenupolicy from Standard to Legacy
| for the GUIDs that have a bootmenupolicy. Bcdedit /set
| {target-guid-here} bootmenupolicy Legacy.
|
| Also remember if using Powershell it's still having trouble
| with the curly brackets so in every case they need to be
| quoted like Bcdedit /set "{target-guid-here}"
| bootmenupolicy Legacy.
| xen2xen1 wrote:
| God bless you for this sir. I've been wanting to get Windows
| iscsi boot working, but there's always one more thing. Did
| you get anything else fun working with ipxe? all the exampled
| online seem so outdated.
| terinjokes wrote:
| Since the goal was just to have non-virtualized Windows to
| satisfy game developers, I haven't tried much else. Sorry.
|
| > all the exampled online seem so outdated.
|
| Many of the guides felt like they've been only slightly
| dusted since the Windows XP days. I've seen none that even
| hinted at making the DVD ISO a target, leading me to
| believe it wasn't well supported 20 years ago.
| wolrah wrote:
| > I've seen none that even hinted at making the DVD ISO a
| target, leading me to believe it wasn't well supported 20
| years ago.
|
| One of the most underrated things about UEFI in my
| opinion is how it made booting simple. No longer did you
| need to use special tools (or arcane knowledge) to stick
| a special real-mode binary in to a normally hidden area
| of your intended media, now you just drop a few PE
| binaries in a filesystem your UEFI can read and let it
| figure itself out.
|
| I just created a bootable restore image for one of my
| clients and it's literally a zip file that can be
| extracted in to the root of a FAT32 disk drive and will
| boot on any x86-64 UEFI PC, with Secure Boot fully
| operational. No special tools required, just drop files
| on any disk using a filesystem any computer that matters
| can read/write.
| lmz wrote:
| I guess for removable media that's fine, but Windows
| still likes to hide the ESP.
| RulerOf wrote:
| I may have written one of those dusty guides in my
| younger days.
|
| >I've seen none that even hinted at making the DVD ISO a
| target, leading me to believe it wasn't well supported 20
| years ago.
|
| iPXE has always supported this, but the devil is always
| in the details.
|
| Windows Setup has a disk-bootability-validation step that
| it runs before it'll let you install Windows. It checks
| that the firmware can actually see the drive (i.e. it was
| present during system boot) and understands the boot
| chain, looking for the ESP or MBR that will actually load
| Windows once Setup completes.
|
| There's something about loading more than one disk into
| the iBFT[1] and hooking those disks into INT13 on a BIOS-
| based system that very frequently breaks one of those
| checks. You try to select a disk and Windows Setup
| complains about not being able to verify that it's
| bootable.
|
| It'd all likely work just fine on a typical UEFI system
| these days.
|
| Of course, since it's Windows, there's no "I checked and
| it's okay please install anyway" button. But I'm pretty
| sure you can just apply the wim manually from the command
| line and install the bootloader, but I was never clear if
| that process implemented the driver reflection step
| necessary to ensure the NIC would start on boot...
|
| It's unfortunate that Windows Setup itself isn't open-
| source. It'd probably have made the time I spent doing
| this stuff a lot more interesting.
|
| [1]: iSCSI Boot Firmware Table is a memory structure used
| to pass iSCSI disks from a firmware-level software
| initiator to the OS-level software initiator after the
| kernel takes over the hardware
|
| [2]: https://ipxe.org/wimboot
| toast0 wrote:
| If you get (i)pxe running, you can chain to
| https://netboot.xyz/ which lets you boot lots of open
| source stuff.
|
| It's a bit of a mixed bag, because pxe environments have a
| way of not always being useful. On bios boot, there's tools
| from isolinux to memory load disk images and hook the bios
| calls... but if your OS of choice doesn't use bios calls
| for storage, it needs a driver that can find the disk image
| in memory.
|
| For uefi boot, there's not a good way to do this,
| supposedly some uefi environments can load disk images from
| the network, but afaik, it's not something you can do from
| ipxe. Instead, for UEFI, the netboot.xyz folks have some
| other approaches; typically fetching the kernel and initrd
| separately or otherwise repackaging things rather than
| using official ISO images.
|
| And I've run into lots of cases where while pxe seems to
| work, maybe the keyboard doesn't work in pxe, or something
| else doesn't get properly initialized and you end up having
| a better time if you give up and boot from USB.
| westurner wrote:
| System Rescue CD and Clonezilla are PXE-bootable.
|
| "OneFileLinux: A 20MB Alpine metadistro that fits into
| the ESP" https://news.ycombinator.com/item?id=40915199 :
|
| > Ventoy, signed EFIstubs, USI, UKI
|
| TIL about https://netboot.xyz/
| xen2xen1 wrote:
| Also interesting using it as an EFI executable. Remind me to
| start adding that to the menu of ... well, everything.
| kennysoona wrote:
| Did you run into any licensing issues with your setup?
| kentonv wrote:
| Microsoft keeps track of licenses in some cloud database
| keyed by machine fingerprint, so if you run the same image on
| multiple machines it's supposed to work fine. I have actually
| purchased licenses for all the machines.
|
| I have found, though, that sometimes machines don't actually
| query Microsoft for activation info, and instead complain
| about activation after a few hours. If I then manually go to
| the activation settings and click "troubleshoot activation",
| _then_ it realizes that it 's already registered in the
| Microsoft database and it stops complaining.
|
| But this is really annoying to have to do for every machine
| at every party, so more commonly we just don't bother, and
| some people end up with the annoying "Activate Windows"
| overlay in the corner. But it's actually far enough into the
| corner and subtle enough that often people just don't even
| notice it's there and it's fine...
| jmb99 wrote:
| If you've already purchased licenses, have you considered
| using more "illicit" means keep the machines activated?
| fuzzfactor wrote:
| Love that party house !
|
| Looks like you have a similar soundbar to my LP-S08.
|
| I had to open it up to access the battery, and interestingly
| the audio output on the PCB was naturally laid out for stereo
| operation like you would expect.
|
| Except on mine only one channel of the PCB was populated from
| the factory, the other channel empty, and it was output to both
| speakers as mono not stereo.
| INTPenis wrote:
| In my experience booting VMs over iSCSI at home the network speed
| made it practically unusable. I'm happy Terin got Windows working
| though.
|
| Clearly it depends a lot on the network backend.
| nicman23 wrote:
| if you do not do rdma the latency kills everything from what i
| understand.
| muhehe wrote:
| Instead of hiding which game it is, the company should be
| properly shamed.
| glitchc wrote:
| There's more than one. I originally thought it was LoL, but
| then he mentions Steam at the end.
| acgh213 wrote:
| if I had to guess from personal experience, Destiny 2 is in
| this exact situation.
|
| someone even patched out the checks and got it running and
| was still caught and banned permanently [1]
|
| it's a shame too because it ran on stadia for years before
| Google killed it and that ran Linux as it's main system. so
| there's clearly the ability. just a desire to prevent it at
| all levels
|
| [1] https://www.protondb.com/app/1085660#leulnGaiUV
| ahepp wrote:
| I'm going to try this out later! Hopefully it works with a Unix
| iSCSI daemon.
|
| I have been doing this with a lot of my machines at home, and
| it's been awesome. Swapping what OS boots on what machine is as
| simple as symlinking a MAC address to a directory on my TFTP
| server.
|
| I wrote up some notes for doing this on Unix systems:
| https://ahepp.dev/2024/08/05/linux-without-hdds.html
|
| Looking at it again now, it seems as though these instructions
| assume the system is being booted in a VM. It's not substantially
| more difficult to boot on real hardware, as long as you add the
| relevant network drivers to your initramfs and handle changes to
| any interface names.
| tehcopec wrote:
| I ran a bunch of workstations with Windows 7 for many years this
| way, also using the predecessor of PrimoCache with local SSDs
| with a cache. I had an OpenSolaris server for the backing storage
| with ZFS/COMSTAR.
|
| I've been pretty much Linux only for the last ~15 years or so,
| but NVMe/TCP is so much faster. I would look at doing a secondary
| disk applications using NVMe/TCP once booted.
| mrspuratic wrote:
| I used to netboot Windows 95 on a campus Novell network... you
| can probably work out how long ago that was ;) It started
| booting a 1.44MB floppy boot image of DOS 7 via (IIRC) BootWare
| boot ROM (pre-PXE) and switching to Windows using Joe Doupnik's
| installation method. In fact I was lucky enough to get Joe to
| help out in person on his way back from a nearby conference.
|
| Performance was not where it needed to be though, we switched
| to a netboot way of imaging Windows 95 onto the local disk --
| actually 2 images, a "good" copy on a hidden partition that
| could quickly replace the live version when it hit cruft force
| 4 or thereabouts; and running (almost) everything else over the
| network. Fun fact: ~1200 PCs used to run an awk (gawk.exe v3.0)
| configuration script during boot.
| tehcopec wrote:
| Haha, that's awesome!
___________________________________________________________________
(page generated 2025-02-28 23:00 UTC)