+------------------------------------------------------------------ +title: OPNSense Upgrade failure +date: Mon, 20 Jul 2026 06:32:17 +0200 +author: -fab- fab@redterminal.org +------------------------------------------------------------------ ▄████▄ ▄▄▄▄ ▄▄ ▄▄ ▄█████ ▄▄▄▄▄ ▄▄ ▄▄ ▄▄▄▄ ▄▄▄▄▄ ██ ██ ██▄█▀ ███▄██ ▀▀▀▄▄▄ ██▄▄ ███▄██ ███▄▄ ██▄▄ ▀████▀ ██ ██ ▀██ █████▀ ██▄▄▄ ██ ▀██ ▄▄██▀ ██▄▄▄ ██ ██ ▄▄▄▄ ▄▄▄▄ ▄▄▄▄ ▄▄▄ ▄▄▄▄ ▄▄▄▄▄ ██ ██ ██▄█▀ ██ ▄▄ ██▄█▄ ██▀██ ██▀██ ██▄▄ ▀████▀ ██ ▀███▀ ██ ██ ██▀██ ████▀ ██▄▄▄ ██████ ▄▄▄ ▄▄ ▄▄ ▄▄ ▄▄ ▄▄▄▄ ▄▄▄▄▄ ██▄▄ ██▀██ ██ ██ ██ ██ ██▄█▄ ██▄▄ ██ ██▀██ ██ ██▄▄▄ ▀███▀ ██ ██ ██▄▄▄ This is a post about my yesterdays update "adventures". <== General Backup/Update process ==> Like (nearly) every Sunday, I tried to do my backup/update duties for all my workstation/laptops and my whole homelab. Normally all goes smoothly without too much trouble. Daily automated backups of my mail server and SyncThing data are performed using BorgBackup to be stored on my 5x 4TB HDD ZFS raidz2. The VMs/containers are backed up on Sundays in the early morning. I just send/receive the incremental backups from this raid to my offline 8TB USB Harddisk. And I also copy my "Relax-and-Recover"[1][2] (ReaR) images for my Proxmox system disks to restore my Proxmox nodes in case of a desaster to my Desktop. The images are generated every Sunday at 04:00 and can recover the system disks in headless mode from a Ventoy USB stick. Highly recommended. [1]: https://relax-and-recover.org/ [2]: gemini://redterminal.org/gemlog/2025-09-17-Relax-and-Recover_on_Proxmox.gmi I use Ansible to update my Proxmox Cluster, its VMs/containers, my 4x Raspberry Pi 4 Incus cluster and my InfoScreen, which is just a Raspberry Pi Zero 2 W to monitor my network on a console screen. But my workstations and my external FreeBSD VPSes and my OpnSense router are always updated manually. And I'm always nervous if router updates are available and this time I've been caught! <== The OpnSense router update desaster ==> I have to admit that I didn't do my backups/updates last Sunday, because reasons (too lazy). And I was already prepared for an update of OpnSense and so there was an update to version 26.1.11. As always I was nervous but it went smoothly. But after that I searched for updates again after the 26.1.11 update was through. And then this well-presented, trustworthy message appeared about an upgrade to version 26.7, which also happens to be based on FreeBSD 15.1. Strangely, I wasn't as nervous this time, and I clicked the attractive green button with the checkmark "✔ Upgrade.". And then after the kernel update it tried to reboot and... nothing! Router unreachable. <== Upgrade in half state ==> After waiting for about 20 minutes, I still thought, maybe the router was doing the update while offline, but I was already nearly in panic. I tried to reboot my router manually by pressing the power button. To my surprise it really booted. But after some checks I found out that it wasn't working correctly anymore. My dnsmasq DNS/DHCP server wasn't reachable from my VLANs, just from my Desktop on my main/untagged LAN. So my update was in a half-done state. <== How I made things worse ==> After a short search I found out, that the Intel microcode plugin was the culprit. I should try disabling it and retry the upgrade. So I uninstalled the Intel microcode package and tried again to upgrade... to no avail. It told me that I was not on a FreeBSD 14 version. Sure, the kernel/system was partly upgraded to FreeBSD 15.1. I also tried an "opnsense-bootstrap -r 26.7 -f" which also gave me the error that I wasn't using a FreeBSD 14 version, even with the force option. Then I took the *very bad* advise from the Brave Search AI to download the "opnsense-bootstrap" script from the OpnSense 26.7 version as a fix and ran "sh opnsense-bootstrap.in -r 26.7". And it *did* run! The first (and last) thing it did, was deleting all my remaining packages. After deleting package 125 (openssl) it suddenly stopped. And froze! And that was it. It didn't boot at all anymore. Complete desaster! The only thing i could do was to reinstall the whole system. <== The reinstallation ==> At that point I already was very upset and *very* angry! I had to take the router out from my closet which was buried under a lot of cable spaghetti and connect it to my InfoScreen monitor and a keyboard and prepare some USB stick with the new OpnSense install image. Luckily I have a smartphone plan with 10GB internet volume, so I downloaded the new USB image from the OpnSense website and dd'd it to my USB stick. After reconfiguring the UEFI BIOS it booted strait into the installer from the USB stick and the installation was easy and strait forward. And this time I didn't made the mistake to choose UFS instead of ZFS. So I installed a ZFS system. After rebooting I was a little calmer. The system booted as expected. But I still had no configuration and I really didn't want to set up all the firewall rules and VLAN configuration and all the little things, which I mostly already forgot from ground up again. Fortunately I still had all the backup configuration files distributed to all my workstations over SyncThing, when it was still working. After some troubles finding out how to set up the backup config.xml on an USB stick alone (you can't simply choose the xml file from the filesystem!), I finally managed to install the configuration backup with "opnsense-import". After a reboot it seemed to have all the config back running. So after some cleaning of my closet and the reinstallation of my InfoScreen, I put the router back on its place, connected all the network and power cables, I did a restart and hoped for the best. And to my surprise it worked! I had internet again and all my servers, VMs and containers became IPv4 and IPv6 (also ULAs) addresses and were reachable by their DNS name. Phew! I just had to reinstall the dynDNS client (ddclient) and do some small configuration I missed to make a backup of and now my whole homelab is running again as if nothing happened. <== The Result ==> What was supposed to be a relatively short update to my home network (no more than an hour, as usual) turned into a hours-long disaster recovery process. I don't know how long it took me to get through the whole ordeal (I was yelling with frustration), and I needed some time to calm down, even though things eventually worked out okay. At least now I have a decent ZFS system and can always take nice snapshots before updates. But I don't understand how OpnSense could release an update process with such errors. I almost made a forum post with words like "idiots", but I managed to restrain myself in time. That's all I want to say about my lovely Sunday. All in all - Have fun anyway! -fab-