Any considerations for upgrading from 12 to 13 regarding ZFS? reddit / freebsd ---------------------------------------------------------------------- jllauser · 2021-02-08 21:30 Is anyone aware of any specific documentation around upgrading from FreeBSD 12.2 to 13.0-BETA1 regarding ZFS? I see this note in `/usr/src/UPGRADING` that very slightly concerns me: 20200824: OpenZFS support has been integrated. Do not upgrade root pools until the loader is updated to support zstd. Furthermore, we caution against 'zpool upgrade' for the next few weeks. The change should be transparent unless you want to use new features. Not all "NO_CLEAN" build scenarios work across these changes. Many scenarios have been tested and fixed, but rebuilding kernels without rebuilding world may fail. The ZFS cache file has moved from /boot to /etc to match the OpenZFS upstream default. A fallback to /boot has been added for mountroot. Pool auto import behavior at boot has been moved from the kernel module to an explicit "zpool import -a" in one of the rc scripts enabled by zfs_enable=YES. This means your non-root zpools won't auto import until you upgrade your /etc/rc.d files. I have root on ZFS, but also I have `/usr` on a *different* ZFS pool, so I'm wondering if I'm going to have a small amount of trouble after the first reboot in the typical upgrade (from source) model where you install the kernel but not world or mergemaster. I also assume the bit about `Do not upgrade root pools until the loader is updated` refers to the bootloaders that get installed using `gpart bootcode`? ---------------------------------------------------------------------- edthesmokebeard · 2021-02-08 22:23 From last August, maybe a newer 12.x version already has the updates needed? This seems like a big deal. ---------------------------------------------------------------------- jllauser · 2021-02-08 22:24 I thought they went into CURRENT last year, but not into 12.x. ---------------------------------------------------------------------- jllauser · 2021-02-08 22:29 But regardless, my question still stands. Even if I'm not upgrading to 13, is there documentation for going from a version of FreeBSD with the "old" ZFS to one with OpenZFS? ---------------------------------------------------------------------- daemonpenguin · 2021-02-08 23:52 I think the file you posted tells you everything you need to know. Since you have a separate pool for /usr then you need to add the import command to your start-up process. This is probably done automatically, but double-check before you reboot. Otherwise you should be fine. I don't see anything in there to be concerned about. ---------------------------------------------------------------------- grahamperrin · 2021-02-09 00:06 > everything you need to know Not quite everything. FreeBSD bug [244831 – sysutils/openzfs-kmod 2020022700 mount of encrypted dataset fails with I/O error](https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=244831) ---------------------------------------------------------------------- [deleted] · 2021-02-09 00:11 [deleted] ---------------------------------------------------------------------- jllauser · 2021-02-09 00:12 No, nothing encrypted. ---------------------------------------------------------------------- jllauser · 2021-02-09 02:22 I don't really know that it does tell me everything I need to know. This seems like something that's going to affect an awful lot of users. I feel like some step-by-step instructions might be warranted here. Like if I need to install a new bootloader at some point, where in the upgrade process should that happen? Do I need to boot into single user mode after installing the kernel but before upgrading my rc scripts to manually import the pools? ---------------------------------------------------------------------- grahamperrin · 2021-02-09 02:35 From FreeBSD bug [253364 – boot partition documentation (FreeBSD could not boot where a non-boot OpenZFS pool included a filesystem that uses ztsd compression)](https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=2 53364): > … I assume that the issue will also be observable with `12.2-RELEASE-p3` … tl;dr – not specific to `13`: * think carefully before using zstd with OpenZFS on any part of any *internal* drive * be prepared to manually update boot partitions. ---------------------------------------------------------------------- daemonpenguin · 2021-02-09 02:38 What do you think is going to affect users? And why do you think you need to install a new bootloader? The updating text doesn't say anything about that. ---------------------------------------------------------------------- jllauser · 2021-02-09 02:40 The text literally says "Do not upgrade root pools until the loader is updated". ---------------------------------------------------------------------- daemonpenguin · 2021-02-09 02:46 Yes.... Exactly my point. Why do you think this is going to be an issue? In case you're not aware, what this is saying is don't upgrade your ZFS pool (the ZFS feature set) before the boot loader supports it. It doesn't mean you have to upgrade your boot loader before or during the OS upgrade. This wouldn't affect a OS upgrade, it just means you should hold off running ZFS upgrade until the boot loader support zstd compression. A root pool isn't the OS, it's the feature set of the filesystem. ---------------------------------------------------------------------- jllauser · 2021-02-09 02:49 I don't have any compression enabled on my root pool, but I literally only have / on my root pool. Everything else, including /usr, is on my "big" pool, and I do intend to use zstd once I have it available. ---------------------------------------------------------------------- jllauser · 2021-02-09 02:52 Yes, I understand that. But I typically don't upgrade my bootloader during upgrades. The last time I even thought about the bootloader was the last time I replaced a boot disk on a machine. So the fact that it needs to explicitly be upgraded some time during the upgrade (or never run zpool upgrade on that pool) seems significant. ---------------------------------------------------------------------- grahamperrin · 2021-02-09 04:23 Thanks, > … Everything else, including /usr, is on my "big" pool, and I do intend to use zstd once I have it available. If this is a separate pool then you might like to begin using zstd now. Add the `sysutils/openzfs` package, install `sysutils/openzfs- kmod` from ports then edit your `/boot/loader.conf` `sysutils/openzfs-kmod` was built successfully in very recent http://b eefy2.nyi.freebsd.org/build.html?mastername=122amd64- quarterly&build=563981 and http://beefy6.nyi.freebsd.org/build.html?ma stername=122amd64-default&build=563982 (both jails run `12.2`) however I don't know when `pkg upgrade` will benefit from the `12.2`-oriented builds. ---------------------------------------------------------------------- avgapon · 2021-02-09 11:14 With ZFS, I just use boot environments and install new world and kernel at the same time. Of course, having /usr on a separate \_pool\_ is a very unconventional setup, so you will need to prepare beforehand. Except for the zpool import issue described in the UPDATING, my upgrade was completely uneventful. During the first boot to the 13 BE I sent to single user mode first and manually mounted my data pool. ---------------------------------------------------------------------- daemonpenguin · 2021-02-09 14:13 You don't need to upgrade the boot loader manually or explicitly. It'll probably just be part of the regular update process when 13.0 hits -RELEASE status. I don't see how you're getting these ideas. ---------------------------------------------------------------------- jllauser · 2021-02-09 15:36 Perhaps we're talking about two different things? I assume writing the updated files in `/boot` takes place during `installkernel` during a source upgrade or... at some point during `freebsd-update`. But I'm assuming that this note in the documentation is talking about the actual loader that gets written into sector 1 on the boot drive, which is installed using `gpart` and I didn't think that got touched by either process. Am I wrong here? ---------------------------------------------------------------------- grahamperrin · 2021-02-09 18:33 > … bootloader? The updating text doesn't say anything about that. … When manual attention to a loader file is appropriate (or required), it's not always in the context of `/usr/src/UPDATING`. Manual attention with regard to OpenZFS is appropriate, for some users, both with and without this context. ---------------------------------------------------------------------- grahamperrin · 2021-02-09 18:53 > You don't need to upgrade the boot loader manually or explicitly. … In my case (not upgrading) it was necessary for me to manually replace `BOOTx64.efi` on the `efi` partition. % lsblk ada0 DEVICE MAJ:MIN SIZE TYPE LABEL MOUNT ada0 0:133 466G GPT - - ada0p1 0:135 200M efi gpt/efiboot0 - ada0p2 0:137 512K freebsd-boot gpt/gptboot0 - -:- 492K - - - ada0p3 0:139 16G freebsd-swap gpt/swap0 SWAP ada0p3.eli 2:28 16G freebsd-swap - SWAP ada0p4 0:141 450G freebsd-zfs gpt/zfs0 ada0p4.eli 0:152 450G - - - -:- 4.0K - - - % gpart show /dev/ada0 => 40 976773088 ada0 GPT (466G) 40 409600 1 efi (200M) 409640 1024 2 freebsd-boot (512K) 410664 984 - free - (492K) 411648 33554432 3 freebsd-swap (16G) 33966080 942807040 4 freebsd-zfs (450G) 976773120 8 - free - (4.0K) % With regard to automation, quoting a couple of lines from IRC a few hours ago: > nope, we're not that good yet – and: > We don't do a good job of advertising this kind of stuff ---------------------------------------------------------------------- throwaway920311 · 2021-02-12 13:04 What to consider is that FreeBSD 13 is still a moving target in beta and it would be wise of you not to be a first mover either, when v13 is eventually released. Why not wait for FreeBSD-13p3-RELEASE and have a good nights sleep without worrying? ---------------------------------------------------------------------- jllauser · 2021-02-12 14:14 True, but I don't think much is going to change about how to migrate the filesystem between now and release. ---------------------------------------------------------------------- grahamperrin · 2021-02-12 20:32 > 13p3 p3? That might be a *long* wait … ---------------------------------------------------------------------- Xerxero · 2021-02-14 09:49 It’s in beta. Hardly a moving target. It just bugs now not switching major things.