boot error: zfs: out of temporary buffer space reddit / freebsd ---------------------------------------------------------------------- matamoroos · 2021-05-29 13:41 Hi all, I ran into this issue this morning upgrading my VPS from 13.0-RELEASE to p1. A quick web search turned up this post here from a month ago: [https://www.reddit.com/r/freebsd/comments/mtx5cf/freebsd\_130\_upgrad e\_zfs\_out\_of\_temporary\_buffer/](https://www.reddit.com/r/freebsd/ comments/mtx5cf/freebsd_130_upgrade_zfs_out_of_temporary_buffer/) Following the instructions there, I rebooted the VPS using the hosting company's FreeBSD emergency console and issued this command: gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 vtbd0 and rebooted. No luck. I'm stuck at the `boot:` prompt (see below). Anyone have a suggestion for me? I created a ZFS snapshot using `bectl` before running `freebsd- update`. How would I try rebooting from the older snapshot? Would that even work or would the same zfs error occur? Please note: I accidentally named the new active partition 12.2-RELEASE-p7, but it is actually 13.0-RELEASE-p1. (I was upgrading several 12.2 systems this morning as well) TIA https://preview.redd.it/6zpznw9pa2271.png?width=1142&format=png&auto=w ebp&s=6407416431c6e4c3383358b2614c4ce9cafb16f2 ---------------------------------------------------------------------- dlangille · 2021-05-29 13:53 > I created a ZFS snapshot using bectl before running freebsd-update. How would I try rebooting from the older snapshot? During the boot process, on the boot menu, there is an option for selecting the boot environment to use. > Would that even work or would the same zfs error occur? Perhaps. It doesn't sound like a `zpool upgrade` has been done, so perhaps it will boot. ---------------------------------------------------------------------- dlangille · 2021-05-29 13:53 Be patient. I know how frustrating a problem like this can be. Don't rush. ---------------------------------------------------------------------- matamoroos · 2021-05-29 16:17 I don''t get as far as the boot menu, alas. ---------------------------------------------------------------------- dlangille · 2021-05-29 16:19 Verify bootfs on the pool. Make sure it’s what you want. Not near a computer now to give you the precise command. ---------------------------------------------------------------------- matamoroos · 2021-05-29 16:20 Thanks. This community has been very helpful in the past, and I trust I may be allowed to draw on the collective wisdom here once again 🤞🙏 ---------------------------------------------------------------------- grahamperrin · 2021-05-29 18:59 There are methods that can be used at the loader command prompt, ~~but (sorry) I don't recall the details~~. ~~Key **Escape** just once soon after starting the computer to get the prompt.~~ A visual hint: https://i.imgur.com/4ZcE8SL.png To remind yourself of the names of your boot environments: `lszfs zroot/ROOT/` – then: `set vfs.root.mountfrom=zfs:zroot/ROOT/13.0-RELEASE` `boot -s` – and in single user mode: 1. key Return or Enter 2. `bectl list -c creation` 3. `bectl activate 13.0-RELEASE` … I'm not sure about step 3, can't recall whether the setting at the loader prompt has a persistent effect. ### Postscript Assuming boot environment name `13.0-RELEASE` based on other comments. ---------------------------------------------------------------------- grahamperrin · 2021-05-29 19:18 boot loader aside … if you can boot FreeBSD from another available disk, zfsbootcfg(8) might be our friend. https://man.freebsd.org/zfsbootcfg(8) in particular, https://man.freebsd.org/zfsbootcfg(8)#EXAMPLES **Postscript** Probably simpler to set a `bootfs` property. https://old.reddit.com/r/freebsd/comments/nnnpxm/-/gzvtmo5/ ---------------------------------------------------------------------- grahamperrin · 2021-05-29 20:03 > Verify bootfs on the pool. Yay! Boot environments become less mysterious when it's realised that there's a bootfs property. https://man.freebsd.org/zpool-get(8) ▶ `zpool set` Here, for example: % bectl list -c creation BE Active Mountpoint Space Created r357746-Waterfox - - 59.2G 2020-03-10 18:24 n246499-097e8701c9f-d - - 503M 2021-05-12 16:33 n246689-91f251b2ab3-f - - 315M 2021-05-25 03:24 n246689-91f251b2ab3-g - - 17.2M 2021-05-29 05:04 n247020-e0fa04e257c-a NR / 115G 2021-05-29 20:06 % zpool get bootfs copperbowl NAME PROPERTY VALUE SOURCE copperbowl bootfs copperbowl/ROOT/n247020-e0fa04e257c-a local % zfs list | grep ROOT copperbowl/ROOT 115G 197G 88K none copperbowl/ROOT/n246499-097e8701c9f-d 2.38M 197G 40.7G / copperbowl/ROOT/n246689-91f251b2ab3-f 14.5M 197G 40.3G / copperbowl/ROOT/n246689-91f251b2ab3-g 3.75M 197G 40.2G / copperbowl/ROOT/n247020-e0fa04e257c-a 115G 197G 40.2G / copperbowl/ROOT/r357746-Waterfox 348M 197G 59.3G / % zpool list -v NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT Transcend 464G 355G 109G - - 39% 76% 1.00x ONLINE - gpt/FreeBSD%20ZFS 464G 355G 109G - - 39% 76.4% - ONLINE cache - - - - - - - - - da0 14.4G 14.4G 53.0M - - 0% 99.6% - ONLINE copperbowl 448G 238G 210G - - 53% 53% 1.00x ONLINE - ada0p4.eli 448G 238G 210G - - 53% 53.1% - ONLINE cache - - - - - - - - - gpt/cache-copperbowl 28.8G 24.3G 4.54G - - 0% 84.2% - ONLINE % ---------------------------------------------------------------------- dlangille · 2021-05-29 20:04 Success! ---------------------------------------------------------------------- dlangille · 2021-05-29 20:04 I am glad you could figure that out based upon my short reply. ---------------------------------------------------------------------- grahamperrin · 2021-05-29 21:03 > I created a ZFS snapshot using bectl before running freebsd-update. … Did you also activate then boot the environment before beginning the update? ---------------------------------------------------------------------- matamoroos · 2021-05-29 21:15 Hi, This looks promising, but I can't get beyond the `boot:` prompt. I see this: FreeBSD/x86 boot Default: zroot/ROOT/12.2-RELEASE-p7 boot: The default listed is my latest boot environment. What I am supposed to type here? Nothing seems to work. Thanks ---------------------------------------------------------------------- grahamperrin · 2021-05-29 21:20 Did you set what's required at the loader prompt? (Before the `boot:` prompt.) Also, maybe a more important question: https://old.reddit.com/r/freebsd/comments/nnnpxm/-/gzwrb48/ ---------------------------------------------------------------------- matamoroos · 2021-05-29 21:24 OK, I boot from the hosting company's emergency FreeBSD rescue mode. I type: # zfsbootcfg "zfs:zroot/ROOT/13.0-RELEASE" zfsbootcfg: command not found. ?? I see that the rescue mode boots FreeBSD v10.3. Maybe that command isn't available with this version? # ---------------------------------------------------------------------- matamoroos · 2021-05-29 21:24 Yes, i did activate it. ---------------------------------------------------------------------- matamoroos · 2021-05-29 21:26 Thanks. Still no luck. :/ ---------------------------------------------------------------------- matamoroos · 2021-05-29 21:33 the boot: prompt is the first thing that appears. I don't see any opportunity to configure anything before it is displayed. Pressing ESC doesn't help. Yes, as I mentioned below, I did activate the new BE. ---------------------------------------------------------------------- grahamperrin · 2021-05-30 04:52 > Maybe that command isn't available with this version? True. : >> **zfsbootcfg** appeared in FreeBSD 11.1. ---------------------------------------------------------------------- grahamperrin · 2021-05-30 05:02 Taking the boot environment name from https://old.reddit.com/r/freebsd/comments/nnnpxm/-/gzwtrt5/ In your 10.3 rescue environment, try: 1. `zpool import -N zroot` 2. `zpool set bootfs=zroot/ROOT/13.0-RELEASE zroot` https://man.freebsd.org/zpool-import(8) https://man.freebsd.org/zpool-get(8) It's also possible to set the property at import time but for now, let's take things step by step … ---------------------------------------------------------------------- matamoroos · 2021-05-30 12:15 I'm getting an error message, apparently because the hosting company's rescue boot environment is so old: # zpool import -fN zroot This pool uses the following feature(s) not supported by this system: com.delphix:spacemap_v2 org.zfsonlinux:project_quota org.zfsonlinux:userobj_accounting com.delphix:log_spacemap and tells me this pool can only be mounted read-only. This works: # zpool import -o readonly=on -fN zroot However, this does not: # zpool set bootfs=zroot/ROOT/13.0-RELEASE zroot Cannot set property for 'zroot': pool is read-only. One cannot boot from a read-only ZFS zpool, apparently. ---------------------------------------------------------------------- grahamperrin · 2021-05-30 13:33 Initial support for *feature flags* was added in 2012 but I can't tell when support for those four features was added. 2020: [Merge OpenZFS support in to HEAD.](https://cgit.freebsd.org/src /commit/?id=9e5787d2284e187abb5b654d924394a65772e004) 10.3 reached end of life 2018-04-30 so the hosting company could do better, but let's not dwell on that. ---- ▶ https://old.reddit.com/r/freebsd/comments/nnnpxm//gzwciwj/ and focus on the getting a loader command prompt … ---------------------------------------------------------------------- grahamperrin · 2021-05-30 13:57 > the boot: prompt is the first thing that appears. I don't see any opportunity to configure anything before it is displayed. Pressing ESC doesn't help. My bad. It seems that **Escape** is effective only after the boot loader menu appears: https://docs- dev.freebsd.org/en/books/handbook/bsdinstall/#bsdinstall-view-probe (I was previously confused by helloSystem, which hides things). Whilst booted from the 10.3 rescue environment, please run: `gpart show` ---------------------------------------------------------------------- matamoroos · 2021-05-31 14:37 My VPS is back in action. Via the rescue console, I copied the gtzfsboot file from another VPS running v13 and installing that manually with gpart bootcode command. I could then reboot. The hosting company says they will eventually update the FreeBSD rescue console to a more recent version, but not in the short-term. I've learned the hard way that running a version of the OS more recent than the company's rescue environment probably is not such a great idea. Thanks for your responses here, much appreciated. ---------------------------------------------------------------------- grahamperrin · 2021-05-31 19:40 Thanks for the follow-up. If there had been an EFI system partition, I would have suggested making good there (because there's debatably less 'risk' of the wrong partition being targeted when things are written.) I'm glad that you got it sorted with gpart!