Linux+KVM moving to FreeBSD+Bhyve reddit / freebsd ---------------------------------------------------------------------- edthesmokebeard · 2022-08-14 01:37 I'm looking to migrate a VM host over to FreeBSD, as a pure learning exercise. Outside of the ability to do filesystem-based snapshots using ZFS (instead of using libvrt/qemu snapshots), are there any solid reasons for or against Bhyve over KVM? Performance, manageability, etc? Like I said I want to move it over for the exercise, but I'm looking for legit reasons as well. Edit: update, trying to move KVM VMs to Byhve, I can create them, when I try and start them using an existing disk image it stays in the "Bootloader" state forever and never boots. Anyone know how to diagnose? ---------------------------------------------------------------------- IanArcad · 2022-08-14 03:44 I have some experience with bhyve and have been using it to run a few virtual servers for about two years. Unfortunately its the only VM setup I have ever used, so I don't know how it compares with anything else. There's not a lot to say about it, since it just works. I also use cpdup to back up file systems periodically (a good freebsd tip in general). I think the only real surprise I had was VM disk usage consumes host CPU. That may be because of my setup (NFS mounting the hosts file system), or maybe that's just how virtualization is, I don't know. That's made me start to think that using a jail might be better for some disk-intensive applications. Other than that, performance has been great, especially CPU performance, and upgrading the virtual machines to newer FreeBSD versions worked flawlessly. I also liked how you could use PF to firewall individual machines, but that's more a a FreeBSD thing than anything specific to bhyve really. ---------------------------------------------------------------------- Epicure_BE · 2022-08-14 06:07 Such a setup works well for me for over 4 years adding also zrepl to the mix. Be prepared for command line management of vms. I would consider that bhyve is more for hard core enthusiasts than the average Jo using a graphical interface. Good luck! ---------------------------------------------------------------------- rykolepl · 2022-08-14 07:24 Running non-uefi VMs in bhyve is pain in the ass. Running UEFI VMs in bhyve is easy. I only run UEFI VMs, and I use bash scripts (that are basically 4 commands) to start them. Basically it's: - create tap interface - attach tap interface to bridge - start vm itself - destroy tap interface since this script stays in foreground, i use daemon(8) to run that script. This may help: https://wiki.freebsd.org/bhyve/UEFI I ommited the "-l com1,stdio" as I didn't need the serial port. And I replaced ahci-hd with virtio-blk to get better performance. I use zfs zvol instead of file as VM disks. The dataset on host uses native zfs encryption. ---------------------------------------------------------------------- _w62_ · 2022-08-14 10:25 The latest and greatest hardware has better support in Linux. So it is easier to establish a virtualization platform with Linux. Most, if not all, VM management infrastructure are assuming taking to Linux machines. There is no immediate benefit for the guest VM to migrate from Linux platforms to FreeBSD platforms. ---------------------------------------------------------------------- Diligent_Ad_9060 · 2022-08-14 11:00 I recommend to look into some third-party tooling to manage VMs. I prefer vm-bhyve, https://github.com/churchers/vm-bhyve ---------------------------------------------------------------------- VastAd1765 · 2022-08-14 11:18 My company runs the latest and greatest hardware and we use FreeBSD. ---------------------------------------------------------------------- IanArcad · 2022-08-14 13:22 Right, that's the one I use and it's pretty straightforward. ---------------------------------------------------------------------- ruyrybeyro · 2022-08-14 14:44 Are using virtio for virtual I/O and an i7+ processor? ---------------------------------------------------------------------- _w62_ · 2022-08-14 16:02 Could you share with us the exact hardware, if you don't mind? ---------------------------------------------------------------------- IanArcad · 2022-08-14 23:14 Yes, virtio-net and virtio-blk for network interface and disk. And the CPU is a i9-7920X (Skylake-X) although planning on changing it out sooner or later. ---------------------------------------------------------------------- 120r · 2022-08-14 23:16 Don’t know if this has changed by my understanding is that PCI pass through is not that great so depending on what you want to do. ---------------------------------------------------------------------- Agreeable_Egg4260 · 2022-08-15 14:12 I think bhyve seems to be faster than kvm when running windows 2019. However kvm has more tools. There is a project called Bhyve VM Control Panel that is really good for managing bhyve vm's. ---------------------------------------------------------------------- knight_to_bishop · 2022-10-12 14:21 To make your life easier, FreeBSD supports libvirt, so you can just virt-manager install your guests just like old Centos days. So all normal commands work, virsh etc ---------------------------------------------------------------------- PhoxFyre007 · 2023-12-07 04:40 Does virtmanager interface with bhyve?