FreeBSD Guest - Nvidia GPU Passthrough - Error starting Xorg reddit / freebsd ---------------------------------------------------------------------- toasty_fe · 2022-04-28 01:15 Hey everyone! So I'm an Arch Linux user and I've recently learned about KVM/QEMU and PCI Passthrough. As I was learning about FreeBSD, I decided to give it a try as a VM. I currently have a Microsoft Surface Book (1st Gen with Performance Base) which has an Intel iGPU and a Nvidia GeForce GTX 965M dGPU. It's basically an Optimus Laptop with 2 GPUs For being able to properly identify the Nvidia GPU and other functionalities, I use a custom kernel for Linux, made specifically for Surface devices. Here is the [Linux Surface](https://github.com/linux-surface/linux-surface) project. With it and a couple of other scripts I was able to detect the Nvidia GPU and use it normally. So back to the VM project, I decided to use the PCI Passthrough tecnology available on Linux through KVM/QEMU to power FreeBSD with the Nvidia GPU and leave the iGPU for the Arch Linux Host. I was able to isolate the GPU properly for the Host, as the `dmesg` states when `dmesg | grep -i vfio`:`[ 0.898938] VFIO - User Level meta-driver version: 0.3` `[ 0.918740] vfio_pci: add [10de:1427[ffffffff:ffffffff]] class 0x000000/00000000` And also `lspci -k` outputs for NVIDIA:`01:00.0 3D controller: NVIDIA Corporation GM206M [GeForce GTX 965M] (rev a1)` `Subsystem: Microsoft Corporation Device 0008` `Kernel driver in use: vfio-pci` `Kernel modules: nouveau, nvidia_drm, nvidia` By booting the VM with it and installing xorg and [nvidia-secondary- driver](https://www.freshports.org/x11/nvidia-secondary-driver) as it seems to be correct Nvidia driver for the project. My current `/etc/rc.conf` has:`kld_list="linux nvidia-modeset i915kms"` `nvidia_xorg_enable="YES"` By doing pciconf on the FreeBSD also shows that NVIDIA was properly detected as the only VGA: vgapci0@pci0:4:0:0: class=0x030200 rev=0xa1 hdr=0x00 vendor=0x10de device=0x1427 subvendor=0x1414 subdevice=0x0008 vendor = 'NVIDIA Corporation' device = 'GM206M [GeForce GTX 965M]' class = display subclass = 3D And the `kldstat` shows that both nvidia and nvidia-modeset has been loaded: Id Refs Address Size Name 1 48 0xffffffff80200000 1f11f90 kernel 2 1 0xffffffff82113000 6800a0 zfs.ko 3 1 0xffffffff82794000 ae38 cryptodev.ko 4 3 0xffffffff82918000 388f8 linux.ko 5 3 0xffffffff82951000 db70 linux_common.ko 6 1 0xffffffff8295f000 107310 nvidia-modeset.ko 7 1 0xffffffff82c00000 1faac70 nvidia.ko 8 1 0xffffffff82a67000 158430 i915kms.ko 9 1 0xffffffff84bab000 7e020 drm.ko 10 2 0xffffffff82bc0000 cbc8 linuxkpi_gplv2.ko 11 1 0xffffffff82bcd000 3250 ichsmb.ko 12 1 0xffffffff82bd1000 2180 smbus.ko And I cannot start X so I can check if it's working properly before installing a Desktop Environment. Everytime I boot, I get the following warning which leads me to think that is the reason why it doesn't start Xorg:`ACPI Warning: \_SB.PCI0.S12.S00._DSM: Argument #4 type mismatch - Found [Buffer], ACPI requires [Package] (20201113/nsarguments-212)` If I force starting X with `startx` this is the output: xauth: file /root/.serverauth.844 does not exist xauth: file /root/.Xauthority does not exist xauth: file /root/.Xauthority does not exist X.Org X Server 1.20.14 X Protocol Version 11, Revision 0 Build Operating System: FreeBSD 13.0-RELEASE-p11 amd64 Current Operating System: FreeBSD freebsd.mydomain 13.0-RELEASE-p11 FreeBSD 13.0-RELEASE-p11 #0: Tue Apr 5 18:54:35 UTC 2022 root@amd64- builder.daemonology.net:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64 Build Date: 12 April 2022 11:45:24AM Current version of pixman: 0.40.0 Before reporting problems, check http://wiki.x.org to make sure that you have the latest version. Markers: (--) probed, (**) from config file, (==) default setting, (++) from command line, (!!) notice, (II) informational, (WW) warning, (EE) error, (NI) not implemented, (??) unknown. (==) Log file: "/var/log/Xorg.0.log", Time: Wed Apr 27 22:57:44 2022 (==) Using system config directory "/usr/local/share/X11/xorg.conf.d" scfb trace: probe start scfb trace: probe done scfb trace: probe start scfb trace: probe done (EE) Fatal server error: (EE) no screens found(EE) (EE) Please consult the The X.Org Foundation support at http://wiki.x.org for help. (EE) Please also check the log file at "/var/log/Xorg.0.log" for additional information. (EE) (EE) Server terminated with error (1). Closing log file. Cxinit: giving up xinit: unable to connect to X server: Connection refused xinit: unexpected signal 2 **Do you guys know how can I proceed and Start X properly? I tried with the default nvidia driver and it did not work.** In case you want to check, the `/var/log/Xorg.0.log` log is in the comments. ​ ---------------------------------------------------------------------- toasty_fe · 2022-04-28 02:07 here is the `Xorg.0.log` \[ 47.013\] X.Org X Server 1.20.14 X Protocol Version 11, Revision 0 \[ 47.013\] Build Operating System: FreeBSD 13.0-RELEASE-p11 amd64 \[ 47.013\] Current Operating System: FreeBSD freebsd.mydomain 13.0-RELEASE-p11 FreeBSD 13.0-RELEASE-p11 #0: Tue Apr 5 18:54:35 UTC 2022 root@amd64- builder.daemonology.net:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64 \[ 47.013\] Build Date: 12 April 2022 11:45:24AM \[ 47.013\] \[ 47.013\] Current version of pixman: 0.40.0 \[ 47.013\] Before reporting problems, check http://wiki.x.org to make sure that you have the latest version. \[ 47.023\] Markers: (--) probed, (\*\*) from config file, (==) default setting, (++) from command line, (!!) notice, (II) informational, (WW) warning, (EE) error, (NI) not implemented, (??) unknown. \[ 47.023\] (==) Log file: "/var/log/Xorg.0.log", Time: Wed Apr 27 22:57:44 2022 \[ 47.023\] (==) Using system config directory "/usr/local/share/X11/xorg.conf.d" \[ 47.023\] (==) No Layout section. Using the first Screen section. \[ 47.023\] (==) No screen section available. Using defaults. \[ 47.023\] (\*\*) |-->Screen "Default Screen Section" (0) \[ 47.023\] (\*\*) | |-->Monitor "" \[ 47.023\] (==) No monitor specified for screen "Default Screen Section". Using a default monitor configuration. \[ 47.023\] (==) Automatically adding devices \[ 47.023\] (==) Automatically enabling devices \[ 47.023\] (==) Not automatically adding GPU devices \[ 47.023\] (==) Max clients allowed: 256, resource mask: 0x1fffff \[ 47.023\] (==) FontPath set to: /usr/local/share/fonts/misc/, /usr/local/share/fonts/TTF/, /usr/local/share/fonts/OTF/, /usr/local/share/fonts/Type1/, /usr/local/share/fonts/100dpi/, /usr/local/share/fonts/75dpi/, catalogue:/usr/local/etc/X11/fontpath.d \[ 47.023\] (==) ModulePath set to "/usr/local/lib/xorg/modules" \[ 47.023\] (II) The server relies on udev to provide the list of input devices. If no devices become available, reconfigure udev or disable AutoAddDevices. \[ 47.023\] (II) Loader magic: 0x435f60 \[ 47.023\] (II) Module ABI versions: \[ 47.023\] X.Org ANSI C Emulation: 0.4 \[ 47.023\] X.Org Video Driver: 24.1 \[ 47.023\] X.Org XInput driver : 24.1 \[ 47.023\] X.Org Server Extension : 10.0 \[ 47.023\] (--) PCI: (4@0:0:0) 10de:1427:1414:0008 rev 161, Mem @ 0xc0000000/16777216, 0x800000000/268435456, 0x810000000/33554432, I/O @ 0x00009000/128 \[ 47.023\] (II) LoadModule: "glx" \[ 47.023\] (II) Loading /usr/local/lib/xorg/modules/extensions/libglx.so \[ 47.023\] (II) Module glx: vendor="X.Org Foundation" \[ 47.023\] compiled for 1.20.14, module version = 1.0.0 \[ 47.023\] ABI class: X.Org Server Extension, version 10.0 \[ 47.023\] (==) Matched modesetting as autoconfigured driver 0 \[ 47.023\] (==) Matched scfb as autoconfigured driver 1 \[ 47.023\] (==) Matched vesa as autoconfigured driver 2 \[ 47.023\] (==) Assigned the driver to the xf86ConfigLayout \[ 47.023\] (II) LoadModule: "modesetting" \[ 47.023\] (II) Loading /usr/local/lib/xorg/modules/drivers/modesetting\_drv.so \[ 47.033\] (II) Module modesetting: vendor="X.Org Foundation" \[ 47.033\] compiled for 1.20.14, module version = 1.20.14 \[ 47.033\] Module class: X.Org Video Driver \[ 47.033\] ABI class: X.Org Video Driver, version 24.1 \[ 47.033\] (II) LoadModule: "scfb" \[ 47.033\] (II) Loading /usr/local/lib/xorg/modules/drivers/scfb\_drv.so \[ 47.033\] (II) Module scfb: vendor="X.Org Foundation" \[ 47.033\] compiled for 1.20.14, module version = 0.0.5 \[ 47.033\] ABI class: X.Org Video Driver, version 24.1 \[ 47.033\] (II) LoadModule: "vesa" \[ 47.033\] (II) Loading /usr/local/lib/xorg/modules/drivers/vesa\_drv.so \[ 47.033\] (II) Module vesa: vendor="X.Org Foundation" \[ 47.033\] compiled for 1.20.14, module version = 2.5.0 \[ 47.033\] Module class: X.Org Video Driver \[ 47.033\] ABI class: X.Org Video Driver, version 24.1 \[ 47.033\] (II) modesetting: Driver for Modesetting Kernel Drivers: kms \[ 47.033\] (II) scfb: driver for wsdisplay framebuffer: scfb \[ 47.033\] (II) VESA: driver for VESA chipsets: vesa \[ 47.033\] (--) Using syscons driver with X support (version 2.0) \[ 47.033\] (--) using VT number 9 \[ 47.033\] (WW) Falling back to old probe method for modesetting \[ 47.033\] (EE) open /dev/dri/card0: No such file or directory \[ 47.033\] (WW) Falling back to old probe method for scfb \[ 47.033\] scfb trace: probe start \[ 47.033\] scfb trace: probe done \[ 47.033\] (WW) Falling back to old probe method for modesetting \[ 47.033\] (EE) open /dev/dri/card0: No such file or directory \[ 47.033\] (WW) Falling back to old probe method for scfb \[ 47.033\] scfb trace: probe start \[ 47.033\] scfb trace: probe done \[ 47.033\] (EE) No devices detected. \[ 47.033\] (EE) Fatal server error: \[ 47.033\] (EE) no screens found(EE) \[ 47.033\] (EE) Please consult the The X.Org Foundation support at http://wiki.x.org for help. \[ 47.033\] (EE) Please also check the log file at "/var/log/Xorg.0.log" for additional information. \[ 47.033\] (EE) \[ 47.043\] (EE) Server terminated with error (1). Closing log file. ---------------------------------------------------------------------- Plump_Apparatus · 2022-04-28 04:31 ACPI Warning: \_SB.PCI0.S12.S00._DSM: Argument #4 type mismatch - Found [Buffer], ACPI requires [Package] (20201113/nsarguments-212) That appears to be a issue with nVidia's ACPI implementation. I'm not sure if it's relevant. I've never dinked around with nVidia's Optimus, much less on a VM. `x11/nvidia-hybrid-graphics` would be the right package for Optimus, `x11/nvidia-driver` for standard nVidia cards. Some quick googling tells me that `10de:1427:1414:0008` is a PCI ID for specifically for the GeForce 965M on the Microsoft Surface. It's possible the FreeBSD proprietary nVidia module doesn't recognize this ID. IIRC when `nvideo.ko` is loaded it'll output to dmesg what device(s) it has attached to. `nvidia.ko` needs to be loaded by `/boot/loader.conf`. All of this is covered in the [Handbook](https://docs.freebsd.org/en/books/handbook/x11/). You should post your dmesg and kldstat outputs if you want people to be able to help. ---------------------------------------------------------------------- toasty_fe · 2022-04-28 12:33 Hi, thanks for replying. Before editing the `/boot/loader.conf` I've added some outputs to the post to clarify some information, such as the `dmesg` from my Host, the `pciconf` from FreeBSD with the VGA output and the `kldstat` ---------------------------------------------------------------------- grahamperrin · 2022-04-28 16:23 > … covered in the Handbook. … FreeBSD bug [258264 – Following handbook on nvidia drivers makes system get stuck on boot.](https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=258264) ---------------------------------------------------------------------- Plump_Apparatus · 2022-04-28 16:29 That's what I get for commenting when I haven't used FreeBSD for anything but CLI for years now. Should have kept my mouth closed. ---------------------------------------------------------------------- grahamperrin · 2022-04-28 16:47 Hey, don't shoot the messenger …