Intel i350 broken in FreeBSD 12? reddit / freebsd ---------------------------------------------------------------------- cs_marshall · 2020-10-19 22:29 I wanted to migrate to FreeBSD 12 for my firewall, but when I did a clean install on a new disk it seems like once I DHCP from my ISP (comcast) the NIC resets after a very short period of time (a few minutes), then the NIC resets. I've seen the behavior in FreeBSD 12, as well as OpnSense and PFSense. I'm ignoring the MTU from the ISP and I've tried various tweaks and still it's not stable, the motherboard has an Intel i354 NIC which has been rock solid on FreeBSD 11 for a few years. [https://www.supermicro.com/products/motherboard/Atom/X10/A1SAi- 2550F.cfm](https://www.supermicro.com/products/motherboard/Atom/X10/A1 SAi-2550F.cfm) Right now there look to be a few open bugs for the migration/merging with the EM driver, but nothing specific to my problem. I'm kinda at a loss on where to start :/ ​ Update: 2020-11-01 Tried Freebsd 12.2 RELEASE and it \*seems\* to have solved my issue of the NIC bouncing, however I can't tell whether the oddities I'm seeing in running \`\`\`pkg\`\`\` commands are Comcast being...odd...or something still weird with the NIC ​ Update: 2021-04-12 Tried FreeBSD 12.2 again thinking maybe an underlying point release might have fixed the problem, I'm pretty sure it's some oddity with the en driver, I just don't know what all I need to turn off, and given that it's also my firewall I get into this weird position where I'm tethering off my phone while wired into my FW to try and debug it. I guess I'll just stay on a FreeBSD 11 rev until it gets fixed ¯\\\_(ツ)\_/¯ ---------------------------------------------------------------------- DependentVegetable · 2020-10-19 23:22 Did you install 12.2 or 12.1 ? 12.2 has a few fixes to em and iflib Another possibility thats a little less intrusive is the try the kld from /usr/ports/net/intel-em-kmod maybe? ---------------------------------------------------------------------- cs_marshall · 2020-10-20 02:17 I tried: ``` % ls Free\*12\*img FreeBSD-12.0-RELEASE-amd64-mini-memstick.img FreeBSD-12.1-RELEASE- amd64-mini-memstick.img FreeBSD-12.2-BETA3-amd64-mini-memstick.img FreeBSD-12.1-RELEASE-amd64-memstick.img FreeBSD-12.2-BETA3-amd64-memstick.img ``` The reason I had to download both the mini *AND* the memstick you might ask? Because the nic is so unusable, that I started by trying to use the memstick, then downloaded the full stick so I could get off the ground, only to find out once I was in the full OS that it was still failing :/ ---------------------------------------------------------------------- opseceu · 2020-10-20 10:56 Thanks for the pointer. It looks like I have the same problem here. One box, igb0 to igb3. igb0 is stable, igb1 and igb2 have problems. pciconf -lBbcevV igb0 shows this: igb0@pci0:3:0:0: class=0x020000 card=0x152115d9 chip=0x15218086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = 'I350 Gigabit Network Connection' class = network subclass = ethernet bar [10] = type Memory, range 32, base 0xef760000, size 131072, enabled bar [18] = type I/O Port, range 32, base 0x5060, size 32, enabled bar [1c] = type Memory, range 32, base 0xef78c000, size 16384, enabled cap 01[40] = powerspec 3 supports D0 D3 current D0 cap 05[50] = MSI supports 1 message, 64 bit, vector masks cap 11[70] = MSI-X supports 10 messages, enabled Table in map 0x1c[0x0], PBA in map 0x1c[0x2000] cap 10[a0] = PCI-Express 2 endpoint max data 512(512) FLR RO NS link x4(x4) speed 5.0(5.0) ASPM L1(L0s/L1) ecap 0001[100] = AER 2 0 fatal 0 non-fatal 1 corrected ecap 0003[140] = Serial 1 3cecefffff45909a ecap 000e[150] = ARI 1 ecap 0010[160] = SR-IOV 1 IOV disabled, Memory Space disabled, ARI disabled 0 VFs configured out of 8 supported First VF RID Offset 0x0180, VF RID Stride 0x0004 VF Device ID 0x1520 Page Sizes: 4096 (enabled), 8192, 65536, 262144, 1048576, 4194304 ecap 0017[1a0] = TPH Requester 1 ecap 0018[1c0] = LTR 1 ecap 000d[1d0] = ACS 1 PCI-e errors = Correctable Error Detected Unsupported Request Detected Corrected = Advisory Non-Fatal Error igb1 shows this: igb1@pci0:3:0:1: class=0x020000 card=0x152115d9 chip=0x15218086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = 'I350 Gigabit Network Connection' class = network subclass = ethernet bar [10] = type Memory, range 32, base 0xef740000, size 131072, enabled bar [18] = type I/O Port, range 32, base 0x5040, size 32, enabled bar [1c] = type Memory, range 32, base 0xef788000, size 16384, enabled cap 01[40] = powerspec 3 supports D0 D3 current D0 cap 05[50] = MSI supports 1 message, 64 bit, vector masks cap 11[70] = MSI-X supports 10 messages, enabled Table in map 0x1c[0x0], PBA in map 0x1c[0x2000] cap 10[a0] = PCI-Express 2 endpoint max data 512(512) FLR RO NS link x4(x4) speed 5.0(5.0) ASPM L1(L0s/L1) ecap 0001[100] = AER 2 0 fatal 0 non-fatal 1 corrected ecap 0003[140] = Serial 1 3cecefffff45909a ecap 000e[150] = ARI 1 ecap 0010[160] = SR-IOV 1 IOV disabled, Memory Space disabled, ARI disabled 0 VFs configured out of 8 supported First VF RID Offset 0x0180, VF RID Stride 0x0004 VF Device ID 0x1520 Page Sizes: 4096 (enabled), 8192, 65536, 262144, 1048576, 4194304 ecap 0017[1a0] = TPH Requester 1 ecap 000d[1d0] = ACS 1 PCI-e errors = Correctable Error Detected Unsupported Request Detected Corrected = Advisory Non-Fatal Error ---------------------------------------------------------------------- DependentVegetable · 2020-10-20 12:51 Stick with 12.2-BETA as you will have more luck getting someone to try and fix it. I saw ref to this bug the other day. Not sure if it makes a diff for you or not https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=230996 Try and disable the driver in the kernel and see if you can use the driver from the ports. ---------------------------------------------------------------------- opseceu · 2020-10-20 19:20 One idea (unsure, if it helps): Try sysctl dev.igb | grep driver I'm not sure if installing net/intel-em-kmod also matches igb, but in 12.1 we have 7.6.1-k, and the port is at 7.7.8. ---------------------------------------------------------------------- cs_marshall · 2020-10-20 22:48 Yea, I saw that but thought that was a little bit more advanced than my problem since I don't think my Cable modem is handing back tagged DHCP offers, but maybe it's related? ---------------------------------------------------------------------- cs_marshall · 2020-10-20 22:51 Is it possible that your igb0 is statically assigned and the others are receiving DHCP offers? My LAN facing NIC on the same controller didn't \*seem\* to have any problems, and it's the one statically assigned. I wondered also if there was some sort of issue with the driver taking MTU or other adjustments, but when I told DHClient to ignore MTU it still happened :( ---------------------------------------------------------------------- cs_marshall · 2020-10-20 22:52 The hard part of this is that to install a port I have to have network, or find a way to piggy back all the binaries I might need onto the machine since the controller that I use to get to the internet is the one that is problematic. I've purchased a USB ethernet controller and was going to try it over the weekend. ---------------------------------------------------------------------- zachsandberg · 2020-10-21 19:03 I believe that I'm seeing a variant of this as well in some capacity. I am on 12.2 and have Linux, Windows and FreeBSD Bhyve guests. After a few hours of the host being up all the guests stop resolving DNS from the internet. Only internal names resolve from my pfSense box. Shutting down and restarting the guests including pfsense has no impact, but rebooting the host fixes the issue....for a while. I have an Intel i350-T4 card. Very odd behavior. ---------------------------------------------------------------------- opseceu · 2020-10-22 17:37 Wierd as it sounds, replacing the patch cable fixed my problem. ---------------------------------------------------------------------- opseceu · 2020-10-22 17:37 No DHCP anywhere, I hope 8-}