Post AkpGCbzP5uM2NMbnYe by jeffpc@mastodon.radio
 (DIR) More posts by jeffpc@mastodon.radio
 (DIR) Post #AkpGCUcoTV7nXGz9sG by jeffpc@mastodon.radio
       2024-08-10T13:41:45Z
       
       1 likes, 1 repeats
       
       Weird.  I have some #FreeBSD jails that are using VNET to get access to the LAN.  At least as far as I can tell, some TCP packets sent to it exceed the MTU by quite a bit.Everything on the server is set up with a 1500 byte MTU but I see TCP packets with 2896 bytes in the packet trace.  The jail doesn't ack them, and ~0.3 seconds later the first 1448 bytes get re-transmitted.After a few 1448-byte packets, the remote end tries another 2896 which causes a 0.3s hiccup,... yielding about 16kB/s 👎
       
 (DIR) Post #AkpGCbzP5uM2NMbnYe by jeffpc@mastodon.radio
       2024-08-10T13:47:43Z
       
       0 likes, 0 repeats
       
       The host got upgraded to 14.1 last night, but I occasionally saw this on 14.0.  I just finally dug into it today :)This happens even with a freshly installed jail.
       
 (DIR) Post #AkpGRZfdztwxWoV7b6 by jeffpc@mastodon.radio
       2024-08-10T13:51:44Z
       
       0 likes, 0 repeats
       
       @feld epair
       
 (DIR) Post #AkpH91KB3bVRkNB19s by jeffpc@mastodon.radio
       2024-08-10T13:53:24Z
       
       0 likes, 0 repeats
       
       @feld Specifically, the host has 2 e1000e NICs that I combine in a lagg, and then I add lagg0.31 to a bridge which gets the epairs for jails.
       
 (DIR) Post #AkpIcetN7mD2oeKqEi by jeffpc@mastodon.radio
       2024-08-10T14:11:59Z
       
       0 likes, 0 repeats
       
       @feld Hm...TSO is off, but LRO seems to be on:options=4e524bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS,MEXTPG>
       
 (DIR) Post #AkpIcfhM7wBxJfil96 by jeffpc@mastodon.radio
       2024-08-10T14:17:41Z
       
       1 likes, 0 repeats
       
       @feld Aaaand disabling LRO "fixes" the transfer speed problem.  Now, I reliably get many MB/s.The largest packets now are 1518 bytes.Maybe there is some weird interactions with LRO and vnet tcp stack not seeing an LRO flag on the epair and stubbornly ignoring the over-MTU packet.
       
 (DIR) Post #AkpLO0bePQd3gDkhay by jeffpc@mastodon.radio
       2024-08-10T14:18:52Z
       
       0 likes, 0 repeats
       
       @feld Indeed.  Thanks for the suggestion to look at the interface options!
       
 (DIR) Post #AkpLO1OZTXlE7wdlqa by jeffpc@mastodon.radio
       2024-08-10T14:39:26Z
       
       1 likes, 0 repeats
       
       @feld I'm going to poke more at it, and file a bug as needed.