From nobody@FreeBSD.org  Sun Feb 12 21:16:30 2012
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 275EC106564A
	for <freebsd-gnats-submit@FreeBSD.org>; Sun, 12 Feb 2012 21:16:30 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from red.freebsd.org (red.freebsd.org [IPv6:2001:4f8:fff6::22])
	by mx1.freebsd.org (Postfix) with ESMTP id EB19D8FC12
	for <freebsd-gnats-submit@FreeBSD.org>; Sun, 12 Feb 2012 21:16:29 +0000 (UTC)
Received: from red.freebsd.org (localhost [127.0.0.1])
	by red.freebsd.org (8.14.4/8.14.4) with ESMTP id q1CLGT5g017657
	for <freebsd-gnats-submit@FreeBSD.org>; Sun, 12 Feb 2012 21:16:29 GMT
	(envelope-from nobody@red.freebsd.org)
Received: (from nobody@localhost)
	by red.freebsd.org (8.14.4/8.14.4/Submit) id q1CLGTdH017656;
	Sun, 12 Feb 2012 21:16:29 GMT
	(envelope-from nobody)
Message-Id: <201202122116.q1CLGTdH017656@red.freebsd.org>
Date: Sun, 12 Feb 2012 21:16:29 GMT
From: David Talkington <t42@pobox.com>
To: freebsd-gnats-submit@FreeBSD.org
Subject: virtio_kmod: networking breaks with a router using virtio net driver on KVM host
X-Send-Pr-Version: www-3.1
X-GNATS-Notify: jmealo@stringtheoryschools.com

>Number:         165059
>Category:       kern
>Synopsis:       [kvm] virtio-kmod: networking breaks with a router using virtio net driver on KVM host
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    bryanv
>State:          feedback
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Sun Feb 12 21:20:11 UTC 2012
>Closed-Date:    
>Last-Modified:  Sun May 04 03:34:35 UTC 2014
>Originator:     David Talkington
>Release:        9.0-RELEASE
>Organization:
>Environment:
FreeBSD atlantis02.flyingjoke.org 9.0-RELEASE FreeBSD 9.0-RELEASE #0: Tue Jan  3 07:46:30 UTC 2012     root@farrell.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC  amd64
>Description:
When the router for a FreeBSD guest on KVM is also a FreeBSD guest on
the same KVM host, and which is using the virtio network driver from
virtio_kmod, ping will work between guests on different subnets, but
no userland network daemons will respond. If I switch to the e1000
driver on the router, but change nothing else, everything works correctly.


>How-To-Repeat:
I created three FreeBSD guests on one Linux KVM host. I am using bridged
networking on the KVM host, as br0 and br1. One of the guests has two
network interfaces and acts as a router between two subnets, as follows:

router1: br0, 192.168.1.1; br1, 192.168.2.1
client1: br0, 192.168.1.100; default route 192.168.1.1
client2: br1, 192.168.2.100; default route 192.168.2.1

I configured virtio network interfaces on all three hosts. I enabled
forwarding on router1, but no packet filtering. No NAT is in use.

Result:

    * client1 can ping client2, and vice versa.
    * ssh works from router1 to client1 and vice versa, and from router1
      to client2 and vice versa.
    * ssh from client1 to client2 will fail (and vice versa); the client
      simply hangs indefinitely while trying to connect. 
    * tcpdump on client2 will show that the SYN is arriving at client2
      port 22, but client2 never replies, nor generates any debug or log
      output that suggests it ever saw the connection attempt.
    * any other userland network service I try (both tcp and udp) will
      show the same thing -- packets arrive at client2 from client1, but
      the daemon seems to never see them. Since ping works, I know the
      kernel is getting them.
    * If I switch back to the e1000 driver on router1, but make no other
      changes, and make no changes at all to client1 and client2, then
      ssh will work properly from client1 to client2 and the problem is resolved.
    * If I let router1 continue to use virtio interfaces, but move router1
      onto a different KVM host -- so that the traffic from client1 to client2
      must leave the KVM host via the bridged interface and then return on a
      different interface - then ssh will work properly from client1 to
      client2 and the problem is resolved.

KVM guests: FreeBSD 9
virtio-kmod: 0.228301
KVM host: Ubuntu 11.10
qemu-kvm: 0.14.1
>Fix:
Unknown.

>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->freebsd-ports-bugs 
Responsible-Changed-By: linimon 
Responsible-Changed-When: Thu Apr 5 01:43:24 UTC 2012 
Responsible-Changed-Why:  
reclassify. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=165059 
Responsible-Changed-From-To: freebsd-ports-bugs->kuriyama 
Responsible-Changed-By: edwin 
Responsible-Changed-When: Mon May 28 02:17:30 UTC 2012 
Responsible-Changed-Why:  
Over to maintainer (via the GNATS Auto Assign Tool) 

http://www.freebsd.org/cgi/query-pr.cgi?pr=165059 
Responsible-Changed-From-To: kuriyama->bryanv 
Responsible-Changed-By: crees 
Responsible-Changed-When: Wed Aug 7 19:33:18 UTC 2013 
Responsible-Changed-Why:  
Hi Bryan, 

Is this something you may be acquainted with, as the virtio maintainer? 
Do you have any recommendations? 

Please accept my apologies if this isn't something for you... 

Chris 

http://www.freebsd.org/cgi/query-pr.cgi?pr=165059 

From: Jeffrey Mealo <jmealo@stringtheoryschools.com>
To: bug-followup@FreeBSD.org, t42@pobox.com
Cc:  
Subject: Re: kern/165059: virtio-kmod: networking breaks with a router using
 virtio net driver on KVM host
Date: Wed, 7 Aug 2013 15:46:18 -0400

 --089e0158c4328e352704e360cbe3
 Content-Type: text/plain; charset=UTF-8
 
 I wanted to confirm that this bug is present in FreeBSD 8.3-RELEASE_p9
 running on SmartOS KVM (a different implementation than the linux KVM).
 
 --089e0158c4328e352704e360cbe3
 Content-Type: text/html; charset=UTF-8
 
 <div dir="ltr">I wanted to confirm that this bug is present in FreeBSD 8.3-RELEASE_p9 running on SmartOS KVM (a different implementation than the linux KVM).<br></div>
 
 --089e0158c4328e352704e360cbe3--
State-Changed-From-To: open->feedback 
State-Changed-By: crees 
State-Changed-When: Wed Aug 7 20:22:58 UTC 2013 
State-Changed-Why:  
Fantastic, thanks for your quick response. 

Jeffrey, does disabling checksum offloading work for you? 

http://www.freebsd.org/cgi/query-pr.cgi?pr=165059 

From: Jeffrey Mealo <jmealo@stringtheoryschools.com>
To: bug-followup@FreeBSD.org, David Talkington <t42@pobox.com>
Cc:  
Subject: Re: kern/165059: virtio-kmod: networking breaks with a router using
 virtio net driver on KVM host
Date: Mon, 12 Aug 2013 20:30:53 -0400

 --001a113366bc846eee04e3c95afd
 Content-Type: text/plain; charset=UTF-8
 
 This did not resolve my issue.
 
 Thanks,
 Jeff
 
 --001a113366bc846eee04e3c95afd
 Content-Type: text/html; charset=UTF-8
 
 <div dir="ltr">This did not resolve my issue.
 <div><br>Thanks,</div><div>Jeff</div></div>
 
 --001a113366bc846eee04e3c95afd--

From: Phil Regnauld <regnauld@x0.dk>
To: bug-followup@FreeBSD.org, t42@pobox.com
Cc:  
Subject: Re: kern/165059: virtio-kmod: networking breaks with a router using
 virtio net driver on KVM host
Date: Tue, 22 Oct 2013 01:22:30 +0200

 Env:
 	Host OS: Debian 7.1 3.2.0-4-amd64 #1 SMP Debian 3.2.46-1+deb7u1 x86_64 GNU/Linux
 	KVMM: QEMU emulator version 1.1.2 (qemu-kvm-1.1.2+dfsg-6, Debian)
 	Guest: FreeBSD 9.2-R amd64
 
 Disabling checksum offload with ifconfig vtnetX -rxcsum -txcsum on both
 interfaces (this is a router) solves the issue, but performance becomes
 terrible (150 KB/sec uses 100% CPU on host).
 
 vtnet interfaces are, Host side, bridged to VLANs.
 
 Problem does not appear if the traffic is to/from the router itself. Only
 forwarded traffic is a problem.
 
 Can provide more info/feedback if needed.

From: Phil Regnauld <regnauld@x0.dk>
To: bug-followup@FreeBSD.org, t42@pobox.com
Cc:  
Subject: Re: kern/165059: virtio-kmod: networking breaks with a router using
 virtio net driver on KVM host
Date: Tue, 7 Jan 2014 15:54:53 +0100

 Phil Regnauld (regnauld) writes:
 > Env:
 > 	Host OS: Debian 7.1 3.2.0-4-amd64 #1 SMP Debian 3.2.46-1+deb7u1 x86_64 GNU/Linux
 > 	KVMM: QEMU emulator version 1.1.2 (qemu-kvm-1.1.2+dfsg-6, Debian)
 > 	Guest: FreeBSD 9.2-R amd64
 > 
 > Disabling checksum offload with ifconfig vtnetX -rxcsum -txcsum on both
 > interfaces (this is a router) solves the issue, but performance becomes
 > terrible (150 KB/sec uses 100% CPU on host).
 > 
 > vtnet interfaces are, Host side, bridged to VLANs.
 > 
 > Problem does not appear if the traffic is to/from the router itself. Only
 > forwarded traffic is a problem.
 > 
 > Can provide more info/feedback if needed.
 
 	Same problem has been observed with 10.0-RC4.
 
 	kern/166645 may be related.
 
 	This is causing FreeBSD (and pfSense) to be unusable as a network
 	appliance / router on KVM platforms.
 
 	Phil
>Unformatted:
