From root@crepe4.monkeybrains.net  Fri Apr  8 06:07:07 2011
Return-Path: <root@crepe4.monkeybrains.net>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 6618F106564A
	for <FreeBSD-gnats-submit@freebsd.org>; Fri,  8 Apr 2011 06:07:07 +0000 (UTC)
	(envelope-from root@crepe4.monkeybrains.net)
Received: from crepe4.monkeybrains.net (crepe4.monkeybrains.net [208.69.40.123])
	by mx1.freebsd.org (Postfix) with ESMTP id 3C05F8FC08
	for <FreeBSD-gnats-submit@freebsd.org>; Fri,  8 Apr 2011 06:07:07 +0000 (UTC)
Received: from crepe4.monkeybrains.net (localhost [127.0.0.1])
	by crepe4.monkeybrains.net (8.14.4/8.14.4) with ESMTP id p385ffgE017384;
	Thu, 7 Apr 2011 22:41:42 -0700 (PDT)
	(envelope-from root@crepe4.monkeybrains.net)
Received: (from root@localhost)
	by crepe4.monkeybrains.net (8.14.4/8.14.4/Submit) id p385ff4h017383;
	Thu, 7 Apr 2011 22:41:41 -0700 (PDT)
	(envelope-from root)
Message-Id: <201104080541.p385ff4h017383@crepe4.monkeybrains.net>
Date: Thu, 7 Apr 2011 22:41:41 -0700 (PDT)
From: crapsh@monkeybrains.net
Reply-To: crapsh@monkeybrains.net
To: FreeBSD-gnats-submit@freebsd.org
Cc: crapsh@monkeybrains.net
Subject: jails don't use routing table
X-Send-Pr-Version: 3.113
X-GNATS-Notify:

>Number:         156268
>Category:       kern
>Synopsis:       jails don't use routing table
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    freebsd-bugs
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Fri Apr 08 06:10:11 UTC 2011
>Closed-Date:    Sat Apr 09 20:16:36 UTC 2011
>Last-Modified:  Sat Apr 09 20:16:36 UTC 2011
>Originator:     Rudy
>Release:        FreeBSD 8.2-RELEASE amd64
>Organization:
MonkeyBrains.net
>Environment:
System: FreeBSD crepe4.monkeybrains.net 8.2-RELEASE FreeBSD 8.2-RELEASE #0: Wed Apr 6 01:45:24 PDT 2011 root@crepe4:/usr/obj/usr/src/sys/CREPE4 amd64
Jail environment
>Description:
Jails can support multiple IPs.  When picking which IP as a 'source IP' the jail does not 
take into consideration the routing table as the host system does.
>How-To-Repeat:
Setup:
  Set 2 IPs on your system: 1.1.1.100/24 and 2.2.2.200/24
  set your jail to use both IPs
  export jail_example_monkeybrains_net_ip="1.1.1.1,2.2.2.2"
  Set default route to 1.1.1.1
  Set a static route to 3.3.3.3 to route through 2.2.2.1
Test:
  run "tcpdump -n icmp" in one window
  run "ping 3.3.3.3" in 'host' -- source packet is 2.2.2.200
  run "ping 3.3.3.3" in 'jailed host' -- source packet is 1.1.1.100

I even added /dev/mem and /dev/kmem to the jailed environment so I could run
'netstat -rn' in the jail.  The route for 3.3.3.3 is in the routing table, 
but the kernel picks the wrong source IP.

>Fix:
Run you stuff outside of jails.  :(
>Release-Note:
>Audit-Trail:

From: Rudy <crapsh@monkeybrains.net>
To: bug-followup@FreeBSD.org, crapsh@monkeybrains.net
Cc:  
Subject: Re: kern/156268: jails don't use routing table
Date: Fri, 08 Apr 2011 16:04:42 -0700

 Looks like just ICMP has issues...
 
 
 TCP OK
 16:00:26.303086 IP 2.2.2.200.51884 > 3.3.3.3.80: Flags [F.], seq 
 1176726875, ack 3364247674, win 8326, options [nop,nop,
 TS val 188728165 ecr 659781298], length 0
 16:00:26.325530 IP 3.3.3.3.80 > 2.2.2.200.51884: Flags [F.], seq 1, ack 
 1, win 2896, options [nop,nop,TS val 659796605 e
 cr 188728165], length 0
 
 ICMP NOT OK
 16:00:32.824971 IP 1.1.1.1 > 3.3.3.3: ICMP echo request, id 47701, seq 
 0, length 64
 16:00:33.825828 IP 1.1.1.1 > 3.3.3.3: ICMP echo request, id 47701, seq 
 1, length 64
 
 UDP OK  (there is not dns on target... just a bogus UDP packet to see if 
 src IP was OK)
 16:00:41.826238 IP 2.2.2.200.43454 > 3.3.3.3.53: 63527+ PTR? 
 5.5.5.10.in-addr.arpa. (39)
 16:00:41.838730 IP 3.3.3.3 > 2.2.2.200: ICMP 3.3.3.3 udp port 53 
 unreachable, length 75
 
 
 
 Of course, I have
   security.jail.allow_raw_sockets: 1
 or else I wouldn't be able to do ICMP at all.  Looks like the raw 
 sockets ignores routing table.
 
 

From: "Bjoern A. Zeeb" <bzeeb-lists@lists.zabbadoz.net>
To: bug-followup@FreeBSD.org, crapsh@monkeybrains.net
Cc:  
Subject: Re: kern/156268: jails don't use routing table
Date: Sat, 9 Apr 2011 01:58:39 +0000 (UTC)

 This is almost a FAQ as it comes up every second month on the list as
 well.  The answer always is - if you enable raw socket, do NOT use ping.
 Ping does it's own way to pick your source address.  Try telnet
 without -s and see what the SYN thinks for example.  You might even
 be able to check with netstat inside the jail not needing to tcpdump
 on base.
 
 Does this help?
 
 -- 
 Bjoern A. Zeeb                                 You have to have visions!
           Stop bit received. Insert coin for new address family.
State-Changed-From-To: open->closed 
State-Changed-By: linimon 
State-Changed-When: Sat Apr 9 20:16:04 UTC 2011 
State-Changed-Why:  
note that this is a FAQ. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=156268 
>Unformatted:
