From nobody@FreeBSD.org  Fri May  7 08:35:01 2010
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52])
	by hub.freebsd.org (Postfix) with ESMTP id 8DC361065691
	for <freebsd-gnats-submit@FreeBSD.org>; Fri,  7 May 2010 08:35:01 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (www.freebsd.org [69.147.83.33])
	by mx1.freebsd.org (Postfix) with ESMTP id 7256B8FC18
	for <freebsd-gnats-submit@FreeBSD.org>; Fri,  7 May 2010 08:35:01 +0000 (UTC)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by www.freebsd.org (8.14.3/8.14.3) with ESMTP id o478Z0dO096609
	for <freebsd-gnats-submit@FreeBSD.org>; Fri, 7 May 2010 08:35:00 GMT
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.14.3/8.14.3/Submit) id o478Z0Wp096607;
	Fri, 7 May 2010 08:35:00 GMT
	(envelope-from nobody)
Message-Id: <201005070835.o478Z0Wp096607@www.freebsd.org>
Date: Fri, 7 May 2010 08:35:00 GMT
From: "Alexander V. Chernikov" <melifaro@ipfw.ru>
To: freebsd-gnats-submit@FreeBSD.org
Subject: ipfw setfib does not work on local outgoing connections
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         146372
>Category:       kern
>Synopsis:       [ipfw] ipfw setfib does not work on local outgoing connections
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    freebsd-ipfw
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Fri May 07 08:40:01 UTC 2010
>Closed-Date:    Fri May 07 15:04:38 PDT 2010
>Last-Modified:  Fri May 07 15:04:38 PDT 2010
>Originator:     Alexander V. Chernikov
>Release:        7.2-STABLE amd64
>Organization:
JSC Meganet
>Environment:
FreeBSD gw.su29.net 7.2-STABLE FreeBSD 7.2-STABLE #19: Sun Nov 15 16:14:31 MSK 2009     root@gw.su29.net:/usr/obj/usr/src/sys/ROUTER  amd64

>Description:
ipfw setfib doesn't change fib for (TCP?) outgoing packets 


Diagnostics:

12:38 [0] m@gw route -n get default
   route to: default
destination: default
       mask: default
    gateway: 81.200.11.1
  interface: vlan12
      flags: <UP,GATEWAY,DONE,STATIC>
 recvpipe  sendpipe  ssthresh  rtt,msec    rttvar  hopcount      mtu     expire
       0         0         0         0         0         0      1500         0

(vlan12)

12:38 [0] m@gw setfib 13 route -n get default
   route to: default
destination: default
       mask: default
    gateway: 92.243.163.1
  interface: vlan13
      flags: <UP,GATEWAY,DONE,STATIC>
 recvpipe  sendpipe  ssthresh  rtt,msec    rttvar  hopcount      mtu     expire
       0         0         0         0         0         0      1500         0

(vlan13)


12:25 [1] m@gw s tcpdump -i vlan13 -lnvs0 host www.ru &
[2] 62372                                              
12:26 [2] m@gw tcpdump: listening on vlan13, link-type EN10MB (Ethernet), capture size 65535 bytes

12:26 [2] m@gw setfib 13 telnet www.ru 80
Trying 194.87.0.50...                    
Connected to www.ru.                     
Escape character is '^]'.                
12:26:10.117204 IP (tos 0x10, ttl 64, id 27808, offset 0, flags [DF], proto TCP (6), length 60) 92.243.163.128.61882 > 194.87.0.50.80: S, cksum 0x80d0 (correct), 1602640083:1602640083(0) win 65535 <mss 1460,nop,wscale 3,sackOK,timestamp 1371867149 0>                                                                                                                                                                                                                                
12:26:10.124662 IP (tos 0x8, ttl 248, id 0, offset 0, flags [DF], proto TCP (6), length 60) 194.87.0.50.80 > 92.243.163.128.61882: S, cksum 0xf3ec (correct), 3712081403:3712081403(0) ack 1602640084 win 5792 <mss 1460,sackOK,timestamp 172077231 1371867149,nop,wscale 7>                                                                                                                                                                                                              
12:26:10.124684 IP (tos 0x10, ttl 64, id 27810, offset 0, flags [DF], proto TCP (6), length 52) 92.243.163.128.61882 > 194.87.0.50.80: ., cksum 0x18cb (correct), ack 1 win 8326 <nop,nop,timestamp 1371867157 172077231>                    
quit                                                                            ....                                           
Connection closed by foreign host.

12:26 [2] m@gw ipfw show 1-10
Password:                    
00001      2240       262576 allow tcp from 10.0.0.0/24 to me dst-port 3389
00002       505        48965 allow tcp from 10.0.0.0/24 to me dst-port 8082
12:26 [2] m@gw ipfw add 3 setfib 13 tcp from me to www.ru 80 out
00003 setfib 13 tcp from me to 194.87.0.50 dst-port 80 out      
12:26 [2] m@gw telnet www.ru 80              
Trying 194.87.0.50...                        
Connected to www.ru.                         
Escape character is '^]'.                    
^]                                           
telnet> quit                                 
Connection closed.                           
12:26 [2] m@gw ipfw show 3
00003         4          216 setfib 13 tcp from me to 194.87.0.50 dst-port 80 out


>How-To-Repeat:
1) Setup an alternative fib table
2) setup ipfw rule like 'setfib X tcp from me to ... out'
3) try to establish TCP connection matching the rule
>Fix:


>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->freebsd-ipfw 
Responsible-Changed-By: linimon 
Responsible-Changed-When: Fri May 7 21:37:54 UTC 2010 
Responsible-Changed-Why:  
Over to maintainer(s). 

http://www.freebsd.org/cgi/query-pr.cgi?pr=146372 
State-Changed-From-To: open->closed 
State-Changed-By: julian 
State-Changed-When: Fri May 7 15:00:00 PDT 2010 
State-Changed-Why:  
Unfortunatly this is mostly unavoidable. The routing decision has already been made by the time that  
ipfw is called. There is a small possibility that  a change of fib could make the stack do teh same thing 
as ipfw fwd and force he packet to go back to the start and get rerouted, but it's not quite as easy as that. 

You should look at possibly setting the fib for the application in question or maybe jsut its socket 
if you have source. contact me directly to get some ideas we may be able to work with you on. 


http://www.freebsd.org/cgi/query-pr.cgi?pr=146372 
>Unformatted:
