From jc@irbs.com Mon Mar 15 18:10:29 1999
Return-Path: <jc@irbs.com>
Received: from irbs.irbs.com (irbs.irbs.com [209.36.62.129])
	by hub.freebsd.org (Postfix) with ESMTP id 400C915039
	for <FreeBSD-gnats-submit@freebsd.org>; Mon, 15 Mar 1999 18:06:50 -0800 (PST)
	(envelope-from jc@irbs.com)
Received: (from jc@localhost)
	by irbs.irbs.com (8.8.5/8.8.5) id TAA25708;
	Mon, 15 Mar 1999 19:46:54 -0500 (EST)
Message-Id: <199903160046.TAA25708@irbs.irbs.com>
Date: Mon, 15 Mar 1999 19:46:54 -0500 (EST)
From: jc@irbs.com
Reply-To: jc@irbs.com
To: FreeBSD-gnats-submit@freebsd.org
Subject: Fast forwarding breaks when arp cache expires
X-Send-Pr-Version: 3.2

>Number:         10607
>Category:       kern
>Synopsis:       Fast forwarding breaks when arp cache expires
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    freebsd-bugs
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Mon Mar 15 18:20:00 PST 1999
>Closed-Date:    Tue Jun 26 02:00:43 PDT 2001
>Last-Modified:  Tue Jun 26 02:01:51 PDT 2001
>Originator:     John Capo
>Release:        FreeBSD 3.1-STABLE i386
>Organization:
IRBS Engineering
>Environment:

3.1 router with two fxp interfaces between a Cisco 7000 and a LAN
of 5 web servers forwarding around 750 pps.

>Description:

When the arp entry for the Cisco expires, arp requests are issued
for the destination IP rather than the next hop IP which is th
Cisco.

The interface address is 209.36.62.211/29, the next hop Cisco is
209.36.62.210.

12:50:30.006992 arp who-has 198.108.225.232 tell 209.36.62.211
12:50:31.018596 arp who-has 212.210.246.39 tell 209.36.62.211
12:50:32.011115 arp who-has 193.227.69.106 tell 209.36.62.211
12:50:33.008656 arp who-has 193.4.198.10 tell 209.36.62.211
12:50:34.012108 arp who-has 194.51.131.253 tell 209.36.62.211
12:50:55.012083 arp who-has 209.36.62.209 tell 209.36.62.211
12:50:55.014307 arp reply 209.36.62.209 is-at 0:0:c:30:35:45

Also, the arp request for the next hop router manages to be delayed
for host_down_time for some reason.  The machine stops routing for
20 seconds.

>How-To-Repeat:

Fast forward packets at a good rate, several hundred per second
probably.  Wait for the next hop router arp entry to expire or
delete it from the arp table.

>Fix:

Not sure if this is the correct way to do this but it does solve
the problem.


Index: src/sys/netinet/ip_flow.c
===================================================================
RCS file: /usr/cvs/src/sys/netinet/ip_flow.c,v
retrieving revision 1.6
diff -c -r1.6 ip_flow.c
*** ip_flow.c	1998/12/07 21:58:38	1.6
--- ip_flow.c	1999/03/16 00:12:29
***************
*** 109,114 ****
--- 109,115 ----
  	struct ip *ip;
  	struct ipflow *ipf;
  	struct rtentry *rt;
+ 	struct sockaddr *dest;
  	int error;
  
  	/*
***************
*** 153,164 ****
  		ip->ip_sum += htons(IPTTLDEC << 8);
  	}
  
  	/*
  	 * Send the packet on its way.  All we can get back is ENOBUFS
  	 */
  	ipf->ipf_uses++;
  	ipf->ipf_timer = IPFLOW_TIMER;
! 	if ((error = (*rt->rt_ifp->if_output)(rt->rt_ifp, m, &ipf->ipf_ro.ro_dst, rt)) != 0) {
  		if (error == ENOBUFS)
  			ipf->ipf_dropped++;
  		else
--- 154,169 ----
  		ip->ip_sum += htons(IPTTLDEC << 8);
  	}
  
+ 	if (rt->rt_flags & RTF_GATEWAY)
+ 		dest = rt->rt_gateway;
+ 	else
+ 		dest = &ipf->ipf_ro.ro_dst;
  	/*
  	 * Send the packet on its way.  All we can get back is ENOBUFS
  	 */
  	ipf->ipf_uses++;
  	ipf->ipf_timer = IPFLOW_TIMER;
! 	if ((error = (*rt->rt_ifp->if_output)(rt->rt_ifp, m, dest, rt)) != 0) {
  		if (error == ENOBUFS)
  			ipf->ipf_dropped++;
  		else

>Release-Note:
>Audit-Trail:
State-Changed-From-To: open->closed 
State-Changed-By: ru 
State-Changed-When: Tue Jun 26 02:00:43 PDT 2001 
State-Changed-Why:  
Fixed in 5.0-CURRENT, ip_flow.c,v 1.12. 
Merge into 4.3-STABLE within 2 weeks. 

http://www.FreeBSD.org/cgi/query-pr.cgi?pr=10607 
>Unformatted:
