From ken@panzer.kdm.org  Mon May 22 13:53:51 2000
Return-Path: <ken@panzer.kdm.org>
Received: from panzer.kdm.org (panzer.kdm.org [216.160.178.169])
	by hub.freebsd.org (Postfix) with ESMTP id D53B437B55D
	for <FreeBSD-gnats-submit@freebsd.org>; Mon, 22 May 2000 13:53:44 -0700 (PDT)
	(envelope-from ken@panzer.kdm.org)
Received: (from ken@localhost)
	by panzer.kdm.org (8.9.3/8.9.1) id OAA08749;
	Mon, 22 May 2000 14:53:43 -0600 (MDT)
	(envelope-from ken)
Message-Id: <200005222053.OAA08749@panzer.kdm.org>
Date: Mon, 22 May 2000 14:53:43 -0600 (MDT)
From: ken@kdm.org
Sender: ken@panzer.kdm.org
Reply-To: ken@kdm.org
To: FreeBSD-gnats-submit@freebsd.org
Subject: [PATCH] fxp driver doesn't enable flow control
X-Send-Pr-Version: 3.2

>Number:         18757
>Category:       kern
>Synopsis:       [PATCH] fxp driver doesn't enable flow control
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    andre
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Mon May 22 14:00:00 PDT 2000
>Closed-Date:    Sat Dec 27 06:53:38 PST 2003
>Last-Modified:  Sat Dec 27 06:53:38 PST 2003
>Originator:     Kenneth D. Merry
>Release:        FreeBSD 5.0-CURRENT i386
>Organization:
KDM Enterprises
>Environment:

Dual P6-200 box, Intel InBusiness 82559 board, and an Intel 410T switch.

Copyright (c) 1992-2000 The FreeBSD Project.
Copyright (c) 1982, 1986, 1989, 1991, 1993
        The Regents of the University of California. All rights reserved.
FreeBSD 5.0-CURRENT #0: Sun May 21 17:02:27 MDT 2000
    ken@panzer.kdm.org:/a/ken/perforce2/FreeBSD-ken/src/sys/compile/panzer
Timecounter "i8254"  frequency 1193182 Hz
CPU: Pentium Pro (199.43-MHz 686-class CPU)
  Origin = "GenuineIntel"  Id = 0x617  Stepping = 7
Features=0xfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV>
real memory  = 201326592 (196608K bytes)
avail memory = 191889408 (187392K bytes)
Programming 24 pins in IOAPIC #0
IOAPIC #0 intpin 2 -> irq 0
FreeBSD/SMP: Multiprocessor motherboard
 cpu0 (BSP): apic id:  1, version: 0x00040011, at 0xfee00000
 cpu1 (AP):  apic id:  0, version: 0x00040011, at 0xfee00000
 io0 (APIC): apic id:  2, version: 0x00170011, at 0xfec00000
[ ... ]
fxp0: <Intel InBusiness 10/100 Ethernet> port 0xc400-0xc43f mem 0xe3000000-0xe30fffff,0xe3800000-0xe3800fff irq 16 at device 12.0 on pci0
fxp0: Ethernet address 00:90:27:d1:77:76


>Description:

By default, the fxp driver doesn't enable flow control.  The 82559 chip on
my ethernet card, however, negotiates flow control with my Intel 410T
switch.

The switch proceeds to send flow control ethernet packets to the card, but
since flow control isn't enabled on the card, the flow control packets are
routed up to the kernel.  They look like this from tcpdump:

22:15:53.313742 0:0:0:0:0:10 1:80:c2:0:0:1 8808 60:
                         0001 0000 0000 8808 0001 0000 0000 8808
                         0001 0000 0000 8808 0001 0000 0000 8808
                         0001 0000 0000 8808 0001 0000 0000
22:15:53.355673 0:0:0:0:0:10 1:80:c2:0:0:1 8808 60:
                         0001 0000 0000 8808 0001 0000 0000 8808
                         0001 0000 0000 8808 0001 0000 0000 8808
                         0001 0000 0000 8808 0001 0000 0000

The last octet of the source hardware address changes based on what port in
the switch I'm plugged into.  (It's in port 10 now.  If I plug it into say
port 11, it'll be 0x20, 12, 0x30, and so on.)

There are several screens full of the above packets in tcpdump every second
without the patch below.  netstat -i shows packet counts increasing
significantly every second.

>How-To-Repeat:

Get an Intel 410T switch (haven't tried any other switch) and an Intel
82559 board, watch packet counts and run tcpdump.

>Fix:

The patch below is also located here:

http://people.FreeBSD.ORG/~ken/if_fxp.flow_control.20000522

I figured out how to enable flow control from the Linux driver for the
8255x boards.  Intel's documentation is rather un-helpful.

I have been running with this patch on -current since early February 2000.

There are a couple of caveats for this patch:

1.	I don't know what the bits do, or whether all of them are necessary
	for flow control.  Someone with access to the (real) documentation
	for the 8225x boards should take a look at this.  The documentation
	for these boards is only available under NDA, from what I have
	heard.  The documentation available on thier web site isn't
	sufficient to investigate or fix this problem.

2.	I have only tested this on my 82559 "InBusiness" board, so I don't
	know what it will do to say a vanilla 82557.


==== //depot/FreeBSD-ken/src/sys/pci/if_fxp.c#4 - /a/ken/perforce/FreeBSD-ken/src/sys/pci/if_fxp.c ====
*** /tmp/tmp.8481.0	Mon May 22 14:33:40 2000
--- /a/ken/perforce/FreeBSD-ken/src/sys/pci/if_fxp.c	Sun May 21 16:47:39 2000
***************
*** 1456,1461 ****
--- 1456,1462 ----
  	cbp->stripping =	!prm;	/* truncate rx packet to byte count */
  	cbp->padding =		1;	/* (do) pad short tx packets */
  	cbp->rcv_crc_xfer =	0;	/* (don't) xfer CRC to host */
+ 	cbp->flow_control =	0x3d;	/* turn on 802.3x flow control */
  	cbp->force_fdx =	0;	/* (don't) force full duplex */
  	cbp->fdx_pin_en =	1;	/* (enable) FDX# pin */
  	cbp->multi_ia =		0;	/* (don't) accept multiple IAs */
==== //depot/FreeBSD-ken/src/sys/pci/if_fxpreg.h#2 - /a/ken/perforce/FreeBSD-ken/src/sys/pci/if_fxpreg.h ====
*** /tmp/tmp.8481.1	Mon May 22 14:33:40 2000
--- /a/ken/perforce/FreeBSD-ken/src/sys/pci/if_fxpreg.h	Sat Feb  5 02:52:29 2000
***************
*** 161,167 ****
  				padding:1,
  				rcv_crc_xfer:1,
  				:5;
! 	volatile u_int		:6,
  				force_fdx:1,
  				fdx_pin_en:1;
  	volatile u_int		:6,
--- 161,172 ----
  				padding:1,
  				rcv_crc_xfer:1,
  				:5;
! 	/*
! 	 * IEEE 802.3x flow control:
! 	 * 0     == off
! 	 * 0x3d  == on
! 	 */
! 	volatile u_int		flow_control:6,
  				force_fdx:1,
  				fdx_pin_en:1;
  	volatile u_int		:6,

>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->dg 
Responsible-Changed-By: ken 
Responsible-Changed-When: Tue May 23 16:57:35 PDT 2000 
Responsible-Changed-Why:  
Over to the driver author. 
State-Changed-From-To: open->feedback 
State-Changed-By: silby 
State-Changed-When: Sat Jan 5 14:46:43 PST 2002 
State-Changed-Why:  
Is this PR still applicable?  It looks to me like jlemon committed 
such support (in a different way) in March, 2001.  Can this PR 
now be closed? 


Responsible-Changed-From-To: dg->jlemon 
Responsible-Changed-By: silby 
Responsible-Changed-When: Sat Jan 5 14:46:43 PST 2002 
Responsible-Changed-Why:  

http://www.FreeBSD.org/cgi/query-pr.cgi?pr=18757 
State-Changed-From-To: feedback->closed 
State-Changed-By: andre 
State-Changed-When: Sat Dec 27 06:51:14 PST 2003 
State-Changed-Why:  
I'm closing this PR because it open for too long and this 
issue seems to fixed, at least together with my Cisco Catalyst 
switch flow control seems to work.  I don't know which exact 
commit fixed it. 


Responsible-Changed-From-To: jlemon->andre 
Responsible-Changed-By: andre 
Responsible-Changed-When: Sat Dec 27 06:51:14 PST 2003 
Responsible-Changed-Why:  
I'm closing this PR because it open for too long and this 
issue seems to fixed, at least together with my Cisco Catalyst 
switch flow control seems to work.  I don't know which exact 
commit fixed it. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=18757 
>Unformatted:
