From nobody@FreeBSD.org  Mon Dec  6 20:27:02 2004
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id 797FB16A4CE
	for <freebsd-gnats-submit@FreeBSD.org>; Mon,  6 Dec 2004 20:27:02 +0000 (GMT)
Received: from www.freebsd.org (www.freebsd.org [216.136.204.117])
	by mx1.FreeBSD.org (Postfix) with ESMTP id 3191343D2F
	for <freebsd-gnats-submit@FreeBSD.org>; Mon,  6 Dec 2004 20:27:02 +0000 (GMT)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by www.freebsd.org (8.13.1/8.13.1) with ESMTP id iB6KR1fm096685
	for <freebsd-gnats-submit@FreeBSD.org>; Mon, 6 Dec 2004 20:27:01 GMT
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.13.1/8.13.1/Submit) id iB6KR1jE096684;
	Mon, 6 Dec 2004 20:27:01 GMT
	(envelope-from nobody)
Message-Id: <200412062027.iB6KR1jE096684@www.freebsd.org>
Date: Mon, 6 Dec 2004 20:27:01 GMT
From: Mike Tancsa <mike@sentex.net>
To: freebsd-gnats-submit@FreeBSD.org
Subject: Smartlink Modem causes interrupt storm on RELENG_4 and RELENG_5
X-Send-Pr-Version: www-2.3

>Number:         74786
>Category:       kern
>Synopsis:       [irq] [patch] Smartlink Modem causes interrupt storm on RELENG_4 and RELENG_5
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Mon Dec 06 20:30:23 GMT 2004
>Closed-Date:    
>Last-Modified:  Tue Oct 25 01:57:11 GMT 2005
>Originator:     Mike Tancsa
>Release:        RELENG_4 and RELENG_5
>Organization:
Sentex Communications
>Environment:
4.10-STABLE FreeBSD 4.10-STABLE #6: Fri Nov 26
and
FreeBSD releng5-865.sentex.ca 5.3-STABLE FreeBSD 5.3-STABLE #1: Wed Dec  1 
>Description:
I think we have been bouncing around this issue for the past few months both on RELENG_4 and RELENG_5.  In the past it has been somewhat difficult to reproduce, but now we can do it reliably.    I dont think its a hardware issue as I can take the exact same 2 boxes with the exact same IRQ assignments and boot with OpenBSD and not run into an interrupt storm or freeze up the box.  Swap back the RELENG_4 or RELENG_5 HD and again, I can produce an interrupt storm at will.

I can also reproduce it on 2 different chipsets as well (VIA and Intel).  The problem seems to be around how a PUC device (either a PCI modem or a PCI serial card) and the sharing of an interrupt (usually an USB controller).  

On RELENG_4, the box just locks up in a race trying to service an interrupt on IRQ 12 but remains unhandled.

On RELENG_5, I actually catch an interrupt storm. e.g. I attach to sio4 (PUC modem) and 

Interrupt storm detected on "irq12: uhci1"; throttling interrupt source

Looking at vmstat -i does indeed show a the rate getting throttled

releng-5-pioneer# vmstat -i
interrupt                          total       rate
irq0: clk                         596719         99
irq1: atkbd0                           2          0
irq4: sio0                          1079          0
irq6: fdc0                             1          0
irq8: rtc                         763812        127
irq12: uhci1                        5825          0
irq13: npx0                            1          0
irq14: ata0                        38727          6
irq15: vr0 ata1                     1984          0
Total                            1408150        235
releng-5-pioneer# 

where irq12 is the IRQ shared by the modem and the USB port.  However, because all IRQ 12s get throttled, the modem is unusable. e.g. trying to cu -l /dev/cuaa4 and typing atz takes about 5 seconds.

The problem is that the modem is not being seen as a PCI / PUC device and instead is being seen as an ISA SIO device ??  The following RELENG_5 and RELENG_4 patches seem to fix the problem.  I wonder if the other modems listed in sio.c suffer the same fate ? 


# cat /var/run/dmesg.boot
Copyright (c) 1992-2004 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD 4.10-STABLE #6: Fri Nov 26 13:52:22 EST 2004
    mdtancsa@station.sentex.ca:/usr/obj/usr/src/sys/gas
Timecounter "i8254"  frequency 1193182 Hz
Timecounter "TSC"  frequency 2400411816 Hz
CPU: Intel(R) Celeron(R) CPU 2.40GHz (2400.41-MHz 686-class CPU)
  Origin = "GenuineIntel"  Id = 0xf33  Stepping = 3
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
real memory  = 267321344 (261056K bytes)
config> q
avail memory = 256221184 (250216K bytes)
Preloaded elf kernel "kernel" at 0xc03d5000.
Preloaded userconfig_script "/boot/kernel.conf" at 0xc03d509c.
Warning: Pentium 4 CPU: PSE disabled
Pentium Pro MTRR support enabled
md0: Malloc disk
Using $PIR table, 8 entries at 0xc00fded0
npx0: <math processor> on motherboard
npx0: INT 16 interface
pcib0: <Host to PCI bridge> on motherboard
pci0: <PCI bus> on pcib0
agp0: <Intel 82865G (865G GMCH) SVGA controller> port 0xd000-0xd007 mem 0xfa400000-0xfa47ffff,0xf0000000-0xf7ffffff at device 2.0 on pci0
agp0: detected 892k stolen memory
agp0: aperture size is 128M
uhci0: <Intel 82801EB (ICH5) USB controller USB-A> port 0xc000-0xc01f irq 15 at device 29.0 on pci0
usb0: <Intel 82801EB (ICH5) USB controller USB-A> on uhci0
usb0: USB revision 1.0
uhub0: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub0: 2 ports with 2 removable, self powered
uhci1: <Intel 82801EB (ICH5) USB controller USB-B> port 0xc400-0xc41f irq 5 at device 29.1 on pci0
usb1: <Intel 82801EB (ICH5) USB controller USB-B> on uhci1
usb1: USB revision 1.0
uhub1: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub1: 2 ports with 2 removable, self powered
uhid0: APC Back-UPS ES 500 FW:801.e5.D USB FW:e5, rev 1.10/1.06, addr 2, iclass 3/0
uhci2: <Intel 82801EB (ICH5) USB controller USB-C> port 0xc800-0xc81f irq 10 at device 29.2 on pci0
usb2: <Intel 82801EB (ICH5) USB controller USB-C> on uhci2
usb2: USB revision 1.0
uhub2: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub2: 2 ports with 2 removable, self powered
uhci3: <Intel 82801EB (ICH5) USB controller USB-D> port 0xcc00-0xcc1f irq 15 at device 29.3 on pci0
usb3: <Intel 82801EB (ICH5) USB controller USB-D> on uhci3
usb3: USB revision 1.0
uhub3: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub3: 2 ports with 2 removable, self powered
pcib1: <Intel 82801BA/BAM (ICH2) Hub to PCI bridge> at device 30.0 on pci0
pci1: <PCI bus> on pcib1
rl0: <RealTek 8139 10/100BaseTX> port 0xa000-0xa0ff mem 0xf9000000-0xf90000ff irq 15 at device 4.0 on pci1
rl0: Ethernet address: 00:50:fc:c7:c2:f8
miibus0: <MII bus> on rl0
rlphy0: <RealTek internal media interface> on miibus0
rlphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
puc0: <SmartLink 5634PCV SurfRider> port 0xa400-0xa407 irq 12 at device 5.0 on pci1
sio2: type 16550A
fxp0: <Intel 82801BA (D865) Pro/100 VE Ethernet> port 0xa800-0xa83f mem 0xf9001000-0xf9001fff irq 11 at device 8.0 on pci1
fxp0: Ethernet address 00:01:80:56:75:7a
inphy0: <i82562ET 10/100 media interface> on miibus1
inphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
puc1: <Lava Computers Quattro-PCI serial port> port 0xb000-0xb007,0xac00-0xac07 irq 10 at device 10.0 on pci1
sio3: type 16550A
sio4: type 16550A
puc2: <Lava Computers Quattro-PCI serial port> port 0xb800-0xb807,0xb400-0xb407 irq 10 at device 10.1 on pci1
sio5: type 16550A
sio6: type 16550A
isab0: <PCI to ISA bridge (vendor=8086 device=24d0)> at device 31.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <Intel ICH5 ATA100 controller> port 0xf000-0xf00f,0-0x3,0-0x7,0-0x3,0-0x7 irq 0 at device 31.1 on pci0
ata0: at 0x1f0 irq 14 on atapci0
ata1: at 0x170 irq 15 on atapci0
ichsmb0: <Intel 82801EB (ICH5) SMBus controller> port 0x5000-0x501f irq 12 at device 31.3 on pci0
smbus0: <System Management Bus> on ichsmb0
smb0: <SMBus general purpose I/O> on smbus0
orm0: <Option ROM> at iomem 0xc0000-0xc9fff on isa0
pmtimer0 on isa0
fdc0: <NEC 72065B or clone> at port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on isa0
atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
sio0 at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0
sio0: type 16550A
sio1 at port 0x2f8-0x2ff irq 3 on isa0
sio1: type 16550A
ipfw2 initialized, divert enabled, rule-based forwarding enabled, default to accept, logging limited to 100 packets/entry by default
IPsec: Initialized Security Association Processing.
ad0: 38166MB <ST340014A> [77545/16/63] at ata0-master UDMA100
Mounting root from ufs:/dev/ad0s1a

>How-To-Repeat:
boot a box with a smartlink PCI modem and have it share its interrupt with a usb controller. 
>Fix:

# diff -u sys/isa/sio.c.prev sys/isa/sio.c
--- sys/isa/sio.c.prev  Thu Sep  9 20:54:24 2004
+++ sys/isa/sio.c       Thu Sep  9 20:54:38 2004
@@ -602,7 +602,6 @@
        { 0x048011c1, "Lucent kermit based PCI Modem", 0x14 },
        { 0x95211415, "Oxford Semiconductor PCI Dual Port Serial", 0x10 },
        { 0x7101135e, "SeaLevel Ultra 530.PCI Single Port Serial", 0x18 },
-       { 0x0000151f, "SmartLink 5634PCV SurfRider", 0x10 },
        { 0x98459710, "Netmos Nm9845 PCI Bridge with Dual UART", 0x10 },
        { 0x00000000, NULL, 0 }
 };


--- sys/dev/puc/pucdata.c.prev  Thu Sep  9 21:01:30 2004
+++ sys/dev/puc/pucdata.c       Thu Sep  9 21:02:48 2004
@@ -804,6 +804,15 @@
            },
        },
 
+        {   "SmartLink 5634PCV SurfRider",
+            {   0x151f, 0x0000, 0,      0       },
+            {   0xffff, 0xffff, 0,      0       },
+            {
+                { PUC_PORT_TYPE_COM, 0x10, 0x00, COM_FREQ },
+            },
+        },
+
+
        /* Actiontec  56K PCI Master */
        {   "Actiontec 56K PCI Master",
            {   0x11c1, 0x0480, 0x0,    0x0     }, 


and for releng5

releng5-865# diff -u sys/dev/puc/pucdata.c.orig sys/dev/puc/pucdata.c
--- sys/dev/puc/pucdata.c.orig  Mon May 17 08:57:30 2004
+++ sys/dev/puc/pucdata.c       Mon Dec  6 15:13:14 2004
@@ -827,6 +827,15 @@
            },
        },
 
+       /* "SmartLink 5634PCV SurfRider */
+       {   "SmartLink 5634PCV SurfRider",
+           {   0x151f, 0x0000, 0,      0       },
+           {   0xffff, 0xffff, 0,      0       },
+           {
+               { PUC_PORT_TYPE_COM, 0x10, 0x00, COM_FREQ },
+           },
+       },
+
        /* Actiontec 56K PCI Master */
        {   "Actiontec 56K PCI Master",
            {   0x11c1, 0x0480, 0,      0       },
releng5-865# 

releng5-865# diff -u sys/dev/sio/sio_pci.c.orig sys/dev/sio/sio_pci.c
--- sys/dev/sio/sio_pci.c.orig  Mon Dec  6 15:15:17 2004
+++ sys/dev/sio/sio_pci.c       Mon Dec  6 15:20:22 2004
@@ -76,7 +76,6 @@
        { 0x048011c1, "Lucent kermit based PCI Modem", 0x14 },
        { 0x95211415, "Oxford Semiconductor PCI Dual Port Serial", 0x10 },
        { 0x7101135e, "SeaLevel Ultra 530.PCI Single Port Serial", 0x18 },
-       { 0x0000151f, "SmartLink 5634PCV SurfRider", 0x10 },
        { 0x0103115d, "Xircom Cardbus modem", 0x10 },
        { 0x98459710, "Netmos Nm9845 PCI Bridge with Dual UART", 0x10 },
        { 0x432214e4, "Broadcom 802.11g/GPRS CardBus (Serial)", 0x10 },
releng5-865# 
>Release-Note:
>Audit-Trail:

From: Bruce Evans <bde@zeta.org.au>
To: Mike Tancsa <mike@sentex.net>
Cc: freebsd-gnats-submit@freebsd.org, freebsd-bugs@freebsd.org
Subject: Re: misc/74786: Smartlink Modem causes interrupt storm on RELENG_4
 and RELENG_5
Date: Tue, 7 Dec 2004 10:55:40 +1100 (EST)

 On Mon, 6 Dec 2004, Mike Tancsa wrote:
 
 > >Description:
 > I think we have been bouncing around this issue for the past few months both on RELENG_4 and RELENG_5.  In the past it has been somewhat difficult to reproduce, but now we can do it reliably.    I dont think its a hardware issue as I can take the exact same 2 boxes with the exact same IRQ assignments and boot with OpenBSD and not run into an interrupt storm or freeze up the box.  Swap back the RELENG_4 or RELENG_5 HD and again, I can produce an interrupt storm at will.
 >
 > I can also reproduce it on 2 different chipsets as well (VIA and Intel).  The problem seems to be around how a PUC device (either a PCI modem or a PCI serial card) and the sharing of an interrupt (usually an USB controller).
 >
 > On RELENG_4, the box just locks up in a race trying to service an interrupt on IRQ 12 but remains unhandled.
 
 This is because interrupt storms are fatal in RELENG_4 (if they happen).
 
 > On RELENG_5, I actually catch an interrupt storm. e.g. I attach to sio4 (PUC modem) and
 >
 > Interrupt storm detected on "irq12: uhci1"; throttling interrupt source
 >
 > Looking at vmstat -i does indeed show a the rate getting throttled
 >
 > releng-5-pioneer# vmstat -i
 > interrupt                          total       rate
 > irq0: clk                         596719         99
 > irq1: atkbd0                           2          0
 > irq4: sio0                          1079          0
 > irq6: fdc0                             1          0
 > irq8: rtc                         763812        127
 > irq12: uhci1                        5825          0
 
 This seems to be from a machine without the problem.  There is no sign
 of a storm here, and no sign of a puc or sio device sharing irq12.
 
 > irq13: npx0                            1          0
 > irq14: ata0                        38727          6
 > irq15: vr0 ata1                     1984          0
 
 The shared case should look like this.  The irq "name" string is too short
 to show more than 1 or 2 devices but I think it would show 2 devices OK
 like it does here.
 
 > Total                            1408150        235
 > releng-5-pioneer#
 >
 > where irq12 is the IRQ shared by the modem and the USB port.  However, because all IRQ 12s get throttled, the modem is unusable. e.g. trying to cu -l /dev/cuaa4 and typing atz takes about 5 seconds.
 
 Does a storm occur when both devices are successfully attached?  Hmm, the
 above is consistent with the following combination:
 1. only usb being attached
 2. the sio device still driving the interrupt but sio not being called to
    handle the interrupt
 3. a very old version of 5.x that has interrupt storm handling with only 1
    interrupt handler call per second for the storming interrupt (later
    versions have HZ interrupts/second)
 4. the old but current rounding bug in vmstat which results in interrupt
    rates of 1-epsilon being displayed as 0 and the clock interrupt rates
    of 100-epsilon and 128-epsilon being displayed as 99 and 127.  The
    above shows a usb rate of approx. 5825/(596719/100) = 0.976.  systat
    would show correctly rounded values.  My version of vmstat has an
    compile-time option to display more precision.
 
 Point (2) shouldn't happen if the device is probed and the probe can find
 the device's irq.  Then the probe turns off the irq.  This is unlikely
 to be a problem for pci devices, but for isa devices it is essential
 to tell the kernel about all devices since interrupts can easily be left
 active after a warm boot from another or the same OS that was using the
 devices.  Such interrupts are deactivated as a side effect of the probe.
 
 > The problem is that the modem is not being seen as a PCI / PUC device and instead is being seen as an ISA SIO device ??  The following RELENG_5 and RELENG_4 patches seem to fix the problem.  I wonder if the other modems listed in sio.c suffer the same fate ?
 
 The primary bug is that bus_setup_intr() still doesn't support dynamic
 choice between fast and normal interrupt handling modes.  All devices
 sharing an irq must use the same mode.  Normal mode must be used unless
 all the relevant drivers support fast mode.  The mode can't be decided
 correctly at attach time or reasonably by drivers at all since the
 full set of drivers is not known at attach time (except for the last
 device, if any).
 
 sio just tries for fast mode first.  If this succeeds then it breaks
 all other devices on the irq that want normal mode.  Minimal breakage
 is for the other devices to not be available.  If their probe or attach
 is buggy or not done then they may cause interrupt storms by driving
 the interrupt.  Whether the try for fast mode succeeds in the shared
 case is too dependent on attach order and upper layers.
 
 Using puc combined with not using PUC_FASTINTR "works" by breaking any
 possibility of using fast mode for puc sio devices.  It makes sio's
 try for fast mode always fail for pci devices.  CY_PCI_FASTINTR does
 the same thing for pci cy devices.  The default is to fail safe (try
 for normal mode only) but to try for fast mode first if *_FASTINTR is
 configured.  The pci layer of sio could implement a similar hack, but
 the layering is not set up for this to be easy, and drivers shouldn't
 have special hacks for this.  The isa layer should only try for fast
 mode since isa irqs can't be shared without special support.
 
 There is only a small relevant difference between PCI and ISA sio
 devices.  It is supposed to be possible to configure sio devices to
 use no irq at all (then they use polled mode and won't cause interrupt
 conflicts) by omitting their irq from the configuration.  Unfortunately,
 configuration became too smart starting with PCI.  The BIOS may support
 moving or not using irqs for PCI devices, but FreeBSD doesn't.
 
 The quickest fix is to change sioattach() to only try for normal mode.
 Normal mode would work better in -current than in RELENG_4 (good enough
 for most configurations, since the latency bugs are reduced), except
 sioattach bogusly asks for non-MPSAFE mode which greatly increases the
 latency bugs relative to RELENG_4.  Fix (?):
 
 %%%
 Index: sio.c
 ===================================================================
 RCS file: /home/ncvs/src/sys/dev/sio/sio.c,v
 retrieving revision 1.442
 diff -u -2 -r1.442 sio.c
 --- sio.c	25 Jun 2004 10:51:33 -0000	1.442
 +++ sio.c	26 Jun 2004 23:11:13 -0000
 @@ -1173,5 +1315,6 @@
  		if (ret) {
  			ret = BUS_SETUP_INTR(device_get_parent(dev), dev,
 -					     com->irqres, INTR_TYPE_TTY,
 +					     com->irqres,
 +					     INTR_TYPE_CLK | INTR_MPSAFE,
  					     siointr, com, &com->cookie);
  			if (ret == 0)
 %%%
 
 This also hacks around the use of the low priority level INTR_TYPE_TTY
 when a very high priority level is preferred.  Using INTR_TYPE_CLK is a
 hack.  A level even higher than that of clocks is preferred.
 
 There may be some brokenness involving layers here.  I thought that
 the above worked, but it shouldn't for puc devices because puc still
 uses INTR_TYPE_TTY and doesn't use INTR_MPSAFE.  It seems to be hard
 for puc to use INTR_TYPE_FASTTTY and INTR_MPSAFE even if all subdevices
 support them.  Whether all subdevices support them is attach ordering
 dependent in the same way as for INTR_FAST.
 
 
 > # cat /var/run/dmesg.boot
 > Copyright (c) 1992-2004 The FreeBSD Project.
 > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
 >         The Regents of the University of California. All rights reserved.
 > FreeBSD 4.10-STABLE #6: Fri Nov 26 13:52:22 EST 2004
 >     mdtancsa@station.sentex.ca:/usr/obj/usr/src/sys/gas
 
 Is this with the fixed version?  The dmesg for the interrupt storms version
 would be more informative.
 
 > ...
 > puc0: <SmartLink 5634PCV SurfRider> port 0xa400-0xa407 irq 12 at device 5.0 on pci1
 > sio2: type 16550A
 > ...
 > ichsmb0: <Intel 82801EB (ICH5) SMBus controller> port 0x5000-0x501f irq 12 at device 31.3 on pci0
 > smbus0: <System Management Bus> on ichsmb0
 
 The conflict seems to be with ichsmb, not with usb.  I don't know if
 either of ichsmb or usb turn off interrupts properly if their attach is
 not reached or fails.
 
 > >How-To-Repeat:
 > boot a box with a smartlink PCI modem and have it share its interrupt with a usb controller.
 > >Fix:
 >
 > # diff -u sys/isa/sio.c.prev sys/isa/sio.c
 > --- sys/isa/sio.c.prev  Thu Sep  9 20:54:24 2004
 > +++ sys/isa/sio.c       Thu Sep  9 20:54:38 2004
 > @@ -602,7 +602,6 @@
 >         { 0x048011c1, "Lucent kermit based PCI Modem", 0x14 },
 >         { 0x95211415, "Oxford Semiconductor PCI Dual Port Serial", 0x10 },
 >         { 0x7101135e, "SeaLevel Ultra 530.PCI Single Port Serial", 0x18 },
 > -       { 0x0000151f, "SmartLink 5634PCV SurfRider", 0x10 },
 >         { 0x98459710, "Netmos Nm9845 PCI Bridge with Dual UART", 0x10 },
 >         { 0x00000000, NULL, 0 }
 >  };
 
 Removing pci support for one of the few pci sio devices that doesn't need
 puc is not good.
 
 > --- sys/dev/puc/pucdata.c.prev  Thu Sep  9 21:01:30 2004
 > +++ sys/dev/puc/pucdata.c       Thu Sep  9 21:02:48 2004
 > @@ -804,6 +804,15 @@
 >             },
 >         },
 >
 > +        {   "SmartLink 5634PCV SurfRider",
 > +            {   0x151f, 0x0000, 0,      0       },
 > +            {   0xffff, 0xffff, 0,      0       },
 > +            {
 > +                { PUC_PORT_TYPE_COM, 0x10, 0x00, COM_FREQ },
 > +            },
 > +        },
 > +
 > +
 >         /* Actiontec  56K PCI Master */
 >         {   "Actiontec 56K PCI Master",
 >             {   0x11c1, 0x0480, 0x0,    0x0     },
 
 ISTR that the pci support must be removed for this to work.  If so,
 then there is another large ordering bug: things apparently break
 because the pci attach happens to be first.  pci first is probably
 right except for the interrupt mode race since it needs less layers
 at runtime, but the order is undocumented AFAIK.
 
 Bruce

From: Mike Tancsa <mike@sentex.net>
To: Bruce Evans <bde@zeta.org.au>
Cc: freebsd-gnats-submit@freebsd.org, freebsd-bugs@freebsd.org
Subject: Re: misc/74786: Smartlink Modem causes interrupt storm on
  RELENG_4 and RELENG_5
Date: Tue, 07 Dec 2004 11:34:38 -0500

 At 06:55 PM 06/12/2004, Bruce Evans wrote:
 >On Mon, 6 Dec 2004, Mike Tancsa wrote:
 >
 > > >Description:
 > > I think we have been bouncing around this issue for the past few months 
 > both on RELENG_4 and RELENG_5.  In the past it has been somewhat 
 > difficult to reproduce, but now we can do it reliably.    I dont think 
 > its a hardware issue as I can take the exact same 2 boxes with the exact 
 > same IRQ assignments and boot with OpenBSD and not run into an interrupt 
 > storm or freeze up the box.  Swap back the RELENG_4 or RELENG_5 HD and 
 > again, I can produce an interrupt storm at will.
 > >
 > > I can also reproduce it on 2 different chipsets as well (VIA and 
 > Intel).  The problem seems to be around how a PUC device (either a PCI 
 > modem or a PCI serial card) and the sharing of an interrupt (usually an 
 > USB controller).
 > >
 > > On RELENG_4, the box just locks up in a race trying to service an 
 > interrupt on IRQ 12 but remains unhandled.
 >
 >This is because interrupt storms are fatal in RELENG_4 (if they happen).
 
 
 Yes, for sure.  The patch does make the box and hardware usable however.
 
 
 
 
 > > On RELENG_5, I actually catch an interrupt storm. e.g. I attach to sio4 
 > (PUC modem) and
 > >
 > > Interrupt storm detected on "irq12: uhci1"; throttling interrupt source
 > >
 > > Looking at vmstat -i does indeed show a the rate getting throttled
 > >
 > > releng-5-pioneer# vmstat -i
 > > interrupt                          total       rate
 > > irq0: clk                         596719         99
 > > irq1: atkbd0                           2          0
 > > irq4: sio0                          1079          0
 > > irq6: fdc0                             1          0
 > > irq8: rtc                         763812        127
 > > irq12: uhci1                        5825          0
 >
 >This seems to be from a machine without the problem.  There is no sign
 >of a storm here, and no sign of a puc or sio device sharing irq12.
 
 I hooked up a RELENG_5 box with the original config to demonstrate / 
 recreate the problem once more
 Here is the dmesg. Notice the "cant reuse leaf" stuff which also gets 
 "fixed" by the patches
 
 releng5-865# cat /var/run/dmesg.boot
 Copyright (c) 1992-2004 The FreeBSD Project.
 Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
          The Regents of the University of California. All rights reserved.
 FreeBSD 5.3-STABLE #1: Wed Dec  1 20:03:29 EST 2004
      mdtancsa@releng5-865.sentex.ca:/usr/obj/usr/src/sys/test
 Timecounter "i8254" frequency 1193182 Hz quality 0
 CPU: Intel(R) Celeron(R) CPU 2.40GHz (2400.41-MHz 686-class CPU)
    Origin = "GenuineIntel"  Id = 0xf29  Stepping = 9
    Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
 real memory  = 267190272 (254 MB)
 avail memory = 251912192 (240 MB)
 npx0: [FAST]
 npx0: <math processor> on motherboard
 npx0: INT 16 interface
 acpi0: <AOpen AWRDACPI> on motherboard
 acpi0: Power Button (fixed)
 Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x4008-0x400b on acpi0
 cpu0: <ACPI CPU> on acpi0
 acpi_tz0: <Thermal Zone> on acpi0
 acpi_button0: <Power Button> on acpi0
 pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
 pci0: <ACPI PCI bus> on pcib0
 agp0: <Intel 82865G (865G GMCH) SVGA controller> port 0xd000-0xd007 mem 
 0xfa000000-0xfa07ffff,0xf0000000-0xf7ffffff at device 2.0 on pci0
 agp0: detected 892k stolen memory
 agp0: aperture size is 128M
 uhci0: <Intel 82801EB (ICH5) USB controller USB-A> port 0xc000-0xc01f irq 
 12 at device 29.0 on pci0
 uhci0: [GIANT-LOCKED]
 usb0: <Intel 82801EB (ICH5) USB controller USB-A> on uhci0
 usb0: USB revision 1.0
 uhub0: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
 uhub0: 2 ports with 2 removable, self powered
 uhci1: <Intel 82801EB (ICH5) USB controller USB-B> port 0xc400-0xc41f irq 5 
 at device 29.1 on pci0
 uhci1: [GIANT-LOCKED]
 usb1: <Intel 82801EB (ICH5) USB controller USB-B> on uhci1
 usb1: USB revision 1.0
 uhub1: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
 uhub1: 2 ports with 2 removable, self powered
 uhci2: <Intel 82801EB (ICH5) USB controller USB-C> port 0xc800-0xc81f irq 
 10 at device 29.2 on pci0
 uhci2: [GIANT-LOCKED]
 usb2: <Intel 82801EB (ICH5) USB controller USB-C> on uhci2
 usb2: USB revision 1.0
 uhub2: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
 uhub2: 2 ports with 2 removable, self powered
 uhci3: <Intel 82801EB (ICH5) USB controller USB-D> port 0xcc00-0xcc1f irq 
 12 at device 29.3 on pci0
 uhci3: [GIANT-LOCKED]
 usb3: <Intel 82801EB (ICH5) USB controller USB-D> on uhci3
 usb3: USB revision 1.0
 uhub3: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
 uhub3: 2 ports with 2 removable, self powered
 pcib1: <ACPI PCI-PCI bridge> at device 30.0 on pci0
 pci1: <ACPI PCI bus> on pcib1
 sio0: <SmartLink 5634PCV SurfRider> port 0xa000-0xa007 irq 12 at device 4.0 
 on pci1
 sio0: moving to sio4
 sio4: type 16550A
 rl0: <RealTek 8139 10/100BaseTX> port 0xa400-0xa4ff mem 
 0xf9000000-0xf90000ff irq 15 at device 5.0 on pci1
 miibus0: <MII bus> on rl0
 rlphy0: <RealTek internal media interface> on miibus0
 rlphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
 rl0: Ethernet address: 00:50:fc:24:2b:4d
 fxp0: <Intel 82801BA (D865) Pro/100 VE Ethernet> port 0xa800-0xa83f mem 
 0xf9001000-0xf9001fff irq 11 at device 8.0 on pci1
 miibus1: <MII bus> on fxp0
 inphy0: <i82562ET 10/100 media interface> on miibus1
 inphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
 fxp0: Ethernet address: 00:01:80:54:b7:09
 puc0: <Lava Computers Quattro-PCI serial port> port 
 0xb000-0xb007,0xac00-0xac07 irq 10 at device 10.0 on pci1
 sio5: <Lava Computers Quattro-PCI serial port> on puc0
 sio5: type 16550A
 sio5: unable to activate interrupt in fast mode - using normal mode
 sio6: <Lava Computers Quattro-PCI serial port> on puc0
 sio6: type 16550A
 sio6: unable to activate interrupt in fast mode - using normal mode
 puc1: <Lava Computers Quattro-PCI serial port> port 
 0xb800-0xb807,0xb400-0xb407 irq 10 at device 10.1 on pci1
 sio7: <Lava Computers Quattro-PCI serial port> on puc1
 sio7: type 16550A
 sio7: unable to activate interrupt in fast mode - using normal mode
 sio8: <Lava Computers Quattro-PCI serial port> on puc1
 sio8: type 16550A
 sio8: unable to activate interrupt in fast mode - using normal mode
 isab0: <PCI-ISA bridge> at device 31.0 on pci0
 isa0: <ISA bus> on isab0
 atapci0: <Intel ICH5 UDMA100 controller> port 
 0xf000-0xf00f,0x376,0x170-0x177,0x3f6,0x1f0-0x1f7 at device 31.1 on pci0
 ata0: channel #0 on atapci0
 ata1: channel #1 on atapci0
 pci0: <serial bus, SMBus> at device 31.3 (no driver attached)
 can't re-use a leaf (%desc)!
 can't re-use a leaf (%driver)!
 can't re-use a leaf (%location)!
 can't re-use a leaf (%pnpinfo)!
 can't re-use a leaf (%parent)!
 sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
 sio0: type 16550A, console
 sio1: <16550A-compatible COM port> port 0x2f8-0x2ff irq 3 on acpi0
 sio1: type 16550A
 atkbdc0: <Keyboard controller (i8042)> port 0x64,0x60 irq 1 on acpi0
 atkbd0: <AT Keyboard> irq 1 on atkbdc0
 kbd0 at atkbd0
 atkbd0: [GIANT-LOCKED]
 orm0: <ISA Option ROM> at iomem 0xc0000-0xc9fff on isa0
 ppc0: parallel port not found.
 sc0: <System console> at flags 0x100 on isa0
 sc0: VGA <16 virtual consoles, flags=0x100>
 vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
 Timecounter "TSC" frequency 2400414032 Hz quality 800
 Timecounters tick every 10.000 msec
 Fast IPsec: Initialized Security Association Processing.
 ad0: 38166MB <ST340014A/3.06> [77545/16/63] at ata0-master UDMA100
 Mounting root from ufs:/dev/ad0s1a
 releng5-865#
 
 Attaching to the modem which is shared with the USB immediately causes the 
 storm
 
 
 
 releng5-865# vmstat -i
 interrupt                          total       rate
 irq0: clk                          28936         99
 irq4: sio0                           207          0
 irq8: rtc                          37039        127
 irq10: uhci2 puc0+                     6          0
 irq11: fxp0                          276          0
 irq12: uhci0 uhci3                 15858         54
 irq13: npx0                            1          0
 irq14: ata0                         1060          3
 Total                              83383        287
 releng5-865#
 
 
 Dec  7 10:04:02 releng5-865 kernel: Interrupt storm detected on "irq12: 
 uhci0 uhci3"; throttling interrupt source
 
 uhci0, uhci3... But no mention of the modem which is on irq 12 as well.
 
 releng5-865# cu -l /dev/cuaa4
 Connected
 atz
 OK
 atz
 OK
 ati3
 TP560 Data/Fa
 
 Although it "works" the modem is unusable as typing "atz" takes about 1 
 second for each char to echo to the screen.  Typing ati3 doesnt work 
 properly as all the data never makes it back
 
 ati3
 TP560 Data/Fa
 
 > > irq13: npx0                            1          0
 > > irq14: ata0                        38727          6
 > > irq15: vr0 ata1                     1984          0
 >
 >The shared case should look like this.  The irq "name" string is too short
 >to show more than 1 or 2 devices but I think it would show 2 devices OK
 >like it does here.
 >
 > > Total                            1408150        235
 > > releng-5-pioneer#
 > >
 > > where irq12 is the IRQ shared by the modem and the USB port.  However, 
 > because all IRQ 12s get throttled, the modem is unusable. e.g. trying to 
 > cu -l /dev/cuaa4 and typing atz takes about 5 seconds.
 >
 >Does a storm occur when both devices are successfully attached?  Hmm, the
 >above is consistent with the following combination:
 >1. only usb being attached
 >2. the sio device still driving the interrupt but sio not being called to
 >    handle the interrupt
 Yes, that sounds correct.
 
 
 
 >3. a very old version of 5.x that has interrupt storm handling with only 1
 >    interrupt handler call per second for the storming interrupt (later
 >    versions have HZ interrupts/second)
 
 I am using FreeBSD releng5-865.sentex.ca 5.3-STABLE FreeBSD 5.3-STABLE #1: 
 Wed Dec  1 20:03:29 EST 2004
 
 
 
 > > The problem is that the modem is not being seen as a PCI / PUC device 
 > and instead is being seen as an ISA SIO device ??  The following RELENG_5 
 > and RELENG_4 patches seem to fix the problem.  I wonder if the other 
 > modems listed in sio.c suffer the same fate ?
 >
 >The primary bug is that bus_setup_intr() still doesn't support dynamic
 >choice between fast and normal interrupt handling modes.  All devices
 >sharing an irq must use the same mode.  Normal mode must be used unless
 >all the relevant drivers support fast mode.  The mode can't be decided
 >correctly at attach time or reasonably by drivers at all since the
 >full set of drivers is not known at attach time (except for the last
 >device, if any).
 >
 >sio just tries for fast mode first.  If this succeeds then it breaks
 >all other devices on the irq that want normal mode.  Minimal breakage
 >is for the other devices to not be available.  If their probe or attach
 >is buggy or not done then they may cause interrupt storms by driving
 >the interrupt.  Whether the try for fast mode succeeds in the shared
 >case is too dependent on attach order and upper layers.
 >
 >Using puc combined with not using PUC_FASTINTR "works" by breaking any
 >possibility of using fast mode for puc sio devices.  It makes sio's
 >try for fast mode always fail for pci devices.  CY_PCI_FASTINTR does
 >the same thing for pci cy devices.  The default is to fail safe (try
 >for normal mode only) but to try for fast mode first if *_FASTINTR is
 >configured.  The pci layer of sio could implement a similar hack, but
 >the layering is not set up for this to be easy, and drivers shouldn't
 >have special hacks for this.  The isa layer should only try for fast
 >mode since isa irqs can't be shared without special support.
 >
 >There is only a small relevant difference between PCI and ISA sio
 >devices.  It is supposed to be possible to configure sio devices to
 >use no irq at all (then they use polled mode and won't cause interrupt
 >conflicts) by omitting their irq from the configuration.  Unfortunately,
 >configuration became too smart starting with PCI.  The BIOS may support
 >moving or not using irqs for PCI devices, but FreeBSD doesn't.
 >
 >The quickest fix is to change sioattach() to only try for normal mode.
 >Normal mode would work better in -current than in RELENG_4 (good enough
 >for most configurations, since the latency bugs are reduced), except
 >sioattach bogusly asks for non-MPSAFE mode which greatly increases the
 >latency bugs relative to RELENG_4.  Fix (?):
 >
 >%%%
 >Index: sio.c
 >===================================================================
 >RCS file: /home/ncvs/src/sys/dev/sio/sio.c,v
 >retrieving revision 1.442
 >diff -u -2 -r1.442 sio.c
 >--- sio.c       25 Jun 2004 10:51:33 -0000      1.442
 >+++ sio.c       26 Jun 2004 23:11:13 -0000
 >@@ -1173,5 +1315,6 @@
 >                 if (ret) {
 >                         ret = BUS_SETUP_INTR(device_get_parent(dev), dev,
 >-                                            com->irqres, INTR_TYPE_TTY,
 >+                                            com->irqres,
 >+                                            INTR_TYPE_CLK | INTR_MPSAFE,
 >                                              siointr, com, &com->cookie);
 >                         if (ret == 0)
 >%%%
 
 
 OK, here is the dmesg with your patch
 
 It does not fix the interrupt storm issue.
 Interrupt storm detected on "irq12: uhci0 uhci3"; throttling interrupt source
 
 
 releng5-865# cat /var/run/dmesg.boot
 Copyright (c) 1992-2004 The FreeBSD Project.
 Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
          The Regents of the University of California. All rights reserved.
 FreeBSD 5.3-STABLE #4: Tue Dec  7 10:44:00 EST 2004
      mdtancsa@releng5-865.sentex.ca:/usr/obj/usr/src/sys/test
 Timecounter "i8254" frequency 1193182 Hz quality 0
 CPU: Intel(R) Celeron(R) CPU 2.40GHz (2400.41-MHz 686-class CPU)
    Origin = "GenuineIntel"  Id = 0xf29  Stepping = 9
    Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
 real memory  = 267190272 (254 MB)
 avail memory = 251912192 (240 MB)
 npx0: [FAST]
 npx0: <math processor> on motherboard
 npx0: INT 16 interface
 acpi0: <AOpen AWRDACPI> on motherboard
 acpi0: Power Button (fixed)
 Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x4008-0x400b on acpi0
 cpu0: <ACPI CPU> on acpi0
 acpi_tz0: <Thermal Zone> on acpi0
 acpi_button0: <Power Button> on acpi0
 pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
 pci0: <ACPI PCI bus> on pcib0
 agp0: <Intel 82865G (865G GMCH) SVGA controller> port 0xd000-0xd007 mem 
 0xfa000000-0xfa07ffff,0xf0000000-0xf7ffffff at device 2.0 on pci0
 agp0: detected 892k stolen memory
 agp0: aperture size is 128M
 uhci0: <Intel 82801EB (ICH5) USB controller USB-A> port 0xc000-0xc01f irq 
 12 at device 29.0 on pci0
 uhci0: [GIANT-LOCKED]
 usb0: <Intel 82801EB (ICH5) USB controller USB-A> on uhci0
 usb0: USB revision 1.0
 uhub0: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
 uhub0: 2 ports with 2 removable, self powered
 uhid0: APC Back-UPS ES 725 FW:802.n2.D USB FW:n2, rev 1.10/1.06, addr 2, 
 iclass 3/0
 uhci1: <Intel 82801EB (ICH5) USB controller USB-B> port 0xc400-0xc41f irq 5 
 at device 29.1 on pci0
 uhci1: [GIANT-LOCKED]
 usb1: <Intel 82801EB (ICH5) USB controller USB-B> on uhci1
 usb1: USB revision 1.0
 uhub1: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
 uhub1: 2 ports with 2 removable, self powered
 uhci2: <Intel 82801EB (ICH5) USB controller USB-C> port 0xc800-0xc81f irq 
 10 at device 29.2 on pci0
 uhci2: [GIANT-LOCKED]
 usb2: <Intel 82801EB (ICH5) USB controller USB-C> on uhci2
 usb2: USB revision 1.0
 uhub2: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
 uhub2: 2 ports with 2 removable, self powered
 uhci3: <Intel 82801EB (ICH5) USB controller USB-D> port 0xcc00-0xcc1f irq 
 12 at device 29.3 on pci0
 uhci3: [GIANT-LOCKED]
 usb3: <Intel 82801EB (ICH5) USB controller USB-D> on uhci3
 usb3: USB revision 1.0
 uhub3: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
 uhub3: 2 ports with 2 removable, self powered
 pcib1: <ACPI PCI-PCI bridge> at device 30.0 on pci0
 pci1: <ACPI PCI bus> on pcib1
 sio0: <SmartLink 5634PCV SurfRider> port 0xa000-0xa007 irq 12 at device 4.0 
 on pci1
 sio0: moving to sio4
 sio4: type 16550A
 rl0: <RealTek 8139 10/100BaseTX> port 0xa400-0xa4ff mem 
 0xf9000000-0xf90000ff irq 15 at device 5.0 on pci1
 miibus0: <MII bus> on rl0
 rlphy0: <RealTek internal media interface> on miibus0
 rlphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
 rl0: Ethernet address: 00:50:fc:24:2b:4d
 fxp0: <Intel 82801BA (D865) Pro/100 VE Ethernet> port 0xa800-0xa83f mem 
 0xf9001000-0xf9001fff irq 11 at device 8.0 on pci1
 miibus1: <MII bus> on fxp0
 inphy0: <i82562ET 10/100 media interface> on miibus1
 inphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
 fxp0: Ethernet address: 00:01:80:54:b7:09
 puc0: <Lava Computers Quattro-PCI serial port> port 
 0xb000-0xb007,0xac00-0xac07 irq 10 at device 10.0 on pci1
 sio5: <Lava Computers Quattro-PCI serial port> on puc0
 sio5: type 16550A
 sio5: unable to activate interrupt in fast mode - using normal mode
 sio6: <Lava Computers Quattro-PCI serial port> on puc0
 sio6: type 16550A
 sio6: unable to activate interrupt in fast mode - using normal mode
 puc1: <Lava Computers Quattro-PCI serial port> port 
 0xb800-0xb807,0xb400-0xb407 irq 10 at device 10.1 on pci1
 sio7: <Lava Computers Quattro-PCI serial port> on puc1
 sio7: type 16550A
 sio7: unable to activate interrupt in fast mode - using normal mode
 sio8: <Lava Computers Quattro-PCI serial port> on puc1
 sio8: type 16550A
 sio8: unable to activate interrupt in fast mode - using normal mode
 isab0: <PCI-ISA bridge> at device 31.0 on pci0
 isa0: <ISA bus> on isab0
 atapci0: <Intel ICH5 UDMA100 controller> port 
 0xf000-0xf00f,0x376,0x170-0x177,0x3f6,0x1f0-0x1f7 at device 31.1 on pci0
 ata0: channel #0 on atapci0
 ata1: channel #1 on atapci0
 pci0: <serial bus, SMBus> at device 31.3 (no driver attached)
 can't re-use a leaf (%desc)!
 can't re-use a leaf (%driver)!
 can't re-use a leaf (%location)!
 can't re-use a leaf (%pnpinfo)!
 can't re-use a leaf (%parent)!
 sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
 sio0: type 16550A, console
 sio1: <16550A-compatible COM port> port 0x2f8-0x2ff irq 3 on acpi0
 sio1: type 16550A
 atkbdc0: <Keyboard controller (i8042)> port 0x64,0x60 irq 1 on acpi0
 atkbd0: <AT Keyboard> irq 1 on atkbdc0
 kbd0 at atkbd0
 atkbd0: [GIANT-LOCKED]
 orm0: <ISA Option ROM> at iomem 0xc0000-0xc9fff on isa0
 ppc0: parallel port not found.
 sc0: <System console> at flags 0x100 on isa0
 sc0: VGA <16 virtual consoles, flags=0x100>
 vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
 Timecounter "TSC" frequency 2400409804 Hz quality 800
 Timecounters tick every 10.000 msec
 Fast IPsec: Initialized Security Association Processing.
 ad0: 38166MB <ST340014A/3.06> [77545/16/63] at ata0-master UDMA100
 Mounting root from ufs:/dev/ad0s1a
 releng5-865#
 
 releng5-865# vmstat -i
 interrupt                          total       rate
 irq0: clk                          22924         99
 irq4: sio0                           181          0
 irq8: rtc                          29344        127
 irq10: uhci2 puc0+                     6          0
 irq11: fxp0                          382          1
 irq12: uhci0 uhci3                 12318         53
 irq13: npx0                            1          0
 irq14: ata0                         1136          4
 Total                              66292        288
 releng5-865#
 
 
 And then with my patches, the dmesg looks like
 
 Copyright (c) 1992-2004 The FreeBSD Project.
 Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
          The Regents of the University of California. All rights reserved.
 FreeBSD 5.3-STABLE #0: Tue Dec  7 11:29:41 EST 2004
      mdtancsa@releng5-865.sentex.ca:/usr/obj/usr/src/sys/test
 Timecounter "i8254" frequency 1193182 Hz quality 0
 CPU: Intel(R) Celeron(R) CPU 2.40GHz (2400.41-MHz 686-class CPU)
    Origin = "GenuineIntel"  Id = 0xf29  Stepping = 9
    Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
 real memory  = 267190272 (254 MB)
 avail memory = 251912192 (240 MB)
 npx0: [FAST]
 npx0: <math processor> on motherboard
 npx0: INT 16 interface
 acpi0: <AOpen AWRDACPI> on motherboard
 acpi0: Power Button (fixed)
 Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x4008-0x400b on acpi0
 cpu0: <ACPI CPU> on acpi0
 acpi_tz0: <Thermal Zone> on acpi0
 acpi_button0: <Power Button> on acpi0
 pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
 pci0: <ACPI PCI bus> on pcib0
 agp0: <Intel 82865G (865G GMCH) SVGA controller> port 0xd000-0xd007 mem 
 0xfa000000-0xfa07ffff,0xf0000000-0xf7ffffff at device 2.0 on pci0
 agp0: detected 892k stolen memory
 agp0: aperture size is 128M
 uhci0: <Intel 82801EB (ICH5) USB controller USB-A> port 0xc000-0xc01f irq 
 12 at device 29.0 on pci0
 uhci0: [GIANT-LOCKED]
 usb0: <Intel 82801EB (ICH5) USB controller USB-A> on uhci0
 usb0: USB revision 1.0
 uhub0: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
 uhub0: 2 ports with 2 removable, self powered
 uhid0: APC Back-UPS ES 725 FW:802.n2.D USB FW:n2, rev 1.10/1.06, addr 2, 
 iclass 3/0
 uhci1: <Intel 82801EB (ICH5) USB controller USB-B> port 0xc400-0xc41f irq 5 
 at device 29.1 on pci0
 uhci1: [GIANT-LOCKED]
 usb1: <Intel 82801EB (ICH5) USB controller USB-B> on uhci1
 usb1: USB revision 1.0
 uhub1: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
 uhub1: 2 ports with 2 removable, self powered
 uhci2: <Intel 82801EB (ICH5) USB controller USB-C> port 0xc800-0xc81f irq 
 10 at device 29.2 on pci0
 uhci2: [GIANT-LOCKED]
 usb2: <Intel 82801EB (ICH5) USB controller USB-C> on uhci2
 usb2: USB revision 1.0
 uhub2: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
 uhub2: 2 ports with 2 removable, self powered
 uhci3: <Intel 82801EB (ICH5) USB controller USB-D> port 0xcc00-0xcc1f irq 
 12 at device 29.3 on pci0
 uhci3: [GIANT-LOCKED]
 usb3: <Intel 82801EB (ICH5) USB controller USB-D> on uhci3
 usb3: USB revision 1.0
 uhub3: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
 uhub3: 2 ports with 2 removable, self powered
 pcib1: <ACPI PCI-PCI bridge> at device 30.0 on pci0
 pci1: <ACPI PCI bus> on pcib1
 puc0: <SmartLink 5634PCV SurfRider> port 0xa000-0xa007 irq 12 at device 4.0 
 on pci1
 sio4: <SmartLink 5634PCV SurfRider> on puc0
 sio4: type 16550A
 sio4: unable to activate interrupt in fast mode - using normal mode
 rl0: <RealTek 8139 10/100BaseTX> port 0xa400-0xa4ff mem 
 0xf9000000-0xf90000ff irq 15 at device 5.0 on pci1
 miibus0: <MII bus> on rl0
 rlphy0: <RealTek internal media interface> on miibus0
 rlphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
 rl0: Ethernet address: 00:50:fc:24:2b:4d
 fxp0: <Intel 82801BA (D865) Pro/100 VE Ethernet> port 0xa800-0xa83f mem 
 0xf9001000-0xf9001fff irq 11 at device 8.0 on pci1
 miibus1: <MII bus> on fxp0
 inphy0: <i82562ET 10/100 media interface> on miibus1
 inphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
 fxp0: Ethernet address: 00:01:80:54:b7:09
 puc1: <Lava Computers Quattro-PCI serial port> port 
 0xb000-0xb007,0xac00-0xac07 irq 10 at device 10.0 on pci1
 sio5: <Lava Computers Quattro-PCI serial port> on puc1
 sio5: type 16550A
 sio5: unable to activate interrupt in fast mode - using normal mode
 sio6: <Lava Computers Quattro-PCI serial port> on puc1
 sio6: type 16550A
 sio6: unable to activate interrupt in fast mode - using normal mode
 puc2: <Lava Computers Quattro-PCI serial port> port 
 0xb800-0xb807,0xb400-0xb407 irq 10 at device 10.1 on pci1
 sio7: <Lava Computers Quattro-PCI serial port> on puc2
 sio7: type 16550A
 sio7: unable to activate interrupt in fast mode - using normal mode
 sio8: <Lava Computers Quattro-PCI serial port> on puc2
 sio8: type 16550A
 sio8: unable to activate interrupt in fast mode - using normal mode
 isab0: <PCI-ISA bridge> at device 31.0 on pci0
 isa0: <ISA bus> on isab0
 atapci0: <Intel ICH5 UDMA100 controller> port 
 0xf000-0xf00f,0x376,0x170-0x177,0x3f6,0x1f0-0x1f7 at device 31.1 on pci0
 ata0: channel #0 on atapci0
 ata1: channel #1 on atapci0
 pci0: <serial bus, SMBus> at device 31.3 (no driver attached)
 sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
 sio0: type 16550A, console
 sio1: <16550A-compatible COM port> port 0x2f8-0x2ff irq 3 on acpi0
 sio1: type 16550A
 atkbdc0: <Keyboard controller (i8042)> port 0x64,0x60 irq 1 on acpi0
 atkbd0: <AT Keyboard> irq 1 on atkbdc0
 kbd0 at atkbd0
 atkbd0: [GIANT-LOCKED]
 orm0: <ISA Option ROM> at iomem 0xc0000-0xc9fff on isa0
 ppc0: parallel port not found.
 sc0: <System console> at flags 0x100 on isa0
 sc0: VGA <16 virtual consoles, flags=0x100>
 vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
 Timecounter "TSC" frequency 2400409704 Hz quality 800
 Timecounters tick every 10.000 msec
 Fast IPsec: Initialized Security Association Processing.
 ad0: 38166MB <ST340014A/3.06> [77545/16/63] at ata0-master UDMA100
 Mounting root from ufs:/dev/ad0s1a
 
 And
 releng5-865# vmstat -i
 interrupt                          total       rate
 irq0: clk                          12964         98
 irq4: sio0                           164          1
 irq8: rtc                          16594        126
 irq10: uhci2 puc1+                     6          0
 irq11: fxp0                          266          2
 irq12: uhci0 uhci3+                   34          0
 irq13: npx0                            1          0
 irq14: ata0                          999          7
 Total                              31028        236
 releng5-865#
 indicated there are more things on irq12
 
 
          ---Mike
 
 >This also hacks around the use of the low priority level INTR_TYPE_TTY
 >when a very high priority level is preferred.  Using INTR_TYPE_CLK is a
 >hack.  A level even higher than that of clocks is preferred.
 >
 >There may be some brokenness involving layers here.  I thought that
 >the above worked, but it shouldn't for puc devices because puc still
 >uses INTR_TYPE_TTY and doesn't use INTR_MPSAFE.  It seems to be hard
 >for puc to use INTR_TYPE_FASTTTY and INTR_MPSAFE even if all subdevices
 >support them.  Whether all subdevices support them is attach ordering
 >dependent in the same way as for INTR_FAST.
 >
 >Removing pci support for one of the few pci sio devices that doesn't need
 >puc is not good.
 
 
 
 > > --- sys/dev/puc/pucdata.c.prev  Thu Sep  9 21:01:30 2004
 > > +++ sys/dev/puc/pucdata.c       Thu Sep  9 21:02:48 2004
 > > @@ -804,6 +804,15 @@
 > >             },
 > >         },
 > >
 > > +        {   "SmartLink 5634PCV SurfRider",
 > > +            {   0x151f, 0x0000, 0,      0       },
 > > +            {   0xffff, 0xffff, 0,      0       },
 > > +            {
 > > +                { PUC_PORT_TYPE_COM, 0x10, 0x00, COM_FREQ },
 > > +            },
 > > +        },
 > > +
 > > +
 > >         /* Actiontec  56K PCI Master */
 > >         {   "Actiontec 56K PCI Master",
 > >             {   0x11c1, 0x0480, 0x0,    0x0     },
 >
 >ISTR that the pci support must be removed for this to work.  If so,
 >then there is another large ordering bug: things apparently break
 >because the pci attach happens to be first.  pci first is probably
 >right except for the interrupt mode race since it needs less layers
 >at runtime, but the order is undocumented AFAIK.
 >
 >Bruce
 

From: Bruce Evans <bde@zeta.org.au>
To: Mike Tancsa <mike@sentex.net>
Cc: freebsd-gnats-submit@freebsd.org, freebsd-bugs@freebsd.org
Subject: Re: misc/74786: Smartlink Modem causes interrupt storm on  RELENG_4
 and RELENG_5
Date: Wed, 8 Dec 2004 18:22:18 +1100 (EST)

 On Tue, 7 Dec 2004, Mike Tancsa wrote:
 
 > At 06:55 PM 06/12/2004, Bruce Evans wrote:
 > >On Mon, 6 Dec 2004, Mike Tancsa wrote:
 > ...
 > > > where irq12 is the IRQ shared by the modem and the USB port.  However,
 > > because all IRQ 12s get throttled, the modem is unusable. e.g. trying to
 > > cu -l /dev/cuaa4 and typing atz takes about 5 seconds.
 > >
 > >Does a storm occur when both devices are successfully attached?  Hmm, the
 > >above is consistent with the following combination:
 > >1. only usb being attached
 > >2. the sio device still driving the interrupt but sio not being called to
 > >    handle the interrupt
 > Yes, that sounds correct.
 
 I think I understand this now.  sio can indeed drive the interrupt (after
 you open an sio device, but not immediately at the end of the attach
 except in the serial console case).  The main bugs are:
 1. sio asks for exclusive access to the interrupt for no good reason
    (some buses like isa might only support exclusive accesses, but sio
    doesn't care).  uhci gets access first in your configuration, so
    allocation of the interrupt resource fails.
 2. Error handling for the failure in (1) is null, so both devices are
    "successfully" attached.
 3. sio sets a flag to tell it to use polling if there is no interrupt
    resource, but it doesn't set the flag if the interrupt resource
    couldn't be allocated or if the interrupt couldn't be set up.
 4. Upper layers provide negative help for debugging (3) using their
    own version of (3).  They print "irq N" in boot messages if an
    interrupt resource justs exists.  This doesn't mean that the device
    is using it.
 5. Device interrupts are still enabled in polling mode.  This depends
    on nothing else sucessfully setting up the (shared) interrupt.
 
 Try this fix.  For the uhci-sio conflict, it should avoid problem (1)
 (please try it with the RF_SHAREABLE flag removed so that if this is
 indeed the primary problem then the new diagnostic will verify it).
 This should move the conflict later, to one between usb's use of
 normal interrupt mode and sio's request for fast interrupt mode.
 There were already sufficient diagnostics for this.  The patch adds
 some more.  Since usb got the interrupt first, sio will fall back
 to normal mode.  If sio goes first then the attach of usb would
 probably fail.  The patch doesn't change these bevhaviours.  The
 patch adds another fallback to polled mode if setup of normal mode
 mode fails and fixes some related resource leaks.  It also increases
 the polling frequency if possible.  Polled mode mostly works at
 115200 bps provided the UART is buffered and HZ >= 1000.  With
 unbuffered UARTs and HZ = 100, only 4800 bos mostly works.
 
 I only tested with some fudged isa conflicts because all my machines
 are configured to not have any shared irqs for serial devices and
 hints are just hints so they can't be used to force conflicts excet
 for isa.
 
 %%%
 Index: sio.c
 ===================================================================
 RCS file: /home/ncvs/src/sys/dev/sio/sio.c,v
 retrieving revision 1.442
 diff -u -2 -r1.442 sio.c
 --- sio.c	25 Jun 2004 10:51:33 -0000	1.442
 +++ sio.c	8 Dec 2004 06:09:12 -0000
 @@ -1165,7 +1165,14 @@
  	pps_init(&com->pps);
 
 +	if (com->no_irq)
 +		goto over_irq_setup;
  	rid = 0;
 -	com->irqres = bus_alloc_resource_any(dev, SYS_RES_IRQ, &rid, RF_ACTIVE);
 -	if (com->irqres) {
 +	com->irqres = bus_alloc_resource_any(dev, SYS_RES_IRQ, &rid,
 +					     RF_SHAREABLE | RF_ACTIVE);
 +	if (com->irqres == NULL) {
 +		device_printf(dev, "interrupt resource allocation failed\n");
 +		device_printf(dev, "falling back to polled mode\n");
 +		com->no_irq = TRUE;
 +	} else {
  		ret = BUS_SETUP_INTR(device_get_parent(dev), dev, com->irqres,
  				     INTR_TYPE_TTY | INTR_FAST,
 @@ -1178,6 +1185,11 @@
  				device_printf(dev, "unable to activate interrupt in fast mode - using normal mode\n");
  		}
 -		if (ret)
 +		if (ret != 0) {
  			device_printf(dev, "could not activate interrupt\n");
 +			bus_release_resource(dev, SYS_RES_IRQ, 0, com->irqres);
 +			com->irqres = NULL;
 +			device_printf(dev, "falling back to polled mode\n");
 +			com->no_irq = TRUE;
 +		}
  #if defined(DDB) && (defined(BREAK_TO_DEBUGGER) || \
      defined(ALT_BREAK_TO_DEBUGGER))
 @@ -1191,4 +1203,9 @@
  #endif
  	}
 +over_irq_setup:
 +	if (com->no_irq) {
 +		outb(com->modem_ctl_port, com->mcr_image &= ~MCR_IENABLE);
 +		device_printf(dev, "polled mode\n");
 +	}
 
  	return (0);
 @@ -2647,5 +2664,5 @@
  	/*
  	 * Set our timeout period to 1 second if no polled devices are open.
 -	 * Otherwise set it to max(1/200, 1/hz).
 +	 * Otherwise, set it to max(1/1000, 1/hz).
  	 * Enable timeouts iff some device is open.
  	 */
 @@ -2659,5 +2676,5 @@
  			someopen = TRUE;
  			if (com->poll || com->poll_output) {
 -				sio_timeout = hz > 200 ? hz / 200 : 1;
 +				sio_timeout = hz > 1000 ? hz / 1000 : 1;
  				break;
  			}
 %%%
 
 Bruce

From: Mike Tancsa <mike@sentex.net>
To: Bruce Evans <bde@zeta.org.au>
Cc: freebsd-gnats-submit@freebsd.org, freebsd-bugs@freebsd.org
Subject: Re: misc/74786: Smartlink Modem causes interrupt storm on 
  RELENG_4 and RELENG_5
Date: Wed, 08 Dec 2004 10:35:17 -0500

 At 02:22 AM 08/12/2004, Bruce Evans wrote:
 >Try this fix.  For the uhci-sio conflict, it should avoid problem (1)
 
 Hi,
 
 The dmesg still complains about the "cant reuse leaf" as well as the other 
 PUC card now complains as well about "unable to activate interrupt in fast 
 mode - using normal mode"
 
 releng5-865% cat /var/run/dmesg.boot
 Copyright (c) 1992-2004 The FreeBSD Project.
 Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
          The Regents of the University of California. All rights reserved.
 FreeBSD 5.3-STABLE #1: Wed Dec  8 10:05:17 EST 2004
      mdtancsa@releng5-865.sentex.ca:/usr/obj/usr/src/sys/test
 Timecounter "i8254" frequency 1193182 Hz quality 0
 CPU: Intel(R) Celeron(R) CPU 2.40GHz (2400.41-MHz 686-class CPU)
    Origin = "GenuineIntel"  Id = 0xf29  Stepping = 9
    Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
 real memory  = 267190272 (254 MB)
 avail memory = 251912192 (240 MB)
 npx0: [FAST]
 npx0: <math processor> on motherboard
 npx0: INT 16 interface
 acpi0: <AOpen AWRDACPI> on motherboard
 acpi0: Power Button (fixed)
 Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x4008-0x400b on acpi0
 cpu0: <ACPI CPU> on acpi0
 acpi_tz0: <Thermal Zone> on acpi0
 acpi_button0: <Power Button> on acpi0
 pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
 pci0: <ACPI PCI bus> on pcib0
 agp0: <Intel 82865G (865G GMCH) SVGA controller> port 0xd000-0xd007 mem 
 0xfa000000-0xfa07ffff,0xf0000000-0xf7ffffff at device 2.0 on pci0
 agp0: detected 892k stolen memory
 agp0: aperture size is 128M
 uhci0: <Intel 82801EB (ICH5) USB controller USB-A> port 0xc000-0xc01f irq 
 12 at device 29.0 on pci0
 uhci0: [GIANT-LOCKED]
 usb0: <Intel 82801EB (ICH5) USB controller USB-A> on uhci0
 usb0: USB revision 1.0
 uhub0: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
 uhub0: 2 ports with 2 removable, self powered
 uhid0: APC Back-UPS ES 725 FW:802.n2.D USB FW:n2, rev 1.10/1.06, addr 2, 
 iclass 3/0
 uhci1: <Intel 82801EB (ICH5) USB controller USB-B> port 0xc400-0xc41f irq 5 
 at device 29.1 on pci0
 uhci1: [GIANT-LOCKED]
 usb1: <Intel 82801EB (ICH5) USB controller USB-B> on uhci1
 usb1: USB revision 1.0
 uhub1: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
 uhub1: 2 ports with 2 removable, self powered
 uhci2: <Intel 82801EB (ICH5) USB controller USB-C> port 0xc800-0xc81f irq 
 10 at device 29.2 on pci0
 uhci2: [GIANT-LOCKED]
 usb2: <Intel 82801EB (ICH5) USB controller USB-C> on uhci2
 usb2: USB revision 1.0
 uhub2: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
 uhub2: 2 ports with 2 removable, self powered
 uhci3: <Intel 82801EB (ICH5) USB controller USB-D> port 0xcc00-0xcc1f irq 
 12 at device 29.3 on pci0
 uhci3: [GIANT-LOCKED]
 usb3: <Intel 82801EB (ICH5) USB controller USB-D> on uhci3
 usb3: USB revision 1.0
 uhub3: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
 uhub3: 2 ports with 2 removable, self powered
 pcib1: <ACPI PCI-PCI bridge> at device 30.0 on pci0
 pci1: <ACPI PCI bus> on pcib1
 sio0: <SmartLink 5634PCV SurfRider> port 0xa000-0xa007 irq 12 at device 4.0 
 on pci1
 sio0: moving to sio4
 sio4: type 16550A
 sio4: unable to activate interrupt in fast mode - using normal mode
 rl0: <RealTek 8139 10/100BaseTX> port 0xa400-0xa4ff mem 
 0xf9000000-0xf90000ff irq 15 at device 5.0 on pci1
 miibus0: <MII bus> on rl0
 rlphy0: <RealTek internal media interface> on miibus0
 rlphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
 rl0: Ethernet address: 00:50:fc:24:2b:4d
 fxp0: <Intel 82801BA (D865) Pro/100 VE Ethernet> port 0xa800-0xa83f mem 
 0xf9001000-0xf9001fff irq 11 at device 8.0 on pci1
 miibus1: <MII bus> on fxp0
 inphy0: <i82562ET 10/100 media interface> on miibus1
 inphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
 fxp0: Ethernet address: 00:01:80:54:b7:09
 puc0: <Lava Computers Quattro-PCI serial port> port 
 0xb000-0xb007,0xac00-0xac07 irq 10 at device 10.0 on pci1
 sio5: <Lava Computers Quattro-PCI serial port> on puc0
 sio5: type 16550A
 sio5: unable to activate interrupt in fast mode - using normal mode
 sio6: <Lava Computers Quattro-PCI serial port> on puc0
 sio6: type 16550A
 sio6: unable to activate interrupt in fast mode - using normal mode
 puc1: <Lava Computers Quattro-PCI serial port> port 
 0xb800-0xb807,0xb400-0xb407 irq 10 at device 10.1 on pci1
 sio7: <Lava Computers Quattro-PCI serial port> on puc1
 sio7: type 16550A
 sio7: unable to activate interrupt in fast mode - using normal mode
 sio8: <Lava Computers Quattro-PCI serial port> on puc1
 sio8: type 16550A
 sio8: unable to activate interrupt in fast mode - using normal mode
 isab0: <PCI-ISA bridge> at device 31.0 on pci0
 isa0: <ISA bus> on isab0
 atapci0: <Intel ICH5 UDMA100 controller> port 
 0xf000-0xf00f,0x376,0x170-0x177,0x3f6,0x1f0-0x1f7 at device 31.1 on pci0
 ata0: channel #0 on atapci0
 ata1: channel #1 on atapci0
 pci0: <serial bus, SMBus> at device 31.3 (no driver attached)
 can't re-use a leaf (%desc)!
 can't re-use a leaf (%driver)!
 can't re-use a leaf (%location)!
 can't re-use a leaf (%pnpinfo)!
 can't re-use a leaf (%parent)!
 sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
 sio0: type 16550A, console
 sio1: <16550A-compatible COM port> port 0x2f8-0x2ff irq 3 on acpi0
 sio1: type 16550A
 atkbdc0: <Keyboard controller (i8042)> port 0x64,0x60 irq 1 on acpi0
 atkbd0: <AT Keyboard> irq 1 on atkbdc0
 kbd0 at atkbd0
 atkbd0: [GIANT-LOCKED]
 orm0: <ISA Option ROM> at iomem 0xc0000-0xc9fff on isa0
 ppc0: parallel port not found.
 sc0: <System console> at flags 0x100 on isa0
 sc0: VGA <16 virtual consoles, flags=0x100>
 vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
 Timecounter "TSC" frequency 2400411284 Hz quality 800
 Timecounters tick every 10.000 msec
 Fast IPsec: Initialized Security Association Processing.
 ad0: 38166MB <ST340014A/3.06> [77545/16/63] at ata0-master UDMA100
 Mounting root from ufs:/dev/ad0s1a
 releng5-865%
 
 
 But it seems to work in that I dont get an interrupt storm when attaching 
 to the modem.
 
 releng5-865# vmstat -i
 interrupt                          total       rate
 irq0: clk                          18220         99
 irq4: sio0                           170          0
 irq8: rtc                          23322        127
 irq10: uhci2 puc0+                     6          0
 irq11: fxp0                          892          4
 irq12: uhci0 uhci3+                  122          0
 irq13: npx0                            1          0
 irq14: ata0                         1110          6
 Total                              43843        239
 releng5-865#
 
 
 >(please try it with the RF_SHAREABLE flag removed so that if this is
 >indeed the primary problem then the new diagnostic will verify it).
 
 
 releng5-865# cat /var/run/dmesg.boot
 Copyright (c) 1992-2004 The FreeBSD Project.
 Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
          The Regents of the University of California. All rights reserved.
 FreeBSD 5.3-STABLE #2: Wed Dec  8 10:20:43 EST 2004
      mdtancsa@releng5-865.sentex.ca:/usr/obj/usr/src/sys/test
 Timecounter "i8254" frequency 1193182 Hz quality 0
 CPU: Intel(R) Celeron(R) CPU 2.40GHz (2400.41-MHz 686-class CPU)
    Origin = "GenuineIntel"  Id = 0xf29  Stepping = 9
    Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
 real memory  = 267190272 (254 MB)
 avail memory = 251912192 (240 MB)
 npx0: [FAST]
 npx0: <math processor> on motherboard
 npx0: INT 16 interface
 acpi0: <AOpen AWRDACPI> on motherboard
 acpi0: Power Button (fixed)
 Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x4008-0x400b on acpi0
 cpu0: <ACPI CPU> on acpi0
 acpi_tz0: <Thermal Zone> on acpi0
 acpi_button0: <Power Button> on acpi0
 pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
 pci0: <ACPI PCI bus> on pcib0
 agp0: <Intel 82865G (865G GMCH) SVGA controller> port 0xd000-0xd007 mem 
 0xfa000000-0xfa07ffff,0xf0000000-0xf7ffffff at device 2.0 on pci0
 agp0: detected 892k stolen memory
 agp0: aperture size is 128M
 uhci0: <Intel 82801EB (ICH5) USB controller USB-A> port 0xc000-0xc01f irq 
 12 at device 29.0 on pci0
 uhci0: [GIANT-LOCKED]
 usb0: <Intel 82801EB (ICH5) USB controller USB-A> on uhci0
 usb0: USB revision 1.0
 uhub0: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
 uhub0: 2 ports with 2 removable, self powered
 uhid0: APC Back-UPS ES 725 FW:802.n2.D USB FW:n2, rev 1.10/1.06, addr 2, 
 iclass 3/0
 uhci1: <Intel 82801EB (ICH5) USB controller USB-B> port 0xc400-0xc41f irq 5 
 at device 29.1 on pci0
 uhci1: [GIANT-LOCKED]
 usb1: <Intel 82801EB (ICH5) USB controller USB-B> on uhci1
 usb1: USB revision 1.0
 uhub1: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
 uhub1: 2 ports with 2 removable, self powered
 uhci2: <Intel 82801EB (ICH5) USB controller USB-C> port 0xc800-0xc81f irq 
 10 at device 29.2 on pci0
 uhci2: [GIANT-LOCKED]
 usb2: <Intel 82801EB (ICH5) USB controller USB-C> on uhci2
 usb2: USB revision 1.0
 uhub2: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
 uhub2: 2 ports with 2 removable, self powered
 uhci3: <Intel 82801EB (ICH5) USB controller USB-D> port 0xcc00-0xcc1f irq 
 12 at device 29.3 on pci0
 uhci3: [GIANT-LOCKED]
 usb3: <Intel 82801EB (ICH5) USB controller USB-D> on uhci3
 usb3: USB revision 1.0
 uhub3: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
 uhub3: 2 ports with 2 removable, self powered
 pcib1: <ACPI PCI-PCI bridge> at device 30.0 on pci0
 pci1: <ACPI PCI bus> on pcib1
 sio0: <SmartLink 5634PCV SurfRider> port 0xa000-0xa007 irq 12 at device 4.0 
 on pci1
 sio0: moving to sio4
 sio4: type 16550A
 sio4: interrupt resource allocation failed
 sio4: falling back to polled mode
 sio4: polled mode
 rl0: <RealTek 8139 10/100BaseTX> port 0xa400-0xa4ff mem 
 0xf9000000-0xf90000ff irq 15 at device 5.0 on pci1
 miibus0: <MII bus> on rl0
 rlphy0: <RealTek internal media interface> on miibus0
 rlphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
 rl0: Ethernet address: 00:50:fc:24:2b:4d
 fxp0: <Intel 82801BA (D865) Pro/100 VE Ethernet> port 0xa800-0xa83f mem 
 0xf9001000-0xf9001fff irq 11 at device 8.0 on pci1
 miibus1: <MII bus> on fxp0
 inphy0: <i82562ET 10/100 media interface> on miibus1
 inphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
 fxp0: Ethernet address: 00:01:80:54:b7:09
 puc0: <Lava Computers Quattro-PCI serial port> port 
 0xb000-0xb007,0xac00-0xac07 irq 10 at device 10.0 on pci1
 sio5: <Lava Computers Quattro-PCI serial port> on puc0
 sio5: type 16550A
 sio5: unable to activate interrupt in fast mode - using normal mode
 sio6: <Lava Computers Quattro-PCI serial port> on puc0
 sio6: type 16550A
 sio6: unable to activate interrupt in fast mode - using normal mode
 puc1: <Lava Computers Quattro-PCI serial port> port 
 0xb800-0xb807,0xb400-0xb407 irq 10 at device 10.1 on pci1
 sio7: <Lava Computers Quattro-PCI serial port> on puc1
 sio7: type 16550A
 sio7: unable to activate interrupt in fast mode - using normal mode
 sio8: <Lava Computers Quattro-PCI serial port> on puc1
 sio8: type 16550A
 sio8: unable to activate interrupt in fast mode - using normal mode
 isab0: <PCI-ISA bridge> at device 31.0 on pci0
 isa0: <ISA bus> on isab0
 atapci0: <Intel ICH5 UDMA100 controller> port 
 0xf000-0xf00f,0x376,0x170-0x177,0x3f6,0x1f0-0x1f7 at device 31.1 on pci0
 ata0: channel #0 on atapci0
 ata1: channel #1 on atapci0
 pci0: <serial bus, SMBus> at device 31.3 (no driver attached)
 can't re-use a leaf (%desc)!
 can't re-use a leaf (%driver)!
 can't re-use a leaf (%location)!
 can't re-use a leaf (%pnpinfo)!
 can't re-use a leaf (%parent)!
 sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
 sio0: type 16550A, console
 sio1: <16550A-compatible COM port> port 0x2f8-0x2ff irq 3 on acpi0
 sio1: type 16550A
 atkbdc0: <Keyboard controller (i8042)> port 0x64,0x60 irq 1 on acpi0
 atkbd0: <AT Keyboard> irq 1 on atkbdc0
 kbd0 at atkbd0
 atkbd0: [GIANT-LOCKED]
 orm0: <ISA Option ROM> at iomem 0xc0000-0xc9fff on isa0
 ppc0: parallel port not found.
 sc0: <System console> at flags 0x100 on isa0
 sc0: VGA <16 virtual consoles, flags=0x100>
 vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
 Timecounter "TSC" frequency 2400411356 Hz quality 800
 Timecounters tick every 10.000 msec
 Fast IPsec: Initialized Security Association Processing.
 ad0: 38166MB <ST340014A/3.06> [77545/16/63] at ata0-master UDMA100
 Mounting root from ufs:/dev/ad0s1a
 releng5-865#
 
 releng5-865# vmstat -i
 interrupt                          total       rate
 irq0: clk                          39748         99
 irq4: sio0                           170          0
 irq8: rtc                          50879        127
 irq10: uhci2 puc0+                     6          0
 irq11: fxp0                          482          1
 irq12: uhci0 uhci3                    86          0
 irq13: npx0                            1          0
 irq14: ata0                         1115          2
 Total                              92487        231
 releng5-865#
 
 implies its not sharing the interrupt but it goes into polled mode.
 
 >This should move the conflict later, to one between usb's use of
 >normal interrupt mode and sio's request for fast interrupt mode.
 >There were already sufficient diagnostics for this.  The patch adds
 >some more.  Since usb got the interrupt first, sio will fall back
 >to normal mode.  If sio goes first then the attach of usb would
 >probably fail.  The patch doesn't change these bevhaviours.  The
 >patch adds another fallback to polled mode if setup of normal mode
 >mode fails and fixes some related resource leaks.  It also increases
 >the polling frequency if possible.  Polled mode mostly works at
 >115200 bps provided the UART is buffered and HZ >= 1000.  With
 >unbuffered UARTs and HZ = 100, only 4800 bos mostly works.
 
 I am using RELENG_5, so I am not sure if the HZ default has changed 
 yet.  Not sure how much of this applies to RELENG_4 either where the 
 problem is particularly acute.
 
 
 
 >         return (0);
 >@@ -2647,5 +2664,5 @@
 >         /*
 >         * Set our timeout period to 1 second if no polled devices are open.
 >-       * Otherwise set it to max(1/200, 1/hz).
 >+       * Otherwise, set it to max(1/1000, 1/hz).
 >         * Enable timeouts iff some device is open.
 >         */
 >@@ -2659,5 +2676,5 @@
 
 
 BTW, The comment part of the patch does not apply cleanly
 
 releng5-865# cat sio.c.rej
 ***************
 *** 2645,2649 ****
          /*
          * Set our timeout period to 1 second if no polled devices are open.
 -       * Otherwise set it to max(1/200, 1/hz).
          * Enable timeouts iff some device is open.
          */
 --- 2662,2666 ----
          /*
          * Set our timeout period to 1 second if no polled devices are open.
 +       * Otherwise, set it to max(1/1000, 1/hz).
          * Enable timeouts iff some device is open.
          */
 releng5-865#  
 

From: Mike Tancsa <mike@sentex.net>
To: Bruce Evans <bde@zeta.org.au>
Cc: freebsd-gnats-submit@freebsd.org, freebsd-bugs@freebsd.org
Subject: Re: misc/74786: Smartlink Modem causes interrupt storm on 
  RELENG_4 and RELENG_5
Date: Wed, 08 Dec 2004 11:08:38 -0500

 At 02:22 AM 08/12/2004, Bruce Evans wrote:
 
 >Try this fix.  For the uhci-sio conflict, it should avoid problem (1)
 >(please try it with the RF_SHAREABLE flag removed so that if this is
 ....
 >115200 bps provided the UART is buffered and HZ >= 1000.  With
 >unbuffered UARTs and HZ = 100, only 4800 bos mostly works.
 
 Note,
 When removing the RF_SHAREABLE flag from the patch as applied to the 
 default src code for RELENG_5, the modem is not usable.  With RF_SHAREABLE, 
 all *seems* to work with the modem, but I dont have any real tests, other 
 than dialing up and doing a fetch of a large file.
 
          ---Mike 
 

From: Mike Tancsa <mike@sentex.net>
To: Bruce Evans <bde@zeta.org.au>
Cc: freebsd-gnats-submit@freebsd.org, freebsd-bugs@freebsd.org
Subject: Re: misc/74786: Smartlink Modem causes interrupt storm on 
  RELENG_4 and RELENG_5
Date: Wed, 08 Dec 2004 11:12:01 -0500

 One more thing that does not quite seem right.  Why would it show the modem 
 on sio.0 ?  It should be on 4 no ?
 
 
 releng5-865# sysctl -a dev.sio
 dev.sio.0.%desc: SmartLink 5634PCV SurfRider
 dev.sio.0.%driver: sio
 dev.sio.0.%location: slot=4 function=0
 dev.sio.0.%pnpinfo: vendor=0x151f device=0x0000 subvendor=0x151f 
 subdevice=0x0000 class=0x078000
 dev.sio.0.%parent: pci1
 dev.sio.0.wake: 0
 dev.sio.5.%desc: Lava Computers Quattro-PCI serial port
 dev.sio.5.%driver: sio
 dev.sio.5.%parent: puc0
 dev.sio.6.%desc: Lava Computers Quattro-PCI serial port
 dev.sio.6.%driver: sio
 dev.sio.6.%parent: puc0
 dev.sio.7.%desc: Lava Computers Quattro-PCI serial port
 dev.sio.7.%driver: sio
 dev.sio.7.%parent: puc1
 dev.sio.8.%desc: Lava Computers Quattro-PCI serial port
 dev.sio.8.%driver: sio
 dev.sio.8.%parent: puc1
 dev.sio.1.%desc: 16550A-compatible COM port
 dev.sio.1.%driver: sio
 dev.sio.1.%location: handle=\_SB_.PCI0.PX40.UAR2
 dev.sio.1.%pnpinfo: _HID=PNP0501 _UID=2
 dev.sio.1.%parent: acpi0
 dev.sio.1.wake: 0
 releng5-865# 
 

From: Mike Tancsa <mike@sentex.net>
To: Bruce Evans <bde@zeta.org.au>
Cc: freebsd-gnats-submit@freebsd.org, freebsd-stable@freebsd.org
Subject: Re: misc/74786: Smartlink Modem causes interrupt storm on 
  RELENG_4 and RELENG_5
Date: Thu, 09 Dec 2004 13:43:32 -0500

 [cc'ing to FreeBSD-Stable]
 
 Hi,
 
 I know that the proposed patches I submitted are not the best patches, but 
 given that the next release of RELENG_4 is coming out, would it not be 
 better to commit those to RELENG_4 as they allow the modem to work when it 
 shares an interrupt with another device ?  The sio and interrupt handling 
 code in RELENG_5 is different enough that I doubt the patches you are 
 proposing would make it back to RELENG_4.  This at least lets the modem 
 work and prevents the machine from locking up in an interrupt storm on 
 RELENG_4 without breaking any functionality (as far as I know).
 
          ---Mike
 
 
 
 
 At 02:22 AM 08/12/2004, Bruce Evans wrote:
 
 >I think I understand this now.  sio can indeed drive the interrupt (after
 >you open an sio device, but not immediately at the end of the attach
 >except in the serial console case).  The main bugs are:
 >1. sio asks for exclusive access to the interrupt for no good reason
 >    (some buses like isa might only support exclusive accesses, but sio
 >    doesn't care).  uhci gets access first in your configuration, so
 >    allocation of the interrupt resource fails.
 >2. Error handling for the failure in (1) is null, so both devices are
 >    "successfully" attached.
 >3. sio sets a flag to tell it to use polling if there is no interrupt
 >    resource, but it doesn't set the flag if the interrupt resource
 >    couldn't be allocated or if the interrupt couldn't be set up.
 >4. Upper layers provide negative help for debugging (3) using their
 >    own version of (3).  They print "irq N" in boot messages if an
 >    interrupt resource justs exists.  This doesn't mean that the device
 >    is using it.
 >5. Device interrupts are still enabled in polling mode.  This depends
 >    on nothing else sucessfully setting up the (shared) interrupt.
 

From: Bruce Evans <bde@zeta.org.au>
To: Mike Tancsa <mike@sentex.net>
Cc: freebsd-gnats-submit@freebsd.org, freebsd-bugs@freebsd.org
Subject: Re: misc/74786: Smartlink Modem causes interrupt storm on   RELENG_4
 and RELENG_5
Date: Fri, 10 Dec 2004 11:38:57 +1100 (EST)

 On Wed, 8 Dec 2004, Mike Tancsa wrote:
 
 > At 02:22 AM 08/12/2004, Bruce Evans wrote:
 >
 > >Try this fix.  For the uhci-sio conflict, it should avoid problem (1)
 > >(please try it with the RF_SHAREABLE flag removed so that if this is
 > ....
 > >115200 bps provided the UART is buffered and HZ >= 1000.  With
 > >unbuffered UARTs and HZ = 100, only 4800 bos mostly works.
 
 Oops.  "4800 bos" should be "600 bps" (a 1-byte fifo drained 100 times
 per second gives at most 100 cps or 1000 bps; 600 gives a safety margin).
 
 >
 > Note,
 > When removing the RF_SHAREABLE flag from the patch as applied to the
 > default src code for RELENG_5, the modem is not usable.
 
 RELENG_5 stil has a reasonable default for HZ (100) so the maximum cps
 wth a 16-byte fifo is 16 * 100 * 10 = 16000.  I guess you use the default
 and don't use a 2400 bps modem that would work with a max of 16000.
 
 > With RF_SHAREABLE,
 > all *seems* to work with the modem, but I dont have any real tests, other
 > than dialing up and doing a fetch of a large file.
 
 Lost characters would be reported as silo overflows.  Any protocol with
 error handling would recover from a few OK.
 
 Bruce
>Unformatted:
