From nate@netapp.com  Fri Sep  9 02:44:38 2005
Return-Path: <nate@netapp.com>
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id EA3B916A41F
	for <FreeBSD-gnats-submit@freebsd.org>; Fri,  9 Sep 2005 02:44:38 +0000 (GMT)
	(envelope-from nate@netapp.com)
Received: from mx1.netapp.com (mx1.netapp.com [216.240.18.38])
	by mx1.FreeBSD.org (Postfix) with ESMTP id 806B543D49
	for <FreeBSD-gnats-submit@freebsd.org>; Fri,  9 Sep 2005 02:44:38 +0000 (GMT)
	(envelope-from nate@netapp.com)
Received: from smtp2.corp.netapp.com ([10.57.159.114])
  by mx1.netapp.com with ESMTP; 08 Sep 2005 19:44:38 -0700
Received: from defender-fe.eng.netapp.com (defender-fe.eng.netapp.com [10.56.11.139])
	by smtp2.corp.netapp.com (8.13.1/8.13.1/NTAP-1.6) with ESMTP id j892ibiI001128;
	Thu, 8 Sep 2005 19:44:37 -0700 (PDT)
Received: from defender-fe.eng.netapp.com (localhost [127.0.0.1])
	by defender-fe.eng.netapp.com (8.13.3/8.13.3) with ESMTP id j892ib7Q048356;
	Thu, 8 Sep 2005 19:44:37 -0700 (PDT)
	(envelope-from nate@defender-fe.eng.netapp.com)
Received: (from nate@localhost)
	by defender-fe.eng.netapp.com (8.13.4/8.13.3/Submit) id j892ibd2048355;
	Thu, 8 Sep 2005 19:44:37 -0700 (PDT)
	(envelope-from nate)
Message-Id: <200509090244.j892ibd2048355@defender-fe.eng.netapp.com>
Date: Thu, 8 Sep 2005 19:44:37 -0700 (PDT)
From: Nathan Smith <nate@netapp.com>
Reply-To: Nathan Smith <nate@netapp.com>
To: FreeBSD-gnats-submit@freebsd.org
Cc: nahthan@gmail.com
Subject: nfs_timer / nfs_socket.c:1146 panic
X-Send-Pr-Version: 3.113
X-GNATS-Notify:

>Number:         85894
>Category:       kern
>Synopsis:       [nfs] [panic] nfs_timer / nfs_socket.c:1146 panic
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    jh
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Fri Sep 09 02:50:17 GMT 2005
>Closed-Date:    Sun Oct 10 07:51:03 UTC 2010
>Last-Modified:  Sun Oct 10 07:51:03 UTC 2010
>Originator:     Nathan Smith <nate@netapp.com
>Release:        FreeBSD 5.3-RELEASE i386
>Organization:
Network Appliance, Inc.
>Environment:
FreeBSD bldf9.spin.eng.netapp.com 5.3-RELEASE FreeBSD 5.3-RELEASE #1: Tue Jul  5 17:23:01 EDT 2005     root@bldf9.spin.eng.netapp.com:/usr/obj/usr/src/sys/SMP  i386

>Description:
 	Kernel panic.  NFS bug.

--- backtrace begins here ---
(kgdb) bt
#0  doadump () at pcpu.h:159
#1  0xc05ed557 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:397
#2  0xc05ed87d in panic (fmt=0xc07d7a40 "%s")
    at /usr/src/sys/kern/kern_shutdown.c:553
#3  0xc07988f4 in trap_fatal (frame=0xe8b5bc70, eva=1701145739)
    at /usr/src/sys/i386/i386/trap.c:809
#4  0xc0798637 in trap_pfault (frame=0xe8b5bc70, usermode=0, eva=1701145739)
    at /usr/src/sys/i386/i386/trap.c:727
#5  0xc0798271 in trap (frame=
      {tf_fs = -992149480, tf_es = 16, tf_ds = -992083952, tf_edi = 0, tf_esi =                                 1702130553, tf_ebp = -390742844, tf_isp = -390742884, tf_ebx = 1701145715, tf_ed                                x = 79722, tf_ecx = -1065060448, tf_eax = 13059410, tf_trapno = 12, tf_err = 0,                                 tf_eip = -1066733936, tf_cs = 8, tf_eflags = 66050, tf_esp = 1305945, tf_ss = 79                                722}) at /usr/src/sys/i386/i386/trap.c:417
#6  0xc078655a in calltrap () at /usr/src/sys/i386/i386/exception.s:140
#7  0xc4dd0018 in ?? ()
#8  0x00000010 in ?? ()
#9  0xc4de0010 in ?? ()
#10 0x00000000 in ?? ()
#11 0x65747379 in ?? ()
#12 0xe8b5bcc4 in ?? ()
#13 0xe8b5bc9c in ?? ()
#14 0x65656c73 in ?? ()
---Type <return> to continue, or q <return> to quit---
#15 0x0001376a in ?? ()
#16 0xc08477a0 in th3 ()
#17 0x00c74552 in ?? ()
#18 0x0000000c in ?? ()
#19 0x00000000 in ?? ()
#20 0xc06aee90 in nfs_timer (arg=0x0)
    at /usr/src/sys/nfsclient/nfs_socket.c:1146
#21 0xc05f9c0f in softclock (dummy=0x0) at /usr/src/sys/kern/kern_timeout.c:259
#22 0xc05d9445 in ithread_loop (arg=0xc4df8580)
    at /usr/src/sys/kern/kern_intr.c:547
#23 0xc05d84f5 in fork_exit (callout=0xc05d92ec <ithread_loop>,
    arg=0xc4df8580, frame=0xe8b5bd48) at /usr/src/sys/kern/kern_fork.c:811
#24 0xc07865bc in fork_trampoline () at /usr/src/sys/i386/i386/exception.s:209
--- backtrace ends here ---

--- dmesg-output begins here ---
Copyright (c) 1992-2004 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	The Regents of the University of California. All rights reserved.
FreeBSD 5.3-RELEASE #1: Tue Jul  5 17:23:01 EDT 2005
    root@xxxxx.spin.eng.netapp.com:/usr/obj/usr/src/sys/SMP
WARNING: Network stack Giant-free, but aio requires Giant.
    Consider adding 'options NET_WITH_GIANT' or setting debug.mpsafenet=0
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: Intel(R) Xeon(TM) CPU 3.00GHz (3000.12-MHz 686-class CPU)
  Origin = "GenuineIntel"  Id = 0xf41  Stepping = 1
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
real memory  = 3623608320 (3455 MB)
avail memory = 3546157056 (3381 MB)
ACPI APIC Table: <IBM    SERBLADE>
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
 cpu0 (BSP): APIC ID:  0
 cpu1 (AP): APIC ID:  6
ioapic3 <Version 2.0> irqs 72-95 on motherboard
ioapic2 <Version 2.0> irqs 48-71 on motherboard
ioapic1 <Version 2.0> irqs 24-47 on motherboard
ioapic0 <Version 2.0> irqs 0-23 on motherboard
npx0: [FAST]
npx0: <math processor> on motherboard
npx0: INT 16 interface
acpi0: <IBM SERBLADE> on motherboard
acpi_bus_number: can't get _ADR
acpi_bus_number: can't get _ADR
acpi_bus_number: can't get _ADR
acpi_bus_number: can't get _ADR
acpi_bus_number: can't get _ADR
acpi_bus_number: can't get _ADR
acpi0: Power Button (fixed)
acpi_bus_number: can't get _ADR
acpi_bus_number: can't get _ADR
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x588-0x58b on acpi0
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
cpu1: Failed to attach throttling P_CNT
pcib0: <ACPI Host-PCI bridge> on acpi0
pci0: <ACPI PCI bus> on pcib0
pci0: <unknown> at device 0.1 (no driver attached)
pcib1: <ACPI PCI-PCI bridge> at device 3.0 on pci0
pcib1: could not get PCI interrupt routing table for \\_SB_.PCI0.PCI1 - AE_NOT_FOUND
pci4: <ACPI PCI bus> on pcib1
pcib2: <ACPI PCI-PCI bridge> at device 0.0 on pci4
pci6: <ACPI PCI bus> on pcib2
pcib3: <ACPI PCI-PCI bridge> at device 0.2 on pci4
pci5: <ACPI PCI bus> on pcib3
bge0: <Broadcom BCM5704S Dual Gigabit Ethernet, ASIC rev. 0x2100> mem 0xdcff0000-0xdcffffff irq 77 at device 1.0 on pci5
bge0: Ethernet address: 00:11:25:9b:38:6e
bge1: <Broadcom BCM5704S Dual Gigabit Ethernet, ASIC rev. 0x2100> mem 0xdcfe0000-0xdcfeffff irq 78 at device 1.1 on pci5
bge1: Ethernet address: 00:11:25:9b:38:6f
pci0: <base peripheral> at device 8.0 (no driver attached)
pcib4: <ACPI PCI-PCI bridge> at device 28.0 on pci0
pci2: <ACPI PCI bus> on pcib4
mpt0: <LSILogic 1030 Ultra4 Adapter> port 0x4000-0x40ff mem 0xdefe0000-0xdefeffff,0xdeff0000-0xdeffffff irq 24 at device 1.0 on pci2
mpt0: [GIANT-LOCKED]
uhci0: <UHCI (generic) USB controller> port 0x2200-0x221f irq 16 at device 29.0 on pci0
uhci0: [GIANT-LOCKED]
usb0: <UHCI (generic) USB controller> on uhci0
usb0: USB revision 1.0
uhub0: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub0: 2 ports with 2 removable, self powered
uhub1: Cypress Semiconductor 4 Port Hub, class 9/0, rev 1.10/0.01, addr 2
uhub1: 4 ports with 4 removable, bus powered
ukbd0: IBM PPC I/F, rev 1.10/0.01, addr 3, iclass 3/1
kbd0 at ukbd0
ums0: IBM PPC I/F, rev 1.10/0.01, addr 3, iclass 3/1
ums0: X report 0x0002 not supported
device_attach: ums0 attach returned 6
ukbd1: IBM HIDK/M, rev 1.10/0.01, addr 4, iclass 3/1
kbd1 at ukbd1
ums0: IBM HIDK/M, rev 1.10/0.01, addr 4, iclass 3/1
ums0: 3 buttons and Z dir.
uhci1: <UHCI (generic) USB controller> port 0x2600-0x261f irq 19 at device 29.1 on pci0
uhci1: [GIANT-LOCKED]
usb1: <UHCI (generic) USB controller> on uhci1
usb1: USB revision 1.0
uhub2: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub2: 2 ports with 2 removable, self powered
pci0: <base peripheral> at device 29.4 (no driver attached)
pci0: <base peripheral, interrupt controller> at device 29.5 (no driver attached)
pcib5: <ACPI PCI-PCI bridge> at device 30.0 on pci0
pci1: <ACPI PCI bus> on pcib5
pci1: <display, VGA> at device 1.0 (no driver attached)
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
pci0: <mass storage, ATA> at device 31.1 (no driver attached)
pci0: <serial bus, SMBus> at device 31.3 (no driver attached)
sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
sio0: type 16550A
orm0: <ISA Option ROM> at iomem 0xc0000-0xc8fff on isa0
pmtimer0 on isa0
ppc0: parallel port not found.
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
sio1: configured irq 3 not in bitmap of probed irqs 0
sio1: port may not be enabled
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
Timecounters tick every 10.000 msec
Waiting 15 seconds for SCSI devices to settle
acpi_cpu: throttling enabled, 8 steps (100% to 12.5%), currently 100.0%
SMP: AP CPU #1 Launched!
da0 at mpt0 bus 0 target 0 lun 0
da0: <IBM-ESXS ST936701LC    FN B415> Fixed Direct Access SCSI-4 device 
da0: 320.000MB/s transfers (160.000MHz, offset 63, 16bit), Tagged Queueing Enabled
da0: 34715MB (71096640 512 byte sectors: 255H 63S/T 4425C)
Mounting root from ufs:/dev/da0s1a
WARNING: / was not properly dismounted
Pre-seeding PRNG:
 kickstart
.
Loading configuration files.
Entropy harvesting:
 interrupts
 ethernet
 point_to_point
 kickstart
.
kernel dumps on /dev/da0s1b
swapon: adding /dev/da0s1b as swap device
Starting file system checks:
/dev/da0s1a: INCORRECT BLOCK COUNT I=447499 (4 should be 0) (CORRECTED)
/dev/da0s1a: UNREF FILE  I=448084  OWNER=root MODE=100600
/dev/da0s1a: SIZE=0 MTIME=Sep  7 03:01 2005  (CLEARED)
/dev/da0s1a: UNREF FILE  I=448086  OWNER=root MODE=100600
/dev/da0s1a: SIZE=0 MTIME=Sep  7 03:01 2005  (CLEARED)
/dev/da0s1a: FREE BLK COUNT(S) WRONG IN SUPERBLK (SALVAGED)
/dev/da0s1a: SUMMARY INFORMATION BAD (SALVAGED)
/dev/da0s1a: BLK(S) MISSING IN BIT MAPS (SALVAGED)
/dev/da0s1a: 2435 files, 72669 used, 1956362 free (1146 frags, 244402 blocks, 0.1% fragmentation)
/dev/da0s1g: DEFER FOR BACKGROUND CHECKING
/dev/da0s1f: DEFER FOR BACKGROUND CHECKING
/dev/da0s1e: DEFER FOR BACKGROUND CHECKING
/dev/da0s1d: DEFER FOR BACKGROUND CHECKING
WARNING: /space was not properly dismounted
WARNING: /t was not properly dismounted
WARNING: /usr was not properly dismounted
WARNING: /var was not properly dismounted
/var: mount pending error: blocks 4 files 1
kern.maxfiles: 
12328
 -> 
65536

kern.ipc.somaxconn: 
128
 -> 
8192

net.inet.tcp.sendspace: 
32768
 -> 
65535

net.inet.tcp.recvspace: 
65536
 -> 
65535

Setting hostname: xxxxx.spin.eng.netapp.com.
bge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
	options=1a<TXCSUM,VLAN_MTU,VLAN_HWTAGGING>
	inet 10.98.17.120 netmask 0xffffff00 broadcast 10.98.17.255
	inet6 fe80::211:25ff:fe9b:386e%bge0 prefixlen 64 tentative scopeid 0x1 
	ether 00:11:25:9b:38:6e
	media: Ethernet autoselect (1000baseSX <full-duplex>)
	status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384
	inet 127.0.0.1 netmask 0xff000000 
	inet6 ::1 prefixlen 128 
	inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 
add net default: gateway 10.98.17.1
Additional routing options:
 broadcast ping responses=YES
.
Starting devd.
kbd1
    ukbd1, type:generic (0)
Mounting NFS file systems:
.
Starting syslogd.
Sep  7 09:21:32 xxxxx syslogd: kernel boot file is /boot/kernel/kernel
Checking for core dump on /dev/da0s1b ...
savecore: reboot after panic: vrele: negative ref cnt
Sep  7 09:21:32 xxxxx savecore: reboot after panic: vrele: negative ref cnt
savecore: no dump, not enough free space on device (3512528 available, need 3538682)
savecore: unsaved dumps found but not saved
Setting date via ntp.
Looking for host timehost1.eng.netapp.com and service ntp
host found : dewey-fe.eng.netapp.com
Looking for host timehost1.eng.netapp.com and service ntp
host found : dewey-fe.eng.netapp.com
Looking for host timehost2.eng.netapp.com and service ntp
host found : louie-fe.eng.netapp.com
 7 Sep 09:21:16 ntpdate[367]: step time server 10.56.10.223 offset -20.717671 sec
Starting rpcbind.
Setting NIS domain: netapp.com.
Starting ypbind.
NFS access cache time=2
Starting amd.
ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/X11R6/lib /usr/local/lib
a.out ldconfig path: /usr/lib/aout /usr/lib/compat/aout /usr/X11R6/lib/aout
Starting statd.
Starting lockd.
Starting usbd.
Starting local daemons:
.
Updating motd
.
Starting ntpd.
Configuring syscons:
 blanktime
.
Starting sshd.
Initial i386 initialization:
.
Additional ABI support:
.
Starting cron.
Local package initialization:
/usr/local/etc/rc.d/snmpd.sh: /usr/local/bin/bsnmpd: not found
.
Additional TCP options:
.
Starting inetd.
Starting background file system checks in 60 seconds.

Wed Sep  7 09:21:19 EDT 2005
uhub1: at uhub0 port 2 (addr 2) disconnected
ukbd0: at uhub1 port 1 (addr 3) disconnected
ukbd0: detached
ukbd1: at uhub1 port 3 (addr 4) disconnected
ukbd1: detached
ums0: at uhub1 port 3 (addr 4) disconnected
ums0: detached
uhub1: detached
Sep  7 09:21:58 xxxxx moused: unable to open /dev/ums0: No such file or directory
nfs server enterprise.sim.spin.eng.netapp.com:/ptc_sim/schellap: not responding
Sep  8 11:22:16 xxxxx kernel: nfs server enterprise.sim.spin.eng.netapp.com:/ptc_sim/schellap: not responding
nfs server enterprise.sim.spin.eng.netapp.com:/ptc_sim/schellap: is alive again
Sep  8 11:22:20 xxxxx kernel: nfs server enterprise.sim.spin.eng.netapp.com:/ptc_sim/schellap: is alive again
nfs server enterprise.sim.spin.eng.netapp.com:/ptc_sim/schellap: not responding
Sep  8 14:14:19 xxxxx kernel: nfs server enterprise.sim.spin.eng.netapp.com:/ptc_sim/schellap: not responding
nfs server enterprise.sim.spin.eng.netapp.com:/ptc_sim/schellap: is alive again
Sep  8 14:14:21 xxxxx kernel: nfs server enterprise.sim.spin.eng.netapp.com:/ptc_sim/schellap: is alive again
--- dmesg-output ends here ---

--- uname-output begins here ---
FreeBSD xxxxx.spin.eng.netapp.com 5.3-RELEASE FreeBSD 5.3-RELEASE #1: Tue Jul  5 17:23:01 EDT 2005     root@xxxxx.spin.eng.netapp.com:/usr/obj/usr/src/sys/SMP  i386
--- uname-output ends here ---

>How-To-Repeat:

>Fix:

>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->cel 
Responsible-Changed-By: cel 
Responsible-Changed-When: Fri May 12 21:33:22 UTC 2006 
Responsible-Changed-Why:  
Hi Nate- 

We need reproducers for these problems.  Thanks. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=85894 
Responsible-Changed-From-To: cel->freebsd-bugs 
Responsible-Changed-By: cel 
Responsible-Changed-When: Mon Mar 12 15:35:12 UTC 2007 
Responsible-Changed-Why:  
Back to the public pool. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=85894 
State-Changed-From-To: open->feedback 
State-Changed-By: jh 
State-Changed-When: Fri Aug 27 12:12:43 UTC 2010 
State-Changed-Why:  
Can you still reproduce this on recent FreeBSD versions? 


Responsible-Changed-From-To: freebsd-bugs->jh 
Responsible-Changed-By: jh 
Responsible-Changed-When: Fri Aug 27 12:12:43 UTC 2010 
Responsible-Changed-Why:  
Track. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=85894 
State-Changed-From-To: feedback->closed 
State-Changed-By: jh 
State-Changed-When: Sun Oct 10 07:51:02 UTC 2010 
State-Changed-Why:  
Feedback timeout. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=85894 
>Unformatted:
