From nobody@FreeBSD.org  Fri Nov 25 20:36:35 2005
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id 1125116A41F
	for <freebsd-gnats-submit@FreeBSD.org>; Fri, 25 Nov 2005 20:36:35 +0000 (GMT)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (www.freebsd.org [216.136.204.117])
	by mx1.FreeBSD.org (Postfix) with ESMTP id 18D4143D5F
	for <freebsd-gnats-submit@FreeBSD.org>; Fri, 25 Nov 2005 20:36:34 +0000 (GMT)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by www.freebsd.org (8.13.1/8.13.1) with ESMTP id jAPKaYYS005606
	for <freebsd-gnats-submit@FreeBSD.org>; Fri, 25 Nov 2005 20:36:34 GMT
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.13.1/8.13.1/Submit) id jAPKaYKL005605;
	Fri, 25 Nov 2005 20:36:34 GMT
	(envelope-from nobody)
Message-Id: <200511252036.jAPKaYKL005605@www.freebsd.org>
Date: Fri, 25 Nov 2005 20:36:34 GMT
From: Aaron Wallace <awallac3@uiuc.edu>
To: freebsd-gnats-submit@FreeBSD.org
Subject: [amd64] sym0: VTOBUS failed (6.0 Release)
X-Send-Pr-Version: www-2.3

>Number:         89550
>Category:       amd64
>Synopsis:       [sym] [patch] sym0: VTOBUS failed (6.0 Release)
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    freebsd-amd64
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Fri Nov 25 20:40:02 GMT 2005
>Closed-Date:    Fri Jul 20 23:03:48 GMT 2007
>Last-Modified:  Fri Jul 20 23:10:03 GMT 2007
>Originator:     Aaron Wallace
>Release:        6.0 Release
>Organization:
n/a
>Environment:
>Description:
Ever since I switched to amd64, I have been having problems with my
symbios / lsi logic 21010 scsi card.  I tried it initially on an nforce2
chipset, and was getting this error message:

panic: sym0 : VTOBUS FAILED!

About a year ago someone was helping address the problem, and I thought
that a patch he had made (6.0-current at the time) had corrected the
problem.  I think maybe this was just a matter of luck though, because
now the problem seems to have resurfaced, this time on an nforce4 chipset.

dmesg output follows:

Copyright (c) 1992-2005 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	The Regents of the University of California. All rights reserved.
FreeBSD 6.0-RELEASE #0: Wed Nov  2 19:07:38 UTC 2005
    root@rat.samsco.home:/usr/obj/usr/src/sys/GENERIC
ACPI APIC Table: <Nvidia AWRDACPI>
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: AMD Athlon(tm) 64 Processor 3200+ (2120.03-MHz K8-class CPU)
  Origin = "AuthenticAMD"  Id = 0x20ff0  Stepping = 0
  Features=0x78bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2>
  Features2=0x1<SSE3>
  AMD Features=0xe2500800<SYSCALL,NX,MMX+,<b25>,LM,3DNow+,3DNow>
real memory  = 1073676288 (1023 MB)
avail memory = 1025216512 (977 MB)
ioapic0 <Version 1.1> irqs 0-23 on motherboard
acpi0: <Nvidia AWRDACPI> on motherboard
acpi_bus_number: can't get _ADR
acpi_bus_number: can't get _ADR
acpi0: Power Button (fixed)
acpi_bus_number: can't get _ADR
acpi_bus_number: can't get _ADR
acpi_bus_number: can't get _ADR
acpi_bus_number: can't get _ADR
pci_link0: <ACPI PCI Link LNK1> irq 10 on acpi0
pci_link1: <ACPI PCI Link LNK2> irq 11 on acpi0
pci_link2: <ACPI PCI Link LNK3> irq 5 on acpi0
pci_link3: <ACPI PCI Link LNK4> irq 10 on acpi0
pci_link4: <ACPI PCI Link LNK5> on acpi0
pci_link5: <ACPI PCI Link LUBA> irq 10 on acpi0
pci_link6: <ACPI PCI Link LUBB> on acpi0
pci_link7: <ACPI PCI Link LMAC> irq 5 on acpi0
pci_link8: <ACPI PCI Link LACI> irq 11 on acpi0
pci_link9: <ACPI PCI Link LMCI> on acpi0
pci_link10: <ACPI PCI Link LSMB> irq 5 on acpi0
pci_link11: <ACPI PCI Link LUB2> irq 11 on acpi0
pci_link12: <ACPI PCI Link LIDE> on acpi0
pci_link13: <ACPI PCI Link LSID> on acpi0
pci_link14: <ACPI PCI Link LFID> on acpi0
pci_link15: <ACPI PCI Link LPCA> on acpi0
pci_link16: <ACPI PCI Link APC1> irq 0 on acpi0
pci_link17: <ACPI PCI Link APC2> irq 0 on acpi0
pci_link18: <ACPI PCI Link APC3> irq 0 on acpi0
pci_link19: <ACPI PCI Link APC4> irq 0 on acpi0
pci_link20: <ACPI PCI Link APC5> irq 16 on acpi0
pci_link21: <ACPI PCI Link APCF> irq 0 on acpi0
pci_link22: <ACPI PCI Link APCG> irq 0 on acpi0
pci_link23: <ACPI PCI Link APCH> irq 0 on acpi0
pci_link24: <ACPI PCI Link APCJ> irq 0 on acpi0
pci_link25: <ACPI PCI Link APCK> irq 0 on acpi0
pci_link26: <ACPI PCI Link APCS> irq 0 on acpi0
pci_link27: <ACPI PCI Link APCL> irq 0 on acpi0
pci_link28: <ACPI PCI Link APCZ> irq 0 on acpi0
pci_link29: <ACPI PCI Link APSI> irq 0 on acpi0
pci_link30: <ACPI PCI Link APSJ> irq 0 on acpi0
pci_link31: <ACPI PCI Link APCP> irq 0 on acpi0
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x4008-0x400b on acpi0
cpu0: <ACPI CPU> on acpi0
acpi_throttle0: <ACPI CPU Throttling> on cpu0
acpi_button0: <Power Button> on acpi0
acpi_button1: <Sleep Button> on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci_link26: BIOS IRQ 5 for -2145774616.1.INTA is invalid
pci_link21: BIOS IRQ 10 for -2145774616.2.INTA is invalid
pci_link27: BIOS IRQ 11 for -2145774616.2.INTB is invalid
pci_link23: BIOS IRQ 5 for -2145774616.10.INTA is invalid
pci_link24: BIOS IRQ 11 for -2145774616.4.INTA is invalid
pci0: <ACPI PCI bus> on pcib0
pci0: <memory> at device 0.0 (no driver attached)
isab0: <PCI-ISA bridge> at device 1.0 on pci0
isa0: <ISA bus> on isab0
pci0: <serial bus, SMBus> at device 1.1 (no driver attached)
ohci0: <OHCI (generic) USB controller> mem 0xfebff000-0xfebfffff irq 21 at device 2.0 on pci0
ohci0: [GIANT-LOCKED]
usb0: OHCI version 1.0, legacy support
usb0: SMM does not respond, resetting
usb0: <OHCI (generic) USB controller> on ohci0
usb0: USB revision 1.0
uhub0: nVidia OHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub0: 10 ports with 10 removable, self powered
ehci0: <EHCI (generic) USB 2.0 controller> mem 0xfebfe000-0xfebfe0ff irq 22 at device 2.1 on pci0
ehci0: [GIANT-LOCKED]
usb1: EHCI version 1.0
usb1: companion controller, 4 ports each: usb0
usb1: <EHCI (generic) USB 2.0 controller> on ehci0
usb1: USB revision 2.0
uhub1: nVidia EHCI root hub, class 9/0, rev 2.00/1.00, addr 1
uhub1: 10 ports with 10 removable, self powered
pci0: <multimedia, audio> at device 4.0 (no driver attached)
atapci0: <nVidia nForce4 UDMA133 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xe000-0xe00f at device 6.0 on pci0
ata0: <ATA channel 0> on atapci0
ata1: <ATA channel 1> on atapci0
pcib1: <ACPI PCI-PCI bridge> at device 9.0 on pci0
pci_link18: BIOS IRQ 21 for 0.10.INTA is invalid
pci1: <ACPI PCI bus> on pcib1
pci1: <multimedia, audio> at device 7.0 (no driver attached)
pci1: <input device> at device 7.1 (no driver attached)
sym0: <1010-33> port 0xc400-0xc4ff mem 0xfeaff000-0xfeaff3ff,0xfeafc000-0xfeafdfff irq 16 at device 8.0 on pci1
sym0: Symbios NVRAM, ID 7, Fast-80, LVD, parity checking
sym0: open drain IRQ line driver, using on-chip SRAM
sym0: using LOAD/STORE-based firmware.
sym0: handling phase mismatch from SCRIPTS.
sym0: SCAN AT BOOT disabled for targets 1 2 4 5 8 9 10 11 12 13 14 15.
sym0: SCAN FOR LUNS disabled for targets 0 3 6.
sym0: [GIANT-LOCKED]
sym1: <1010-33> port 0xc000-0xc0ff mem 0xfeafe000-0xfeafe3ff,0xfeafa000-0xfeafbfff irq 17 at device 8.1 on pci1
sym1: Symbios NVRAM, ID 7, Fast-80, SE, parity checking
sym1: open drain IRQ line driver, using on-chip SRAM
sym1: using LOAD/STORE-based firmware.
sym1: handling phase mismatch from SCRIPTS.
sym1: SCAN AT BOOT disabled for targets 1 2 3 4 5 6 8 9 10 11 12 13 14 15.
sym1: SCAN FOR LUNS disabled for targets 0 1 2 3 4 5 6 8 9 10 11 12 13 14 15.
sym1: [GIANT-LOCKED]
sym2: <810a> port 0xbc00-0xbcff mem 0xfe9ff000-0xfe9ff0ff irq 18 at device 10.0 on pci1
sym2: No NVRAM, ID 7, Fast-10, SE, parity checking
sym2: [GIANT-LOCKED]
nve0: <NVIDIA nForce MCP9 Networking Adapter> port 0xdc00-0xdc07 mem 0xfebfb000-0xfebfbfff irq 21 at device 10.0 on pci0
nve0: Ethernet address 00:50:70:56:4b:68
miibus0: <MII bus> on nve0
ciphy0: <Cicada CS8201 10/100/1000TX PHY> on miibus0
ciphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto
nve0: Ethernet address: 00:50:70:56:4b:68
nve0: [GIANT-LOCKED]
pcib2: <ACPI PCI-PCI bridge> at device 11.0 on pci0
pci2: <ACPI PCI bus> on pcib2
pcib3: <ACPI PCI-PCI bridge> at device 12.0 on pci0
pci3: <ACPI PCI bus> on pcib3
pcib4: <ACPI PCI-PCI bridge> at device 13.0 on pci0
pci4: <ACPI PCI bus> on pcib4
pcib5: <ACPI PCI-PCI bridge> at device 14.0 on pci0
pci5: <ACPI PCI bus> on pcib5
pci5: <display, VGA> at device 0.0 (no driver attached)
acpi_tz0: <Thermal Zone> on acpi0
fdc0: <floppy drive controller> port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0
fdc0: [FAST]
fd0: <1440-KB 3.5" drive> on fdc0 drive 0
sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
sio0: type 16550A
sio1: <16550A-compatible COM port> port 0x2f8-0x2ff irq 3 on acpi0
sio1: type 16550A
ppc0: <ECP parallel printer port> port 0x378-0x37f,0x778-0x77b irq 7 drq 3 on acpi0
ppc0: Generic chipset (ECP/PS2/NIBBLE) in COMPATIBLE mode
ppc0: FIFO with 16/16/16 bytes threshold
ppbus0: <Parallel port bus> on ppc0
plip0: <PLIP network interface> on ppbus0
lpt0: <Printer> on ppbus0
lpt0: Interrupt-driven port
ppi0: <Parallel I/O> on ppbus0
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> flags 0x1 irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
orm0: <ISA Option ROMs> at iomem 0xc0000-0xcefff,0xd0000-0xd3fff,0xd4000-0xd57ff,0xd6000-0xd9fff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
ums0: Logitech USB Mouse, rev 1.10/6.10, addr 2, iclass 3/1
ums0: 3 buttons and Z dir.
umass0: Generic USB Storage Device, rev 1.10/30.0a, addr 3
Timecounter "TSC" frequency 2120029086 Hz quality 800
Timecounters tick every 1.000 msec
Waiting 5 seconds for SCSI devices to settle
(noperiph:sym0:0:-1:-1): SCSI BUS reset delivered.
(noperiph:sym1:0:-1:-1): SCSI BUS reset delivered.
acd0: DVDR <PIONEER DVD-RW DVR-106D/1.05> at ata0-slave UDMA33
(probe32:sym2:0:2:0): phase change 6-7 6@3db89198 resid=4.
pass4 at sym2 bus 0 target 2 lun 0
pass4: <HP C5110A 3701> Fixed Processor SCSI-2 device 
pass4: 3.300MB/s transfers
da0 at sym0 bus 0 target 0 lun 0
da0: <COMPAQ BF03665223 B014> Fixed Direct Access SCSI-3 device 
da0: 160.000MB/s transfers (80.000MHz, offset 31, 16bit)
da0: 34732MB (71132000 512 byte sectors: 255H 63S/T 4427C)
da1 at sym0 bus 0 target 3 lun 0
da1: <QUANTUM ATLAS10K2-TY184L DDD6> Fixed Direct Access SCSI-3 device 
da1: 160.000MB/s transfers (80.000MHz, offset 62, 16bit)
da1: 17510MB (35860910 512 byte sectors: 255H 63S/T 2232C)
da2 at sym0 bus 0 target 6 lun 0
da2: <IBM IC35L036UWDY10-0 S29A> Fixed Direct Access SCSI-3 device 
da2: 160.000MB/s transfers (80.000MHz, offset 62, 16bit)
da2: 35003MB (71687340 512 byte sectors: 255H 63S/T 4462C)
cd0 at sym1 bus 0 target 0 lun 0
cd0: <TEAC CD-ROM CD-532S 1.0A> Removable CD-ROM SCSI-2 device 
cd0: 10.000MB/s transfers (10.000MHz, offset 16)
cd0: Attempt to query device size failed: NOT READY, Medium not present
da3 at umass-sim0 bus 0 target 0 lun 0
da3: <CBOX3 USB Storage-SMC 300A> Removable Direct Access SCSI-0 device 
da3: 1.000MB/s transfers
da3: Attempt to query device size failed: NOT READY, Medium not present
da4 at umass-sim0 bus 0 target 0 lun 1
da4: <CBOX3 USB Storage-CFC 300A> Removable Direct Access SCSI-0 device 
da4: 1.000MB/s transfers
da4: Attempt to query device size failed: NOT READY, Medium not present
da5 at umass-sim0 bus 0 target 0 lun 2
da5: <CBOX3 USB Storage-MMC 300A> Removable Direct Access SCSI-0 device 
da5: 1.000MB/s transfers
da5: Attempt to query device size failed: NOT READY, Medium not present
da6 at umass-sim0 bus 0 target 0 lun 3
da6: <CBOX3 USB Storage-MSC 300A> Removable Direct Access SCSI-0 device 
da6: 1.000MB/s transfers
da6: Attempt to query device size failed: NOT READY, Medium not present
(da3:umass-sim0:0:0:0): READ CAPACITY. CDB: 25 0 0 0 0 0 0 0 0 0 
(da3:umass-sim0:0:0:0): CAM Status: SCSI Status Error
(da3:umass-sim0:0:0:0): SCSI Status: Check Condition
(da3:umass-sim0:0:0:0): NOT READY asc:3a,0
(da3:umass-sim0:0:0:0): Medium not present
(da3:umass-sim0:0:0:0): Unretryable error
Opened disk da3 -> 6
(da3:umass-sim0:0:0:0): READ CAPACITY. CDB: 25 0 0 0 0 0 0 0 0 0 
(da3:umass-sim0:0:0:0): CAM Status: SCSI Status Error
(da3:umass-sim0:0:0:0): SCSI Status: Check Condition
(da3:umass-sim0:0:0:0): NOT READY asc:3a,0
(da3:umass-sim0:0:0:0): Medium not present
(da3:umass-sim0:0:0:0): Unretryable error
Opened disk da3 -> 6
(da3:umass-sim0:0:0:0): READ CAPACITY. CDB: 25 0 0 0 0 0 0 0 0 0 
(da3:umass-sim0:0:0:0): CAM Status: SCSI Status Error
(da3:umass-sim0:0:0:0): SCSI Status: Check Condition
(da3:umass-sim0:0:0:0): NOT READY asc:3a,0
(da3:umass-sim0:0:0:0): Medium not present
(da3:umass-sim0:0:0:0): Unretryable error
Opened disk da3 -> 6
(da4:umass-sim0:0:0:1): READ CAPACITY. CDB: 25 20 0 0 0 0 0 0 0 0 
(da4:umass-sim0:0:0:1): CAM Status: SCSI Status Error
(da4:umass-sim0:0:0:1): SCSI Status: Check Condition
(da4:umass-sim0:0:0:1): NOT READY asc:3a,0
(da4:umass-sim0:0:0:1): Medium not present
(da4:umass-sim0:0:0:1): Unretryable error
Opened disk da4 -> 6
(da4:umass-sim0:0:0:1): READ CAPACITY. CDB: 25 20 0 0 0 0 0 0 0 0 
(da4:umass-sim0:0:0:1): CAM Status: SCSI Status Error
(da4:umass-sim0:0:0:1): SCSI Status: Check Condition
(da4:umass-sim0:0:0:1): NOT READY asc:3a,0
(da4:umass-sim0:0:0:1): Medium not present
(da4:umass-sim0:0:0:1): Unretryable error
Opened disk da4 -> 6
(da4:umass-sim0:0:0:1): READ CAPACITY. CDB: 25 20 0 0 0 0 0 0 0 0 
(da4:umass-sim0:0:0:1): CAM Status: SCSI Status Error
(da4:umass-sim0:0:0:1): SCSI Status: Check Condition
(da4:umass-sim0:0:0:1): NOT READY asc:3a,0
(da4:umass-sim0:0:0:1): Medium not present
(da4:umass-sim0:0:0:1): Unretryable error
Opened disk da4 -> 6
(da5:umass-sim0:0:0:2): READ CAPACITY. CDB: 25 40 0 0 0 0 0 0 0 0 
(da5:umass-sim0:0:0:2): CAM Status: SCSI Status Error
(da5:umass-sim0:0:0:2): SCSI Status: Check Condition
(da5:umass-sim0:0:0:2): NOT READY asc:3a,0
(da5:umass-sim0:0:0:2): Medium not present
(da5:umass-sim0:0:0:2): Unretryable error
Opened disk da5 -> 6
(da5:umass-sim0:0:0:2): READ CAPACITY. CDB: 25 40 0 0 0 0 0 0 0 0 
(da5:umass-sim0:0:0:2): CAM Status: SCSI Status Error
(da5:umass-sim0:0:0:2): SCSI Status: Check Condition
(da5:umass-sim0:0:0:2): NOT READY asc:3a,0
(da5:umass-sim0:0:0:2): Medium not present
(da5:umass-sim0:0:0:2): Unretryable error
Opened disk da5 -> 6
(da5:umass-sim0:0:0:2): READ CAPACITY. CDB: 25 40 0 0 0 0 0 0 0 0 
(da5:umass-sim0:0:0:2): CAM Status: SCSI Status Error
(da5:umass-sim0:0:0:2): SCSI Status: Check Condition
(da5:umass-sim0:0:0:2): NOT READY asc:3a,0
(da5:umass-sim0:0:0:2): Medium not present
(da5:umass-sim0:0:0:2): Unretryable error
Opened disk da5 -> 6
(da6:umass-sim0:0:0:3): READ CAPACITY. CDB: 25 60 0 0 0 0 0 0 0 0 
(da6:umass-sim0:0:0:3): CAM Status: SCSI Status Error
(da6:umass-sim0:0:0:3): SCSI Status: Check Condition
(da6:umass-sim0:0:0:3): NOT READY asc:3a,0
(da6:umass-sim0:0:0:3): Medium not present
(da6:umass-sim0:0:0:3): Unretryable error
Opened disk da6 -> 6
(da6:umass-sim0:0:0:3): READ CAPACITY. CDB: 25 60 0 0 0 0 0 0 0 0 
(da6:umass-sim0:0:0:3): CAM Status: SCSI Status Error
(da6:umass-sim0:0:0:3): SCSI Status: Check Condition
(da6:umass-sim0:0:0:3): NOT READY asc:3a,0
(da6:umass-sim0:0:0:3): Medium not present
(da6:umass-sim0:0:0:3): Unretryable error
Opened disk da6 -> 6
(da6:umass-sim0:0:0:3): READ CAPACITY. CDB: 25 60 0 0 0 0 0 0 0 0 
(da6:umass-sim0:0:0:3): CAM Status: SCSI Status Error
(da6:umass-sim0:0:0:3): SCSI Status: Check Condition
(da6:umass-sim0:0:0:3): NOT READY asc:3a,0
(da6:umass-sim0:0:0:3): Medium not present
(da6:umass-sim0:0:0:3): Unretryable error
Opened disk da6 -> 6
Trying to mount root from ufs:/dev/da1s1a

Typically, I will either get the message immediately after detection
of the sym0 <1010> chip or after acd0 (following scsi bus reset).  Of
course at either of these points the boot sequence halts and it gives
me 15 seconds to reboot.
>How-To-Repeat:
I've tried several combinations of enabling/disabling various motherboard
components, but haven't found the exact recipe to repeat the problem
consistently.  There is clearly some randomness to the problem because
I can boot twice in a row with the same settings and it will work one
time and not the other.  Some factors that seem to have some influence
on the problem are the enabling/disabling of the onboard USB in BIOS
and perhaps whether or not I enable or disable ACPI on the boot menu
(you can see there are some issues with acpi from dmesg, but I can't
tell if this is because of the symbios card or not).  When I first
booted up with the installation cdrom, I had to disable the onboard
USB controller to get it to the install menu - this seemed to work
consistently.  Then once I started booting up from the hard drive,
disabling the onboard USB didn't help.  I re-enabled onboard USB and
tried it again with ACPI disabled from the boot menu and it failed
again.  I rebooted one more time with USB enabled and ACPI enabled and
it worked - the very same scenario in which it failed immediately after
installation.
>Fix:
So far I don't have a fix to the problem, but would be willing to try
any suggestions or patches.  I should note here also that my nvidia
ethernet controller (nve) isn't working at the moment either, so no
networking available atm either.
>Release-Note:
>Audit-Trail:

From: Scott Long <scottl@samsco.org>
To: Aaron Wallace <awallac3@uiuc.edu>
Cc: freebsd-gnats-submit@freebsd.org
Subject: Re: amd64/89550: [amd64] sym0: VTOBUS failed (6.0 Release)
Date: Fri, 25 Nov 2005 13:43:15 -0700

 I'm already working on fixing this.
 
 Scott

From: Cedric Ware <cedric.ware@enst.fr>
To: bug-followup@FreeBSD.org, awallac3@uiuc.edu
Cc:  
Subject: Re: amd64/89550: [amd64] sym0: VTOBUS failed (6.0 Release)
Date: Mon, 20 Mar 2006 23:07:19 +0100

 	Hello,
 
 On Fri, 25 Nov 2005, Scott Long wrote:
 > I'm already working on fixing this.
 
 Any success?
 
 I think I've been hit by a similar problem trying to upgrade from
 FreeBSD 5.x/i386 to 6.x/amd64 on an AND Athlon 64 X2 3800+, with a
 Tekram DC-390 SCSI adapter.
 
 When booting from the FreeBSD install CD-ROM (6.1-BETA3 as well as
 6.1-20060320-STABLE-amd64), the boot stops with a panic; I'm not
 sure it always does at the same place, I noted one at device scan:
 ______________________________________________________
 
 Waiting 5 seconds for SCSI devices to settle
 (noperiph:sym0:0:-1:-1): SCSI BUS reset delivered.
 (noperiph:sym1:0:-1:-1): SCSI BUS reset delivered.
 panic: sym: VTOBUS FAILED!
 
 Uptime: 6s
 Cannot dump.  No dump device defined.
 Automatic reboot in 15 seconds - press any key (etc.)
 ______________________________________________________
 
 Another happened after card detection; from memory, I think it was:
 ______________________________________________________
 
 sym0: <1010-33> port (etc.)
 panic: sym: VTOBUS FAILED!
 
 Uptime: 1s
 (etc.)
 ______________________________________________________
 
 My dmesg with FreeBSD 5 follows.  I haven't yet tried 6.x/i386.
 
 					Thanks in advance,
 					Cedric Ware.
 
 Copyright (c) 1992-2005 The FreeBSD Project.
 Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
         The Regents of the University of California. All rights reserved.
 FreeBSD 5.4-STABLE #15: Tue Nov  1 21:16:14 CET 2005
     ware@krakatoa:/data/usrobj/data/usrsrc/sys/KRAKATOA
 ACPI APIC Table: <A M I  OEMAPIC >
 Timecounter "i8254" frequency 1193182 Hz quality 0
 CPU: AMD Athlon(tm) 64 X2 Dual Core Processor 3800+ (2002.58-MHz 686-class CPU)
   Origin = "AuthenticAMD"  Id = 0x20f32  Stepping = 2
   Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
   Hyperthreading: 2 logical CPUs
   AMD Features=0xe0500000<NX,AMIE,LM,DSP,3DNow!>
 real memory  = 1073414144 (1023 MB)
 avail memory = 1040859136 (992 MB)
 FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
  cpu0 (BSP): APIC ID:  0
  cpu1 (AP): APIC ID:  1
 MADT: Forcing active-low polarity and level trigger for SCI
 ioapic0 <Version 0.3> irqs 0-23 on motherboard
 npx0: <math processor> on motherboard
 npx0: INT 16 interface
 acpi0: <A M I OEMRSDT> on motherboard
 acpi0: Power Button (fixed)
 Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0
 cpu0: <ACPI CPU> on acpi0
 cpu1: <ACPI CPU> on acpi0
 pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
 pci0: <ACPI PCI bus> on pcib0
 agp0: <VIA K8T800Pro host to PCI bridge> mem 0xe4000000-0xe7ffffff at device 0.0 on pci0
 pcib1: <ACPI PCI-PCI bridge> at device 1.0 on pci0
 pci1: <ACPI PCI bus> on pcib1
 drm0: <ATI Radeon RV280 9200> port 0xe000-0xe0ff mem 0xfbe00000-0xfbe0ffff,0xe8000000-0xefffffff irq 16 at device 0.0 on pci1
 info: [drm] AGP at 0xe4000000 64MB
 info: [drm] Initialized radeon 1.11.0 20020828 on minor 0
 pci1: <display> at device 0.1 (no driver attached)
 fwohci0: <VIA Fire II (VT6306)> port 0x7400-0x747f mem 0xfb000000-0xfb0007ff irq 16 at device 7.0 on pci0
 fwohci0: OHCI version 1.0 (ROM=1)
 fwohci0: No. of Isochronous channels is 4.
 fwohci0: EUI64 00:11:d8:00:00:58:8f:4a
 fwohci0: Phy 1394a available S400, 2 ports.
 fwohci0: Link S400, max_rec 2048 bytes.
 firewire0: <IEEE1394(FireWire) bus> on fwohci0
 fwohci0: Initiate bus reset
 fwohci0: node_id=0xc800ffc0, gen=1, CYCLEMASTER mode
 firewire0: 1 nodes, maxhop <= 0, cable IRM = 0 (me)
 firewire0: bus manager 0 (me)
 atapci0: <Promise PDC20378 SATA150 controller> port 0x7800-0x787f,0x8800-0x880f,0x9800-0x983f mem 0xfb100000-0xfb11ffff,0xfb200000-0xfb200fff irq 18 at device 8.0 on pci0
 atapci0: failed: rid 0x20 is memory, requested 4
 ata2: channel #0 on atapci0
 ata3: channel #1 on atapci0
 ata4: channel #2 on atapci0
 skc0: <Marvell Gigabit Ethernet> port 0xa000-0xa0ff mem 0xfb400000-0xfb403fff irq 17 at device 10.0 on pci0
 skc0: Marvell Yukon Lite Gigabit Ethernet rev. (0x9)
 sk0: <Marvell Semiconductor, Inc. Yukon> on skc0
 sk0: Ethernet address: 00:13:d4:9b:6d:eb
 miibus0: <MII bus> on sk0
 e1000phy0: <Marvell 88E1000 Gigabit PHY> on miibus0
 e1000phy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX-FDX, auto
 sym0: <1010-33> port 0x8000-0x80ff mem 0xfb600000-0xfb601fff,0xfb700000-0xfb7003ff irq 18 at device 13.0 on pci0
 sym0: Symbios NVRAM, ID 7, Fast-80, LVD, parity checking
 sym0: open drain IRQ line driver, using on-chip SRAM
 sym0: using LOAD/STORE-based firmware.
 sym0: handling phase mismatch from SCRIPTS.
 sym0: SCAN AT BOOT disabled for targets 1 2 3 8 9 11 12 13 14 15.
 sym1: <1010-33> port 0x9000-0x90ff mem 0xfb900000-0xfb901fff,0xfba00000-0xfba003ff irq 18 at device 13.1 on pci0
 sym1: Symbios NVRAM, ID 7, Fast-80, SE, parity checking
 sym1: open drain IRQ line driver, using on-chip SRAM
 sym1: using LOAD/STORE-based firmware.
 sym1: handling phase mismatch from SCRIPTS.
 sym1: SCAN AT BOOT disabled for targets 2 5 8 9 10 11 12 13 14 15.
 atapci1: <VIA 6420 SATA150 controller> port 0xa400-0xa4ff,0xa800-0xa80f,0xb000-0xb003,0xb400-0xb407,0xb800-0xb803,0xc000-0xc007 irq 20 at device 15.0 on pci0
 ata5: channel #0 on atapci1
 ata6: channel #1 on atapci1
 atapci2: <VIA 8237 UDMA133 controller> port 0xfc00-0xfc0f,0x376,0x170-0x177,0x3f6,0x1f0-0x1f7 at device 15.1 on pci0
 ata0: channel #0 on atapci2
 ata1: channel #1 on atapci2
 uhci0: <VIA 83C572 USB controller> port 0xc400-0xc41f irq 21 at device 16.0 on pci0
 usb0: <VIA 83C572 USB controller> on uhci0
 usb0: USB revision 1.0
 uhub0: VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
 uhub0: 2 ports with 2 removable, self powered
 uhci1: <VIA 83C572 USB controller> port 0xc800-0xc81f irq 21 at device 16.1 on pci0
 usb1: <VIA 83C572 USB controller> on uhci1
 usb1: USB revision 1.0
 uhub1: VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
 uhub1: 2 ports with 2 removable, self powered
 uhci2: <VIA 83C572 USB controller> port 0xd000-0xd01f irq 21 at device 16.2 on pci0
 usb2: <VIA 83C572 USB controller> on uhci2
 usb2: USB revision 1.0
 uhub2: VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
 uhub2: 2 ports with 2 removable, self powered
 uhci3: <VIA 83C572 USB controller> port 0xd400-0xd41f irq 21 at device 16.3 on pci0
 usb3: <VIA 83C572 USB controller> on uhci3
 usb3: USB revision 1.0
 uhub3: VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
 uhub3: 2 ports with 2 removable, self powered
 pci0: <serial bus, USB> at device 16.4 (no driver attached)
 isab0: <PCI-ISA bridge> at device 17.0 on pci0
 isa0: <ISA bus> on isab0
 pcm0: <VIA VT8237> port 0xd800-0xd8ff irq 22 at device 17.5 on pci0
 pcm0: <Avance Logic ALC850 AC97 Codec>
 acpi_button0: <Power Button> on acpi0
 acpi_button1: <Sleep Button> on acpi0
 atkbdc0: <Keyboard controller (i8042)> port 0x64,0x60 irq 1 on acpi0
 atkbd0: <AT Keyboard> irq 1 on atkbdc0
 kbd0 at atkbd0
 psm0: <PS/2 Mouse> irq 12 on atkbdc0
 psm0: model MouseMan+, device ID 0
 fdc0: <floppy drive controller (FDE)> port 0x3f7,0x3f0-0x3f5 irq 6 drq 2 on acpi0
 fd0: <1440-KB 3.5" drive> on fdc0 drive 0
 sio0: <16550A-compatible COM port> port 0x2f8-0x2ff irq 3 flags 0x10 on acpi0
 sio0: type 16550A
 sio1: configured irq 4 not in bitmap of probed irqs 0
 sio1: port may not be enabled
 sio1: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 on acpi0
 sio1: type 16550A
 orm0: <ISA Option ROMs> at iomem 0xcd000-0xd0fff,0xc0000-0xccfff on isa0
 pmtimer0 on isa0
 ppc0: parallel port not found.
 sc0: <System console> at flags 0x100 on isa0
 sc0: VGA <16 virtual consoles, flags=0x300>
 vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
 Timecounters tick every 10.000 msec
 ad0: 152627MB <ST3160023A/8.01> [310101/16/63] at ata0-master UDMA100
 acd0: DVDR <PIONEER DVD-RW DVR-109/1.17> at ata1-master UDMA66
 Waiting 5 seconds for SCSI devices to settle
 (noperiph:sym0:0:-1:-1): SCSI BUS reset delivered.
 (noperiph:sym1:0:-1:-1): SCSI BUS reset delivered.
 da0 at sym0 bus 0 target 0 lun 0
 da0: <SEAGATE ST336754LW 0002> Fixed Direct Access SCSI-3 device 
 da0: 160.000MB/s transfers (80.000MHz, offset 62, 16bit), Tagged Queueing Enabled
 da0: 35003MB (71687372 512 byte sectors: 255H 63S/T 4462C)
 SMP: AP CPU #1 Launched!
 cd0 at ata1 bus 0 target 0 lun 0
 cd0: <PIONEER DVD-RW  DVR-109 1.17> Removable CD-ROM SCSI-0 device 
 cd0: 66.000MB/s transfers
 cd0: Attempt to query device size failed: NOT READY, Medium not present
 Mounting root from ufs:/dev/ad0s2a

From: "Jan Mikkelsen" <janm@transactionware.com>
To: <bug-followup@FreeBSD.org>,
	<awallac3@uiuc.edu>
Cc:  
Subject: Re: amd64/89550: [amd64] sym0: VTOBUS failed (6.0 Release)
Date: Mon, 18 Sep 2006 11:39:42 +1000

 I am seeing this too.
 
 Tyan S2865 motherboard, Athlon X2, 6.1-RELEASE-p5.  Booting from a 
 6.0-RELEASE/i386 CD seems to work fine, at least as far as the fixit shell.
 
 The behaviour below is consistent;  it always dies with "panic: sym: VTOBUS 
 FAILED!" after finding the sym0 controller.
 
 dmesg below.
 
 Thanks,
 
 Jan Mikkelsen
 janm@transactionware.com
 
 
 
 Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
  The Regents of the University of California. All rights reserved.
 FreeBSD 6.1-RELEASE-p5 #0: Tue Sep 12 13:32:17 EST 2006
     root@test.transactionware.com:/work/home/janm/p4/image-freebsd-std-fileserv/work/base-freebsd/work/home/janm/p4/image-freebsd-std-fileserv/FreeBSD/src/sys/TW-SMP
 ACPI APIC Table: <Nvidia AWRDACPI>
 Timecounter "i8254" frequency 1193182 Hz quality 0
 CPU: AMD Athlon(tm) 64 X2 Dual Core Processor 4200+ (2210.20-MHz K8-class 
 CPU)
   Origin = "AuthenticAMD"  Id = 0x20fb1  Stepping = 1
   Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
   Features2=0x1<SSE3>
   AMD Features=0xe2500800<SYSCALL,NX,MMX+,FFXSR,LM,3DNow+,3DNow>
   AMD Features2=0x3<LAHF,CMP>
   Cores per package: 2
 real memory  = 2146304000 (2046 MB)
 avail memory = 2061185024 (1965 MB)
 FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
  cpu0 (BSP): APIC ID:  0
  cpu1 (AP): APIC ID:  1
 ioapic0: Changing APIC ID to 2
 ioapic0 <Version 1.1> irqs 0-23 on motherboard
 kbd1 at kbdmux0
 acpi0: <Nvidia AWRDACPI> on motherboard
 acpi_bus_number: can't get _ADR
 acpi_bus_number: can't get _ADR
 acpi_bus_number: can't get _ADR
 acpi_bus_number: can't get _ADR
 acpi0: Power Button (fixed)
 acpi_bus_number: can't get _ADR
 acpi_bus_number: can't get _ADR
 acpi_bus_number: can't get _ADR
 acpi_bus_number: can't get _ADR
 acpi_bus_number: can't get _ADR
 acpi_bus_number: can't get _ADR
 unknown: I/O range not supported
 Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0
 cpu0: <ACPI CPU> on acpi0
 cpu1: <ACPI CPU> on acpi0
 acpi_button0: <Power Button> on acpi0
 pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
 pci0: <ACPI PCI bus> on pcib0
 pci0: <memory> at device 0.0 (no driver attached)
 isab0: <PCI-ISA bridge> at device 1.0 on pci0
 isa0: <ISA bus> on isab0
 pci0: <serial bus, SMBus> at device 1.1 (no driver attached)
 ohci0: <OHCI (generic) USB controller> mem 0xfebff000-0xfebfffff irq 21 at 
 device 2.0 on pci0
 ohci0: [GIANT-LOCKED]
 usb0: OHCI version 1.0, legacy support
 usb0: <OHCI (generic) USB controller> on ohci0
 usb0: USB revision 1.0
 uhub0: nVidia OHCI root hub, class 9/0, rev 1.00/1.00, addr 1
 uhub0: 8 ports with 8 removable, self powered
 ehci0: <NVIDIA nForce4 USB 2.0 controller> mem 0xfeb00000-0xfeb000ff irq 22 
 at device 2.1 on pci0
 ehci0: [GIANT-LOCKED]
 usb1: waiting for BIOS to give up control
 usb1: timed out waiting for BIOS
 usb1: EHCI version 1.0
 usb1: companion controller, 4 ports each: usb0
 usb1: <NVIDIA nForce4 USB 2.0 controller> on ehci0
 usb1: USB revision 2.0
 uhub1: nVidia EHCI root hub, class 9/0, rev 2.00/1.00, addr 1
 uhub1: 8 ports with 8 removable, self powered
 pci0: <multimedia, audio> at device 4.0 (no driver attached)
 atapci0: <nVidia nForce CK804 UDMA133 controller> port 
 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xe000-0xe00f at device 6.0 on pci0
 ata0: <ATA channel 0> on atapci0
 ata1: <ATA channel 1> on atapci0
 atapci1: <nVidia nForce CK804 SATA300 controller> port 
 0x9f0-0x9f7,0xbf0-0xbf3,0x970-0x977,0xb70-0xb73,0xcc00-0xcc0f mem 
 0xfebfb000-0xfebfbfff irq 21 at device 7.0 on pci0
 ata2: <ATA channel 0> on atapci1
 ata3: <ATA channel 1> on atapci1
 atapci2: <nVidia nForce CK804 SATA300 controller> port 
 0x9e0-0x9e7,0xbe0-0xbe3,0x960-0x967,0xb60-0xb63,0xb800-0xb80f mem 
 0xfebfa000-0xfebfafff irq 22 at device 8.0 on pci0
 ata4: <ATA channel 0> on atapci2
 ata5: <ATA channel 1> on atapci2
 pcib1: <ACPI PCI-PCI bridge> at device 9.0 on pci0
 pci1: <ACPI PCI bus> on pcib1
 pci2: <display, VGA> at device 5.0 (no driver attached)
 fwohci0: <VIA Fire II (VT6306)> port 0xa800-0xa87f mem 0xfcffe000-0xfcffe7ff 
 irq 18 at device 6.0 on pci1
 fwohci0: OHCI version 1.10 (ROM=1)
 fwohci0: No. of Isochronous channels is 4.
 fwohci0: EUI64 00:e0:81:00:00:26:37:41
 fwohci0: Phy 1394a available S400, 2 ports.
 fwohci0: Link S400, max_rec 2048 bytes.
 firewire0: <IEEE1394(FireWire) bus> on fwohci0
 fwe0: <Ethernet over FireWire> on firewire0
 if_fwe0: Fake Ethernet address: 02:e0:81:26:37:41
 fwe0: Ethernet address: 02:e0:81:26:37:41
 fwe0: if_start running deferred for Giant
 sbp0: <SBP-2/SCSI over FireWire> on firewire0
 fwohci0: Initiate bus reset
 fwohci0: node_id=0xc800ffc0, gen=1, CYCLEMASTER mode
 firewire0: 1 nodes, maxhop <= 0, cable IRM = 0 (me)
 firewire0: bus manager 0 (me)
 sym0: <1010-66> port 0xa400-0xa4ff mem 
 0xfcffd000-0xfcffd3ff,0xfcffa000-0xfcffbfff irq 18 at device 9.0 on pci1
 panic: sym: VTOBUS FAILED!
 
 cpuid = 0
 Uptime: 1s
 

From: "Jan Mikkelsen" <janm@transactionware.com>
To: <bug-followup@FreeBSD.org>,
	<awallac3@uiuc.edu>
Cc:  
Subject: Re: amd64/89550: [amd64] sym0: VTOBUS failed (6.0 Release)
Date: Tue, 19 Sep 2006 15:39:32 +1000

 I've just been looking at this.
 
 The problem seems to be caused by the sym driver assuming that the virtual
 address of the memory allocated by bus_dmamem_alloc will be aligned to an
 address 2*PAGE_SIZE.  The physical address is aligned like that, but the
 virtual address isn't.  This also breaks the buddy algorithm.
 
 Here are the changes I have made to make it work on my system.  Summary:
 Allocate additional pages and adjust the virtual address if necessary to get
 memory with the right alignment.
 
 Other changes (possibly incorrect):
 
 - Adjusted the contigmalloc call because the boundary argument didn't make
 sense to me compared to the man page.
 
 - Adjusted the highaddr argument in the bus_dma_tag_create() call, again,
 the usage didn't really match what I expected from the man page.
 
 Little testing;  I have used it to boot and dd over an old 4GB drive.  With
 the original version it never made it past sym_pci_attach.
 
 
 *** FreeBSD/src/sys/dev/sym/sym_hipd.c	Tue Aug  8 00:43:31 2006
 --- /home/janm/sym_hipd.c	Tue Sep 19 15:27:55 2006
 ***************
 *** 439,445 ****
   #define free_pages(p)		free((p), M_DEVBUF)
   #else
   #define get_pages()		contigmalloc(MEMO_CLUSTER_SIZE, M_DEVBUF, \
 ! 				    0, 0, 1LL << 32, PAGE_SIZE, 1LL << 32)
   #define free_pages(p)		contigfree((p), MEMO_CLUSTER_SIZE, M_DEVBUF)
   #endif
   
 --- 439,446 ----
   #define free_pages(p)		free((p), M_DEVBUF)
   #else
   #define get_pages()		contigmalloc(MEMO_CLUSTER_SIZE, M_DEVBUF, \
 ! 				    0, 0, 1LL << 32, PAGE_SIZE, \
 ! 				    MEMO_CLUSTER_SIZE)
   #define free_pages(p)		contigfree((p), MEMO_CLUSTER_SIZE, M_DEVBUF)
   #endif
   
 ***************
 *** 453,458 ****
 --- 454,462 ----
   	struct m_vtob	*next;
   	bus_dmamap_t	dmamap;	/* Map for this chunk */
   	m_addr_t	vaddr;	/* Virtual address */
 + #if PAGE_SIZE < MEMO_CLUSTER_SIZE
 + 	m_addr_t	adjusted_vaddr;	/* Virtual address adjusted for
 buddy */
 + #endif
   	m_addr_t	baddr;	/* Bus physical address */
   } m_vtob_s;
   /* Hash this stuff a bit to speed up translations */
 ***************
 *** 674,685 ****
   	if (bus_dmamem_alloc(mp->dmat, &vaddr,
   			      BUS_DMA_NOWAIT, &vbp->dmamap))
   		goto out_err;
   	bus_dmamap_load(mp->dmat, vbp->dmamap, vaddr,
   			MEMO_CLUSTER_SIZE, getbaddrcb, &baddr, 0);
   	if (baddr) {
 ! 		int hc = VTOB_HASH_CODE(vaddr);
   		vbp->vaddr = (m_addr_t) vaddr;
   		vbp->baddr = (m_addr_t) baddr;
   		vbp->next = mp->vtob[hc];
   		mp->vtob[hc] = vbp;
   		++mp->nump;
 --- 678,711 ----
   	if (bus_dmamem_alloc(mp->dmat, &vaddr,
   			      BUS_DMA_NOWAIT, &vbp->dmamap))
   		goto out_err;
 + #if PAGE_SIZE < MEMO_CLUSTER_SIZE
 + 	bus_dmamap_load(mp->dmat, vbp->dmamap, vaddr,
 + 			MEMO_CLUSTER_SIZE * 2, getbaddrcb, &baddr, 0);
 + #else
   	bus_dmamap_load(mp->dmat, vbp->dmamap, vaddr,
   			MEMO_CLUSTER_SIZE, getbaddrcb, &baddr, 0);
 + #endif
   	if (baddr) {
 ! 		int hc;
 ! 
   		vbp->vaddr = (m_addr_t) vaddr;
   		vbp->baddr = (m_addr_t) baddr;
 + #if PAGE_SIZE < MEMO_CLUSTER_SIZE
 + 		/*
 + 		 * Adjusting vaddr is OK here because we do not error out.
 + 		 * If that changes, the bus_dmamem_free() call below would
 + 		 * need to be fixed.
 + 		 *
 + 		 * This code assumes that we will be at most one page away
 + 		 * from having things aligned the way we want.
 + 		 */
 + 
 + 		if (((m_addr_t) vaddr & ~MEMO_CLUSTER_MASK) != (m_addr_t)
 vaddr)
 + 			vaddr = (void*) ((m_addr_t) vaddr + PAGE_SIZE);
 + 
 + 		vbp->adjusted_vaddr = (m_addr_t) vaddr;
 + #endif
 + 	       	hc = VTOB_HASH_CODE(vaddr);
   		vbp->next = mp->vtob[hc];
   		mp->vtob[hc] = vbp;
   		++mp->nump;
 ***************
 *** 733,744 ****
   	mp = __sym_calloc(&mp0, sizeof(*mp), "MPOOL");
   	if (mp) {
   		mp->dev_dmat = dev_dmat;
 ! 		if (!bus_dma_tag_create(dev_dmat, 1, MEMO_CLUSTER_SIZE,
   			       BUS_SPACE_MAXADDR_32BIT,
   			       BUS_SPACE_MAXADDR_32BIT,
   			       NULL, NULL, MEMO_CLUSTER_SIZE, 1,
   			       MEMO_CLUSTER_SIZE, 0,
   			       busdma_lock_mutex, &Giant, &mp->dmat)) {
   			mp->getp = ___dma_getp;
   #ifdef	MEMO_FREE_UNUSED
   			mp->freep = ___dma_freep;
 --- 759,779 ----
   	mp = __sym_calloc(&mp0, sizeof(*mp), "MPOOL");
   	if (mp) {
   		mp->dev_dmat = dev_dmat;
 ! #if PAGE_SIZE < MEMO_CLUSTER_SIZE
 ! 		if (!bus_dma_tag_create(dev_dmat, 1, MEMO_CLUSTER_SIZE * 2,
   			       BUS_SPACE_MAXADDR_32BIT,
 + 			       BUS_SPACE_MAXADDR,
 + 			       NULL, NULL, MEMO_CLUSTER_SIZE * 2, 1,
 + 			       MEMO_CLUSTER_SIZE * 2, 0,
 + 			       busdma_lock_mutex, &Giant, &mp->dmat)) {
 + #else
 + 		if (!bus_dma_tag_create(dev_dmat, 1, MEMO_CLUSTER_SIZE,
   			       BUS_SPACE_MAXADDR_32BIT,
 + 			       BUS_SPACE_MAXADDR,
   			       NULL, NULL, MEMO_CLUSTER_SIZE, 1,
   			       MEMO_CLUSTER_SIZE, 0,
   			       busdma_lock_mutex, &Giant, &mp->dmat)) {
 + #endif
   			mp->getp = ___dma_getp;
   #ifdef	MEMO_FREE_UNUSED
   			mp->freep = ___dma_freep;
 ***************
 *** 815,827 ****
   	mp = ___get_dma_pool(dev_dmat);
   	if (mp) {
   		vp = mp->vtob[hc];
   		while (vp && (m_addr_t) vp->vaddr != a)
   			vp = vp->next;
   	}
   	/* Unlock */
   	if (!vp)
   		panic("sym: VTOBUS FAILED!\n");
 ! 	return vp ? vp->baddr + (((m_addr_t) m) - a) : 0;
   }
   
   
 --- 850,872 ----
   	mp = ___get_dma_pool(dev_dmat);
   	if (mp) {
   		vp = mp->vtob[hc];
 + #if (PAGE_SIZE < MEMO_CLUSTER_SIZE)
 + 		while (vp && (m_addr_t) vp->adjusted_vaddr != a)
 + #else
   		while (vp && (m_addr_t) vp->vaddr != a)
 + #endif
   			vp = vp->next;
   	}
 + 
   	/* Unlock */
   	if (!vp)
   		panic("sym: VTOBUS FAILED!\n");
 ! #if (PAGE_SIZE < MEMO_CLUSTER_SIZE)
 ! 	return vp->baddr + (((m_addr_t) m) - a) +
 ! 		(vp->adjusted_vaddr - vp->vaddr);
 ! #else
 ! 	return vp->baddr + (((m_addr_t) m) - a);
 ! #endif
   }
   
   
 

From: "Jan Mikkelsen" <janm@transactionware.com>
To: <bug-followup@FreeBSD.org>,
	<awallac3@uiuc.edu>
Cc:  
Subject: Re: amd64/89550: [amd64] sym0: VTOBUS failed (6.0 Release)
Date: Tue, 26 Sep 2006 13:06:36 +1000

 Alternative, smaller, cleaner, patch.
 
 Removes amd64 specific case by splitting the sym_hcb struct so that we don't
 have to cross page boundaries.
 
 --- 
 /home/janm/p4/image-freebsd-std-main/FreeBSD/src/sys/dev/sym/sym_hipd.c
 Sun Sep 24 08:06:41 2006
 +++ sym_hipd.c	Mon Sep 25 17:25:02 2006
 @@ -421,11 +421,7 @@
   */
  
  #define MEMO_SHIFT	4	/* 16 bytes minimum memory chunk */
 -#ifndef __amd64__
  #define MEMO_PAGE_ORDER	0	/* 1 PAGE  maximum */
 -#else
 -#define MEMO_PAGE_ORDER	1	/* 2 PAGEs maximum on amd64 */
 -#endif
  #if 0
  #define MEMO_FREE_UNUSED	/* Free unused pages immediately */
  #endif
 @@ -434,14 +430,8 @@
  #define MEMO_CLUSTER_SIZE	(1UL << MEMO_CLUSTER_SHIFT)
  #define MEMO_CLUSTER_MASK	(MEMO_CLUSTER_SIZE-1)
  
 -#ifndef __amd64__
  #define get_pages()		malloc(MEMO_CLUSTER_SIZE, M_DEVBUF,
 M_NOWAIT)
  #define free_pages(p)		free((p), M_DEVBUF)
 -#else
 -#define get_pages()		contigmalloc(MEMO_CLUSTER_SIZE, M_DEVBUF, \
 -				    0, 0, 1LL << 32, PAGE_SIZE, 1LL << 32)
 -#define free_pages(p)		contigfree((p), MEMO_CLUSTER_SIZE, M_DEVBUF)
 -#endif
  
  typedef u_long m_addr_t;	/* Enough bits to bit-hack addresses */
  
 @@ -675,7 +665,7 @@
  			      BUS_DMA_NOWAIT, &vbp->dmamap))
  		goto out_err;
  	bus_dmamap_load(mp->dmat, vbp->dmamap, vaddr,
 -			MEMO_CLUSTER_SIZE, getbaddrcb, &baddr, 0);
 +			MEMO_CLUSTER_SIZE, getbaddrcb, &baddr,
 BUS_DMA_NOWAIT);
  	if (baddr) {
  		int hc = VTOB_HASH_CODE(vaddr);
  		vbp->vaddr = (m_addr_t) vaddr;
 @@ -735,7 +725,7 @@
  		mp->dev_dmat = dev_dmat;
  		if (!bus_dma_tag_create(dev_dmat, 1, MEMO_CLUSTER_SIZE,
  			       BUS_SPACE_MAXADDR_32BIT,
 -			       BUS_SPACE_MAXADDR_32BIT,
 +			       BUS_SPACE_MAXADDR,
  			       NULL, NULL, MEMO_CLUSTER_SIZE, 1,
  			       MEMO_CLUSTER_SIZE, 0,
  			       busdma_lock_mutex, &Giant, &mp->dmat)) {
 @@ -1592,7 +1582,7 @@
  	/*
  	 *  Target data.
  	 */
 -	struct sym_tcb	target[SYM_CONF_MAX_TARGET];
 +	struct sym_tcb	*target;
  
  	/*
  	 *  Target control block bus address array used by the SCRIPT 
 @@ -8698,6 +8688,11 @@
  	np->fw_patch	 = fw->patch;
  	np->fw_name	 = fw->name;
  
 +	np->target = sym_calloc_dma(SYM_CONF_MAX_TARGET *
 sizeof(*(np->target)),
 +			"TARGET");
 +	if (!np->target)
 +		goto attach_failed;
 +
  	/*
  	 * Edit its name.
  	 */
 @@ -9096,6 +9091,9 @@
  			       "LUNMP");
  #endif 
  	}
 +	if (np->target)
 +		sym_mfree_dma(np->target,
 +			SYM_CONF_MAX_TARGET * sizeof(*(np->target)),
 "TARGET");
  	if (np->targtbl)
  		sym_mfree_dma(np->targtbl, 256, "TARGTBL");
  	if (np->data_dmat)
 

From: Mikhail Teterin <mi+kde@aldan.algebra.com>
To: bug-followup@freebsd.org, awallac3@uiuc.edu
Cc: "Jan Mikkelsen" <janm@transactionware.com>
Subject: Re: amd64/89550: [amd64] sym0: VTOBUS failed (6.0 Release)
Date: Wed, 31 Jan 2007 11:29:08 -0500

 Hello!
 
 The 6.2's install shamefully paniced on our amd64-system -- due to this bug. 
 We had to borrow an Adaptec card and then switch back to the Symbios ones 
 after applying Jan's patch-1.diff
 
 Unfortunately, that patch allowed us to collect a number of kernel dumps:
 
 	http://aldan.algebra.com/~mi/bonkers-crash-0.txt - GPF
 	http://aldan.algebra.com/~mi/bonkers-crash-1.txt - spin lock
 	http://aldan.algebra.com/~mi/bonkers-crash-2.txt - GPF
 
 (there were more panics with "spin lock held too long", but no kernel dump 
 took place, perhaps, because the dumpdev is attached to the sym-card too)
 
 The second patch, posted by Jan on Sep 26th seems much better so far, although 
 the driver's use of Giant is still disappointing...
 
 Maybe, Jan's enhancement can be committed to stop the VTOBUS-panics from 
 happening again?
 
 Thanks!
 
 	-mi

From: Mikhail Teterin <mi+kde@aldan.algebra.com>
To: bug-followup@freebsd.org, awallac3@uiuc.edu, jhb@freebsd.org,
        stable@freebsd.org
Cc:  
Subject: Re: amd64/89550: spin-lock panics continue (crash trace)
Date: Sat, 26 May 2007 11:30:51 -0400

 The patch in the PR allows our two Symbios 875 boards to work with 
 FreeBSD-6.2-recent.
 
 However, the machine continues to suffer from the "spin lock held to long" 
 panics every few days. Usually, there is a double-panic, which prevents crash 
 dump from occuring. But this morning it succeeded.
 
 The full trace is available at
 
 	http://aldan.algebra.com/~mi/bonkers-crash-4.txt
 
 The panic is from kern_mutex.c. The value of the delay-counting i is, no 
 surprise, exactly 60000000.
 
 I'm not sure, if it is the sym-enabling patch, that causes the problem, or 
 something else. Either way, it would be very useful for an expert to look at 
 it -- and commit the (possibly modified) patch too.
 
 	-mi
State-Changed-From-To: open->closed 
State-Changed-By: se 
State-Changed-When: Fri Jul 20 23:02:12 UTC 2007 
State-Changed-Why:  
Slightly modified version of the last suggested patch has been committed. 
Thanks! 

http://www.freebsd.org/cgi/query-pr.cgi?pr=89550 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: amd64/89550: commit references a PR
Date: Fri, 20 Jul 2007 23:02:24 +0000 (UTC)

 se          2007-07-20 23:02:01 UTC
 
   FreeBSD src repository
 
   Modified files:
     sys/dev/sym          sym_hipd.c 
   Log:
   Fix Symbios driver on amd64: Since amd64 has 64 bit pointers but the same
   4KB pages as i386, data structures that just fit in one page on i386 (and
   on 64 bit architectures with 8KB pages) can be distributed over two pages
   on amd64. This is a porblem in the case of the Symbios driver, since the
   SCRIPTS engine in the SCSI chip operates on physical addresses and needs
   physically contiguous memory. Earlier patches used contigmalloc on amd64,
   but this version replaces part of a structure by a pointer to that data.
   In order to not introduce an extra indirection for other architectures,
   the change has been made conditional on __amd64__.
   
   Earlier attempts to repair this problem are removed (i.e. the macros that
   made amd64 use contigmalloc). The fix was submitted by Jan Mikkelsen and
   modified by me to only affect amd64.
   
   PR:             89550
   Submitted by:   janm at transactionware dot com (Jan Mikkelsen)
   Approved by:    re (Hiroki Sato)
   MFC after:      2 weeks
   
   Revision  Changes    Path
   1.67      +17 -12    src/sys/dev/sym/sym_hipd.c
 _______________________________________________
 cvs-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/cvs-all
 To unsubscribe, send any mail to "cvs-all-unsubscribe@freebsd.org"
 
>Unformatted:
