From nobody@FreeBSD.org  Sun Jul 16 20:38:26 2006
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id 3A3C216A4DF
	for <freebsd-gnats-submit@FreeBSD.org>; Sun, 16 Jul 2006 20:38:26 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (www.freebsd.org [216.136.204.117])
	by mx1.FreeBSD.org (Postfix) with ESMTP id F420F43D46
	for <freebsd-gnats-submit@FreeBSD.org>; Sun, 16 Jul 2006 20:38:25 +0000 (GMT)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by www.freebsd.org (8.13.1/8.13.1) with ESMTP id k6GKcPlJ098026
	for <freebsd-gnats-submit@FreeBSD.org>; Sun, 16 Jul 2006 20:38:25 GMT
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.13.1/8.13.1/Submit) id k6GKcP93098010;
	Sun, 16 Jul 2006 20:38:25 GMT
	(envelope-from nobody)
Message-Id: <200607162038.k6GKcP93098010@www.freebsd.org>
Date: Sun, 16 Jul 2006 20:38:25 GMT
From: Eric Lakin <elakin@infohell.net>
To: freebsd-gnats-submit@FreeBSD.org
Subject: bge driver disables access to shared iLo on HP DL360 G4
X-Send-Pr-Version: www-2.3

>Number:         100410
>Category:       kern
>Synopsis:       [bge] bge driver disables access to shared iLo on HP DL360 G4
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    bz
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Sun Jul 16 20:40:14 GMT 2006
>Closed-Date:    Tue Nov 17 20:55:20 UTC 2009
>Last-Modified:  Tue Nov 17 20:55:20 UTC 2009
>Originator:     Eric Lakin
>Release:        6.1
>Organization:
>Environment:
FreeBSD amun-ra.priv.infohell.net 6.1-RELEASE-p2 FreeBSD 6.1-RELEASE-p2 #7: Sat Jul 15 20:53:49 PDT 2006     root@amun-ra.priv.infohell.net:/usr/obj/usr/src/sys/AMUN-RA  amd64
>Description:
One of the configuration options of the iLo (integrated lights-out management
device) on the HP DL360 G4 and later is to use a "shared network" setup.
Normally, the iLo device uses a dedicated network interface. With the shared
network, the iLo shares the first onboard ethernet device with the host
computer. Under FreeBSD this is bge0.

When a kernel containing the bge0 driver is booted, or the 'if_bge' module
is loaded, access to the iLo is interrupted - the ilo can no longer be pinged
or ssh'd into.

When if_bge is loaded on the DL360, the following is logged:

bge0: <Broadcom BCM5704C Dual Gigabit Ethernet, ASIC rev. 0x2100> mem 0xfdff0000-0xfdffffff irq 25 at device 2.0 on pci2
bge0: Ethernet address: 00:12:79:8f:8f:54
bge1: <Broadcom BCM5704C Dual Gigabit Ethernet, ASIC rev. 0x2100> mem 0xfdfe0000-0xfdfeffff irq 26 at device 2.1 on pci2
bge1: Ethernet address: 00:12:79:8f:8f:53
miibus0: <MII bus> on bge0
brgphy0: <BCM5704 10/100/1000baseTX PHY> on miibus0
brgphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX, 1000baseTX-FDX, auto
miibus1: <MII bus> on bge1
brgphy1: <BCM5704 10/100/1000baseTX PHY> on miibus1
brgphy1:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX, 1000baseTX-FDX, auto

Also, if a host is pinging the iLo IP address when if_bge is loaded, the
FreeBSD host can see the ICMP packets that should be destined for iLo.
Because of this, it looks as though the FreeBSD driver is accessing the
bge device at a lower level than it needs, and is reading packets from the
HW that should be going to the iLo.

This is kind of a strange setup - basically, two computers are sharing the
same network interface (the iLo and the computer running FreeBSD). However,
it's something that I also tested under linux - and linux is able to utilize
the ethernet interface w/o impacting the iLo using it as well.

The machine that I discovered this on is not yet in active service, and so
I can help test and provide more information for a while, as needed.

>How-To-Repeat:
Load if_bge, or boot a kernel that includes 'device bge'
>Fix:

>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->bz 
Responsible-Changed-By: bz 
Responsible-Changed-When: Mon Oct 12 21:55:59 UTC 2009 
Responsible-Changed-Why:  
Take for the moment. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=100410 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/100410: commit references a PR
Date: Tue, 13 Oct 2009 20:22:27 +0000 (UTC)

 Author: bz
 Date: Tue Oct 13 20:22:12 2009
 New Revision: 198049
 URL: http://svn.freebsd.org/changeset/base/198049
 
 Log:
   Immediately after clearing a pending callout that didn't make it due
   to the lock we hold, disable interrupts, and announce to the firmware
   that we are shutting down. Especially do this before disabling blocks.
   
   This makes some types of machines with asf enabled no longer hang upon
   boot, when we start configuring the interface.
   
   PR:			i386/96382, kern/100410, kern/122252, kern/116328
   Reported by:		erwin
   Hardware provided by:	TDC A/S
   Reviewed by:		stas
   Tested by:		stas
 
 Modified:
   head/sys/dev/bge/if_bge.c
 
 Modified: head/sys/dev/bge/if_bge.c
 ==============================================================================
 --- head/sys/dev/bge/if_bge.c	Tue Oct 13 20:21:17 2009	(r198048)
 +++ head/sys/dev/bge/if_bge.c	Tue Oct 13 20:22:12 2009	(r198049)
 @@ -4270,6 +4270,16 @@ bge_stop(struct bge_softc *sc)
  
  	callout_stop(&sc->bge_stat_ch);
  
 +	/* Disable host interrupts. */
 +	BGE_SETBIT(sc, BGE_PCI_MISC_CTL, BGE_PCIMISCCTL_MASK_PCI_INTR);
 +	bge_writembx(sc, BGE_MBX_IRQ0_LO, 1);
 +
 +	/*
 +	 * Tell firmware we're shutting down.
 +	 */
 +	bge_stop_fw(sc);
 +	bge_sig_pre_reset(sc, BGE_RESET_STOP);
 +
  	/*
  	 * Disable all of the receiver blocks.
  	 */
 @@ -4309,16 +4319,6 @@ bge_stop(struct bge_softc *sc)
  		BGE_CLRBIT(sc, BGE_MARB_MODE, BGE_MARBMODE_ENABLE);
  	}
  
 -	/* Disable host interrupts. */
 -	BGE_SETBIT(sc, BGE_PCI_MISC_CTL, BGE_PCIMISCCTL_MASK_PCI_INTR);
 -	bge_writembx(sc, BGE_MBX_IRQ0_LO, 1);
 -
 -	/*
 -	 * Tell firmware we're shutting down.
 -	 */
 -
 -	bge_stop_fw(sc);
 -	bge_sig_pre_reset(sc, BGE_RESET_STOP);
  	bge_reset(sc);
  	bge_sig_legacy(sc, BGE_RESET_STOP);
  	bge_sig_post_reset(sc, BGE_RESET_STOP);
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/100410: commit references a PR
Date: Tue, 17 Nov 2009 14:30:25 +0000 (UTC)

 Author: bz
 Date: Tue Nov 17 14:30:09 2009
 New Revision: 199381
 URL: http://svn.freebsd.org/changeset/base/199381
 
 Log:
   MFC r198049:
   
     Immediately after clearing a pending callout that didn't make it due
     to the lock we hold, disable interrupts, and announce to the firmware
     that we are shutting down. Especially do this before disabling blocks.
   
     This makes some types of machines with asf enabled no longer hang upon
     boot, when we start configuring the interface.
   
   PR:	i386/96382, kern/100410, kern/122252, kern/116328
 
 Modified:
   stable/8/sys/dev/bge/if_bge.c
 Directory Properties:
   stable/8/sys/   (props changed)
   stable/8/sys/amd64/include/xen/   (props changed)
   stable/8/sys/cddl/contrib/opensolaris/   (props changed)
   stable/8/sys/contrib/dev/acpica/   (props changed)
   stable/8/sys/contrib/pf/   (props changed)
   stable/8/sys/dev/xen/xenpci/   (props changed)
 
 Modified: stable/8/sys/dev/bge/if_bge.c
 ==============================================================================
 --- stable/8/sys/dev/bge/if_bge.c	Tue Nov 17 14:23:09 2009	(r199380)
 +++ stable/8/sys/dev/bge/if_bge.c	Tue Nov 17 14:30:09 2009	(r199381)
 @@ -4218,6 +4218,16 @@ bge_stop(struct bge_softc *sc)
  
  	callout_stop(&sc->bge_stat_ch);
  
 +	/* Disable host interrupts. */
 +	BGE_SETBIT(sc, BGE_PCI_MISC_CTL, BGE_PCIMISCCTL_MASK_PCI_INTR);
 +	bge_writembx(sc, BGE_MBX_IRQ0_LO, 1);
 +
 +	/*
 +	 * Tell firmware we're shutting down.
 +	 */
 +	bge_stop_fw(sc);
 +	bge_sig_pre_reset(sc, BGE_RESET_STOP);
 +
  	/*
  	 * Disable all of the receiver blocks.
  	 */
 @@ -4257,16 +4267,6 @@ bge_stop(struct bge_softc *sc)
  		BGE_CLRBIT(sc, BGE_MARB_MODE, BGE_MARBMODE_ENABLE);
  	}
  
 -	/* Disable host interrupts. */
 -	BGE_SETBIT(sc, BGE_PCI_MISC_CTL, BGE_PCIMISCCTL_MASK_PCI_INTR);
 -	bge_writembx(sc, BGE_MBX_IRQ0_LO, 1);
 -
 -	/*
 -	 * Tell firmware we're shutting down.
 -	 */
 -
 -	bge_stop_fw(sc);
 -	bge_sig_pre_reset(sc, BGE_RESET_STOP);
  	bge_reset(sc);
  	bge_sig_legacy(sc, BGE_RESET_STOP);
  	bge_sig_post_reset(sc, BGE_RESET_STOP);
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/100410: commit references a PR
Date: Tue, 17 Nov 2009 20:48:53 +0000 (UTC)

 Author: bz
 Date: Tue Nov 17 20:48:32 2009
 New Revision: 199433
 URL: http://svn.freebsd.org/changeset/base/199433
 
 Log:
   MFC r198049:
   
     Immediately after clearing a pending callout that didn't make it due
     to the lock we hold, disable interrupts, and announce to the firmware
     that we are shutting down. Especially do this before disabling blocks.
   
     This makes some types of machines with asf enabled no longer hang upon
     boot, when we start configuring the interface.
   
   PR:	i386/96382, kern/100410, kern/122252, kern/116328
 
 Modified:
   stable/7/sys/dev/bge/if_bge.c
 Directory Properties:
   stable/7/sys/   (props changed)
   stable/7/sys/contrib/pf/   (props changed)
 
 Modified: stable/7/sys/dev/bge/if_bge.c
 ==============================================================================
 --- stable/7/sys/dev/bge/if_bge.c	Tue Nov 17 20:45:49 2009	(r199432)
 +++ stable/7/sys/dev/bge/if_bge.c	Tue Nov 17 20:48:32 2009	(r199433)
 @@ -4213,6 +4213,16 @@ bge_stop(struct bge_softc *sc)
  
  	callout_stop(&sc->bge_stat_ch);
  
 +	/* Disable host interrupts. */
 +	BGE_SETBIT(sc, BGE_PCI_MISC_CTL, BGE_PCIMISCCTL_MASK_PCI_INTR);
 +	bge_writembx(sc, BGE_MBX_IRQ0_LO, 1);
 +
 +	/*
 +	 * Tell firmware we're shutting down.
 +	 */
 +	bge_stop_fw(sc);
 +	bge_sig_pre_reset(sc, BGE_RESET_STOP);
 +
  	/*
  	 * Disable all of the receiver blocks.
  	 */
 @@ -4252,16 +4262,6 @@ bge_stop(struct bge_softc *sc)
  		BGE_CLRBIT(sc, BGE_MARB_MODE, BGE_MARBMODE_ENABLE);
  	}
  
 -	/* Disable host interrupts. */
 -	BGE_SETBIT(sc, BGE_PCI_MISC_CTL, BGE_PCIMISCCTL_MASK_PCI_INTR);
 -	bge_writembx(sc, BGE_MBX_IRQ0_LO, 1);
 -
 -	/*
 -	 * Tell firmware we're shutting down.
 -	 */
 -
 -	bge_stop_fw(sc);
 -	bge_sig_pre_reset(sc, BGE_RESET_STOP);
  	bge_reset(sc);
  	bge_sig_legacy(sc, BGE_RESET_STOP);
  	bge_sig_post_reset(sc, BGE_RESET_STOP);
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/100410: commit references a PR
Date: Tue, 17 Nov 2009 20:49:30 +0000 (UTC)

 Author: bz
 Date: Tue Nov 17 20:49:09 2009
 New Revision: 199434
 URL: http://svn.freebsd.org/changeset/base/199434
 
 Log:
   MFC r198049:
   
     Immediately after clearing a pending callout that didn't make it due
     to the lock we hold, disable interrupts, and announce to the firmware
     that we are shutting down. Especially do this before disabling blocks.
   
     This makes some types of machines with asf enabled no longer hang upon
     boot, when we start configuring the interface.
   
   PR:	i386/96382, kern/100410, kern/122252, kern/116328
 
 Modified:
   stable/6/sys/dev/bge/if_bge.c
 Directory Properties:
   stable/6/sys/   (props changed)
   stable/6/sys/conf/   (props changed)
   stable/6/sys/contrib/pf/   (props changed)
   stable/6/sys/dev/cxgb/   (props changed)
 
 Modified: stable/6/sys/dev/bge/if_bge.c
 ==============================================================================
 --- stable/6/sys/dev/bge/if_bge.c	Tue Nov 17 20:48:32 2009	(r199433)
 +++ stable/6/sys/dev/bge/if_bge.c	Tue Nov 17 20:49:09 2009	(r199434)
 @@ -4146,6 +4146,16 @@ bge_stop(struct bge_softc *sc)
  
  	callout_stop(&sc->bge_stat_ch);
  
 +	/* Disable host interrupts. */
 +	BGE_SETBIT(sc, BGE_PCI_MISC_CTL, BGE_PCIMISCCTL_MASK_PCI_INTR);
 +	bge_writembx(sc, BGE_MBX_IRQ0_LO, 1);
 +
 +	/*
 +	 * Tell firmware we're shutting down.
 +	 */
 +	bge_stop_fw(sc);
 +	bge_sig_pre_reset(sc, BGE_RESET_STOP);
 +
  	/*
  	 * Disable all of the receiver blocks.
  	 */
 @@ -4185,16 +4195,6 @@ bge_stop(struct bge_softc *sc)
  		BGE_CLRBIT(sc, BGE_MARB_MODE, BGE_MARBMODE_ENABLE);
  	}
  
 -	/* Disable host interrupts. */
 -	BGE_SETBIT(sc, BGE_PCI_MISC_CTL, BGE_PCIMISCCTL_MASK_PCI_INTR);
 -	bge_writembx(sc, BGE_MBX_IRQ0_LO, 1);
 -
 -	/*
 -	 * Tell firmware we're shutting down.
 -	 */
 -
 -	bge_stop_fw(sc);
 -	bge_sig_pre_reset(sc, BGE_RESET_STOP);
  	bge_reset(sc);
  	bge_sig_legacy(sc, BGE_RESET_STOP);
  	bge_sig_post_reset(sc, BGE_RESET_STOP);
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 
State-Changed-From-To: open->closed 
State-Changed-By: bz 
State-Changed-When: Tue Nov 17 20:53:27 UTC 2009 
State-Changed-Why:  
There have been a lot of driver updates since this report. 
The latest commit seems to eliminate boot hangup problems 
on HP servers. 
Please also find the hw.bge.allow_asf tunable in man 4 bge. 
Thanks a lot for reporting; please feel free to re-open should 
the problems still exist after an update to a latest support branch. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=100410 
>Unformatted:
