From nobody@FreeBSD.org  Mon Nov 12 19:19:37 2012
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52])
	by hub.freebsd.org (Postfix) with ESMTP id 5331659C
	for <freebsd-gnats-submit@FreeBSD.org>; Mon, 12 Nov 2012 19:19:37 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from red.freebsd.org (red.freebsd.org [IPv6:2001:4f8:fff6::22])
	by mx1.freebsd.org (Postfix) with ESMTP id 1F8698FC13
	for <freebsd-gnats-submit@FreeBSD.org>; Mon, 12 Nov 2012 19:19:37 +0000 (UTC)
Received: from red.freebsd.org (localhost [127.0.0.1])
	by red.freebsd.org (8.14.5/8.14.5) with ESMTP id qACJJaxU006784
	for <freebsd-gnats-submit@FreeBSD.org>; Mon, 12 Nov 2012 19:19:36 GMT
	(envelope-from nobody@red.freebsd.org)
Received: (from nobody@localhost)
	by red.freebsd.org (8.14.5/8.14.5/Submit) id qACJJa9t006783;
	Mon, 12 Nov 2012 19:19:36 GMT
	(envelope-from nobody)
Message-Id: <201211121919.qACJJa9t006783@red.freebsd.org>
Date: Mon, 12 Nov 2012 19:19:36 GMT
From: Garrett Cooper <yaneurabeya@gmail.com>
To: freebsd-gnats-submit@FreeBSD.org
Subject: [PATCH] [cxgbe] Add hw.cxgbe.force_firmware_install tunable to force firmware "updates" at module load
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         173584
>Category:       kern
>Synopsis:       [PATCH] [cxgbe] Add hw.cxgbe.force_firmware_install tunable to force firmware "updates" at module load
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    np
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Mon Nov 12 19:20:00 UTC 2012
>Closed-Date:    Tue Mar 18 18:37:56 UTC 2014
>Last-Modified:  Tue Mar 18 18:37:56 UTC 2014
>Originator:     Garrett Cooper
>Release:        9.1-STABLE
>Organization:
EMC Isilon
>Environment:
FreeBSD wf158.west.isilon.com 9.1-PRERELEASE FreeBSD 9.1-PRERELEASE #2: Tue Nov  6 14:00:42 PST 2012     root@wf158.west.isilon.com:/usr/obj/usr/src/sys/GENERIC  amd64
>Description:
The attached patch adds a tunable  which forces a firmware update in order to ensure that when the driver is loaded, if it's running a newer version of firmware it can be downgraded to an older version. Example:

# kenv hw.cxgbe.force_firmware_install=1
# kldload if_cxgbe
# sysctl dev.t4nex.0.firmware_version
dev.t4nex.0.firmware_version: 1.6.2.0

The firmware version provided on the card was newer before (1.7.x IIRC).
>How-To-Repeat:

>Fix:


Patch attached with submission follows:

Index: sys/dev/cxgbe/t4_main.c
===================================================================
--- sys/dev/cxgbe/t4_main.c	(revision 242655)
+++ sys/dev/cxgbe/t4_main.c	(working copy)
@@ -234,6 +234,9 @@
 static int t4_fcoecaps_allowed = 0;
 TUNABLE_INT("hw.cxgbe.fcoecaps_allowed", &t4_fcoecaps_allowed);
 
+static int t4_force_firmware_install = 0;
+TUNABLE_INT("hw.cxgbe.force_firmware_install", &t4_force_firmware_install);
+
 struct intrs_and_queues {
 	int intr_type;		/* INTx, MSI, or MSI-X */
 	int nirq;		/* Number of vectors */
@@ -1576,9 +1579,10 @@
 		/*
 		 * Always upgrade, even for minor/micro/build mismatches.
 		 * Downgrade only for a major version mismatch or if
-		 * force_firmware_install was specified.
+		 * t4_force_firmware_install was specified.
 		 */
-		if (fw != NULL && (rc < 0 || v > sc->params.fw_vers)) {
+		if (fw != NULL &&
+		    (t4_force_firmware_install || rc < 0 || v > sc->params.fw_vers)) {
 			device_printf(sc->dev,
 			    "installing firmware %d.%d.%d.%d on card.\n",
 			    G_FW_HDR_FW_VER_MAJOR(v), G_FW_HDR_FW_VER_MINOR(v),


>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->np 
Responsible-Changed-By: np 
Responsible-Changed-When: Mon Nov 12 19:30:56 UTC 2012 
Responsible-Changed-Why:  
I'll take a look at this. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=173584 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/173584: commit references a PR
Date: Tue, 26 Feb 2013 20:36:08 +0000 (UTC)

 Author: np
 Date: Tue Feb 26 20:35:54 2013
 New Revision: 247347
 URL: http://svnweb.freebsd.org/changeset/base/247347
 
 Log:
   cxgbe(4): Consider all the API versions of the interfaces exported by
   the firmware (instead of just the main firmware version) when evaluating
   firmware compatibility.  Document the new "hw.cxgbe.fw_install" knob
   being introduced here.
   
   This should fix kern/173584 too.  Setting hw.cxgbe.fw_install=2 will
   mostly do what was requested in the PR but it's a bit more intelligent
   in that it won't reinstall the same firmware repeatedly if the knob is
   left set.
   
   PR:		kern/173584
   MFC after:	5 days
 
 Modified:
   head/share/man/man4/cxgbe.4
   head/sys/dev/cxgbe/common/common.h
   head/sys/dev/cxgbe/t4_main.c
 
 Modified: head/share/man/man4/cxgbe.4
 ==============================================================================
 --- head/share/man/man4/cxgbe.4	Tue Feb 26 20:35:40 2013	(r247346)
 +++ head/share/man/man4/cxgbe.4	Tue Feb 26 20:35:54 2013	(r247347)
 @@ -178,6 +178,15 @@ Bit 0 represents INTx (line interrupts),
  The default is 7 (all allowed).
  The driver will select the best possible type out of the allowed types by
  itself.
 +.It Va hw.cxgbe.fw_install
 +0 prohibits the driver from installing a firmware on the card.
 +1 allows the driver to install a new firmware if internal driver
 +heuristics indicate that the new firmware is preferable to the one
 +already on the card.
 +2 instructs the driver to always install the new firmware on the card as
 +long as it is compatible with the driver and is a different version than
 +the one already on the card.
 +The default is 1.
  .It Va hw.cxgbe.config_file
  Select a pre-packaged device configuration file.
  A configuration file contains a recipe for partitioning and configuring the
 
 Modified: head/sys/dev/cxgbe/common/common.h
 ==============================================================================
 --- head/sys/dev/cxgbe/common/common.h	Tue Feb 26 20:35:40 2013	(r247346)
 +++ head/sys/dev/cxgbe/common/common.h	Tue Feb 26 20:35:54 2013	(r247347)
 @@ -68,6 +68,11 @@ enum {
  #define FW_VERSION_MICRO 4
  #define FW_VERSION_BUILD 0
  
 +#define FW_VERSION (V_FW_HDR_FW_VER_MAJOR(FW_VERSION_MAJOR) | \
 +    V_FW_HDR_FW_VER_MINOR(FW_VERSION_MINOR) | \
 +    V_FW_HDR_FW_VER_MICRO(FW_VERSION_MICRO) | \
 +    V_FW_HDR_FW_VER_BUILD(FW_VERSION_BUILD))
 +
  struct port_stats {
  	u64 tx_octets;            /* total # of octets in good frames */
  	u64 tx_frames;            /* all good frames */
 
 Modified: head/sys/dev/cxgbe/t4_main.c
 ==============================================================================
 --- head/sys/dev/cxgbe/t4_main.c	Tue Feb 26 20:35:40 2013	(r247346)
 +++ head/sys/dev/cxgbe/t4_main.c	Tue Feb 26 20:35:54 2013	(r247347)
 @@ -213,6 +213,13 @@ static char t4_cfg_file[32] = "default";
  TUNABLE_STR("hw.cxgbe.config_file", t4_cfg_file, sizeof(t4_cfg_file));
  
  /*
 + * Firmware auto-install by driver during attach (0, 1, 2 = prohibited, allowed,
 + * encouraged respectively).
 + */
 +static unsigned int t4_fw_install = 1;
 +TUNABLE_INT("hw.cxgbe.fw_install", &t4_fw_install);
 +
 +/*
   * ASIC features that will be used.  Disable the ones you don't want so that the
   * chip resources aren't wasted on features that will not be used.
   */
 @@ -1503,6 +1510,33 @@ allocate:
  }
  
  /*
 + * Is the given firmware compatible with the one the driver was compiled with?
 + */
 +static int
 +fw_compatible(const struct fw_hdr *hdr)
 +{
 +
 +	if (hdr->fw_ver == htonl(FW_VERSION))
 +		return (1);
 +
 +	/*
 +	 * XXX: Is this too conservative?  Perhaps I should limit this to the
 +	 * features that are supported in the driver.
 +	 */
 +	if (hdr->intfver_nic == FW_HDR_INTFVER_NIC &&
 +	    hdr->intfver_vnic == FW_HDR_INTFVER_VNIC &&
 +	    hdr->intfver_ofld == FW_HDR_INTFVER_OFLD &&
 +	    hdr->intfver_ri == FW_HDR_INTFVER_RI &&
 +	    hdr->intfver_iscsipdu == FW_HDR_INTFVER_ISCSIPDU &&
 +	    hdr->intfver_iscsi == FW_HDR_INTFVER_ISCSI &&
 +	    hdr->intfver_fcoepdu == FW_HDR_INTFVER_FCOEPDU &&
 +	    hdr->intfver_fcoe == FW_HDR_INTFVER_FCOEPDU)
 +		return (1);
 +
 +	return (0);
 +}
 +
 +/*
   * Install a compatible firmware (if required), establish contact with it (by
   * saying hello), and reset the device.  If we end up as the master driver,
   * partition adapter resources by providing a configuration file to the
 @@ -1512,84 +1546,99 @@ static int
  prep_firmware(struct adapter *sc)
  {
  	const struct firmware *fw = NULL, *cfg = NULL, *default_cfg;
 -	int rc;
 +	int rc, card_fw_usable, kld_fw_usable;
  	enum dev_state state;
 +	struct fw_hdr *card_fw;
 +	const struct fw_hdr *kld_fw;
  
  	default_cfg = firmware_get(T4_CFGNAME);
  
 -	/* Check firmware version and install a different one if necessary */
 -	rc = t4_check_fw_version(sc);
 -	snprintf(sc->fw_version, sizeof(sc->fw_version), "%u.%u.%u.%u",
 -	    G_FW_HDR_FW_VER_MAJOR(sc->params.fw_vers),
 -	    G_FW_HDR_FW_VER_MINOR(sc->params.fw_vers),
 -	    G_FW_HDR_FW_VER_MICRO(sc->params.fw_vers),
 -	    G_FW_HDR_FW_VER_BUILD(sc->params.fw_vers));
 -	if (rc != 0) {
 -		uint32_t v = 0;
 -
 -		fw = firmware_get(T4_FWNAME);
 -		if (fw != NULL) {
 -			const struct fw_hdr *hdr = (const void *)fw->data;
 +	/* Read the header of the firmware on the card */
 +	card_fw = malloc(sizeof(*card_fw), M_CXGBE, M_ZERO | M_WAITOK);
 +	rc = -t4_read_flash(sc, FLASH_FW_START,
 +	    sizeof (*card_fw) / sizeof (uint32_t), (uint32_t *)card_fw, 1);
 +	if (rc == 0)
 +		card_fw_usable = fw_compatible((const void*)card_fw);
 +	else {
 +		device_printf(sc->dev,
 +		    "Unable to read card's firmware header: %d\n", rc);
 +		card_fw_usable = 0;
 +	}
  
 -			v = ntohl(hdr->fw_ver);
 +	/* This is the firmware in the KLD */
 +	fw = firmware_get(T4_FWNAME);
 +	if (fw != NULL) {
 +		kld_fw = (const void *)fw->data;
 +		kld_fw_usable = fw_compatible(kld_fw);
 +	} else {
 +		kld_fw = NULL;
 +		kld_fw_usable = 0;
 +	}
  
 -			/*
 -			 * The firmware module will not be used if it isn't the
 -			 * same major version as what the driver was compiled
 -			 * with.
 -			 */
 -			if (G_FW_HDR_FW_VER_MAJOR(v) != FW_VERSION_MAJOR) {
 -				device_printf(sc->dev,
 -				    "Found firmware image but version %d "
 -				    "can not be used with this driver (%d)\n",
 -				    G_FW_HDR_FW_VER_MAJOR(v), FW_VERSION_MAJOR);
 +	/*
 +	 * Short circuit for the common case: the firmware on the card is an
 +	 * exact match and the KLD is an exact match too, or it's
 +	 * absent/incompatible, or we're prohibited from using it.  Note that
 +	 * t4_fw_install = 2 is ignored here -- use cxgbetool loadfw if you want
 +	 * to reinstall the same firmware as the one on the card.
 +	 */
 +	if (card_fw_usable && card_fw->fw_ver == htonl(FW_VERSION) &&
 +	    (!kld_fw_usable || kld_fw->fw_ver == htonl(FW_VERSION) ||
 +	    t4_fw_install == 0))
 +		goto hello;
 +
 +	if (kld_fw_usable && (!card_fw_usable ||
 +	    ntohl(kld_fw->fw_ver) > ntohl(card_fw->fw_ver) ||
 +	    (t4_fw_install == 2 && kld_fw->fw_ver != card_fw->fw_ver))) {
 +		uint32_t v = ntohl(kld_fw->fw_ver);
  
 -				firmware_put(fw, FIRMWARE_UNLOAD);
 -				fw = NULL;
 -			}
 -		}
 +		device_printf(sc->dev,
 +		    "installing firmware %d.%d.%d.%d on card.\n",
 +		    G_FW_HDR_FW_VER_MAJOR(v), G_FW_HDR_FW_VER_MINOR(v),
 +		    G_FW_HDR_FW_VER_MICRO(v), G_FW_HDR_FW_VER_BUILD(v));
  
 -		if (fw == NULL && rc < 0) {
 -			device_printf(sc->dev, "No usable firmware. "
 -			    "card has %d.%d.%d, driver compiled with %d.%d.%d",
 -			    G_FW_HDR_FW_VER_MAJOR(sc->params.fw_vers),
 -			    G_FW_HDR_FW_VER_MINOR(sc->params.fw_vers),
 -			    G_FW_HDR_FW_VER_MICRO(sc->params.fw_vers),
 -			    FW_VERSION_MAJOR, FW_VERSION_MINOR,
 -			    FW_VERSION_MICRO);
 -			rc = EAGAIN;
 +		rc = -t4_load_fw(sc, fw->data, fw->datasize);
 +		if (rc != 0) {
 +			device_printf(sc->dev,
 +			    "failed to install firmware: %d\n", rc);
  			goto done;
  		}
  
 -		/*
 -		 * Always upgrade, even for minor/micro/build mismatches.
 -		 * Downgrade only for a major version mismatch or if
 -		 * force_firmware_install was specified.
 -		 */
 -		if (fw != NULL && (rc < 0 || v > sc->params.fw_vers)) {
 -			device_printf(sc->dev,
 -			    "installing firmware %d.%d.%d.%d on card.\n",
 -			    G_FW_HDR_FW_VER_MAJOR(v), G_FW_HDR_FW_VER_MINOR(v),
 -			    G_FW_HDR_FW_VER_MICRO(v), G_FW_HDR_FW_VER_BUILD(v));
 -
 -			rc = -t4_load_fw(sc, fw->data, fw->datasize);
 -			if (rc != 0) {
 -				device_printf(sc->dev,
 -				    "failed to install firmware: %d\n", rc);
 -				goto done;
 -			} else {
 -				/* refresh */
 -				(void) t4_check_fw_version(sc);
 -				snprintf(sc->fw_version,
 -				    sizeof(sc->fw_version), "%u.%u.%u.%u",
 -				    G_FW_HDR_FW_VER_MAJOR(sc->params.fw_vers),
 -				    G_FW_HDR_FW_VER_MINOR(sc->params.fw_vers),
 -				    G_FW_HDR_FW_VER_MICRO(sc->params.fw_vers),
 -				    G_FW_HDR_FW_VER_BUILD(sc->params.fw_vers));
 -			}
 -		}
 +		/* Installed successfully, update the cached header too. */
 +		memcpy(card_fw, kld_fw, sizeof(*card_fw));
 +		card_fw_usable = 1;
 +	}
 +
 +	if (!card_fw_usable) {
 +		uint32_t c, k;
 +
 +		c = ntohl(card_fw->fw_ver);
 +		k = kld_fw ? ntohl(kld_fw->fw_ver) : 0;
 +
 +		device_printf(sc->dev, "Cannot find a usable firmware: "
 +		    "fw_install %d, driver compiled with %d.%d.%d.%d, "
 +		    "card has %d.%d.%d.%d, KLD has %d.%d.%d.%d\n",
 +		    t4_fw_install,
 +		    G_FW_HDR_FW_VER_MAJOR(FW_VERSION),
 +		    G_FW_HDR_FW_VER_MINOR(FW_VERSION),
 +		    G_FW_HDR_FW_VER_MICRO(FW_VERSION),
 +		    G_FW_HDR_FW_VER_BUILD(FW_VERSION),
 +		    G_FW_HDR_FW_VER_MAJOR(c), G_FW_HDR_FW_VER_MINOR(c),
 +		    G_FW_HDR_FW_VER_MICRO(c), G_FW_HDR_FW_VER_BUILD(c),
 +		    G_FW_HDR_FW_VER_MAJOR(k), G_FW_HDR_FW_VER_MINOR(k),
 +		    G_FW_HDR_FW_VER_MICRO(k), G_FW_HDR_FW_VER_BUILD(k));
 +		goto done;
  	}
  
 +hello:
 +	/* We're using whatever's on the card and it's known to be good. */
 +	sc->params.fw_vers = ntohl(card_fw->fw_ver);
 +	snprintf(sc->fw_version, sizeof(sc->fw_version), "%u.%u.%u.%u",
 +	    G_FW_HDR_FW_VER_MAJOR(sc->params.fw_vers),
 +	    G_FW_HDR_FW_VER_MINOR(sc->params.fw_vers),
 +	    G_FW_HDR_FW_VER_MICRO(sc->params.fw_vers),
 +	    G_FW_HDR_FW_VER_BUILD(sc->params.fw_vers));
 +
  	/* Contact firmware.  */
  	rc = t4_fw_hello(sc, sc->mbox, sc->mbox, MASTER_MAY, &state);
  	if (rc < 0) {
 @@ -1639,6 +1688,7 @@ prep_firmware(struct adapter *sc)
  	sc->flags |= FW_OK;
  
  done:
 +	free(card_fw, M_CXGBE);
  	if (fw != NULL)
  		firmware_put(fw, FIRMWARE_UNLOAD);
  	if (cfg != NULL)
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 
State-Changed-From-To: open->closed 
State-Changed-By: np 
State-Changed-When: Tue Mar 18 18:36:27 UTC 2014 
State-Changed-Why:  
This was fixed more than 1 year back in r247347. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=173584 
>Unformatted:
