From nobody@FreeBSD.org  Wed Jul 24 07:20:19 2013
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1])
	by hub.freebsd.org (Postfix) with ESMTP id A8019FFF
	for <freebsd-gnats-submit@FreeBSD.org>; Wed, 24 Jul 2013 07:20:19 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from oldred.freebsd.org (oldred.freebsd.org [8.8.178.121])
	(using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
	(No client certificate requested)
	by mx1.freebsd.org (Postfix) with ESMTPS id 9A1862DEE
	for <freebsd-gnats-submit@FreeBSD.org>; Wed, 24 Jul 2013 07:20:19 +0000 (UTC)
Received: from oldred.freebsd.org ([127.0.1.6])
	by oldred.freebsd.org (8.14.5/8.14.7) with ESMTP id r6O7KJ8T075502
	for <freebsd-gnats-submit@FreeBSD.org>; Wed, 24 Jul 2013 07:20:19 GMT
	(envelope-from nobody@oldred.freebsd.org)
Received: (from nobody@localhost)
	by oldred.freebsd.org (8.14.5/8.14.5/Submit) id r6O7KJiH075489;
	Wed, 24 Jul 2013 07:20:19 GMT
	(envelope-from nobody)
Message-Id: <201307240720.r6O7KJiH075489@oldred.freebsd.org>
Date: Wed, 24 Jul 2013 07:20:19 GMT
From: Shahar Klein <shahark@mellanox.com>
To: freebsd-gnats-submit@FreeBSD.org
Subject: Kernel crash on ifdown and kldunload mlxen
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         180791
>Category:       kern
>Synopsis:       [ofed] [patch] Kernel crash on ifdown and kldunload mlxen
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    jhb
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Wed Jul 24 07:30:00 UTC 2013
>Closed-Date:    Wed Aug 07 18:03:11 UTC 2013
>Last-Modified:  Wed Aug 07 18:03:11 UTC 2013
>Originator:     Shahar Klein
>Release:        9.1
>Organization:
Mellanox
>Environment:
FreeBSD 9.1-RELEASE #0 r+95f2241: Mon Jul 15 16:48:47 IDT 2013   .../sys/GENERIC  amd64
>Description:
Two related problems:

1. ifdown on an eth interface(using mlxen) panics the kernel.
2. kldunload mlxen when the interface is up - crashes the kernel
>How-To-Repeat:
1. bring a Mellanox HCA up and try to ifdown it.
2. kldunload mlxen when an eth interface is up(using mlxen)
>Fix:
### first fix ####

<pre>
diff --git a/sys/ofed/drivers/net/mlx4/en_netdev.c b/sys/ofed/drivers/net/mlx4/en_netdev.c
index acb2c12..157f05d 100755
--- a/sys/ofed/drivers/net/mlx4/en_netdev.c
+++ b/sys/ofed/drivers/net/mlx4/en_netdev.c
@@ -495,11 +495,6 @@ static void mlx4_en_do_get_stats(struct work_struct *work)

                queue_delayed_work(mdev->workqueue, &priv->stats_task, STATS_DELAY);
        }
-   if (mdev->mac_removed[MLX4_MAX_PORTS + 1 - priv->port]) {
-           panic("mlx4_en_do_get_stats: Unexpected mac removed for %d\n",
-               priv->port);
-           mdev->mac_removed[MLX4_MAX_PORTS + 1 - priv->port] = 0;
-   }
        mutex_unlock(&mdev->state_lock);
 }
</pre>



### second fix ###

<pre>
diff --git a/sys/ofed/drivers/net/mlx4/en_netdev.c b/sys/ofed/drivers/net/mlx4/en_netdev.c
index 260952b..807858a 100755
--- a/sys/ofed/drivers/net/mlx4/en_netdev.c
+++ b/sys/ofed/drivers/net/mlx4/en_netdev.c
@@ -921,6 +921,8 @@ void mlx4_en_destroy_netdev(struct net_device *dev)
        if (priv->sysctl)
                sysctl_ctx_free(&priv->conf_ctx);

+        mlx4_en_stop_port(dev);
+
        cancel_delayed_work(&priv->stats_task);
        /* flush any pending task for this netdev */
        flush_workqueue(mdev->workqueue);
</pre>

>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-amd64->freebsd-net 
Responsible-Changed-By: linimon 
Responsible-Changed-When: Wed Jul 24 21:48:47 UTC 2013 
Responsible-Changed-Why:  
relcassify. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=180791 

From: John Baldwin <jhb@freebsd.org>
To: bug-followup@freebsd.org,
 shahark@mellanox.com
Cc:  
Subject: Re: kern/180791: [ofed] [patch] Kernel crash on ifdown and kldunload mlxen
Date: Thu, 25 Jul 2013 15:18:06 -0400

 Thanks.  One note is that it seems like the state_lock should be held when
 stop is called.  Also, the callout used for stats should be drained during
 detach.  (If you use callot_init_mtx() instead of callout_init() then
 callout_stop() under the lock is race-free, though I'd still do the
 callout_drain() during detach to be safe.)
 
 Also, I had some other changes I made to this file to make the locking more
 consistent with other NIC drivers in the tree.  Can you look at this and
 test this patch please?
 
 Index: en_netdev.c
 ===================================================================
 --- en_netdev.c	(revision 253547)
 +++ en_netdev.c	(working copy)
 @@ -495,11 +495,6 @@ static void mlx4_en_do_get_stats(struct work_struc
  
  		queue_delayed_work(mdev->workqueue, &priv->stats_task, STATS_DELAY);
  	}
 -	if (mdev->mac_removed[MLX4_MAX_PORTS + 1 - priv->port]) {
 -		panic("mlx4_en_do_get_stats: Unexpected mac removed for %d\n",
 -		    priv->port);
 -		mdev->mac_removed[MLX4_MAX_PORTS + 1 - priv->port] = 0;
 -	}
  	mutex_unlock(&mdev->state_lock);
  }
  
 @@ -688,8 +683,8 @@ int mlx4_en_start_port(struct net_device *dev)
  	mlx4_en_set_multicast(dev);
  
  	/* Enable the queues. */
 -	atomic_clear_int(&dev->if_drv_flags, IFF_DRV_OACTIVE);
 -	atomic_set_int(&dev->if_drv_flags, IFF_DRV_RUNNING);
 +	dev->if_drv_flags &= ~IFF_DRV_OACTIVE;
 +	dev->if_drv_flags |= IFF_DRV_RUNNING;
  
  	callout_reset(&priv->watchdog_timer, MLX4_EN_WATCHDOG_TIMEOUT,
  	    mlx4_en_watchdog_timeout, priv);
 @@ -761,7 +756,7 @@ void mlx4_en_stop_port(struct net_device *dev)
  
  	callout_stop(&priv->watchdog_timer);
  
 -	atomic_clear_int(&dev->if_drv_flags, IFF_DRV_RUNNING);
 +	dev->if_drv_flags &= ~(IFF_DRV_RUNNING | IFF_DRV_OACTIVE);
  }
  
  static void mlx4_en_restart(struct work_struct *work)
 @@ -802,19 +797,30 @@ mlx4_en_init(void *arg)
  {
  	struct mlx4_en_priv *priv;
  	struct mlx4_en_dev *mdev;
 +
 +	priv = arg;
 +	mdev = priv->mdev;
 +	mutex_lock(&mdev->state_lock);
 +	mlx4_en_init_locked(priv);
 +	mutex_unlock(&mdev->state_lock);
 +}
 +
 +static void
 +mlx4_en_init_locked(struct mlx4_en_priv *priv)
 +{
 +
 +	struct mlx4_en_dev *mdev;
  	struct ifnet *dev;
  	int i;
  
 -	priv = arg;
  	dev = priv->dev;
  	mdev = priv->mdev;
 -	mutex_lock(&mdev->state_lock);
  	if (dev->if_drv_flags & IFF_DRV_RUNNING)
  		mlx4_en_stop_port(dev);
  
  	if (!mdev->device_up) {
  		en_err(priv, "Cannot open - device down/disabled\n");
 -		goto out;
 +		return;
  	}
  
  	/* Reset HW statistics and performance counters */
 @@ -835,9 +841,6 @@ mlx4_en_init(void *arg)
  	mlx4_en_set_default_moderation(priv);
  	if (mlx4_en_start_port(dev))
  		en_err(priv, "Failed starting port:%d\n", priv->port);
 -
 -out:
 -	mutex_unlock(&mdev->state_lock);
  }
  
  void mlx4_en_free_resources(struct mlx4_en_priv *priv)
 @@ -927,9 +930,14 @@ void mlx4_en_destroy_netdev(struct net_device *dev
  	if (priv->sysctl)
  		sysctl_ctx_free(&priv->conf_ctx);
  
 +	mutex_lock(&mdev->state_lock);
 +	mlx4_en_stop_port(dev);
 +	mutex_unlock(&mdev->state_lock);
 +
  	cancel_delayed_work(&priv->stats_task);
  	/* flush any pending task for this netdev */
  	flush_workqueue(mdev->workqueue);
 +	callout_drain(&priv->watchdog_timer);
  
  	/* Detach the netdev so tasks would not attempt to access it */
  	mutex_lock(&mdev->state_lock);
 @@ -1091,25 +1099,25 @@ static int mlx4_en_ioctl(struct ifnet *dev, u_long
  		error = -mlx4_en_change_mtu(dev, ifr->ifr_mtu);
  		break;
  	case SIOCSIFFLAGS:
 +		mutex_lock(&mdev->state_lock);
  		if (dev->if_flags & IFF_UP) {
 -			if ((dev->if_drv_flags & IFF_DRV_RUNNING) == 0) {
 -				mutex_lock(&mdev->state_lock);
 +			if ((dev->if_drv_flags & IFF_DRV_RUNNING) == 0)
  				mlx4_en_start_port(dev);
 -				mutex_unlock(&mdev->state_lock);
 -			} else
 +			else
  				mlx4_en_set_multicast(dev);
  		} else {
 -			mutex_lock(&mdev->state_lock);
  			if (dev->if_drv_flags & IFF_DRV_RUNNING) {
  				mlx4_en_stop_port(dev);
  				if_link_state_change(dev, LINK_STATE_DOWN);
  			}
 -			mutex_unlock(&mdev->state_lock);
  		}
 +		mutex_unlock(&mdev->state_lock);
  		break;
  	case SIOCADDMULTI:
  	case SIOCDELMULTI:
 +		mutex_lock(&mdev->state_lock);
  		mlx4_en_set_multicast(dev);
 +		mutex_unlock(&mdev->state_lock);
  		break;
  	case SIOCSIFMEDIA:
  	case SIOCGIFMEDIA:
 @@ -1116,6 +1124,7 @@ static int mlx4_en_ioctl(struct ifnet *dev, u_long
  		error = ifmedia_ioctl(dev, ifr, &priv->media, command);
  		break;
  	case SIOCSIFCAP:
 +		mutex_lock(&mdev->state_lock);
  		mask = ifr->ifr_reqcap ^ dev->if_capenable;
  		if (mask & IFCAP_HWCSUM)
  			dev->if_capenable ^= IFCAP_HWCSUM;
 @@ -1130,7 +1139,8 @@ static int mlx4_en_ioctl(struct ifnet *dev, u_long
  		if (mask & IFCAP_WOL_MAGIC)
  			dev->if_capenable ^= IFCAP_WOL_MAGIC;
  		if (dev->if_drv_flags & IFF_DRV_RUNNING)
 -			mlx4_en_init(priv);
 +			mlx4_en_init_locked(priv);
 +		mutex_unlock(&mdev->state_lock);
  		VLAN_CAPABILITIES(dev);
  		break;
  	default:
 
 
 -- 
 John Baldwin

From: Shahar Klein <shahark@mellanox.com>
To: John Baldwin <jhb@freebsd.org>, "bug-followup@freebsd.org"
	<bug-followup@freebsd.org>
Cc: Oded Shanoon <odeds@mellanox.com>
Subject: RE: kern/180791: [ofed] [patch] Kernel crash on ifdown and
 kldunload mlxen
Date: Mon, 29 Jul 2013 06:46:37 +0000

 Thanks John,
 
 I've reviewed and tested this patch and it's working fine.
 
 Is this patch going to both 10.0 and 9.2?
 
 10x
 Shahar
 
 -----Original Message-----
 From: John Baldwin [mailto:jhb@freebsd.org]=20
 Sent: Thursday, July 25, 2013 10:39 PM
 To: bug-followup@freebsd.org; Shahar Klein
 Subject: Re: kern/180791: [ofed] [patch] Kernel crash on ifdown and kldunlo=
 ad mlxen
 
 Thanks.  One note is that it seems like the state_lock should be held when =
 stop is called.  Also, the callout used for stats should be drained during =
 detach.  (If you use callot_init_mtx() instead of callout_init() then
 callout_stop() under the lock is race-free, though I'd still do the
 callout_drain() during detach to be safe.)
 
 Also, I had some other changes I made to this file to make the locking more=
  consistent with other NIC drivers in the tree.  Can you look at this and t=
 est this patch please?
 
 Index: en_netdev.c
 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
 --- en_netdev.c	(revision 253547)
 +++ en_netdev.c	(working copy)
 @@ -495,11 +495,6 @@ static void mlx4_en_do_get_stats(struct work_struc
 =20
  		queue_delayed_work(mdev->workqueue, &priv->stats_task, STATS_DELAY);
  	}
 -	if (mdev->mac_removed[MLX4_MAX_PORTS + 1 - priv->port]) {
 -		panic("mlx4_en_do_get_stats: Unexpected mac removed for %d\n",
 -		    priv->port);
 -		mdev->mac_removed[MLX4_MAX_PORTS + 1 - priv->port] =3D 0;
 -	}
  	mutex_unlock(&mdev->state_lock);
  }
 =20
 @@ -688,8 +683,8 @@ int mlx4_en_start_port(struct net_device *dev)
  	mlx4_en_set_multicast(dev);
 =20
  	/* Enable the queues. */
 -	atomic_clear_int(&dev->if_drv_flags, IFF_DRV_OACTIVE);
 -	atomic_set_int(&dev->if_drv_flags, IFF_DRV_RUNNING);
 +	dev->if_drv_flags &=3D ~IFF_DRV_OACTIVE;
 +	dev->if_drv_flags |=3D IFF_DRV_RUNNING;
 =20
  	callout_reset(&priv->watchdog_timer, MLX4_EN_WATCHDOG_TIMEOUT,
  	    mlx4_en_watchdog_timeout, priv);
 @@ -761,7 +756,7 @@ void mlx4_en_stop_port(struct net_device *dev)
 =20
  	callout_stop(&priv->watchdog_timer);
 =20
 -	atomic_clear_int(&dev->if_drv_flags, IFF_DRV_RUNNING);
 +	dev->if_drv_flags &=3D ~(IFF_DRV_RUNNING | IFF_DRV_OACTIVE);
  }
 =20
  static void mlx4_en_restart(struct work_struct *work) @@ -802,19 +797,30 @=
 @ mlx4_en_init(void *arg)  {
  	struct mlx4_en_priv *priv;
  	struct mlx4_en_dev *mdev;
 +
 +	priv =3D arg;
 +	mdev =3D priv->mdev;
 +	mutex_lock(&mdev->state_lock);
 +	mlx4_en_init_locked(priv);
 +	mutex_unlock(&mdev->state_lock);
 +}
 +
 +static void
 +mlx4_en_init_locked(struct mlx4_en_priv *priv) {
 +
 +	struct mlx4_en_dev *mdev;
  	struct ifnet *dev;
  	int i;
 =20
 -	priv =3D arg;
  	dev =3D priv->dev;
  	mdev =3D priv->mdev;
 -	mutex_lock(&mdev->state_lock);
  	if (dev->if_drv_flags & IFF_DRV_RUNNING)
  		mlx4_en_stop_port(dev);
 =20
  	if (!mdev->device_up) {
  		en_err(priv, "Cannot open - device down/disabled\n");
 -		goto out;
 +		return;
  	}
 =20
  	/* Reset HW statistics and performance counters */ @@ -835,9 +841,6 @@ ml=
 x4_en_init(void *arg)
  	mlx4_en_set_default_moderation(priv);
  	if (mlx4_en_start_port(dev))
  		en_err(priv, "Failed starting port:%d\n", priv->port);
 -
 -out:
 -	mutex_unlock(&mdev->state_lock);
  }
 =20
  void mlx4_en_free_resources(struct mlx4_en_priv *priv) @@ -927,9 +930,14 @=
 @ void mlx4_en_destroy_netdev(struct net_device *dev
  	if (priv->sysctl)
  		sysctl_ctx_free(&priv->conf_ctx);
 =20
 +	mutex_lock(&mdev->state_lock);
 +	mlx4_en_stop_port(dev);
 +	mutex_unlock(&mdev->state_lock);
 +
  	cancel_delayed_work(&priv->stats_task);
  	/* flush any pending task for this netdev */
  	flush_workqueue(mdev->workqueue);
 +	callout_drain(&priv->watchdog_timer);
 =20
  	/* Detach the netdev so tasks would not attempt to access it */
  	mutex_lock(&mdev->state_lock);
 @@ -1091,25 +1099,25 @@ static int mlx4_en_ioctl(struct ifnet *dev, u_long
  		error =3D -mlx4_en_change_mtu(dev, ifr->ifr_mtu);
  		break;
  	case SIOCSIFFLAGS:
 +		mutex_lock(&mdev->state_lock);
  		if (dev->if_flags & IFF_UP) {
 -			if ((dev->if_drv_flags & IFF_DRV_RUNNING) =3D=3D 0) {
 -				mutex_lock(&mdev->state_lock);
 +			if ((dev->if_drv_flags & IFF_DRV_RUNNING) =3D=3D 0)
  				mlx4_en_start_port(dev);
 -				mutex_unlock(&mdev->state_lock);
 -			} else
 +			else
  				mlx4_en_set_multicast(dev);
  		} else {
 -			mutex_lock(&mdev->state_lock);
  			if (dev->if_drv_flags & IFF_DRV_RUNNING) {
  				mlx4_en_stop_port(dev);
  				if_link_state_change(dev, LINK_STATE_DOWN);
  			}
 -			mutex_unlock(&mdev->state_lock);
  		}
 +		mutex_unlock(&mdev->state_lock);
  		break;
  	case SIOCADDMULTI:
  	case SIOCDELMULTI:
 +		mutex_lock(&mdev->state_lock);
  		mlx4_en_set_multicast(dev);
 +		mutex_unlock(&mdev->state_lock);
  		break;
  	case SIOCSIFMEDIA:
  	case SIOCGIFMEDIA:
 @@ -1116,6 +1124,7 @@ static int mlx4_en_ioctl(struct ifnet *dev, u_long
  		error =3D ifmedia_ioctl(dev, ifr, &priv->media, command);
  		break;
  	case SIOCSIFCAP:
 +		mutex_lock(&mdev->state_lock);
  		mask =3D ifr->ifr_reqcap ^ dev->if_capenable;
  		if (mask & IFCAP_HWCSUM)
  			dev->if_capenable ^=3D IFCAP_HWCSUM;
 @@ -1130,7 +1139,8 @@ static int mlx4_en_ioctl(struct ifnet *dev, u_long
  		if (mask & IFCAP_WOL_MAGIC)
  			dev->if_capenable ^=3D IFCAP_WOL_MAGIC;
  		if (dev->if_drv_flags & IFF_DRV_RUNNING)
 -			mlx4_en_init(priv);
 +			mlx4_en_init_locked(priv);
 +		mutex_unlock(&mdev->state_lock);
  		VLAN_CAPABILITIES(dev);
  		break;
  	default:
 
 
 --
 John Baldwin
State-Changed-From-To: open->patched 
State-Changed-By: jhb 
State-Changed-When: Mon Jul 29 18:49:12 UTC 2013 
State-Changed-Why:  
Patch committed to HEAD, thanks! 


Responsible-Changed-From-To: freebsd-net->jhb 
Responsible-Changed-By: jhb 
Responsible-Changed-When: Mon Jul 29 18:49:12 UTC 2013 
Responsible-Changed-Why:  
Patch committed to HEAD, thanks! 

http://www.freebsd.org/cgi/query-pr.cgi?pr=180791 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/180791: commit references a PR
Date: Mon, 29 Jul 2013 18:45:00 +0000 (UTC)

 Author: jhb
 Date: Mon Jul 29 18:44:52 2013
 New Revision: 253774
 URL: http://svnweb.freebsd.org/changeset/base/253774
 
 Log:
   Various fixes to the mlxen(4) driver:
   - Remove an incorrect assertion that can trigger when downing an interface.
   - Stop the interface during detach to avoid panics when unloading the
     driver.
   - A few locking fixes to be more consistent with other FreeBSD drivers:
     - Protect if_drv_flags with the driver lock, not atomic ops
     - Hold the driver lock when adjusting multicast state.
     - Hold the driver lock while adjusting if_capenable.
   
   PR:		kern/180791 [1,2]
   Submitted by:	Shakar Klein @ Mellanox [1,2]
   MFC after:	3 days
 
 Modified:
   head/sys/ofed/drivers/net/mlx4/en_netdev.c
 
 Modified: head/sys/ofed/drivers/net/mlx4/en_netdev.c
 ==============================================================================
 --- head/sys/ofed/drivers/net/mlx4/en_netdev.c	Mon Jul 29 17:03:42 2013	(r253773)
 +++ head/sys/ofed/drivers/net/mlx4/en_netdev.c	Mon Jul 29 18:44:52 2013	(r253774)
 @@ -495,11 +495,6 @@ static void mlx4_en_do_get_stats(struct 
  
  		queue_delayed_work(mdev->workqueue, &priv->stats_task, STATS_DELAY);
  	}
 -	if (mdev->mac_removed[MLX4_MAX_PORTS + 1 - priv->port]) {
 -		panic("mlx4_en_do_get_stats: Unexpected mac removed for %d\n",
 -		    priv->port);
 -		mdev->mac_removed[MLX4_MAX_PORTS + 1 - priv->port] = 0;
 -	}
  	mutex_unlock(&mdev->state_lock);
  }
  
 @@ -688,8 +683,8 @@ int mlx4_en_start_port(struct net_device
  	mlx4_en_set_multicast(dev);
  
  	/* Enable the queues. */
 -	atomic_clear_int(&dev->if_drv_flags, IFF_DRV_OACTIVE);
 -	atomic_set_int(&dev->if_drv_flags, IFF_DRV_RUNNING);
 +	dev->if_drv_flags &= ~IFF_DRV_OACTIVE;
 +	dev->if_drv_flags |= IFF_DRV_RUNNING;
  
  	callout_reset(&priv->watchdog_timer, MLX4_EN_WATCHDOG_TIMEOUT,
  	    mlx4_en_watchdog_timeout, priv);
 @@ -761,7 +756,7 @@ void mlx4_en_stop_port(struct net_device
  
  	callout_stop(&priv->watchdog_timer);
  
 -	atomic_clear_int(&dev->if_drv_flags, IFF_DRV_RUNNING);
 +	dev->if_drv_flags &= ~(IFF_DRV_RUNNING | IFF_DRV_OACTIVE);
  }
  
  static void mlx4_en_restart(struct work_struct *work)
 @@ -802,19 +797,30 @@ mlx4_en_init(void *arg)
  {
  	struct mlx4_en_priv *priv;
  	struct mlx4_en_dev *mdev;
 +
 +	priv = arg;
 +	mdev = priv->mdev;
 +	mutex_lock(&mdev->state_lock);
 +	mlx4_en_init_locked(priv);
 +	mutex_unlock(&mdev->state_lock);
 +}
 +
 +static void
 +mlx4_en_init_locked(struct mlx4_en_priv *priv)
 +{
 +
 +	struct mlx4_en_dev *mdev;
  	struct ifnet *dev;
  	int i;
  
 -	priv = arg;
  	dev = priv->dev;
  	mdev = priv->mdev;
 -	mutex_lock(&mdev->state_lock);
  	if (dev->if_drv_flags & IFF_DRV_RUNNING)
  		mlx4_en_stop_port(dev);
  
  	if (!mdev->device_up) {
  		en_err(priv, "Cannot open - device down/disabled\n");
 -		goto out;
 +		return;
  	}
  
  	/* Reset HW statistics and performance counters */
 @@ -835,9 +841,6 @@ mlx4_en_init(void *arg)
  	mlx4_en_set_default_moderation(priv);
  	if (mlx4_en_start_port(dev))
  		en_err(priv, "Failed starting port:%d\n", priv->port);
 -
 -out:
 -	mutex_unlock(&mdev->state_lock);
  }
  
  void mlx4_en_free_resources(struct mlx4_en_priv *priv)
 @@ -927,9 +930,14 @@ void mlx4_en_destroy_netdev(struct net_d
  	if (priv->sysctl)
  		sysctl_ctx_free(&priv->conf_ctx);
  
 +	mutex_lock(&mdev->state_lock);
 +	mlx4_en_stop_port(dev);
 +	mutex_unlock(&mdev->state_lock);
 +
  	cancel_delayed_work(&priv->stats_task);
  	/* flush any pending task for this netdev */
  	flush_workqueue(mdev->workqueue);
 +	callout_drain(&priv->watchdog_timer);
  
  	/* Detach the netdev so tasks would not attempt to access it */
  	mutex_lock(&mdev->state_lock);
 @@ -1091,31 +1099,32 @@ static int mlx4_en_ioctl(struct ifnet *d
  		error = -mlx4_en_change_mtu(dev, ifr->ifr_mtu);
  		break;
  	case SIOCSIFFLAGS:
 +		mutex_lock(&mdev->state_lock);
  		if (dev->if_flags & IFF_UP) {
 -			if ((dev->if_drv_flags & IFF_DRV_RUNNING) == 0) {
 -				mutex_lock(&mdev->state_lock);
 +			if ((dev->if_drv_flags & IFF_DRV_RUNNING) == 0)
  				mlx4_en_start_port(dev);
 -				mutex_unlock(&mdev->state_lock);
 -			} else
 +			else
  				mlx4_en_set_multicast(dev);
  		} else {
 -			mutex_lock(&mdev->state_lock);
  			if (dev->if_drv_flags & IFF_DRV_RUNNING) {
  				mlx4_en_stop_port(dev);
  				if_link_state_change(dev, LINK_STATE_DOWN);
  			}
 -			mutex_unlock(&mdev->state_lock);
  		}
 +		mutex_unlock(&mdev->state_lock);
  		break;
  	case SIOCADDMULTI:
  	case SIOCDELMULTI:
 +		mutex_lock(&mdev->state_lock);
  		mlx4_en_set_multicast(dev);
 +		mutex_unlock(&mdev->state_lock);
  		break;
  	case SIOCSIFMEDIA:
  	case SIOCGIFMEDIA:
  		error = ifmedia_ioctl(dev, ifr, &priv->media, command);
  		break;
  	case SIOCSIFCAP:
 +		mutex_lock(&mdev->state_lock);
  		mask = ifr->ifr_reqcap ^ dev->if_capenable;
  		if (mask & IFCAP_HWCSUM)
  			dev->if_capenable ^= IFCAP_HWCSUM;
 @@ -1130,7 +1139,8 @@ static int mlx4_en_ioctl(struct ifnet *d
  		if (mask & IFCAP_WOL_MAGIC)
  			dev->if_capenable ^= IFCAP_WOL_MAGIC;
  		if (dev->if_drv_flags & IFF_DRV_RUNNING)
 -			mlx4_en_init(priv);
 +			mlx4_en_init_locked(priv);
 +		mutex_unlock(&mdev->state_lock);
  		VLAN_CAPABILITIES(dev);
  		break;
  	default:
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 

From: John Baldwin <jhb@freebsd.org>
To: Shahar Klein <shahark@mellanox.com>
Cc: "bug-followup@freebsd.org" <bug-followup@freebsd.org>,
 Oded Shanoon <odeds@mellanox.com>
Subject: Re: kern/180791: [ofed] [patch] Kernel crash on ifdown and kldunload mlxen
Date: Mon, 29 Jul 2013 14:27:34 -0400

 On Monday, July 29, 2013 2:46:37 am Shahar Klein wrote:
 > Thanks John,
 > 
 > I've reviewed and tested this patch and it's working fine.
 > 
 > Is this patch going to both 10.0 and 9.2?
 
 I will see about merging all of the recent ofed changes to 9 today and see if 
 I can get them into 9.2.
 
 > 10x
 > Shahar
 > 
 > -----Original Message-----
 > From: John Baldwin [mailto:jhb@freebsd.org] 
 > Sent: Thursday, July 25, 2013 10:39 PM
 > To: bug-followup@freebsd.org; Shahar Klein
 > Subject: Re: kern/180791: [ofed] [patch] Kernel crash on ifdown and 
 kldunload mlxen
 > 
 > Thanks.  One note is that it seems like the state_lock should be held when 
 stop is called.  Also, the callout used for stats should be drained during 
 detach.  (If you use callot_init_mtx() instead of callout_init() then
 > callout_stop() under the lock is race-free, though I'd still do the
 > callout_drain() during detach to be safe.)
 > 
 > Also, I had some other changes I made to this file to make the locking more 
 consistent with other NIC drivers in the tree.  Can you look at this and test 
 this patch please?
 > 
 > Index: en_netdev.c
 > ===================================================================
 > --- en_netdev.c	(revision 253547)
 > +++ en_netdev.c	(working copy)
 > @@ -495,11 +495,6 @@ static void mlx4_en_do_get_stats(struct work_struc
 >  
 >  		queue_delayed_work(mdev->workqueue, &priv->stats_task, STATS_DELAY);
 >  	}
 > -	if (mdev->mac_removed[MLX4_MAX_PORTS + 1 - priv->port]) {
 > -		panic("mlx4_en_do_get_stats: Unexpected mac removed for %d\n",
 > -		    priv->port);
 > -		mdev->mac_removed[MLX4_MAX_PORTS + 1 - priv->port] = 0;
 > -	}
 >  	mutex_unlock(&mdev->state_lock);
 >  }
 >  
 > @@ -688,8 +683,8 @@ int mlx4_en_start_port(struct net_device *dev)
 >  	mlx4_en_set_multicast(dev);
 >  
 >  	/* Enable the queues. */
 > -	atomic_clear_int(&dev->if_drv_flags, IFF_DRV_OACTIVE);
 > -	atomic_set_int(&dev->if_drv_flags, IFF_DRV_RUNNING);
 > +	dev->if_drv_flags &= ~IFF_DRV_OACTIVE;
 > +	dev->if_drv_flags |= IFF_DRV_RUNNING;
 >  
 >  	callout_reset(&priv->watchdog_timer, MLX4_EN_WATCHDOG_TIMEOUT,
 >  	    mlx4_en_watchdog_timeout, priv);
 > @@ -761,7 +756,7 @@ void mlx4_en_stop_port(struct net_device *dev)
 >  
 >  	callout_stop(&priv->watchdog_timer);
 >  
 > -	atomic_clear_int(&dev->if_drv_flags, IFF_DRV_RUNNING);
 > +	dev->if_drv_flags &= ~(IFF_DRV_RUNNING | IFF_DRV_OACTIVE);
 >  }
 >  
 >  static void mlx4_en_restart(struct work_struct *work) @@ -802,19 +797,30 @@ 
 mlx4_en_init(void *arg)  {
 >  	struct mlx4_en_priv *priv;
 >  	struct mlx4_en_dev *mdev;
 > +
 > +	priv = arg;
 > +	mdev = priv->mdev;
 > +	mutex_lock(&mdev->state_lock);
 > +	mlx4_en_init_locked(priv);
 > +	mutex_unlock(&mdev->state_lock);
 > +}
 > +
 > +static void
 > +mlx4_en_init_locked(struct mlx4_en_priv *priv) {
 > +
 > +	struct mlx4_en_dev *mdev;
 >  	struct ifnet *dev;
 >  	int i;
 >  
 > -	priv = arg;
 >  	dev = priv->dev;
 >  	mdev = priv->mdev;
 > -	mutex_lock(&mdev->state_lock);
 >  	if (dev->if_drv_flags & IFF_DRV_RUNNING)
 >  		mlx4_en_stop_port(dev);
 >  
 >  	if (!mdev->device_up) {
 >  		en_err(priv, "Cannot open - device down/disabled\n");
 > -		goto out;
 > +		return;
 >  	}
 >  
 >  	/* Reset HW statistics and performance counters */ @@ -835,9 +841,6 @@ 
 mlx4_en_init(void *arg)
 >  	mlx4_en_set_default_moderation(priv);
 >  	if (mlx4_en_start_port(dev))
 >  		en_err(priv, "Failed starting port:%d\n", priv->port);
 > -
 > -out:
 > -	mutex_unlock(&mdev->state_lock);
 >  }
 >  
 >  void mlx4_en_free_resources(struct mlx4_en_priv *priv) @@ -927,9 +930,14 @@ 
 void mlx4_en_destroy_netdev(struct net_device *dev
 >  	if (priv->sysctl)
 >  		sysctl_ctx_free(&priv->conf_ctx);
 >  
 > +	mutex_lock(&mdev->state_lock);
 > +	mlx4_en_stop_port(dev);
 > +	mutex_unlock(&mdev->state_lock);
 > +
 >  	cancel_delayed_work(&priv->stats_task);
 >  	/* flush any pending task for this netdev */
 >  	flush_workqueue(mdev->workqueue);
 > +	callout_drain(&priv->watchdog_timer);
 >  
 >  	/* Detach the netdev so tasks would not attempt to access it */
 >  	mutex_lock(&mdev->state_lock);
 > @@ -1091,25 +1099,25 @@ static int mlx4_en_ioctl(struct ifnet *dev, u_long
 >  		error = -mlx4_en_change_mtu(dev, ifr->ifr_mtu);
 >  		break;
 >  	case SIOCSIFFLAGS:
 > +		mutex_lock(&mdev->state_lock);
 >  		if (dev->if_flags & IFF_UP) {
 > -			if ((dev->if_drv_flags & IFF_DRV_RUNNING) == 0) {
 > -				mutex_lock(&mdev->state_lock);
 > +			if ((dev->if_drv_flags & IFF_DRV_RUNNING) == 0)
 >  				mlx4_en_start_port(dev);
 > -				mutex_unlock(&mdev->state_lock);
 > -			} else
 > +			else
 >  				mlx4_en_set_multicast(dev);
 >  		} else {
 > -			mutex_lock(&mdev->state_lock);
 >  			if (dev->if_drv_flags & IFF_DRV_RUNNING) {
 >  				mlx4_en_stop_port(dev);
 >  				if_link_state_change(dev, LINK_STATE_DOWN);
 >  			}
 > -			mutex_unlock(&mdev->state_lock);
 >  		}
 > +		mutex_unlock(&mdev->state_lock);
 >  		break;
 >  	case SIOCADDMULTI:
 >  	case SIOCDELMULTI:
 > +		mutex_lock(&mdev->state_lock);
 >  		mlx4_en_set_multicast(dev);
 > +		mutex_unlock(&mdev->state_lock);
 >  		break;
 >  	case SIOCSIFMEDIA:
 >  	case SIOCGIFMEDIA:
 > @@ -1116,6 +1124,7 @@ static int mlx4_en_ioctl(struct ifnet *dev, u_long
 >  		error = ifmedia_ioctl(dev, ifr, &priv->media, command);
 >  		break;
 >  	case SIOCSIFCAP:
 > +		mutex_lock(&mdev->state_lock);
 >  		mask = ifr->ifr_reqcap ^ dev->if_capenable;
 >  		if (mask & IFCAP_HWCSUM)
 >  			dev->if_capenable ^= IFCAP_HWCSUM;
 > @@ -1130,7 +1139,8 @@ static int mlx4_en_ioctl(struct ifnet *dev, u_long
 >  		if (mask & IFCAP_WOL_MAGIC)
 >  			dev->if_capenable ^= IFCAP_WOL_MAGIC;
 >  		if (dev->if_drv_flags & IFF_DRV_RUNNING)
 > -			mlx4_en_init(priv);
 > +			mlx4_en_init_locked(priv);
 > +		mutex_unlock(&mdev->state_lock);
 >  		VLAN_CAPABILITIES(dev);
 >  		break;
 >  	default:
 > 
 > 
 > --
 > John Baldwin
 > 
 
 -- 
 John Baldwin

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/180791: commit references a PR
Date: Tue,  6 Aug 2013 19:24:22 +0000 (UTC)

 Author: jhb
 Date: Tue Aug  6 19:23:57 2013
 New Revision: 254006
 URL: http://svnweb.freebsd.org/changeset/base/254006
 
 Log:
   MFC 253048,253423,253449,253653,253774,253785:
   - Allow mlx4 devices to switch between Ethernet and Infiniband:
     - Fix sysfs attribute handling by using sysctl_handle_string() and
       properly handling trailing newlines in attribute values.
     - Remove check forbidding requests that would result in one port being
       set to Ethernet and the subsequent port being set to IB.
   - Avoid trashing IP fragments by correctly managing hardware checksumming.
   - Fix panics when downing or unloading the mlx4 driver.
   
   PR:		kern/179999, kern/174213, kern/180430, kern/180791
 
 Modified:
   stable/9/sys/ofed/drivers/net/mlx4/en_netdev.c
   stable/9/sys/ofed/drivers/net/mlx4/en_tx.c
   stable/9/sys/ofed/drivers/net/mlx4/main.c
   stable/9/sys/ofed/include/linux/sysfs.h
 Directory Properties:
   stable/9/sys/   (props changed)
 
 Modified: stable/9/sys/ofed/drivers/net/mlx4/en_netdev.c
 ==============================================================================
 --- stable/9/sys/ofed/drivers/net/mlx4/en_netdev.c	Tue Aug  6 19:14:02 2013	(r254005)
 +++ stable/9/sys/ofed/drivers/net/mlx4/en_netdev.c	Tue Aug  6 19:23:57 2013	(r254006)
 @@ -43,6 +43,7 @@
  #include <net/if_vlan_var.h>
  #include <sys/sockio.h>
  
 +static void mlx4_en_init_locked(struct mlx4_en_priv *priv);
  static void mlx4_en_sysctl_stat(struct mlx4_en_priv *priv);
  
  static void mlx4_en_vlan_rx_add_vid(void *arg, struct net_device *dev, u16 vid)
 @@ -495,11 +496,6 @@ static void mlx4_en_do_get_stats(struct 
  
  		queue_delayed_work(mdev->workqueue, &priv->stats_task, STATS_DELAY);
  	}
 -	if (mdev->mac_removed[MLX4_MAX_PORTS + 1 - priv->port]) {
 -		panic("mlx4_en_do_get_stats: Unexpected mac removed for %d\n",
 -		    priv->port);
 -		mdev->mac_removed[MLX4_MAX_PORTS + 1 - priv->port] = 0;
 -	}
  	mutex_unlock(&mdev->state_lock);
  }
  
 @@ -688,8 +684,8 @@ int mlx4_en_start_port(struct net_device
  	mlx4_en_set_multicast(dev);
  
  	/* Enable the queues. */
 -	atomic_clear_int(&dev->if_drv_flags, IFF_DRV_OACTIVE);
 -	atomic_set_int(&dev->if_drv_flags, IFF_DRV_RUNNING);
 +	dev->if_drv_flags &= ~IFF_DRV_OACTIVE;
 +	dev->if_drv_flags |= IFF_DRV_RUNNING;
  
  	callout_reset(&priv->watchdog_timer, MLX4_EN_WATCHDOG_TIMEOUT,
  	    mlx4_en_watchdog_timeout, priv);
 @@ -761,7 +757,7 @@ void mlx4_en_stop_port(struct net_device
  
  	callout_stop(&priv->watchdog_timer);
  
 -	atomic_clear_int(&dev->if_drv_flags, IFF_DRV_RUNNING);
 +	dev->if_drv_flags &= ~(IFF_DRV_RUNNING | IFF_DRV_OACTIVE);
  }
  
  static void mlx4_en_restart(struct work_struct *work)
 @@ -802,19 +798,30 @@ mlx4_en_init(void *arg)
  {
  	struct mlx4_en_priv *priv;
  	struct mlx4_en_dev *mdev;
 +
 +	priv = arg;
 +	mdev = priv->mdev;
 +	mutex_lock(&mdev->state_lock);
 +	mlx4_en_init_locked(priv);
 +	mutex_unlock(&mdev->state_lock);
 +}
 +
 +static void
 +mlx4_en_init_locked(struct mlx4_en_priv *priv)
 +{
 +
 +	struct mlx4_en_dev *mdev;
  	struct ifnet *dev;
  	int i;
  
 -	priv = arg;
  	dev = priv->dev;
  	mdev = priv->mdev;
 -	mutex_lock(&mdev->state_lock);
  	if (dev->if_drv_flags & IFF_DRV_RUNNING)
  		mlx4_en_stop_port(dev);
  
  	if (!mdev->device_up) {
  		en_err(priv, "Cannot open - device down/disabled\n");
 -		goto out;
 +		return;
  	}
  
  	/* Reset HW statistics and performance counters */
 @@ -835,9 +842,6 @@ mlx4_en_init(void *arg)
  	mlx4_en_set_default_moderation(priv);
  	if (mlx4_en_start_port(dev))
  		en_err(priv, "Failed starting port:%d\n", priv->port);
 -
 -out:
 -	mutex_unlock(&mdev->state_lock);
  }
  
  void mlx4_en_free_resources(struct mlx4_en_priv *priv)
 @@ -927,9 +931,14 @@ void mlx4_en_destroy_netdev(struct net_d
  	if (priv->sysctl)
  		sysctl_ctx_free(&priv->conf_ctx);
  
 +	mutex_lock(&mdev->state_lock);
 +	mlx4_en_stop_port(dev);
 +	mutex_unlock(&mdev->state_lock);
 +
  	cancel_delayed_work(&priv->stats_task);
  	/* flush any pending task for this netdev */
  	flush_workqueue(mdev->workqueue);
 +	callout_drain(&priv->watchdog_timer);
  
  	/* Detach the netdev so tasks would not attempt to access it */
  	mutex_lock(&mdev->state_lock);
 @@ -1091,31 +1100,32 @@ static int mlx4_en_ioctl(struct ifnet *d
  		error = -mlx4_en_change_mtu(dev, ifr->ifr_mtu);
  		break;
  	case SIOCSIFFLAGS:
 +		mutex_lock(&mdev->state_lock);
  		if (dev->if_flags & IFF_UP) {
 -			if ((dev->if_drv_flags & IFF_DRV_RUNNING) == 0) {
 -				mutex_lock(&mdev->state_lock);
 +			if ((dev->if_drv_flags & IFF_DRV_RUNNING) == 0)
  				mlx4_en_start_port(dev);
 -				mutex_unlock(&mdev->state_lock);
 -			} else
 +			else
  				mlx4_en_set_multicast(dev);
  		} else {
 -			mutex_lock(&mdev->state_lock);
  			if (dev->if_drv_flags & IFF_DRV_RUNNING) {
  				mlx4_en_stop_port(dev);
  				if_link_state_change(dev, LINK_STATE_DOWN);
  			}
 -			mutex_unlock(&mdev->state_lock);
  		}
 +		mutex_unlock(&mdev->state_lock);
  		break;
  	case SIOCADDMULTI:
  	case SIOCDELMULTI:
 +		mutex_lock(&mdev->state_lock);
  		mlx4_en_set_multicast(dev);
 +		mutex_unlock(&mdev->state_lock);
  		break;
  	case SIOCSIFMEDIA:
  	case SIOCGIFMEDIA:
  		error = ifmedia_ioctl(dev, ifr, &priv->media, command);
  		break;
  	case SIOCSIFCAP:
 +		mutex_lock(&mdev->state_lock);
  		mask = ifr->ifr_reqcap ^ dev->if_capenable;
  		if (mask & IFCAP_HWCSUM)
  			dev->if_capenable ^= IFCAP_HWCSUM;
 @@ -1130,7 +1140,8 @@ static int mlx4_en_ioctl(struct ifnet *d
  		if (mask & IFCAP_WOL_MAGIC)
  			dev->if_capenable ^= IFCAP_WOL_MAGIC;
  		if (dev->if_drv_flags & IFF_DRV_RUNNING)
 -			mlx4_en_init(priv);
 +			mlx4_en_init_locked(priv);
 +		mutex_unlock(&mdev->state_lock);
  		VLAN_CAPABILITIES(dev);
  		break;
  	default:
 
 Modified: stable/9/sys/ofed/drivers/net/mlx4/en_tx.c
 ==============================================================================
 --- stable/9/sys/ofed/drivers/net/mlx4/en_tx.c	Tue Aug  6 19:14:02 2013	(r254005)
 +++ stable/9/sys/ofed/drivers/net/mlx4/en_tx.c	Tue Aug  6 19:23:57 2013	(r254006)
 @@ -780,8 +780,12 @@ retry:
  	tx_desc->ctrl.srcrb_flags = cpu_to_be32(MLX4_WQE_CTRL_CQ_UPDATE |
  						MLX4_WQE_CTRL_SOLICITED);
  	if (mb->m_pkthdr.csum_flags & (CSUM_IP|CSUM_TCP|CSUM_UDP)) {
 -		tx_desc->ctrl.srcrb_flags |= cpu_to_be32(MLX4_WQE_CTRL_IP_CSUM |
 -							 MLX4_WQE_CTRL_TCP_UDP_CSUM);
 +		if (mb->m_pkthdr.csum_flags & CSUM_IP)
 +			tx_desc->ctrl.srcrb_flags |=
 +			    cpu_to_be32(MLX4_WQE_CTRL_IP_CSUM);
 +		if (mb->m_pkthdr.csum_flags & (CSUM_TCP|CSUM_UDP))
 +			tx_desc->ctrl.srcrb_flags |=
 +			    cpu_to_be32(MLX4_WQE_CTRL_TCP_UDP_CSUM);
  		priv->port_stats.tx_chksum_offload++;
  	}
  
 
 Modified: stable/9/sys/ofed/drivers/net/mlx4/main.c
 ==============================================================================
 --- stable/9/sys/ofed/drivers/net/mlx4/main.c	Tue Aug  6 19:14:02 2013	(r254005)
 +++ stable/9/sys/ofed/drivers/net/mlx4/main.c	Tue Aug  6 19:23:57 2013	(r254006)
 @@ -209,9 +209,6 @@ int mlx4_check_port_params(struct mlx4_d
  					 "on this HCA, aborting.\n");
  				return -EINVAL;
  			}
 -			if (port_type[i] == MLX4_PORT_TYPE_ETH &&
 -			    port_type[i + 1] == MLX4_PORT_TYPE_IB)
 -				return -EINVAL;
  		}
  	}
  
 
 Modified: stable/9/sys/ofed/include/linux/sysfs.h
 ==============================================================================
 --- stable/9/sys/ofed/include/linux/sysfs.h	Tue Aug  6 19:14:02 2013	(r254005)
 +++ stable/9/sys/ofed/include/linux/sysfs.h	Tue Aug  6 19:23:57 2013	(r254006)
 @@ -75,39 +75,42 @@ sysctl_handle_attr(SYSCTL_HANDLER_ARGS)
  	struct kobject *kobj;
  	struct attribute *attr;
  	const struct sysfs_ops *ops;
 -	void *buf;
 +	char *buf;
  	int error;
  	ssize_t len;
  
  	kobj = arg1;
  	attr = (struct attribute *)arg2;
 -	buf = (void *)get_zeroed_page(GFP_KERNEL);
 -	len = 1;	/* Copy out a NULL byte at least. */
  	if (kobj->ktype == NULL || kobj->ktype->sysfs_ops == NULL)
  		return (ENODEV);
 -	ops = kobj->ktype->sysfs_ops;
 +	buf = (char *)get_zeroed_page(GFP_KERNEL);
  	if (buf == NULL)
  		return (ENOMEM);
 +	ops = kobj->ktype->sysfs_ops;
  	if (ops->show) {
  		len = ops->show(kobj, attr, buf);
  		/*
 -		 * It's valid not to have a 'show' so we just return 1 byte
 -		 * of NULL.
 +		 * It's valid to not have a 'show' so just return an
 +		 * empty string.
  	 	 */
  		if (len < 0) {
  			error = -len;
 -			len = 1;
  			if (error != EIO)
  				goto out;
  		}
 +
 +		/* Trim trailing newline. */
 +		len--;
 +		buf[len] = '\0';
  	}
 -	error = SYSCTL_OUT(req, buf, len);
 -	if (error || !req->newptr || ops->store == NULL)
 -		goto out;
 -	error = SYSCTL_IN(req, buf, PAGE_SIZE);
 -	if (error)
 +
 +	/* Leave one trailing byte to append a newline. */
 +	error = sysctl_handle_string(oidp, buf, PAGE_SIZE - 1, req);
 +	if (error != 0 || req->newptr == NULL || ops->store == NULL)
  		goto out;
 -	len = ops->store(kobj, attr, buf, req->newlen);
 +	len = strlcat(buf, "\n", PAGE_SIZE);
 +	KASSERT(len < PAGE_SIZE, ("new attribute truncated"));
 +	len = ops->store(kobj, attr, buf, len);
  	if (len < 0)
  		error = -len;
  out:
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/180791: commit references a PR
Date: Tue,  6 Aug 2013 20:04:59 +0000 (UTC)

 Author: jhb
 Date: Tue Aug  6 20:04:44 2013
 New Revision: 254007
 URL: http://svnweb.freebsd.org/changeset/base/254007
 
 Log:
   MFC 253048,253423,253449,253653,253774,253785:
   - Allow mlx4 devices to switch between Ethernet and Infiniband:
     - Fix sysfs attribute handling by using sysctl_handle_string() and
       properly handling trailing newlines in attribute values.
     - Remove check forbidding requests that would result in one port being
       set to Ethernet and the subsequent port being set to IB.
   - Avoid trashing IP fragments by correctly managing hardware checksumming.
   - Fix panics when downing or unloading the mlx4 driver.
   
   PR:		kern/179999, kern/174213, kern/180430, kern/180791
   Approved by:	re (kib)
 
 Modified:
   releng/9.2/sys/ofed/drivers/net/mlx4/en_netdev.c
   releng/9.2/sys/ofed/drivers/net/mlx4/en_tx.c
   releng/9.2/sys/ofed/drivers/net/mlx4/main.c
   releng/9.2/sys/ofed/include/linux/sysfs.h
 Directory Properties:
   releng/9.2/sys/   (props changed)
 
 Modified: releng/9.2/sys/ofed/drivers/net/mlx4/en_netdev.c
 ==============================================================================
 --- releng/9.2/sys/ofed/drivers/net/mlx4/en_netdev.c	Tue Aug  6 19:23:57 2013	(r254006)
 +++ releng/9.2/sys/ofed/drivers/net/mlx4/en_netdev.c	Tue Aug  6 20:04:44 2013	(r254007)
 @@ -43,6 +43,7 @@
  #include <net/if_vlan_var.h>
  #include <sys/sockio.h>
  
 +static void mlx4_en_init_locked(struct mlx4_en_priv *priv);
  static void mlx4_en_sysctl_stat(struct mlx4_en_priv *priv);
  
  static void mlx4_en_vlan_rx_add_vid(void *arg, struct net_device *dev, u16 vid)
 @@ -495,11 +496,6 @@ static void mlx4_en_do_get_stats(struct 
  
  		queue_delayed_work(mdev->workqueue, &priv->stats_task, STATS_DELAY);
  	}
 -	if (mdev->mac_removed[MLX4_MAX_PORTS + 1 - priv->port]) {
 -		panic("mlx4_en_do_get_stats: Unexpected mac removed for %d\n",
 -		    priv->port);
 -		mdev->mac_removed[MLX4_MAX_PORTS + 1 - priv->port] = 0;
 -	}
  	mutex_unlock(&mdev->state_lock);
  }
  
 @@ -688,8 +684,8 @@ int mlx4_en_start_port(struct net_device
  	mlx4_en_set_multicast(dev);
  
  	/* Enable the queues. */
 -	atomic_clear_int(&dev->if_drv_flags, IFF_DRV_OACTIVE);
 -	atomic_set_int(&dev->if_drv_flags, IFF_DRV_RUNNING);
 +	dev->if_drv_flags &= ~IFF_DRV_OACTIVE;
 +	dev->if_drv_flags |= IFF_DRV_RUNNING;
  
  	callout_reset(&priv->watchdog_timer, MLX4_EN_WATCHDOG_TIMEOUT,
  	    mlx4_en_watchdog_timeout, priv);
 @@ -761,7 +757,7 @@ void mlx4_en_stop_port(struct net_device
  
  	callout_stop(&priv->watchdog_timer);
  
 -	atomic_clear_int(&dev->if_drv_flags, IFF_DRV_RUNNING);
 +	dev->if_drv_flags &= ~(IFF_DRV_RUNNING | IFF_DRV_OACTIVE);
  }
  
  static void mlx4_en_restart(struct work_struct *work)
 @@ -802,19 +798,30 @@ mlx4_en_init(void *arg)
  {
  	struct mlx4_en_priv *priv;
  	struct mlx4_en_dev *mdev;
 +
 +	priv = arg;
 +	mdev = priv->mdev;
 +	mutex_lock(&mdev->state_lock);
 +	mlx4_en_init_locked(priv);
 +	mutex_unlock(&mdev->state_lock);
 +}
 +
 +static void
 +mlx4_en_init_locked(struct mlx4_en_priv *priv)
 +{
 +
 +	struct mlx4_en_dev *mdev;
  	struct ifnet *dev;
  	int i;
  
 -	priv = arg;
  	dev = priv->dev;
  	mdev = priv->mdev;
 -	mutex_lock(&mdev->state_lock);
  	if (dev->if_drv_flags & IFF_DRV_RUNNING)
  		mlx4_en_stop_port(dev);
  
  	if (!mdev->device_up) {
  		en_err(priv, "Cannot open - device down/disabled\n");
 -		goto out;
 +		return;
  	}
  
  	/* Reset HW statistics and performance counters */
 @@ -835,9 +842,6 @@ mlx4_en_init(void *arg)
  	mlx4_en_set_default_moderation(priv);
  	if (mlx4_en_start_port(dev))
  		en_err(priv, "Failed starting port:%d\n", priv->port);
 -
 -out:
 -	mutex_unlock(&mdev->state_lock);
  }
  
  void mlx4_en_free_resources(struct mlx4_en_priv *priv)
 @@ -927,9 +931,14 @@ void mlx4_en_destroy_netdev(struct net_d
  	if (priv->sysctl)
  		sysctl_ctx_free(&priv->conf_ctx);
  
 +	mutex_lock(&mdev->state_lock);
 +	mlx4_en_stop_port(dev);
 +	mutex_unlock(&mdev->state_lock);
 +
  	cancel_delayed_work(&priv->stats_task);
  	/* flush any pending task for this netdev */
  	flush_workqueue(mdev->workqueue);
 +	callout_drain(&priv->watchdog_timer);
  
  	/* Detach the netdev so tasks would not attempt to access it */
  	mutex_lock(&mdev->state_lock);
 @@ -1091,31 +1100,32 @@ static int mlx4_en_ioctl(struct ifnet *d
  		error = -mlx4_en_change_mtu(dev, ifr->ifr_mtu);
  		break;
  	case SIOCSIFFLAGS:
 +		mutex_lock(&mdev->state_lock);
  		if (dev->if_flags & IFF_UP) {
 -			if ((dev->if_drv_flags & IFF_DRV_RUNNING) == 0) {
 -				mutex_lock(&mdev->state_lock);
 +			if ((dev->if_drv_flags & IFF_DRV_RUNNING) == 0)
  				mlx4_en_start_port(dev);
 -				mutex_unlock(&mdev->state_lock);
 -			} else
 +			else
  				mlx4_en_set_multicast(dev);
  		} else {
 -			mutex_lock(&mdev->state_lock);
  			if (dev->if_drv_flags & IFF_DRV_RUNNING) {
  				mlx4_en_stop_port(dev);
  				if_link_state_change(dev, LINK_STATE_DOWN);
  			}
 -			mutex_unlock(&mdev->state_lock);
  		}
 +		mutex_unlock(&mdev->state_lock);
  		break;
  	case SIOCADDMULTI:
  	case SIOCDELMULTI:
 +		mutex_lock(&mdev->state_lock);
  		mlx4_en_set_multicast(dev);
 +		mutex_unlock(&mdev->state_lock);
  		break;
  	case SIOCSIFMEDIA:
  	case SIOCGIFMEDIA:
  		error = ifmedia_ioctl(dev, ifr, &priv->media, command);
  		break;
  	case SIOCSIFCAP:
 +		mutex_lock(&mdev->state_lock);
  		mask = ifr->ifr_reqcap ^ dev->if_capenable;
  		if (mask & IFCAP_HWCSUM)
  			dev->if_capenable ^= IFCAP_HWCSUM;
 @@ -1130,7 +1140,8 @@ static int mlx4_en_ioctl(struct ifnet *d
  		if (mask & IFCAP_WOL_MAGIC)
  			dev->if_capenable ^= IFCAP_WOL_MAGIC;
  		if (dev->if_drv_flags & IFF_DRV_RUNNING)
 -			mlx4_en_init(priv);
 +			mlx4_en_init_locked(priv);
 +		mutex_unlock(&mdev->state_lock);
  		VLAN_CAPABILITIES(dev);
  		break;
  	default:
 
 Modified: releng/9.2/sys/ofed/drivers/net/mlx4/en_tx.c
 ==============================================================================
 --- releng/9.2/sys/ofed/drivers/net/mlx4/en_tx.c	Tue Aug  6 19:23:57 2013	(r254006)
 +++ releng/9.2/sys/ofed/drivers/net/mlx4/en_tx.c	Tue Aug  6 20:04:44 2013	(r254007)
 @@ -780,8 +780,12 @@ retry:
  	tx_desc->ctrl.srcrb_flags = cpu_to_be32(MLX4_WQE_CTRL_CQ_UPDATE |
  						MLX4_WQE_CTRL_SOLICITED);
  	if (mb->m_pkthdr.csum_flags & (CSUM_IP|CSUM_TCP|CSUM_UDP)) {
 -		tx_desc->ctrl.srcrb_flags |= cpu_to_be32(MLX4_WQE_CTRL_IP_CSUM |
 -							 MLX4_WQE_CTRL_TCP_UDP_CSUM);
 +		if (mb->m_pkthdr.csum_flags & CSUM_IP)
 +			tx_desc->ctrl.srcrb_flags |=
 +			    cpu_to_be32(MLX4_WQE_CTRL_IP_CSUM);
 +		if (mb->m_pkthdr.csum_flags & (CSUM_TCP|CSUM_UDP))
 +			tx_desc->ctrl.srcrb_flags |=
 +			    cpu_to_be32(MLX4_WQE_CTRL_TCP_UDP_CSUM);
  		priv->port_stats.tx_chksum_offload++;
  	}
  
 
 Modified: releng/9.2/sys/ofed/drivers/net/mlx4/main.c
 ==============================================================================
 --- releng/9.2/sys/ofed/drivers/net/mlx4/main.c	Tue Aug  6 19:23:57 2013	(r254006)
 +++ releng/9.2/sys/ofed/drivers/net/mlx4/main.c	Tue Aug  6 20:04:44 2013	(r254007)
 @@ -209,9 +209,6 @@ int mlx4_check_port_params(struct mlx4_d
  					 "on this HCA, aborting.\n");
  				return -EINVAL;
  			}
 -			if (port_type[i] == MLX4_PORT_TYPE_ETH &&
 -			    port_type[i + 1] == MLX4_PORT_TYPE_IB)
 -				return -EINVAL;
  		}
  	}
  
 
 Modified: releng/9.2/sys/ofed/include/linux/sysfs.h
 ==============================================================================
 --- releng/9.2/sys/ofed/include/linux/sysfs.h	Tue Aug  6 19:23:57 2013	(r254006)
 +++ releng/9.2/sys/ofed/include/linux/sysfs.h	Tue Aug  6 20:04:44 2013	(r254007)
 @@ -75,39 +75,42 @@ sysctl_handle_attr(SYSCTL_HANDLER_ARGS)
  	struct kobject *kobj;
  	struct attribute *attr;
  	const struct sysfs_ops *ops;
 -	void *buf;
 +	char *buf;
  	int error;
  	ssize_t len;
  
  	kobj = arg1;
  	attr = (struct attribute *)arg2;
 -	buf = (void *)get_zeroed_page(GFP_KERNEL);
 -	len = 1;	/* Copy out a NULL byte at least. */
  	if (kobj->ktype == NULL || kobj->ktype->sysfs_ops == NULL)
  		return (ENODEV);
 -	ops = kobj->ktype->sysfs_ops;
 +	buf = (char *)get_zeroed_page(GFP_KERNEL);
  	if (buf == NULL)
  		return (ENOMEM);
 +	ops = kobj->ktype->sysfs_ops;
  	if (ops->show) {
  		len = ops->show(kobj, attr, buf);
  		/*
 -		 * It's valid not to have a 'show' so we just return 1 byte
 -		 * of NULL.
 +		 * It's valid to not have a 'show' so just return an
 +		 * empty string.
  	 	 */
  		if (len < 0) {
  			error = -len;
 -			len = 1;
  			if (error != EIO)
  				goto out;
  		}
 +
 +		/* Trim trailing newline. */
 +		len--;
 +		buf[len] = '\0';
  	}
 -	error = SYSCTL_OUT(req, buf, len);
 -	if (error || !req->newptr || ops->store == NULL)
 -		goto out;
 -	error = SYSCTL_IN(req, buf, PAGE_SIZE);
 -	if (error)
 +
 +	/* Leave one trailing byte to append a newline. */
 +	error = sysctl_handle_string(oidp, buf, PAGE_SIZE - 1, req);
 +	if (error != 0 || req->newptr == NULL || ops->store == NULL)
  		goto out;
 -	len = ops->store(kobj, attr, buf, req->newlen);
 +	len = strlcat(buf, "\n", PAGE_SIZE);
 +	KASSERT(len < PAGE_SIZE, ("new attribute truncated"));
 +	len = ops->store(kobj, attr, buf, len);
  	if (len < 0)
  		error = -len;
  out:
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 
State-Changed-From-To: patched->closed 
State-Changed-By: jhb 
State-Changed-When: Wed Aug 7 18:02:53 UTC 2013 
State-Changed-Why:  
Merged to 9 and 9.2. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=180791 
>Unformatted:
