From nobody@FreeBSD.org  Mon Apr 23 13:58:57 2012
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52])
	by hub.freebsd.org (Postfix) with ESMTP id AADBA1065741
	for <freebsd-gnats-submit@FreeBSD.org>; Mon, 23 Apr 2012 13:58:57 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from red.freebsd.org (red.freebsd.org [IPv6:2001:4f8:fff6::22])
	by mx1.freebsd.org (Postfix) with ESMTP id 967888FC08
	for <freebsd-gnats-submit@FreeBSD.org>; Mon, 23 Apr 2012 13:58:57 +0000 (UTC)
Received: from red.freebsd.org (localhost [127.0.0.1])
	by red.freebsd.org (8.14.4/8.14.4) with ESMTP id q3NDwv1S042631
	for <freebsd-gnats-submit@FreeBSD.org>; Mon, 23 Apr 2012 13:58:57 GMT
	(envelope-from nobody@red.freebsd.org)
Received: (from nobody@localhost)
	by red.freebsd.org (8.14.4/8.14.4/Submit) id q3NDwuaD042630;
	Mon, 23 Apr 2012 13:58:57 GMT
	(envelope-from nobody)
Message-Id: <201204231358.q3NDwuaD042630@red.freebsd.org>
Date: Mon, 23 Apr 2012 13:58:57 GMT
From: Petr Lampa <lampa@fit.vutbr.cz>
To: freebsd-gnats-submit@FreeBSD.org
Subject: new tbolt mfi driver cannot read sector >= 2^32 or 2^21
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         167226
>Category:       kern
>Synopsis:       [mfi] [patch] new tbolt mfi driver cannot read sector >= 2^32 or 2^21
>Confidential:   no
>Severity:       critical
>Priority:       high
>Responsible:    ambrisko
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Mon Apr 23 14:00:35 UTC 2012
>Closed-Date:    Fri May 04 16:02:54 UTC 2012
>Last-Modified:  Mon May  7 19:20:05 UTC 2012
>Originator:     Petr Lampa
>Release:        9.0-STABLE
>Organization:
BUT FIT
>Environment:
FreeBSD temp 9.0-STABLE FreeBSD 9.0-STABLE #0: Thu Apr 19 11:18:42 CEST 2012     root@temp:/usr/obj/usr/src/sys/TEMP  amd64

>Description:
The new mfi_tbolt_build_cdb() ignores lba_hi when checking if lba address
can be placed into read/write 6/10 CDB. The result is sector address clipped
to 2^21-1 or 2^32-1 (this depends on request length).
>How-To-Repeat:
Connect 9265/9285 LSI MegaRAID and read existing array > 2TB.
>Fix:
Change dev/mfi/mfi_tbolt.c:

 if ((num_lba <= 0xFF) && (lba_lo <= 0x1FFFFF)) {
..
 else if ((num_lba <= 0xFFFF) && (lba_lo <= 0xFFFFFFFF)) {
..

to 

 if (lba_hi == 0 && (num_lba <= 0xFF) && (lba_lo <= 0x1FFFFF)) {
 ...
 else if (lba_hi == 0 && (num_lba <= 0xFFFF) && (lba_lo <= 0xFFFFFFFF)) {
 ...

>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->ambrisko 
Responsible-Changed-By: linimon 
Responsible-Changed-When: Mon Apr 23 14:02:04 UTC 2012 
Responsible-Changed-Why:  
Over to maintainer. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=167226 
State-Changed-From-To: open->closed 
State-Changed-By: ambrisko 
State-Changed-When: Fri May 4 16:01:48 UTC 2012 
State-Changed-Why:  
Fix has been checked into -current.  Thanks for the patch. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=167226 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/167226: commit references a PR
Date: Fri,  4 May 2012 16:00:51 +0000 (UTC)

 Author: ambrisko
 Date: Fri May  4 16:00:39 2012
 New Revision: 235014
 URL: http://svn.freebsd.org/changeset/base/235014
 
 Log:
   First fix pr 167226:
   	ThunderBolt cannot read sector >= 2^32 or 2^21
   with supplied patch.
   
   Second the bigger change, fix RAID operation on ThunderBolt base
   card such as physically removing a disk from a RAID and replacing
   it.  The current situation is the RAID firmware effectively hangs
   waiting for an acknowledgement from the driver.  This is due to
   the firmware support of the driver actually accessing the RAID
   from under the firmware.  This is an interesting feature that
   the FreeBSD driver does not use.  However, when the firmare
   detects the driver has attached it then expects the driver will
   synchronize LD's with the firmware.  If the driver does not sync.
   then the management part of the firmware will hang waiting for
   it so a pulled driver will listed as still there.
   
   The fix for this problem isn't extremely difficult.  However,
   figuring out why some of the code was the way it was and then
   redoing it was involved.  Not have a spec. made it harder to
   try to figure out.  The existing driver would send a
   MFI_DCMD_LD_MAP_GET_INFO command in write mode to acknowledge
   a LD state change.  In read mode it gets the RAID map from the
   firmware.  The FreeBSD driver doesn't do that currently.  It
   could be added in the future with the appropriate structures.
   To simplify things, get the current LD state and then build
   the MFI_DCMD_LD_MAP_GET_INFO/write command so that it sends
   an acknowledgement for each LD.  The map would probably state
   which LD's changed so then the driver could probably just
   acknowledge the LD's that changed versus all.  This doesn't seem
   to be a problem.  When a MFI_DCMD_LD_MAP_GET_INFO/write command
   is sent to the firmware, it will complete later when a change
   to the LD's happen.  So it is very much like an AEN command
   returning when something happened.  When the
   MFI_DCMD_LD_MAP_GET_INFO/write command completes, we refire the
   sync'ing of the LD state.  This needs to be done in as an event
   so that MFI_DCMD_LD_GET_LIST can wait for that command to
   complete before issuing the MFI_DCMD_LD_MAP_GET_INFO/write.
   The prior code didn't use the call-back function and tried
   to intercept the MFI_DCMD_LD_MAP_GET_INFO/write command when
   processing an interrupt.  This added a bunch of code complexity
   to the interrupt handler.  Using the call-back that is done
   for other commands got rid of this need.  So the interrupt
   handler is greatly simplified.  It seems that even commands
   that shouldn't be acknowledged end up in the interrupt handler.
   To deal with this, code was added to check to see if a command
   is in the busy queue or not.  This might have contributed to the
   interrupt storm happening without MSI enabled on these cards.
   
   Note that MFI_DCMD_LD_MAP_GET_INFO/read returns right away.
   
   It would be interesting to see what other complexity could
   be removed from the ThunderBolt driver that really isn't
   needed in our mode of operation.  Letting the RAID firmware
   do all of the I/O to disks is a lot faster since it can
   use its caches.  It greatly simplifies what the driver has
   to do and potential bugs if the driver and firmware are
   not in sync.
   
   Simplify the aen_abort/cm_map_abort and put it in the softc
   versus in the command structure.
   
   This should get merged to 9 before the driver is merged to
   8.
   
   PR:		167226
   Submitted by:	Petr Lampa
   MFC after:	3 days
 
 Modified:
   head/sys/dev/mfi/mfi.c
   head/sys/dev/mfi/mfi_debug.c
   head/sys/dev/mfi/mfi_tbolt.c
   head/sys/dev/mfi/mfireg.h
   head/sys/dev/mfi/mfivar.h
 
 Modified: head/sys/dev/mfi/mfi.c
 ==============================================================================
 --- head/sys/dev/mfi/mfi.c	Fri May  4 16:00:22 2012	(r235013)
 +++ head/sys/dev/mfi/mfi.c	Fri May  4 16:00:39 2012	(r235014)
 @@ -90,8 +90,6 @@ static int	mfi_get_controller_info(struc
  static int	mfi_get_log_state(struct mfi_softc *,
  		    struct mfi_evt_log_state **);
  static int	mfi_parse_entries(struct mfi_softc *, int, int);
 -static int	mfi_dcmd_command(struct mfi_softc *, struct mfi_command **,
 -		    uint32_t, void **, size_t);
  static void	mfi_data_cb(void *, bus_dma_segment_t *, int, int);
  static void	mfi_startup(void *arg);
  static void	mfi_intr(void *arg);
 @@ -377,6 +375,7 @@ mfi_attach(struct mfi_softc *sc)
  	TAILQ_INIT(&sc->mfi_syspd_tqh);
  	TAILQ_INIT(&sc->mfi_evt_queue);
  	TASK_INIT(&sc->mfi_evt_task, 0, mfi_handle_evt, sc);
 +	TASK_INIT(&sc->mfi_map_sync_task, 0, mfi_handle_map_sync, sc);
  	TAILQ_INIT(&sc->mfi_aen_pids);
  	TAILQ_INIT(&sc->mfi_cam_ccbq);
  
 @@ -696,7 +695,6 @@ mfi_attach(struct mfi_softc *sc)
  			return (EINVAL);
  		}
  		sc->mfi_enable_intr(sc);
 -		sc->map_id = 0;
  	} else {
  		if ((error = mfi_comms_init(sc)) != 0)
  			return (error);
 @@ -762,6 +760,10 @@ mfi_attach(struct mfi_softc *sc)
  	callout_reset(&sc->mfi_watchdog_callout, MFI_CMD_TIMEOUT * hz,
  	    mfi_timeout, sc);
  
 +	if (sc->mfi_flags & MFI_FLAGS_TBOLT) {
 +		mfi_tbolt_sync_map_info(sc);
 +	}
 +
  	return (0);
  }
  
 @@ -845,7 +847,7 @@ mfi_release_command(struct mfi_command *
  	mfi_enqueue_free(cm);
  }
  
 -static int
 +int
  mfi_dcmd_command(struct mfi_softc *sc, struct mfi_command **cmp,
      uint32_t opcode, void **bufp, size_t bufsize)
  {
 @@ -1286,8 +1288,8 @@ mfi_shutdown(struct mfi_softc *sc)
  	if (sc->mfi_aen_cm != NULL)
  		mfi_abort(sc, sc->mfi_aen_cm);
  
 -	if (sc->map_update_cmd != NULL)
 -		mfi_abort(sc, sc->map_update_cmd);
 +	if (sc->mfi_map_sync_cm != NULL)
 +		mfi_abort(sc, sc->mfi_map_sync_cm);
  
  	dcmd = &cm->cm_frame->dcmd;
  	dcmd->header.flags = MFI_FRAME_DIR_NONE;
 @@ -1664,9 +1666,9 @@ mfi_aen_complete(struct mfi_command *cm)
  	if (sc->mfi_aen_cm == NULL)
  		return;
  
 -	if (sc->mfi_aen_cm->cm_aen_abort ||
 +	if (sc->cm_aen_abort ||
  	    hdr->cmd_status == MFI_STAT_INVALID_STATUS) {
 -		sc->mfi_aen_cm->cm_aen_abort = 0;
 +		sc->cm_aen_abort = 0;
  		aborted = 1;
  	} else {
  		sc->mfi_aen_triggered = 1;
 @@ -2385,7 +2387,9 @@ mfi_abort(struct mfi_softc *sc, struct m
  	cm->cm_flags = MFI_CMD_POLLED;
  
  	if (sc->mfi_aen_cm)
 -		sc->mfi_aen_cm->cm_aen_abort = 1;
 +		sc->cm_aen_abort = 1;
 +	if (sc->mfi_map_sync_cm)
 +		sc->cm_map_abort = 1;
  	mfi_mapcmd(sc, cm);
  	mfi_release_command(cm);
  
 @@ -2394,6 +2398,11 @@ mfi_abort(struct mfi_softc *sc, struct m
  		    5 * hz);
  		i++;
  	}
 +	while (i < 5 && sc->mfi_map_sync_cm != NULL) {
 +		msleep(&sc->mfi_map_sync_cm, &sc->mfi_io_lock, 0, "mfiabort",
 +		    5 * hz);
 +		i++;
 +	}
  
  	return (0);
  }
 @@ -3549,9 +3558,9 @@ mfi_timeout(void *data)
  	}
  	mtx_lock(&sc->mfi_io_lock);
  	TAILQ_FOREACH(cm, &sc->mfi_busy, cm_link) {
 -		if (sc->mfi_aen_cm == cm)
 +		if (sc->mfi_aen_cm == cm || sc->mfi_map_sync_cm == cm)
  			continue;
 -		if ((sc->mfi_aen_cm != cm) && (cm->cm_timestamp < deadline)) {
 +		if (cm->cm_timestamp < deadline) {
  			if (sc->adpreset != 0 && sc->issuepend_done == 0) {
  				cm->cm_timestamp = time_uptime;
  			} else {
 
 Modified: head/sys/dev/mfi/mfi_debug.c
 ==============================================================================
 --- head/sys/dev/mfi/mfi_debug.c	Fri May  4 16:00:22 2012	(r235013)
 +++ head/sys/dev/mfi/mfi_debug.c	Fri May  4 16:00:39 2012	(r235014)
 @@ -172,6 +172,9 @@ mfi_print_dcmd(struct mfi_softc *sc, dev
  	case MFI_DCMD_CLUSTER_RESET_LD:
  		opcode = "CLUSTER_RESET_LD";
  		break;
 +	case MFI_DCMD_LD_MAP_GET_INFO:
 +		opcode = "LD_MAP_GET_INFO";
 +		break;
  	default:
  		opcode = "UNKNOWN";
  		break;
 
 Modified: head/sys/dev/mfi/mfi_tbolt.c
 ==============================================================================
 --- head/sys/dev/mfi/mfi_tbolt.c	Fri May  4 16:00:22 2012	(r235013)
 +++ head/sys/dev/mfi/mfi_tbolt.c	Fri May  4 16:00:39 2012	(r235014)
 @@ -82,7 +82,8 @@ map_tbolt_cmd_status(struct mfi_command 
  static void mfi_issue_pending_cmds_again (struct mfi_softc *sc);
  static void mfi_kill_hba (struct mfi_softc *sc);
  static void mfi_process_fw_state_chg_isr(void *arg);
 -uint8_t mfi_tbolt_get_map_info(struct mfi_softc *sc);
 +static void mfi_sync_map_complete(struct mfi_command *);
 +static void mfi_queue_map_sync(struct mfi_softc *sc);
  
  #define MFI_FUSION_ENABLE_INTERRUPT_MASK	(0x00000008)
  
 @@ -627,10 +628,11 @@ mfi_tbolt_return_cmd(struct mfi_softc *s
  	TAILQ_INSERT_TAIL(&sc->mfi_cmd_tbolt_tqh, cmd, next);
  }
  
 -void mfi_tbolt_complete_cmd(struct mfi_softc *sc)
 +void
 +mfi_tbolt_complete_cmd(struct mfi_softc *sc)
  {
  	struct mfi_mpi2_reply_header *desc, *reply_desc;
 -	struct mfi_command *cmd_mfi;	/* For MFA Cmds */
 +	struct mfi_command *cmd_mfi, *cmd_mfi_check;	/* For MFA Cmds */
  	struct mfi_cmd_tbolt *cmd_tbolt;
  	uint16_t smid;
  	uint8_t reply_descript_type;
 @@ -657,7 +659,6 @@ void mfi_tbolt_complete_cmd(struct mfi_s
  
  	/* Read Reply descriptor */
  	while ((val.u.low != 0xFFFFFFFF) && (val.u.high != 0xFFFFFFFF)) {
 -
  		smid = reply_desc->SMID;
  		if (!smid || smid > sc->mfi_max_fw_cmds + 1) {
  			device_printf(sc->mfi_dev, "smid is %x. Cannot "
 @@ -669,66 +670,20 @@ void mfi_tbolt_complete_cmd(struct mfi_s
  		cmd_mfi = &sc->mfi_commands[cmd_tbolt->sync_cmd_idx];
  		scsi_io_req = cmd_tbolt->io_request;
  
 -		/* Check if internal commands */
  		status = cmd_mfi->cm_frame->dcmd.header.cmd_status;
  		extStatus = cmd_mfi->cm_frame->dcmd.header.scsi_status;
 +		map_tbolt_cmd_status(cmd_mfi, status, extStatus);
  
 -		switch (scsi_io_req->Function) {
 -		case MPI2_FUNCTION_LD_IO_REQUEST:
 -			/* Regular Path IO. */
 -			/* Map the Fw Error Status. */
 -			map_tbolt_cmd_status(cmd_mfi, status,
 -			    extStatus);
 -			if ((cmd_mfi->cm_frame->dcmd.opcode
 -			    == MFI_DCMD_LD_MAP_GET_INFO)
 -			    && (cmd_mfi->cm_frame->dcmd.mbox[1] == 1)) {
 -					if (cmd_mfi->cm_frame->header.cmd_status
 -					    != 0)
 -						device_printf(sc->mfi_dev,
 -						    "map sync failed\n");
 -					else {
 -						sc->map_id++;
 -						device_printf(sc->mfi_dev,
 -						    "map sync completed\n");
 -						mfi_release_command(cmd_mfi);
 -					}
 -				}
 -			if ((cmd_mfi->cm_flags & MFI_ON_MFIQ_BUSY)
 -			    == MFI_ON_MFIQ_BUSY
 -			    && (cmd_mfi->cm_flags & MFI_CMD_POLLED) == 0) {
 -				/* BHARAT poll workaround */
 -				mfi_remove_busy(cmd_mfi);
 -				cmd_mfi->cm_error = 0;
 -				mfi_complete(sc, cmd_mfi);
 -			}
 -			mfi_tbolt_return_cmd(sc, cmd_tbolt);
 -			break;
 -		case MPI2_FUNCTION_PASSTHRU_IO_REQUEST:
 -			map_tbolt_cmd_status(cmd_mfi, status, extStatus);
 -			if ((cmd_mfi->cm_frame->dcmd.opcode
 -			    == MFI_DCMD_LD_MAP_GET_INFO)
 -			    && (cmd_mfi->cm_frame->dcmd.mbox[1] == 1)) {
 -				if (cmd_mfi->cm_frame->header.cmd_status != 0)
 -					device_printf(sc->mfi_dev,
 -					    "map sync failed\n");
 -				else {
 -					sc->map_id++;
 -					device_printf(sc->mfi_dev,
 -					    "map sync completed\n");
 -					mfi_release_command(cmd_mfi);
 -				}
 -			}
 -			if ((cmd_mfi->cm_flags & MFI_ON_MFIQ_BUSY)
 -			    == MFI_ON_MFIQ_BUSY
 -			    && (cmd_mfi->cm_flags & MFI_CMD_POLLED) == 0) {
 -				/* BHARAT poll workaround */
 +		/* remove command from busy queue if not polled */
 +		TAILQ_FOREACH(cmd_mfi_check, &sc->mfi_busy, cm_link) {
 +			if (cmd_mfi_check == cmd_mfi) {
  				mfi_remove_busy(cmd_mfi);
 -				cmd_mfi->cm_error = 0;
 -				mfi_complete(sc, cmd_mfi);
 +				break;
  			}
 -			mfi_tbolt_return_cmd(sc, cmd_tbolt);
 -			break;
  		}
 +		cmd_mfi->cm_error = 0;
 +		mfi_complete(sc, cmd_mfi);
 +		mfi_tbolt_return_cmd(sc, cmd_tbolt);
  
  		sc->last_reply_idx++;
  		if (sc->last_reply_idx >= sc->mfi_max_fw_cmds) {
 @@ -949,7 +904,7 @@ mfi_tbolt_build_cdb(struct mfi_softc *sc
  	lba_lo = mfi_cmd->cm_frame->io.lba_lo;
  	lba_hi = mfi_cmd->cm_frame->io.lba_hi;
  
 -	if ((num_lba <= 0xFF) && (lba_lo <= 0x1FFFFF)) {
 +	if (lba_hi == 0 && (num_lba <= 0xFF) && (lba_lo <= 0x1FFFFF)) {
  		if (mfi_cmd->cm_frame->header.cmd == MFI_CMD_LD_WRITE)
  			/* Read 6 or Write 6 */
  			cdb[0] = (uint8_t) (0x0A);
 @@ -962,7 +917,7 @@ mfi_tbolt_build_cdb(struct mfi_softc *sc
  		cdb[1] = (uint8_t) ((lba_lo >> 16) & 0x1F);
  		cdb_len = 6;
  	}
 -	else if ((num_lba <= 0xFFFF) && (lba_lo <= 0xFFFFFFFF)) {
 +	else if (lba_hi == 0 && (num_lba <= 0xFFFF) && (lba_lo <= 0xFFFFFFFF)) {
  		if (mfi_cmd->cm_frame->header.cmd == MFI_CMD_LD_WRITE)
  			/* Read 10 or Write 10 */
  			cdb[0] = (uint8_t) (0x2A);
 @@ -1152,8 +1107,8 @@ mfi_tbolt_send_frame(struct mfi_softc *s
  		cm->cm_timestamp = time_uptime;
  		mfi_enqueue_busy(cm);
  	}
 -	else {
 -		hdr->cmd_status = 0xff;
 +	else {	/* still get interrupts for it */
 +		hdr->cmd_status = MFI_STAT_INVALID_STATUS;
  		hdr->flags |= MFI_FRAME_DONT_POST_IN_REPLY_QUEUE;
  	}
  
 @@ -1189,16 +1144,16 @@ mfi_tbolt_send_frame(struct mfi_softc *s
  		return 0;
  
  	/* This is a polled command, so busy-wait for it to complete. */
 -	while (hdr->cmd_status == 0xff) {
 +	while (hdr->cmd_status == MFI_STAT_INVALID_STATUS) {
  		DELAY(1000);
  		tm -= 1;
  		if (tm <= 0)
 -			break;
 +		break;
  	}
 -
 -	if (hdr->cmd_status == 0xff) {
 + 
 +	if (hdr->cmd_status == MFI_STAT_INVALID_STATUS) {
  		device_printf(sc->mfi_dev, "Frame %p timed out "
 -		      "command 0x%X\n", hdr, cm->cm_frame->dcmd.opcode);
 +		    "command 0x%X\n", hdr, cm->cm_frame->dcmd.opcode);
  		return (ETIMEDOUT);
  	}
  	return 0;
 @@ -1308,9 +1263,9 @@ static void mfi_process_fw_state_chg_isr
  			mfi_release_command(sc->mfi_aen_cm);
  			sc->mfi_aen_cm = NULL;
  		}
 -		if (sc->map_update_cmd) {
 -			mfi_release_command(sc->map_update_cmd);
 -			sc->map_update_cmd = NULL;
 +		if (sc->mfi_map_sync_cm) {
 +			mfi_release_command(sc->mfi_map_sync_cm);
 +			sc->mfi_map_sync_cm = NULL;
  		}
  		mfi_issue_pending_cmds_again(sc);
  
 @@ -1337,3 +1292,171 @@ static void mfi_process_fw_state_chg_isr
  	}
  	mtx_unlock(&sc->mfi_io_lock);
  }
 +
 +
 +/*
 + * The ThunderBolt HW has an option for the driver to directly
 + * access the underlying disks and operate on the RAID.  To 
 + * do this there needs to be a capability to keep the RAID controller
 + * and driver in sync.  The FreeBSD driver does not take advantage
 + * of this feature since it adds a lot of complexity and slows down
 + * performance.  Performance is gained by using the controller's
 + * cache etc.
 + *
 + * Even though this driver doesn't access the disks directly, an
 + * AEN like command is used to inform the RAID firmware to "sync"
 + * with all LD's via the MFI_DCMD_LD_MAP_GET_INFO command.  This
 + * command in write mode will return when the RAID firmware has
 + * detected a change to the RAID state.  Examples of this type
 + * of change are removing a disk.  Once the command returns then
 + * the driver needs to acknowledge this and "sync" all LD's again.
 + * This repeats until we shutdown.  Then we need to cancel this
 + * pending command.
 + *
 + * If this is not done right the RAID firmware will not remove a
 + * pulled drive and the RAID won't go degraded etc.  Effectively,
 + * stopping any RAID mangement to functions.
 + *
 + * Doing another LD sync, requires the use of an event since the
 + * driver needs to do a mfi_wait_command and can't do that in an
 + * interrupt thread.
 + *
 + * The driver could get the RAID state via the MFI_DCMD_LD_MAP_GET_INFO
 + * That requires a bunch of structure and it is simplier to just do
 + * the MFI_DCMD_LD_GET_LIST versus walking the RAID map.
 + */
 +
 +void
 +mfi_tbolt_sync_map_info(struct mfi_softc *sc)
 +{
 +	int error = 0, i;
 +	struct mfi_command *cmd;
 +	struct mfi_dcmd_frame *dcmd;
 +	uint32_t context = 0;
 +	union mfi_ld_ref *ld_sync;
 +	size_t ld_size;
 +	struct mfi_frame_header *hdr;
 +	struct mfi_command *cm = NULL;
 +	struct mfi_ld_list *list = NULL;
 +
 +	if (sc->mfi_map_sync_cm != NULL || sc->cm_map_abort)
 +		return;
 +
 +	mtx_lock(&sc->mfi_io_lock);
 +	error = mfi_dcmd_command(sc, &cm, MFI_DCMD_LD_GET_LIST,
 +	    (void **)&list, sizeof(*list));
 +	if (error)
 +		goto out;
 +
 +	cm->cm_flags = MFI_CMD_POLLED | MFI_CMD_DATAIN;
 +	if (mfi_wait_command(sc, cm) != 0) {
 +		device_printf(sc->mfi_dev, "Failed to get device listing\n");
 +		goto out;
 +	}
 +
 +	hdr = &cm->cm_frame->header;
 +	if (hdr->cmd_status != MFI_STAT_OK) {
 +		device_printf(sc->mfi_dev, "MFI_DCMD_LD_GET_LIST failed %x\n",
 +			      hdr->cmd_status);
 +		goto out;
 +	}
 +
 +	ld_size = sizeof(*ld_sync) * list->ld_count;
 +	mtx_unlock(&sc->mfi_io_lock);
 +	ld_sync = (union mfi_ld_ref *) malloc(ld_size, M_MFIBUF,
 +	     M_WAITOK | M_ZERO);
 +	for (i = 0; i < list->ld_count; i++) {
 +		ld_sync[i].ref = list->ld_list[i].ld.ref;
 +	}
 +
 +	mtx_lock(&sc->mfi_io_lock);
 +	if ((cmd = mfi_dequeue_free(sc)) == NULL)
 +		return;
 +	context = cmd->cm_frame->header.context;
 +	bzero(cmd->cm_frame, sizeof(union mfi_frame));
 +	cmd->cm_frame->header.context = context;
 +
 +	dcmd = &cmd->cm_frame->dcmd;
 +	bzero(dcmd->mbox, MFI_MBOX_SIZE);
 +	dcmd->header.cmd = MFI_CMD_DCMD;
 +	dcmd->header.flags = MFI_FRAME_DIR_WRITE;
 +	dcmd->header.timeout = 0;
 +	dcmd->header.data_len = ld_size;
 +	dcmd->header.scsi_status = 0;
 +	dcmd->opcode = MFI_DCMD_LD_MAP_GET_INFO;
 +	cmd->cm_sg = &dcmd->sgl;
 +	cmd->cm_total_frame_size = MFI_DCMD_FRAME_SIZE;
 +	cmd->cm_data = ld_sync;
 +	cmd->cm_private = ld_sync;
 +
 +	cmd->cm_len = ld_size;
 +	cmd->cm_complete = mfi_sync_map_complete;
 +	sc->mfi_map_sync_cm = cmd;
 +
 +	cmd->cm_flags = MFI_CMD_DATAOUT;
 +	cmd->cm_frame->dcmd.mbox[0] = list->ld_count;
 +	cmd->cm_frame->dcmd.mbox[1] = MFI_DCMD_MBOX_PEND_FLAG;
 +
 +	if ((error = mfi_mapcmd(sc, cmd)) != 0) {
 +		device_printf(sc->mfi_dev, "failed to send map sync\n");
 +		return;
 +	}
 +
 +out:
 +	if (list)
 +		free(list, M_MFIBUF);
 +	if (cm)
 +		mfi_release_command(cm);
 +	mtx_unlock(&sc->mfi_io_lock);
 +
 +	return;
 +}
 +
 +
 +static void
 +mfi_sync_map_complete(struct mfi_command *cm)
 +{
 +	struct mfi_frame_header *hdr;
 +	struct mfi_softc *sc;
 +	int aborted = 0;
 +
 +	sc = cm->cm_sc;
 +	mtx_assert(&sc->mfi_io_lock, MA_OWNED);
 +
 +	hdr = &cm->cm_frame->header;
 +
 +	if (sc->mfi_map_sync_cm == NULL)
 +		return;
 +
 +	if (sc->cm_map_abort ||
 +	    hdr->cmd_status == MFI_STAT_INVALID_STATUS) {
 +		sc->cm_map_abort = 0;
 +		aborted = 1;
 +	}
 +
 +	free(cm->cm_data, M_MFIBUF);
 +	sc->mfi_map_sync_cm = NULL;
 +	wakeup(&sc->mfi_map_sync_cm);
 +	mfi_release_command(cm);
 +
 +	/* set it up again so the driver can catch more events */
 +	if (!aborted) {
 +		mfi_queue_map_sync(sc);
 +	}
 +}
 +
 +static void
 +mfi_queue_map_sync(struct mfi_softc *sc)
 +{
 +	mtx_assert(&sc->mfi_io_lock, MA_OWNED);
 +	taskqueue_enqueue(taskqueue_swi, &sc->mfi_map_sync_task);
 +}
 +
 +void
 +mfi_handle_map_sync(void *context, int pending)
 +{
 +	struct mfi_softc *sc;
 +
 +	sc = context;
 +	mfi_tbolt_sync_map_info(sc);
 +}
 
 Modified: head/sys/dev/mfi/mfireg.h
 ==============================================================================
 --- head/sys/dev/mfi/mfireg.h	Fri May  4 16:00:22 2012	(r235013)
 +++ head/sys/dev/mfi/mfireg.h	Fri May  4 16:00:39 2012	(r235014)
 @@ -403,6 +403,7 @@ typedef enum {
  #define MR_EVT_CTRL_HOST_BUS_SCAN_REQUESTED	0x0152
  #define MR_EVT_PD_REMOVED			0x0070
  #define MR_EVT_PD_INSERTED			0x005b
 +#define MR_EVT_LD_CHANGE			0x0051
  
  typedef enum {
  	MR_LD_CACHE_WRITE_BACK =	0x01,
 
 Modified: head/sys/dev/mfi/mfivar.h
 ==============================================================================
 --- head/sys/dev/mfi/mfivar.h	Fri May  4 16:00:22 2012	(r235013)
 +++ head/sys/dev/mfi/mfivar.h	Fri May  4 16:00:39 2012	(r235014)
 @@ -105,7 +105,6 @@ struct mfi_command {
  #define MFI_ON_MFIQ_READY	(1<<6)
  #define MFI_ON_MFIQ_BUSY	(1<<7)
  #define MFI_ON_MFIQ_MASK	((1<<5)|(1<<6)|(1<<7))
 -	int			cm_aen_abort;
  	uint8_t			retry_for_fw_reset;
  	void			(* cm_complete)(struct mfi_command *cm);
  	void			*cm_private;
 @@ -216,9 +215,13 @@ struct mfi_softc {
  
  	TAILQ_HEAD(,mfi_evt_queue_elm)	mfi_evt_queue;
  	struct task			mfi_evt_task;
 +	struct task			mfi_map_sync_task;
  	TAILQ_HEAD(,mfi_aen)		mfi_aen_pids;
  	struct mfi_command		*mfi_aen_cm;
  	struct mfi_command		*mfi_skinny_cm;
 +	struct mfi_command		*mfi_map_sync_cm;
 +	int				cm_aen_abort;
 +	int				cm_map_abort;
  	uint32_t			mfi_aen_triggered;
  	uint32_t			mfi_poll_waiting;
  	uint32_t			mfi_boot_seq_num;
 @@ -303,8 +306,6 @@ struct mfi_softc {
  	/* ThunderBolt */
  	uint32_t			mfi_tbolt;
  	uint32_t			MFA_enabled;
 -	uint64_t			map_id;
 -	struct mfi_command 		*map_update_cmd;
  	/* Single Reply structure size */
  	uint16_t			reply_size;
  	/* Singler message size. */
 @@ -417,7 +418,10 @@ extern int mfi_tbolt_alloc_cmd(struct mf
  extern int mfi_tbolt_send_frame(struct mfi_softc *sc, struct mfi_command *cm);
  extern int mfi_tbolt_adp_reset(struct mfi_softc *sc);
  extern int mfi_tbolt_reset(struct mfi_softc *sc);
 -extern int mfi_tbolt_sync_map_info(struct mfi_softc *sc);
 +extern void mfi_tbolt_sync_map_info(struct mfi_softc *sc);
 +extern void mfi_handle_map_sync(void *context, int pending);
 +extern int mfi_dcmd_command(struct mfi_softc *, struct mfi_command **,
 +		    uint32_t, void **, size_t);
  
  #define MFIQ_ADD(sc, qname)					\
  	do {							\
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/167226: commit references a PR
Date: Mon,  7 May 2012 19:17:31 +0000 (UTC)

 Author: ambrisko
 Date: Mon May  7 19:17:09 2012
 New Revision: 235135
 URL: http://svn.freebsd.org/changeset/base/235135
 
 Log:
   MFC r235014:	Rebuild fix and LBA calculation fix for ThunderBolt
   		based cards.
   
   MFC r235016:	Some style improvements.
   
   MFC r235040:	Fix the returns in mfi_tbolt_sync_map_info that I added.
   
   PR:		167226
 
 Modified:
   stable/9/sys/dev/mfi/mfi.c
   stable/9/sys/dev/mfi/mfi_debug.c
   stable/9/sys/dev/mfi/mfi_disk.c
   stable/9/sys/dev/mfi/mfi_tbolt.c
   stable/9/sys/dev/mfi/mfireg.h
   stable/9/sys/dev/mfi/mfivar.h
 Directory Properties:
   stable/9/sys/   (props changed)
 
 Modified: stable/9/sys/dev/mfi/mfi.c
 ==============================================================================
 --- stable/9/sys/dev/mfi/mfi.c	Mon May  7 18:30:22 2012	(r235134)
 +++ stable/9/sys/dev/mfi/mfi.c	Mon May  7 19:17:09 2012	(r235135)
 @@ -90,8 +90,6 @@ static int	mfi_get_controller_info(struc
  static int	mfi_get_log_state(struct mfi_softc *,
  		    struct mfi_evt_log_state **);
  static int	mfi_parse_entries(struct mfi_softc *, int, int);
 -static int	mfi_dcmd_command(struct mfi_softc *, struct mfi_command **,
 -		    uint32_t, void **, size_t);
  static void	mfi_data_cb(void *, bus_dma_segment_t *, int, int);
  static void	mfi_startup(void *arg);
  static void	mfi_intr(void *arg);
 @@ -377,6 +375,7 @@ mfi_attach(struct mfi_softc *sc)
  	TAILQ_INIT(&sc->mfi_syspd_tqh);
  	TAILQ_INIT(&sc->mfi_evt_queue);
  	TASK_INIT(&sc->mfi_evt_task, 0, mfi_handle_evt, sc);
 +	TASK_INIT(&sc->mfi_map_sync_task, 0, mfi_handle_map_sync, sc);
  	TAILQ_INIT(&sc->mfi_aen_pids);
  	TAILQ_INIT(&sc->mfi_cam_ccbq);
  
 @@ -696,7 +695,6 @@ mfi_attach(struct mfi_softc *sc)
  			return (EINVAL);
  		}
  		sc->mfi_enable_intr(sc);
 -		sc->map_id = 0;
  	} else {
  		if ((error = mfi_comms_init(sc)) != 0)
  			return (error);
 @@ -762,6 +760,10 @@ mfi_attach(struct mfi_softc *sc)
  	callout_reset(&sc->mfi_watchdog_callout, MFI_CMD_TIMEOUT * hz,
  	    mfi_timeout, sc);
  
 +	if (sc->mfi_flags & MFI_FLAGS_TBOLT) {
 +		mfi_tbolt_sync_map_info(sc);
 +	}
 +
  	return (0);
  }
  
 @@ -845,7 +847,7 @@ mfi_release_command(struct mfi_command *
  	mfi_enqueue_free(cm);
  }
  
 -static int
 +int
  mfi_dcmd_command(struct mfi_softc *sc, struct mfi_command **cmp,
      uint32_t opcode, void **bufp, size_t bufsize)
  {
 @@ -1286,8 +1288,8 @@ mfi_shutdown(struct mfi_softc *sc)
  	if (sc->mfi_aen_cm != NULL)
  		mfi_abort(sc, sc->mfi_aen_cm);
  
 -	if (sc->map_update_cmd != NULL)
 -		mfi_abort(sc, sc->map_update_cmd);
 +	if (sc->mfi_map_sync_cm != NULL)
 +		mfi_abort(sc, sc->mfi_map_sync_cm);
  
  	dcmd = &cm->cm_frame->dcmd;
  	dcmd->header.flags = MFI_FRAME_DIR_NONE;
 @@ -1317,7 +1319,7 @@ mfi_syspdprobe(struct mfi_softc *sc)
  	/* Add SYSTEM PD's */
  	error = mfi_dcmd_command(sc, &cm, MFI_DCMD_PD_LIST_QUERY,
  	    (void **)&pdlist, sizeof(*pdlist));
 -	if (error){
 +	if (error) {
  		device_printf(sc->mfi_dev,
  		    "Error while forming SYSTEM PD list\n");
  		goto out;
 @@ -1664,9 +1666,9 @@ mfi_aen_complete(struct mfi_command *cm)
  	if (sc->mfi_aen_cm == NULL)
  		return;
  
 -	if (sc->mfi_aen_cm->cm_aen_abort ||
 +	if (sc->cm_aen_abort ||
  	    hdr->cmd_status == MFI_STAT_INVALID_STATUS) {
 -		sc->mfi_aen_cm->cm_aen_abort = 0;
 +		sc->cm_aen_abort = 0;
  		aborted = 1;
  	} else {
  		sc->mfi_aen_triggered = 1;
 @@ -1956,6 +1958,7 @@ mfi_add_sys_pd_complete(struct mfi_comma
  	mtx_unlock(&Giant);
  	mtx_lock(&sc->mfi_io_lock);
  }
 +
  static struct mfi_command *
  mfi_bio_command(struct mfi_softc *sc)
  {
 @@ -1963,7 +1966,7 @@ mfi_bio_command(struct mfi_softc *sc)
  	struct mfi_command *cm = NULL;
  
  	/*reserving two commands to avoid starvation for IOCTL*/
 -	if (sc->mfi_qstat[MFIQ_FREE].q_length < 2){
 +	if (sc->mfi_qstat[MFIQ_FREE].q_length < 2) {
  		return (NULL);
  	}
  	if ((bio = mfi_dequeue_bio(sc)) == NULL) {
 @@ -2385,7 +2388,9 @@ mfi_abort(struct mfi_softc *sc, struct m
  	cm->cm_flags = MFI_CMD_POLLED;
  
  	if (sc->mfi_aen_cm)
 -		sc->mfi_aen_cm->cm_aen_abort = 1;
 +		sc->cm_aen_abort = 1;
 +	if (sc->mfi_map_sync_cm)
 +		sc->cm_map_abort = 1;
  	mfi_mapcmd(sc, cm);
  	mfi_release_command(cm);
  
 @@ -2394,6 +2399,11 @@ mfi_abort(struct mfi_softc *sc, struct m
  		    5 * hz);
  		i++;
  	}
 +	while (i < 5 && sc->mfi_map_sync_cm != NULL) {
 +		msleep(&sc->mfi_map_sync_cm, &sc->mfi_io_lock, 0, "mfiabort",
 +		    5 * hz);
 +		i++;
 +	}
  
  	return (0);
  }
 @@ -2685,12 +2695,12 @@ static int mfi_check_for_sscd(struct mfi
  	int error = 0;
  
  	if ((cm->cm_frame->dcmd.opcode == MFI_DCMD_CFG_ADD) &&
 -	    (conf_data->ld[0].params.isSSCD == 1)){
 +	    (conf_data->ld[0].params.isSSCD == 1)) {
  		error = 1;
  	} else if (cm->cm_frame->dcmd.opcode == MFI_DCMD_LD_DELETE) {
  		error = mfi_dcmd_command (sc, &ld_cm, MFI_DCMD_LD_GET_INFO,
  		    (void **)&ld_info, sizeof(*ld_info));
 -		if (error){
 +		if (error) {
  			device_printf(sc->mfi_dev, "Failed to allocate"
  			    "MFI_DCMD_LD_GET_INFO %d", error);
  			if (ld_info)
 @@ -2700,7 +2710,7 @@ static int mfi_check_for_sscd(struct mfi
  		ld_cm->cm_flags = MFI_CMD_DATAIN;
  		ld_cm->cm_frame->dcmd.mbox[0]= cm->cm_frame->dcmd.mbox[0];
  		ld_cm->cm_frame->header.target_id = cm->cm_frame->dcmd.mbox[0];
 -		if (mfi_wait_command(sc, ld_cm) != 0){
 +		if (mfi_wait_command(sc, ld_cm) != 0) {
  			device_printf(sc->mfi_dev, "failed to get log drv\n");
  			mfi_release_command(ld_cm);
  			free(ld_info, M_MFIBUF);
 @@ -3549,9 +3559,9 @@ mfi_timeout(void *data)
  	}
  	mtx_lock(&sc->mfi_io_lock);
  	TAILQ_FOREACH(cm, &sc->mfi_busy, cm_link) {
 -		if (sc->mfi_aen_cm == cm)
 +		if (sc->mfi_aen_cm == cm || sc->mfi_map_sync_cm == cm)
  			continue;
 -		if ((sc->mfi_aen_cm != cm) && (cm->cm_timestamp < deadline)) {
 +		if (cm->cm_timestamp < deadline) {
  			if (sc->adpreset != 0 && sc->issuepend_done == 0) {
  				cm->cm_timestamp = time_uptime;
  			} else {
 
 Modified: stable/9/sys/dev/mfi/mfi_debug.c
 ==============================================================================
 --- stable/9/sys/dev/mfi/mfi_debug.c	Mon May  7 18:30:22 2012	(r235134)
 +++ stable/9/sys/dev/mfi/mfi_debug.c	Mon May  7 19:17:09 2012	(r235135)
 @@ -172,6 +172,9 @@ mfi_print_dcmd(struct mfi_softc *sc, dev
  	case MFI_DCMD_CLUSTER_RESET_LD:
  		opcode = "CLUSTER_RESET_LD";
  		break;
 +	case MFI_DCMD_LD_MAP_GET_INFO:
 +		opcode = "LD_MAP_GET_INFO";
 +		break;
  	default:
  		opcode = "UNKNOWN";
  		break;
 
 Modified: stable/9/sys/dev/mfi/mfi_disk.c
 ==============================================================================
 --- stable/9/sys/dev/mfi/mfi_disk.c	Mon May  7 18:30:22 2012	(r235134)
 +++ stable/9/sys/dev/mfi/mfi_disk.c	Mon May  7 19:17:09 2012	(r235135)
 @@ -256,17 +256,17 @@ mfi_disk_strategy(struct bio *bio)
  		return;
  	}
  
 -	if (controller->adpreset){
 +	if (controller->adpreset) {
  		bio->bio_error = EBUSY;
  		return;
  	}
  
 -	if (controller->hw_crit_error){
 +	if (controller->hw_crit_error) {
  		bio->bio_error = EBUSY;
  		return;
  	}
  
 -	if (controller->issuepend_done == 0){
 +	if (controller->issuepend_done == 0) {
  		bio->bio_error = EBUSY;
  		return;
  	}
 
 Modified: stable/9/sys/dev/mfi/mfi_tbolt.c
 ==============================================================================
 --- stable/9/sys/dev/mfi/mfi_tbolt.c	Mon May  7 18:30:22 2012	(r235134)
 +++ stable/9/sys/dev/mfi/mfi_tbolt.c	Mon May  7 19:17:09 2012	(r235135)
 @@ -82,7 +82,8 @@ map_tbolt_cmd_status(struct mfi_command 
  static void mfi_issue_pending_cmds_again (struct mfi_softc *sc);
  static void mfi_kill_hba (struct mfi_softc *sc);
  static void mfi_process_fw_state_chg_isr(void *arg);
 -uint8_t mfi_tbolt_get_map_info(struct mfi_softc *sc);
 +static void mfi_sync_map_complete(struct mfi_command *);
 +static void mfi_queue_map_sync(struct mfi_softc *sc);
  
  #define MFI_FUSION_ENABLE_INTERRUPT_MASK	(0x00000008)
  
 @@ -140,11 +141,12 @@ mfi_tbolt_issue_cmd_ppc(struct mfi_softc
  	MFI_WRITE4(sc, MFI_IQPH, (uint32_t)((uint64_t)bus_add >> 32));
  }
  
 -/**
 +/*
   * mfi_tbolt_adp_reset - For controller reset
   * @regs: MFI register set
   */
 -int mfi_tbolt_adp_reset(struct mfi_softc *sc)
 +int
 +mfi_tbolt_adp_reset(struct mfi_softc *sc)
  {
  	int retry = 0, i = 0;
  	int HostDiag;
 @@ -192,12 +194,10 @@ int mfi_tbolt_adp_reset(struct mfi_softc
  }
  
  /*
 - *******************************************************************************************
 - * Description:
 - *      This routine initialize Thunderbolt specific device information
 - *******************************************************************************************
 + * This routine initialize Thunderbolt specific device information
   */
 -void mfi_tbolt_init_globals(struct mfi_softc *sc)
 +void
 +mfi_tbolt_init_globals(struct mfi_softc *sc)
  {
  	/* Initialize single reply size and Message size */
  	sc->reply_size = MEGASAS_THUNDERBOLT_REPLY_SIZE;
 @@ -239,16 +239,12 @@ void mfi_tbolt_init_globals(struct mfi_s
  }
  
  /*
 - ****************************************************************************
 - * Description:
 - *      This function calculates the memory requirement for Thunderbolt
 - *      controller
 - * Return Value:
 - *      Total required memory in bytes
 - ****************************************************************************
 + * This function calculates the memory requirement for Thunderbolt
 + * controller, returns the total required memory in bytes
   */
  
 -uint32_t mfi_tbolt_get_memory_requirement(struct mfi_softc *sc)
 +uint32_t
 +mfi_tbolt_get_memory_requirement(struct mfi_softc *sc)
  {
  	uint32_t size;
  	size = MEGASAS_THUNDERBOLT_MSG_ALLIGNMENT;	/* for Alignment */
 @@ -260,7 +256,6 @@ uint32_t mfi_tbolt_get_memory_requiremen
  }
  
  /*
 - ****************************************************************************
   * Description:
   *      This function will prepare message pools for the Thunderbolt controller
   * Arguments:
 @@ -269,9 +264,9 @@ uint32_t mfi_tbolt_get_memory_requiremen
   * Return Value:
   *      TRUE if successful
   *      FALSE if failed
 - ****************************************************************************
   */
 -int mfi_tbolt_init_desc_pool(struct mfi_softc *sc, uint8_t* mem_location,
 +int
 +mfi_tbolt_init_desc_pool(struct mfi_softc *sc, uint8_t* mem_location,
      uint32_t tbolt_contg_length)
  {
  	uint32_t     offset = 0;
 @@ -328,10 +323,7 @@ int mfi_tbolt_init_desc_pool(struct mfi_
  }
  
  /*
 - ****************************************************************************
 - * Description:
 - *   This routine prepare and issue INIT2 frame to the Firmware
 - ****************************************************************************
 + * This routine prepare and issue INIT2 frame to the Firmware
   */
  
  int
 @@ -442,7 +434,8 @@ mfi_tbolt_init_MFI_queue(struct mfi_soft
  
  }
  
 -int mfi_tbolt_alloc_cmd(struct mfi_softc *sc)
 +int
 +mfi_tbolt_alloc_cmd(struct mfi_softc *sc)
  {
  	struct mfi_cmd_tbolt *cmd;
  	bus_addr_t io_req_base_phys;
 @@ -515,7 +508,8 @@ int mfi_tbolt_alloc_cmd(struct mfi_softc
  	return 0;
  }
  
 -int mfi_tbolt_reset(struct mfi_softc *sc)
 +int
 +mfi_tbolt_reset(struct mfi_softc *sc)
  {
  	uint32_t fw_state;
  
 @@ -550,7 +544,8 @@ int mfi_tbolt_reset(struct mfi_softc *sc
  /*
   * mfi_intr_tbolt - isr entry point
   */
 -void mfi_intr_tbolt(void *arg)
 +void
 +mfi_intr_tbolt(void *arg)
  {
  	struct mfi_softc *sc = (struct mfi_softc *)arg;
  
 @@ -568,7 +563,7 @@ void mfi_intr_tbolt(void *arg)
  	return;
  }
  
 -/**
 +/*
   * map_cmd_status -	Maps FW cmd status to OS cmd status
   * @cmd :		Pointer to cmd
   * @status :		status of cmd returned by FW
 @@ -581,7 +576,6 @@ map_tbolt_cmd_status(struct mfi_command 
  {
  
  	switch (status) {
 -
  		case MFI_STAT_OK:
  			mfi_cmd->cm_frame->header.cmd_status = 0;
  			mfi_cmd->cm_frame->dcmd.header.cmd_status = 0;
 @@ -614,7 +608,7 @@ map_tbolt_cmd_status(struct mfi_command 
  		}
  }
  
 -/**
 +/*
   * mfi_tbolt_return_cmd -	Return a cmd to free command pool
   * @instance:		Adapter soft state
   * @cmd:		Command packet to be returned to free command pool
 @@ -627,10 +621,11 @@ mfi_tbolt_return_cmd(struct mfi_softc *s
  	TAILQ_INSERT_TAIL(&sc->mfi_cmd_tbolt_tqh, cmd, next);
  }
  
 -void mfi_tbolt_complete_cmd(struct mfi_softc *sc)
 +void
 +mfi_tbolt_complete_cmd(struct mfi_softc *sc)
  {
  	struct mfi_mpi2_reply_header *desc, *reply_desc;
 -	struct mfi_command *cmd_mfi;	/* For MFA Cmds */
 +	struct mfi_command *cmd_mfi, *cmd_mfi_check;	/* For MFA Cmds */
  	struct mfi_cmd_tbolt *cmd_tbolt;
  	uint16_t smid;
  	uint8_t reply_descript_type;
 @@ -657,7 +652,6 @@ void mfi_tbolt_complete_cmd(struct mfi_s
  
  	/* Read Reply descriptor */
  	while ((val.u.low != 0xFFFFFFFF) && (val.u.high != 0xFFFFFFFF)) {
 -
  		smid = reply_desc->SMID;
  		if (!smid || smid > sc->mfi_max_fw_cmds + 1) {
  			device_printf(sc->mfi_dev, "smid is %x. Cannot "
 @@ -669,66 +663,20 @@ void mfi_tbolt_complete_cmd(struct mfi_s
  		cmd_mfi = &sc->mfi_commands[cmd_tbolt->sync_cmd_idx];
  		scsi_io_req = cmd_tbolt->io_request;
  
 -		/* Check if internal commands */
  		status = cmd_mfi->cm_frame->dcmd.header.cmd_status;
  		extStatus = cmd_mfi->cm_frame->dcmd.header.scsi_status;
 +		map_tbolt_cmd_status(cmd_mfi, status, extStatus);
  
 -		switch (scsi_io_req->Function) {
 -		case MPI2_FUNCTION_LD_IO_REQUEST:
 -			/* Regular Path IO. */
 -			/* Map the Fw Error Status. */
 -			map_tbolt_cmd_status(cmd_mfi, status,
 -			    extStatus);
 -			if ((cmd_mfi->cm_frame->dcmd.opcode
 -			    == MFI_DCMD_LD_MAP_GET_INFO)
 -			    && (cmd_mfi->cm_frame->dcmd.mbox[1] == 1)) {
 -					if (cmd_mfi->cm_frame->header.cmd_status
 -					    != 0)
 -						device_printf(sc->mfi_dev,
 -						    "map sync failed\n");
 -					else {
 -						sc->map_id++;
 -						device_printf(sc->mfi_dev,
 -						    "map sync completed\n");
 -						mfi_release_command(cmd_mfi);
 -					}
 -				}
 -			if ((cmd_mfi->cm_flags & MFI_ON_MFIQ_BUSY)
 -			    == MFI_ON_MFIQ_BUSY
 -			    && (cmd_mfi->cm_flags & MFI_CMD_POLLED) == 0) {
 -				/* BHARAT poll workaround */
 -				mfi_remove_busy(cmd_mfi);
 -				cmd_mfi->cm_error = 0;
 -				mfi_complete(sc, cmd_mfi);
 -			}
 -			mfi_tbolt_return_cmd(sc, cmd_tbolt);
 -			break;
 -		case MPI2_FUNCTION_PASSTHRU_IO_REQUEST:
 -			map_tbolt_cmd_status(cmd_mfi, status, extStatus);
 -			if ((cmd_mfi->cm_frame->dcmd.opcode
 -			    == MFI_DCMD_LD_MAP_GET_INFO)
 -			    && (cmd_mfi->cm_frame->dcmd.mbox[1] == 1)) {
 -				if (cmd_mfi->cm_frame->header.cmd_status != 0)
 -					device_printf(sc->mfi_dev,
 -					    "map sync failed\n");
 -				else {
 -					sc->map_id++;
 -					device_printf(sc->mfi_dev,
 -					    "map sync completed\n");
 -					mfi_release_command(cmd_mfi);
 -				}
 -			}
 -			if ((cmd_mfi->cm_flags & MFI_ON_MFIQ_BUSY)
 -			    == MFI_ON_MFIQ_BUSY
 -			    && (cmd_mfi->cm_flags & MFI_CMD_POLLED) == 0) {
 -				/* BHARAT poll workaround */
 +		/* remove command from busy queue if not polled */
 +		TAILQ_FOREACH(cmd_mfi_check, &sc->mfi_busy, cm_link) {
 +			if (cmd_mfi_check == cmd_mfi) {
  				mfi_remove_busy(cmd_mfi);
 -				cmd_mfi->cm_error = 0;
 -				mfi_complete(sc, cmd_mfi);
 +				break;
  			}
 -			mfi_tbolt_return_cmd(sc, cmd_tbolt);
 -			break;
  		}
 +		cmd_mfi->cm_error = 0;
 +		mfi_complete(sc, cmd_mfi);
 +		mfi_tbolt_return_cmd(sc, cmd_tbolt);
  
  		sc->last_reply_idx++;
  		if (sc->last_reply_idx >= sc->mfi_max_fw_cmds) {
 @@ -763,15 +711,15 @@ void mfi_tbolt_complete_cmd(struct mfi_s
  	return;
  }
  
 -/**
 +/*
   * mfi_get_cmd -	Get a command from the free pool
   * @instance:		Adapter soft state
   *
   * Returns a free command from the pool
   */
  
 -struct mfi_cmd_tbolt *mfi_tbolt_get_cmd(struct mfi_softc
 -						  *sc)
 +struct mfi_cmd_tbolt *
 +mfi_tbolt_get_cmd(struct mfi_softc *sc)
  {
  	struct mfi_cmd_tbolt *cmd = NULL;
  
 @@ -876,7 +824,8 @@ mfi_tbolt_build_ldio(struct mfi_softc *s
  	    * MFI_SECTOR_LEN;
  }
  
 -int mfi_tbolt_is_ldio(struct mfi_command *mfi_cmd)
 +int
 +mfi_tbolt_is_ldio(struct mfi_command *mfi_cmd)
  {
  	if (mfi_cmd->cm_frame->header.cmd == MFI_CMD_LD_READ
  	    || mfi_cmd->cm_frame->header.cmd == MFI_CMD_LD_WRITE)
 @@ -886,7 +835,8 @@ int mfi_tbolt_is_ldio(struct mfi_command
  }
  
  int
 -mfi_tbolt_build_io(struct mfi_softc *sc, struct mfi_command *mfi_cmd, struct mfi_cmd_tbolt *cmd)
 +mfi_tbolt_build_io(struct mfi_softc *sc, struct mfi_command *mfi_cmd,
 +    struct mfi_cmd_tbolt *cmd)
  {
  	uint32_t device_id;
  	uint32_t sge_count;
 @@ -949,7 +899,7 @@ mfi_tbolt_build_cdb(struct mfi_softc *sc
  	lba_lo = mfi_cmd->cm_frame->io.lba_lo;
  	lba_hi = mfi_cmd->cm_frame->io.lba_hi;
  
 -	if ((num_lba <= 0xFF) && (lba_lo <= 0x1FFFFF)) {
 +	if (lba_hi == 0 && (num_lba <= 0xFF) && (lba_lo <= 0x1FFFFF)) {
  		if (mfi_cmd->cm_frame->header.cmd == MFI_CMD_LD_WRITE)
  			/* Read 6 or Write 6 */
  			cdb[0] = (uint8_t) (0x0A);
 @@ -962,7 +912,7 @@ mfi_tbolt_build_cdb(struct mfi_softc *sc
  		cdb[1] = (uint8_t) ((lba_lo >> 16) & 0x1F);
  		cdb_len = 6;
  	}
 -	else if ((num_lba <= 0xFFFF) && (lba_lo <= 0xFFFFFFFF)) {
 +	else if (lba_hi == 0 && (num_lba <= 0xFFFF) && (lba_lo <= 0xFFFFFFFF)) {
  		if (mfi_cmd->cm_frame->header.cmd == MFI_CMD_LD_WRITE)
  			/* Read 10 or Write 10 */
  			cdb[0] = (uint8_t) (0x2A);
 @@ -975,8 +925,7 @@ mfi_tbolt_build_cdb(struct mfi_softc *sc
  		cdb[3] = (uint8_t) (lba_lo >> 16);
  		cdb[2] = (uint8_t) (lba_lo >> 24);
  		cdb_len = 10;
 -	}
 -	else if ((num_lba > 0xFFFF) && (lba_hi == 0)) {
 +	} else if ((num_lba > 0xFFFF) && (lba_hi == 0)) {
  		if (mfi_cmd->cm_frame->header.cmd == MFI_CMD_LD_WRITE)
  			/* Read 12 or Write 12 */
  			cdb[0] = (uint8_t) (0xAA);
 @@ -1152,8 +1101,8 @@ mfi_tbolt_send_frame(struct mfi_softc *s
  		cm->cm_timestamp = time_uptime;
  		mfi_enqueue_busy(cm);
  	}
 -	else {
 -		hdr->cmd_status = 0xff;
 +	else {	/* still get interrupts for it */
 +		hdr->cmd_status = MFI_STAT_INVALID_STATUS;
  		hdr->flags |= MFI_FRAME_DONT_POST_IN_REPLY_QUEUE;
  	}
  
 @@ -1189,22 +1138,23 @@ mfi_tbolt_send_frame(struct mfi_softc *s
  		return 0;
  
  	/* This is a polled command, so busy-wait for it to complete. */
 -	while (hdr->cmd_status == 0xff) {
 +	while (hdr->cmd_status == MFI_STAT_INVALID_STATUS) {
  		DELAY(1000);
  		tm -= 1;
  		if (tm <= 0)
 -			break;
 +		break;
  	}
  
 -	if (hdr->cmd_status == 0xff) {
 +	if (hdr->cmd_status == MFI_STAT_INVALID_STATUS) {
  		device_printf(sc->mfi_dev, "Frame %p timed out "
 -		      "command 0x%X\n", hdr, cm->cm_frame->dcmd.opcode);
 +		    "command 0x%X\n", hdr, cm->cm_frame->dcmd.opcode);
  		return (ETIMEDOUT);
  	}
  	return 0;
  }
  
 -static void mfi_issue_pending_cmds_again (struct mfi_softc *sc)
 +static void
 +mfi_issue_pending_cmds_again (struct mfi_softc *sc)
  {
  	struct mfi_command *cm, *tmp;
  
 @@ -1249,7 +1199,8 @@ static void mfi_issue_pending_cmds_again
  	mfi_startio(sc);
  }
  
 -static void mfi_kill_hba (struct mfi_softc *sc)
 +static void
 +mfi_kill_hba (struct mfi_softc *sc)
  {
  	if (sc->mfi_flags & MFI_FLAGS_TBOLT)
  		MFI_WRITE4 (sc, 0x00,MFI_STOP_ADP);
 @@ -1257,7 +1208,8 @@ static void mfi_kill_hba (struct mfi_sof
  		MFI_WRITE4 (sc, MFI_IDB,MFI_STOP_ADP);
  }
  
 -static void mfi_process_fw_state_chg_isr(void *arg)
 +static void
 +mfi_process_fw_state_chg_isr(void *arg)
  {
  	struct mfi_softc *sc= (struct mfi_softc *)arg;
  	struct mfi_cmd_tbolt *cmd;
 @@ -1308,9 +1260,9 @@ static void mfi_process_fw_state_chg_isr
  			mfi_release_command(sc->mfi_aen_cm);
  			sc->mfi_aen_cm = NULL;
  		}
 -		if (sc->map_update_cmd) {
 -			mfi_release_command(sc->map_update_cmd);
 -			sc->map_update_cmd = NULL;
 +		if (sc->mfi_map_sync_cm) {
 +			mfi_release_command(sc->mfi_map_sync_cm);
 +			sc->mfi_map_sync_cm = NULL;
  		}
  		mfi_issue_pending_cmds_again(sc);
  
 @@ -1337,3 +1289,178 @@ static void mfi_process_fw_state_chg_isr
  	}
  	mtx_unlock(&sc->mfi_io_lock);
  }
 +
 +/*
 + * The ThunderBolt HW has an option for the driver to directly
 + * access the underlying disks and operate on the RAID.  To
 + * do this there needs to be a capability to keep the RAID controller
 + * and driver in sync.  The FreeBSD driver does not take advantage
 + * of this feature since it adds a lot of complexity and slows down
 + * performance.  Performance is gained by using the controller's
 + * cache etc.
 + *
 + * Even though this driver doesn't access the disks directly, an
 + * AEN like command is used to inform the RAID firmware to "sync"
 + * with all LD's via the MFI_DCMD_LD_MAP_GET_INFO command.  This
 + * command in write mode will return when the RAID firmware has
 + * detected a change to the RAID state.  Examples of this type
 + * of change are removing a disk.  Once the command returns then
 + * the driver needs to acknowledge this and "sync" all LD's again.
 + * This repeats until we shutdown.  Then we need to cancel this
 + * pending command.
 + *
 + * If this is not done right the RAID firmware will not remove a
 + * pulled drive and the RAID won't go degraded etc.  Effectively,
 + * stopping any RAID mangement to functions.
 + *
 + * Doing another LD sync, requires the use of an event since the
 + * driver needs to do a mfi_wait_command and can't do that in an
 + * interrupt thread.
 + *
 + * The driver could get the RAID state via the MFI_DCMD_LD_MAP_GET_INFO
 + * That requires a bunch of structure and it is simplier to just do
 + * the MFI_DCMD_LD_GET_LIST versus walking the RAID map.
 + */
 +
 +void
 +mfi_tbolt_sync_map_info(struct mfi_softc *sc)
 +{
 +	int error = 0, i;
 +	struct mfi_command *cmd;
 +	struct mfi_dcmd_frame *dcmd;
 +	uint32_t context = 0;
 +	union mfi_ld_ref *ld_sync;
 +	size_t ld_size;
 +	struct mfi_frame_header *hdr;
 +	struct mfi_command *cm = NULL;
 +	struct mfi_ld_list *list = NULL;
 +
 +	if (sc->mfi_map_sync_cm != NULL || sc->cm_map_abort)
 +		return;
 +
 +	mtx_lock(&sc->mfi_io_lock);
 +	error = mfi_dcmd_command(sc, &cm, MFI_DCMD_LD_GET_LIST,
 +	    (void **)&list, sizeof(*list));
 +	if (error)
 +		goto out;
 +
 +	cm->cm_flags = MFI_CMD_POLLED | MFI_CMD_DATAIN;
 +	if (mfi_wait_command(sc, cm) != 0) {
 +		device_printf(sc->mfi_dev, "Failed to get device listing\n");
 +		goto out;
 +	}
 +
 +	hdr = &cm->cm_frame->header;
 +	if (hdr->cmd_status != MFI_STAT_OK) {
 +		device_printf(sc->mfi_dev, "MFI_DCMD_LD_GET_LIST failed %x\n",
 +			      hdr->cmd_status);
 +		goto out;
 +	}
 +
 +	ld_size = sizeof(*ld_sync) * list->ld_count;
 +	mtx_unlock(&sc->mfi_io_lock);
 +	ld_sync = (union mfi_ld_ref *) malloc(ld_size, M_MFIBUF,
 +	     M_WAITOK | M_ZERO);
 +	if (ld_sync == NULL) {
 +		device_printf(sc->mfi_dev, "Failed to allocate sync\n");
 +		goto out;
 +	}
 +	for (i = 0; i < list->ld_count; i++) {
 +		ld_sync[i].ref = list->ld_list[i].ld.ref;
 +	}
 +
 +	mtx_lock(&sc->mfi_io_lock);
 +	if ((cmd = mfi_dequeue_free(sc)) == NULL) {
 +		device_printf(sc->mfi_dev, "Failed to get command\n");
 +		free(ld_sync, M_MFIBUF);
 +		goto out;
 +	}
 +	
 +	context = cmd->cm_frame->header.context;
 +	bzero(cmd->cm_frame, sizeof(union mfi_frame));
 +	cmd->cm_frame->header.context = context;
 +
 +	dcmd = &cmd->cm_frame->dcmd;
 +	bzero(dcmd->mbox, MFI_MBOX_SIZE);
 +	dcmd->header.cmd = MFI_CMD_DCMD;
 +	dcmd->header.flags = MFI_FRAME_DIR_WRITE;
 +	dcmd->header.timeout = 0;
 +	dcmd->header.data_len = ld_size;
 +	dcmd->header.scsi_status = 0;
 +	dcmd->opcode = MFI_DCMD_LD_MAP_GET_INFO;
 +	cmd->cm_sg = &dcmd->sgl;
 +	cmd->cm_total_frame_size = MFI_DCMD_FRAME_SIZE;
 +	cmd->cm_data = ld_sync;
 +	cmd->cm_private = ld_sync;
 +
 +	cmd->cm_len = ld_size;
 +	cmd->cm_complete = mfi_sync_map_complete;
 +	sc->mfi_map_sync_cm = cmd;
 +
 +	cmd->cm_flags = MFI_CMD_DATAOUT;
 +	cmd->cm_frame->dcmd.mbox[0] = list->ld_count;
 +	cmd->cm_frame->dcmd.mbox[1] = MFI_DCMD_MBOX_PEND_FLAG;
 +
 +	if ((error = mfi_mapcmd(sc, cmd)) != 0) {
 +		device_printf(sc->mfi_dev, "failed to send map sync\n");
 +		free(ld_sync, M_MFIBUF);
 +		sc->mfi_map_sync_cm = NULL;
 +		mfi_requeue_ready(cmd);
 +		goto out;
 +	}
 +
 +out:
 +	if (list)
 +		free(list, M_MFIBUF);
 +	if (cm)
 +		mfi_release_command(cm);
 +	mtx_unlock(&sc->mfi_io_lock);
 +}
 +
 +static void
 +mfi_sync_map_complete(struct mfi_command *cm)
 +{
 +	struct mfi_frame_header *hdr;
 +	struct mfi_softc *sc;
 +	int aborted = 0;
 +
 +	sc = cm->cm_sc;
 +	mtx_assert(&sc->mfi_io_lock, MA_OWNED);
 +
 +	hdr = &cm->cm_frame->header;
 +
 +	if (sc->mfi_map_sync_cm == NULL)
 +		return;
 +
 +	if (sc->cm_map_abort ||
 +	    hdr->cmd_status == MFI_STAT_INVALID_STATUS) {
 +		sc->cm_map_abort = 0;
 +		aborted = 1;
 +	}
 +
 +	free(cm->cm_data, M_MFIBUF);
 +	sc->mfi_map_sync_cm = NULL;
 +	wakeup(&sc->mfi_map_sync_cm);
 +	mfi_release_command(cm);
 +
 +	/* set it up again so the driver can catch more events */
 +	if (!aborted) {
 +		mfi_queue_map_sync(sc);
 +	}
 +}
 +
 +static void
 +mfi_queue_map_sync(struct mfi_softc *sc)
 +{
 +	mtx_assert(&sc->mfi_io_lock, MA_OWNED);
 +	taskqueue_enqueue(taskqueue_swi, &sc->mfi_map_sync_task);
 +}
 +
 +void
 +mfi_handle_map_sync(void *context, int pending)
 +{
 +	struct mfi_softc *sc;
 +
 +	sc = context;
 +	mfi_tbolt_sync_map_info(sc);
 +}
 
 Modified: stable/9/sys/dev/mfi/mfireg.h
 ==============================================================================
 --- stable/9/sys/dev/mfi/mfireg.h	Mon May  7 18:30:22 2012	(r235134)
 +++ stable/9/sys/dev/mfi/mfireg.h	Mon May  7 19:17:09 2012	(r235135)
 @@ -403,6 +403,7 @@ typedef enum {
  #define MR_EVT_CTRL_HOST_BUS_SCAN_REQUESTED	0x0152
  #define MR_EVT_PD_REMOVED			0x0070
  #define MR_EVT_PD_INSERTED			0x005b
 +#define MR_EVT_LD_CHANGE			0x0051
  
  typedef enum {
  	MR_LD_CACHE_WRITE_BACK =	0x01,
 
 Modified: stable/9/sys/dev/mfi/mfivar.h
 ==============================================================================
 --- stable/9/sys/dev/mfi/mfivar.h	Mon May  7 18:30:22 2012	(r235134)
 +++ stable/9/sys/dev/mfi/mfivar.h	Mon May  7 19:17:09 2012	(r235135)
 @@ -105,7 +105,6 @@ struct mfi_command {
  #define MFI_ON_MFIQ_READY	(1<<6)
  #define MFI_ON_MFIQ_BUSY	(1<<7)
  #define MFI_ON_MFIQ_MASK	((1<<5)|(1<<6)|(1<<7))
 -	int			cm_aen_abort;
  	uint8_t			retry_for_fw_reset;
  	void			(* cm_complete)(struct mfi_command *cm);
  	void			*cm_private;
 @@ -216,9 +215,13 @@ struct mfi_softc {
  
  	TAILQ_HEAD(,mfi_evt_queue_elm)	mfi_evt_queue;
  	struct task			mfi_evt_task;
 +	struct task			mfi_map_sync_task;
  	TAILQ_HEAD(,mfi_aen)		mfi_aen_pids;
  	struct mfi_command		*mfi_aen_cm;
  	struct mfi_command		*mfi_skinny_cm;
 +	struct mfi_command		*mfi_map_sync_cm;
 +	int				cm_aen_abort;
 +	int				cm_map_abort;
  	uint32_t			mfi_aen_triggered;
  	uint32_t			mfi_poll_waiting;
  	uint32_t			mfi_boot_seq_num;
 @@ -303,8 +306,6 @@ struct mfi_softc {
  	/* ThunderBolt */
  	uint32_t			mfi_tbolt;
  	uint32_t			MFA_enabled;
 -	uint64_t			map_id;
 -	struct mfi_command 		*map_update_cmd;
  	/* Single Reply structure size */
  	uint16_t			reply_size;
  	/* Singler message size. */
 @@ -417,7 +418,10 @@ extern int mfi_tbolt_alloc_cmd(struct mf
  extern int mfi_tbolt_send_frame(struct mfi_softc *sc, struct mfi_command *cm);
  extern int mfi_tbolt_adp_reset(struct mfi_softc *sc);
  extern int mfi_tbolt_reset(struct mfi_softc *sc);
 -extern int mfi_tbolt_sync_map_info(struct mfi_softc *sc);
 +extern void mfi_tbolt_sync_map_info(struct mfi_softc *sc);
 +extern void mfi_handle_map_sync(void *context, int pending);
 +extern int mfi_dcmd_command(struct mfi_softc *, struct mfi_command **,
 +		    uint32_t, void **, size_t);
  
  #define MFIQ_ADD(sc, qname)					\
  	do {							\
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 
>Unformatted:
