From ai@kliksys.ru  Thu Oct 29 20:47:26 2009
Return-Path: <ai@kliksys.ru>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 86225106566C;
	Thu, 29 Oct 2009 20:47:26 +0000 (UTC)
	(envelope-from ai@kliksys.ru)
Received: from gate.kliksys.ru (gate.kliksys.ru [78.110.241.113])
	by mx1.freebsd.org (Postfix) with ESMTP id 306BF8FC0A;
	Thu, 29 Oct 2009 20:47:26 +0000 (UTC)
Received: from [192.168.1.13] (helo=one.kliksys.ru)
	by gate.kliksys.ru with esmtp (Exim 4.69 (FreeBSD))
	(envelope-from <ai@kliksys.ru>)
	id 1N3bF6-0005hY-HA; Thu, 29 Oct 2009 23:05:09 +0300
Received: from one.kliksys.ru (localhost [127.0.0.1])
	by one.kliksys.ru (8.14.3/8.14.3) with ESMTP id n9TK59fo013671;
	Thu, 29 Oct 2009 23:05:09 +0300 (MSK)
	(envelope-from ai@kliksys.ru)
Received: (from ai@localhost)
	by one.kliksys.ru (8.14.3/8.14.3/Submit) id n9TK58rJ013670;
	Thu, 29 Oct 2009 23:05:08 +0300 (MSK)
	(envelope-from ai)
Message-Id: <200910292005.n9TK58rJ013670@one.kliksys.ru>
Date: Thu, 29 Oct 2009 23:05:08 +0300 (MSK)
From: Artemiev Igor <ai@kliksys.ru>
Reply-To: Artemiev Igor <ai@kliksys.ru>
To: FreeBSD-gnats-submit@freebsd.org
Cc: gibbs@freebsd.org, ken@freebsd.org
Subject: [PATCH] allow for da(4) large block transfer than DFLTPHYS
X-Send-Pr-Version: 3.113
X-GNATS-Notify:

>Number:         140091
>Category:       kern
>Synopsis:       [da] [patch] allow for da(4) large block transfer than DFLTPHYS
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    freebsd-scsi
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Thu Oct 29 20:50:01 UTC 2009
>Closed-Date:    Fri Mar 12 06:07:01 UTC 2010
>Last-Modified:  Fri Mar 12 06:07:01 UTC 2010
>Originator:     Artemiev Igor
>Release:        FreeBSD 7.2-STABLE i386
>Organization:
KlikSys
>Environment:
System: FreeBSD one.kliksys.ru 7.2-STABLE FreeBSD 7.2-STABLE #7: Sun May 17 14:54:10 MSD 2009 root@one.kliksys.ru:/usr/obj/usr/src.7/sys/KERNEL i386

>Description:
    By default, one transfer of data block limiting to DFLTPHYS for da(4), but modern controllers allow operate up to 128K per one DMA transaction, i.e. 3ware devices. This patch via loader.conf tunable allow scale d_maxsize by multiple DFLTPHYS.	
>How-To-Repeat:
    On my typical load (~4K active connections)

    Before:
    #iostat -dt da
                 da0
      KB/t tps  MB/s
    61.38 1731 56.31

# netstat -I igb0 -hw1
            input         (igb0)           output
   packets  errs      bytes    packets  errs      bytes colls
       41K     0        10M        30K     0        68M     0
       41K     0        11M        31K     0        68M     0
       42K     0        11M        31K     0        68M     0
       42K     0        11M        31K     0        68M     0
       42K     0       9.2M        31K     0        68M     0
       42K     0       9.8M        30K     0        59M     0
       42K     0        11M        30K     0        60M     0
       42K     0        11M        31K     0        68M     0
       42K     0        11M        31K     0        68M     0

    After:
#iostat -dt da
                 da0
      KB/t tps  MB/s
    127.29 827 80.18

# netstat -I igb0 -hw1
            input         (igb0)           output
   packets  errs      bytes    packets  errs      bytes colls
       46K     0        16M        69K     0        95M     0
       47K     0        18M        70K     0        95M     0
       48K     0        18M        71K     0        97M     0
       44K     0        13M        70K     0        98M     0
       40K     0       9.2M        68K     0        95M     0
       50K     0        21M        74K     0       100M     0
       49K     0        21M        71K     0        97M     0
       46K     0        16M        72K     0        99M     0
       51K     0        22M        75K     0       101M     0

>Fix:
    Apply this patch and reboot with kern.cam.da.max_size="2" in loader.conf

--- sys/cam/scsi/scsi_da.c.orig	2009-10-28 14:53:43.000000000 +0100
+++ sys/cam/scsi/scsi_da.c	2009-10-28 15:16:26.000000000 +0100
@@ -515,12 +515,16 @@
 static int da_retry_count = DA_DEFAULT_RETRY;
 static int da_default_timeout = DA_DEFAULT_TIMEOUT;
 static int da_send_ordered = DA_DEFAULT_SEND_ORDERED;
+static int da_max_size = 1;
 
 SYSCTL_NODE(_kern_cam, OID_AUTO, da, CTLFLAG_RD, 0,
             "CAM Direct Access Disk driver");
 SYSCTL_INT(_kern_cam_da, OID_AUTO, retry_count, CTLFLAG_RW,
            &da_retry_count, 0, "Normal I/O retry count");
 TUNABLE_INT("kern.cam.da.retry_count", &da_retry_count);
+TUNABLE_INT("kern.cam.da.max_size", &da_max_size);
+SYSCTL_INT(_kern_cam_da, OID_AUTO, max_size, CTLFLAG_RD,
+           &da_max_size, 0, "DFLTPHYS scale");
 SYSCTL_INT(_kern_cam_da, OID_AUTO, default_timeout, CTLFLAG_RW,
            &da_default_timeout, 0, "Normal I/O timeout (in seconds)");
 TUNABLE_INT("kern.cam.da.default_timeout", &da_default_timeout);
@@ -1201,7 +1205,10 @@
 	softc->disk->d_dump = dadump;
 	softc->disk->d_name = "da";
 	softc->disk->d_drv1 = periph;
-	softc->disk->d_maxsize = DFLTPHYS; /* XXX: probably not arbitrary */
+
+	if(da_max_size > 2)
+	    da_max_size = 1;
+	softc->disk->d_maxsize = da_max_size*DFLTPHYS; /* XXX: probably not arbitrary */
 	softc->disk->d_unit = periph->unit_number;
 	softc->disk->d_flags = DISKFLAG_NEEDSGIANT;
 	disk_create(softc->disk, DISK_VERSION);

>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->freebsd-scsi 
Responsible-Changed-By: linimon 
Responsible-Changed-When: Thu Oct 29 23:54:55 UTC 2009 
Responsible-Changed-Why:  
Over to maintainer(s). 

http://www.freebsd.org/cgi/query-pr.cgi?pr=140091 
State-Changed-From-To: open->closed 
State-Changed-By: mav 
State-Changed-When: Fri Mar 12 06:03:29 UTC 2010 
State-Changed-Why:  
Support for larger I/O sizes were added to 8-STABLE. 
Not much controller drivers now announce such capability, 
but it is different question. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=140091 
>Unformatted:
