From nobody@FreeBSD.org  Fri Sep  9 08:10:08 2011
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id CE6CF106564A
	for <freebsd-gnats-submit@FreeBSD.org>; Fri,  9 Sep 2011 08:10:08 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from red.freebsd.org (red.freebsd.org [IPv6:2001:4f8:fff6::22])
	by mx1.freebsd.org (Postfix) with ESMTP id A45F98FC18
	for <freebsd-gnats-submit@FreeBSD.org>; Fri,  9 Sep 2011 08:10:08 +0000 (UTC)
Received: from red.freebsd.org (localhost [127.0.0.1])
	by red.freebsd.org (8.14.4/8.14.4) with ESMTP id p898A8rZ077528
	for <freebsd-gnats-submit@FreeBSD.org>; Fri, 9 Sep 2011 08:10:08 GMT
	(envelope-from nobody@red.freebsd.org)
Received: (from nobody@localhost)
	by red.freebsd.org (8.14.4/8.14.4/Submit) id p898A848077527;
	Fri, 9 Sep 2011 08:10:08 GMT
	(envelope-from nobody)
Message-Id: <201109090810.p898A848077527@red.freebsd.org>
Date: Fri, 9 Sep 2011 08:10:08 GMT
From: Garrett Cooper <yaneurabeya@gmail.com>
To: freebsd-gnats-submit@FreeBSD.org
Subject: mfiutil doesn't properly detect dead battery with mfiutil show battery
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         160581
>Category:       bin
>Synopsis:       mfiutil(8) doesn't properly detect dead battery with mfiutil show battery
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    emaste
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Fri Sep 09 08:20:04 UTC 2011
>Closed-Date:    Tue Oct 04 00:29:46 UTC 2011
>Last-Modified:  Sun Feb 03 22:29:25 UTC 2013
>Originator:     Garrett Cooper
>Release:        9.0-BETA1
>Organization:
iXsystems, Inc.
>Environment:
FreeBSD bayonetta.local 9.0-BETA1 FreeBSD 9.0-BETA1 #0 r224946M: Thu Aug 18 10:02:09 PDT 2011     gcooper@bayonetta.local:/usr/obj/usr/src/sys/BAYONETTA  amd64
>Description:
Long story short, my RAID battery died 2 weeks ago. The controller is reporting the failure, along with the mfiutil cache show command, but mfiutil isn't detecting the correct battery state when running mfiutil show battery. Example:

$ sudo mfiutil show battery
mfi0: Battery State:
     Manufacture Date: 9/19/2009
        Serial Number: 1022
         Manufacturer: LS1111001B
                Model: 3598301
            Chemistry: LION
      Design Capacity: 1215 mAh
 Full Charge Capacity: 417 mAh
     Current Capacity: 397 mAh
        Charge Cycles: 58
       Current Charge: 95%
       Design Voltage: 3700 mV
      Current Voltage: 4067 mV
          Temperature: 40 C
               Status: normal <-- HERE
$ dmesg | grep batt
mfi0: 35492 (boot + 4s/0x0008/FATAL) - Battery has failed and cannot support data retention. Please replace the battery
mfi0: 35623 (368111183s/0x0008/FATAL) - Battery has failed and cannot support data retention. Please replace the battery
$ sudo mfiutil cache mfid0
mfi0 volume mfid0 cache settings:
             I/O caching: writes and reads
           write caching: write-back
write cache with bad BBU: disabled
              read ahead: always
       drive write cache: enabled
Cache Disabled Due to Dead Battery

MegaCli does though, of course:

$ sudo ~/MegaCli -AdpBbuCmd -GetBbuS
                                     
BBU status for Adapter: 0

BatteryType: iBBU
Voltage: 4051 mV
Current: 0 mA
Temperature: 40 C

BBU Firmware Status:

  Charging Status              : None
  Voltage                      : OK
  Temperature                  : OK
  Learn Cycle Requested        : No
  Learn Cycle Active           : No
  Learn Cycle Status           : OK
  Learn Cycle Timeout          : No
  I2c Errors Detected          : No
  Battery Pack Missing         : No
  Battery Replacement required : Yes
  Remaining Capacity Low       : Yes
  Periodic Learn Required      : No
  Transparent Learn            : No

Battery state: 

GasGuageStatus:
  Fully Discharged        : No
  Fully Charged           : Yes
  Discharging             : Yes
  Initialized             : Yes
  Remaining Time Alarm    : No
  Remaining Capacity Alarm: No
  Discharge Terminated    : No
  Over Temperature        : No
  Charging Terminated     : No
  Over Charged            : No

Relative State of Charge: 100 %
Charger System State: 49168
Charger System Ctrl: 0
Charging current: 0 mA
Absolute state of charge: 34 %
Max Error: 2 %

Exit Code: 0x00
>How-To-Repeat:

>Fix:


>Release-Note:
>Audit-Trail:

From: Mark <markjdb@gmail.com>
To: bug-followup@FreeBSD.org, gcooper@ixsystems.com
Cc:  
Subject: Re: bin/160581: mfiutil(8) doesn&#39;t properly detect dead battery
 with mfiutil show battery
Date: Sat, 10 Sep 2011 15:58:27 -0400

 --20cf307c9f1ca842bb04ac9bb7de
 Content-Type: text/plain; charset=ISO-8859-1
 
 Hi Garrett,
 
 It looks like mfi(4) is missing a mask or several for the fw_status
 field when it tries to grab info about the battery backup. Maybe the
 attached patch can help a bit in figuring out what mfiutil is
 ignoring. I don't know if there's a publicly available datasheet for
 the firmware; I couldn't find it last time I tried. Maybe jhb@ has
 access to one. There's nothing useful in LSI's FreeBSD or Linux
 drivers - probably the opcodes and masks are just hardcoded into
 MegaCLI.
 
 If there's no way to figure out the masks from a datasheet, it'll
 probably be necessary to disassemble MegaCLI or something. I can try
 and figure it out, but I don't have access to a machine with an LSI
 controller right now, so it'd be tricky. =(
 
 Thanks,
 -Mark
 
 --20cf307c9f1ca842bb04ac9bb7de
 Content-Type: text/plain; charset=US-ASCII; name="mfiutil_debug.patch.txt"
 Content-Disposition: attachment; filename="mfiutil_debug.patch.txt"
 Content-Transfer-Encoding: base64
 X-Attachment-Id: f_gsf0odhw0
 
 ZGlmZiAtLWdpdCBhL3Vzci5zYmluL21maXV0aWwvbWZpX3Nob3cuYyBiL3Vzci5zYmluL21maXV0
 aWwvbWZpX3Nob3cuYwppbmRleCA5ZDlmN2ExLi4wOWUxNGFhIDEwMDY0NAotLS0gYS91c3Iuc2Jp
 bi9tZml1dGlsL21maV9zaG93LmMKKysrIGIvdXNyLnNiaW4vbWZpdXRpbC9tZmlfc2hvdy5jCkBA
 IC0yMjAsNiArMjIwLDcgQEAgc2hvd19iYXR0ZXJ5KGludCBhYywgY2hhciAqKmF2KQogCWlmIChz
 dGF0LmZ3X3N0YXR1cyAmIE1GSV9CQlVfU1RBVEVfRElTQ0hBUkdFX0FDVElWRSkgewogCQlwcmlu
 dGYoIiVzIERJU0NIQVJHSU5HIiwgY29tbWEgPyAiLCIgOiAiIik7CiAJfQorCXByaW50ZigiZncg
 c3RhdHVzOiAleFxuIiwgc3RhdC5md19zdGF0dXMpOwogCWlmICghY29tbWEpCiAJCXByaW50Zigi
 IG5vcm1hbCIpOwogCXByaW50ZigiXG4iKTsK
 --20cf307c9f1ca842bb04ac9bb7de--

From: Garrett Cooper <yaneurabeya@gmail.com>
To: Mark <markjdb@gmail.com>
Cc: bug-followup@FreeBSD.org
Subject: Re: bin/160581: mfiutil(8) doesn&#39;t properly detect dead battery
 with mfiutil show battery
Date: Sat, 10 Sep 2011 17:34:19 -0700 (PDT)

 On Sat, 10 Sep 2011, Mark wrote:
 
 > Hi Garrett,
 >
 > It looks like mfi(4) is missing a mask or several for the fw_status
 > field when it tries to grab info about the battery backup. Maybe the
 > attached patch can help a bit in figuring out what mfiutil is
 > ignoring. I don't know if there's a publicly available datasheet for
 > the firmware; I couldn't find it last time I tried. Maybe jhb@ has
 > access to one. There's nothing useful in LSI's FreeBSD or Linux
 > drivers - probably the opcodes and masks are just hardcoded into
 > MegaCLI.
 >
 > If there's no way to figure out the masks from a datasheet, it'll
 > probably be necessary to disassemble MegaCLI or something. I can try
 > and figure it out, but I don't have access to a machine with an LSI
 > controller right now, so it'd be tricky. =(
 
 Hi Mark,
  	That's exactly what I tried before, but didn't include in my PR 
 email. Here's what I got:
 
 $ sudo mfiutil show battery
 mfi0: Battery State:
       Manufacture Date: 9/19/2009
          Serial Number: 1022
           Manufacturer: LS1111001B
                  Model: 3598301
              Chemistry: LION
        Design Capacity: 1215 mAh
   Full Charge Capacity: 417 mAh
       Current Capacity: 415 mAh
          Charge Cycles: 58
         Current Charge: 100%
         Design Voltage: 3700 mV
        Current Voltage: 4049 mV
            Temperature: 43 C
                 Status:Firmware status: 3072 <-- HERE
   normal
 
  	LSI's copy of the driver is unfortunately unhelpful here, and I 
 tried ktrace'ing/truss'ing the driver to no avail. I guess the next step 
 is to hack the driver and dump the ioctl output sent via MegaCLI 
 (shouldn't be too hard).
 Thanks!
 -Garrett

From: Mark Johnston <markjdb@gmail.com>
To: Garrett Cooper <yaneurabeya@gmail.com>
Cc: bug-followup@FreeBSD.org
Subject: Re: bin/160581: mfiutil(8) doesn&#39;t properly detect dead battery
 with mfiutil show battery
Date: Sun, 11 Sep 2011 15:00:43 -0400

 On Sat, Sep 10, 2011 at 05:34:19PM -0700, Garrett Cooper wrote:
 > [snip]
 > I guess the next step 
 > is to hack the driver and dump the ioctl output sent via MegaCLI 
 > (shouldn't be too hard).
 
 I don't see how that'd help though; presumably MegaCLI grabs battery
 info in the same way as mfiutil - the MFIIO_PASSTHRU ioctl with opcode
 MFI_DCMD_GET_BBU_STATUS.
 
 Thanks,
 -Mark
Responsible-Changed-From-To: freebsd-bugs->emaste 
Responsible-Changed-By: emaste 
Responsible-Changed-When: Fri Sep 23 02:24:42 UTC 2011 
Responsible-Changed-Why:  
Grab this, Garrett reports the patch I posted fixes this issue.  In the queue 
for re@ approval. 


http://www.freebsd.org/cgi/query-pr.cgi?pr=160581 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: bin/160581: commit references a PR
Date: Tue, 27 Sep 2011 14:28:17 +0000 (UTC)

 Author: emaste
 Date: Tue Sep 27 14:28:07 2011
 New Revision: 225798
 URL: http://svn.freebsd.org/changeset/base/225798
 
 Log:
   Improve battery capacity reporting
   
   When a status pointer is passed in mfi_dcmd_command does not return an
   errno (if the ioctl is successful), so move the test for NO_HW_PRESENT
   outside of the error case.  This should fix incorrect reporting for
   systems with a dead or no battery.
   
   Additionally, handle error codes other than NO_HW_PRESENT by omitting
   the battery capacity display.  LSI's supercap-based parts (CV series)
   report their data using the same interface as battery-based parts,
   except that they do not include the capacity stats (state of charge,
   cumulative charge cycles, etc.)
   
   Reviewd by:   jhb, bz
   Tested by:    pluknet@, Garrett Cooper
   PR:           bin/160581
   MFC after:    1 week
 
 Modified:
   head/usr.sbin/mfiutil/mfi_show.c
 
 Modified: head/usr.sbin/mfiutil/mfi_show.c
 ==============================================================================
 --- head/usr.sbin/mfiutil/mfi_show.c	Tue Sep 27 14:00:32 2011	(r225797)
 +++ head/usr.sbin/mfiutil/mfi_show.c	Tue Sep 27 14:28:07 2011	(r225798)
 @@ -141,7 +141,7 @@ show_battery(int ac, char **av)
  	struct mfi_bbu_design_info design;
  	struct mfi_bbu_status stat;
  	uint8_t status;
 -	int comma, error, fd;
 +	int comma, error, fd, show_capacity;
  
  	if (ac != 1) {
  		warnx("show battery: extra arguments");
 @@ -157,16 +157,17 @@ show_battery(int ac, char **av)
  
  	if (mfi_dcmd_command(fd, MFI_DCMD_BBU_GET_CAPACITY_INFO, &cap,
  	    sizeof(cap), NULL, 0, &status) < 0) {
 -		if (status == MFI_STAT_NO_HW_PRESENT) {
 -			printf("mfi%d: No battery present\n", mfi_unit);
 -			close(fd);
 -			return (0);
 -		}
  		error = errno;
  		warn("Failed to get capacity info");
  		close(fd);
  		return (error);
  	}
 +	if (status == MFI_STAT_NO_HW_PRESENT) {
 +		printf("mfi%d: No battery present\n", mfi_unit);
 +		close(fd);
 +		return (0);
 +	}
 +	show_capacity = (status == MFI_STAT_OK);
  
  	if (mfi_dcmd_command(fd, MFI_DCMD_BBU_GET_DESIGN_INFO, &design,
  	    sizeof(design), NULL, 0, NULL) < 0) {
 @@ -192,10 +193,14 @@ show_battery(int ac, char **av)
  	printf("                Model: %s\n", design.device_name);
  	printf("            Chemistry: %s\n", design.device_chemistry);
  	printf("      Design Capacity: %d mAh\n", design.design_capacity);
 -	printf(" Full Charge Capacity: %d mAh\n", cap.full_charge_capacity);
 -	printf("     Current Capacity: %d mAh\n", cap.remaining_capacity);
 -	printf("        Charge Cycles: %d\n", cap.cycle_count);
 -	printf("       Current Charge: %d%%\n", cap.relative_charge);
 +	if (show_capacity) {
 +		printf(" Full Charge Capacity: %d mAh\n",
 +		    cap.full_charge_capacity);
 +		printf("     Current Capacity: %d mAh\n",
 +		    cap.remaining_capacity);
 +		printf("        Charge Cycles: %d\n", cap.cycle_count);
 +		printf("       Current Charge: %d%%\n", cap.relative_charge);
 +	}
  	printf("       Design Voltage: %d mV\n", design.design_voltage);
  	printf("      Current Voltage: %d mV\n", stat.voltage);
  	printf("          Temperature: %d C\n", stat.temperature);
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: bin/160581: commit references a PR
Date: Tue, 27 Sep 2011 17:11:21 +0000 (UTC)

 Author: emaste
 Date: Tue Sep 27 17:11:09 2011
 New Revision: 225804
 URL: http://svn.freebsd.org/changeset/base/225804
 
 Log:
   MFC r225798: Improve battery capacity reporting
   
     When a status pointer is passed in mfi_dcmd_command does not return an
     errno (if the ioctl is successful), so move the test for NO_HW_PRESENT
     outside of the error case.  This should fix incorrect reporting for
     systems with a dead or no battery.
   
     Additionally, handle error codes other than NO_HW_PRESENT by omitting
     the battery capacity display.  LSI's supercap-based parts (CV series)
     report their data using the same interface as battery-based parts,
     except that they do not include the capacity stats (state of charge,
     cumulative charge cycles, etc.)
   
   PR:           bin/160581
   Approved by:  re
 
 Modified:
   stable/9/usr.sbin/mfiutil/mfi_show.c
 Directory Properties:
   stable/9/usr.sbin/mfiutil/   (props changed)
 
 Modified: stable/9/usr.sbin/mfiutil/mfi_show.c
 ==============================================================================
 --- stable/9/usr.sbin/mfiutil/mfi_show.c	Tue Sep 27 17:04:13 2011	(r225803)
 +++ stable/9/usr.sbin/mfiutil/mfi_show.c	Tue Sep 27 17:11:09 2011	(r225804)
 @@ -141,7 +141,7 @@ show_battery(int ac, char **av)
  	struct mfi_bbu_design_info design;
  	struct mfi_bbu_status stat;
  	uint8_t status;
 -	int comma, error, fd;
 +	int comma, error, fd, show_capacity;
  
  	if (ac != 1) {
  		warnx("show battery: extra arguments");
 @@ -157,16 +157,17 @@ show_battery(int ac, char **av)
  
  	if (mfi_dcmd_command(fd, MFI_DCMD_BBU_GET_CAPACITY_INFO, &cap,
  	    sizeof(cap), NULL, 0, &status) < 0) {
 -		if (status == MFI_STAT_NO_HW_PRESENT) {
 -			printf("mfi%d: No battery present\n", mfi_unit);
 -			close(fd);
 -			return (0);
 -		}
  		error = errno;
  		warn("Failed to get capacity info");
  		close(fd);
  		return (error);
  	}
 +	if (status == MFI_STAT_NO_HW_PRESENT) {
 +		printf("mfi%d: No battery present\n", mfi_unit);
 +		close(fd);
 +		return (0);
 +	}
 +	show_capacity = (status == MFI_STAT_OK);
  
  	if (mfi_dcmd_command(fd, MFI_DCMD_BBU_GET_DESIGN_INFO, &design,
  	    sizeof(design), NULL, 0, NULL) < 0) {
 @@ -192,10 +193,14 @@ show_battery(int ac, char **av)
  	printf("                Model: %s\n", design.device_name);
  	printf("            Chemistry: %s\n", design.device_chemistry);
  	printf("      Design Capacity: %d mAh\n", design.design_capacity);
 -	printf(" Full Charge Capacity: %d mAh\n", cap.full_charge_capacity);
 -	printf("     Current Capacity: %d mAh\n", cap.remaining_capacity);
 -	printf("        Charge Cycles: %d\n", cap.cycle_count);
 -	printf("       Current Charge: %d%%\n", cap.relative_charge);
 +	if (show_capacity) {
 +		printf(" Full Charge Capacity: %d mAh\n",
 +		    cap.full_charge_capacity);
 +		printf("     Current Capacity: %d mAh\n",
 +		    cap.remaining_capacity);
 +		printf("        Charge Cycles: %d\n", cap.cycle_count);
 +		printf("       Current Charge: %d%%\n", cap.relative_charge);
 +	}
  	printf("       Design Voltage: %d mV\n", design.design_voltage);
  	printf("      Current Voltage: %d mV\n", stat.voltage);
  	printf("          Temperature: %d C\n", stat.temperature);
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 
State-Changed-From-To: open->patched 
State-Changed-By: emaste 
State-Changed-When: Tue Sep 27 17:32:14 UTC 2011 
State-Changed-Why:  
Committed to HEAD as 225798 and stable/9 as 225804. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=160581 
State-Changed-From-To: patched->closed 
State-Changed-By: emaste 
State-Changed-When: Tue Oct 4 00:29:24 UTC 2011 
State-Changed-Why:  
Merged to stable/8 now as well. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=160581 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: bin/160581: commit references a PR
Date: Tue,  4 Oct 2011 00:29:29 +0000 (UTC)

 Author: emaste
 Date: Tue Oct  4 00:29:10 2011
 New Revision: 225956
 URL: http://svn.freebsd.org/changeset/base/225956
 
 Log:
   MFC r225798: Improve battery capacity reporting
   
     When a status pointer is passed in mfi_dcmd_command does not return an
     errno (if the ioctl is successful), so move the test for NO_HW_PRESENT
     outside of the error case.  This should fix incorrect reporting for
     systems with a dead or no battery.
   
     Additionally, handle error codes other than NO_HW_PRESENT by omitting
     the battery capacity display.  LSI's supercap-based parts (CV series)
     report their data using the same interface as battery-based parts,
     except that they do not include the capacity stats (state of charge,
     cumulative charge cycles, etc.)
   
   PR:	bin/160581
 
 Modified:
   stable/8/usr.sbin/mfiutil/mfi_show.c
 Directory Properties:
   stable/8/usr.sbin/mfiutil/   (props changed)
 
 Modified: stable/8/usr.sbin/mfiutil/mfi_show.c
 ==============================================================================
 --- stable/8/usr.sbin/mfiutil/mfi_show.c	Tue Oct  4 00:15:40 2011	(r225955)
 +++ stable/8/usr.sbin/mfiutil/mfi_show.c	Tue Oct  4 00:29:10 2011	(r225956)
 @@ -141,7 +141,7 @@ show_battery(int ac, char **av)
  	struct mfi_bbu_design_info design;
  	struct mfi_bbu_status stat;
  	uint8_t status;
 -	int comma, error, fd;
 +	int comma, error, fd, show_capacity;
  
  	if (ac != 1) {
  		warnx("show battery: extra arguments");
 @@ -157,16 +157,17 @@ show_battery(int ac, char **av)
  
  	if (mfi_dcmd_command(fd, MFI_DCMD_BBU_GET_CAPACITY_INFO, &cap,
  	    sizeof(cap), NULL, 0, &status) < 0) {
 -		if (status == MFI_STAT_NO_HW_PRESENT) {
 -			printf("mfi%d: No battery present\n", mfi_unit);
 -			close(fd);
 -			return (0);
 -		}
  		error = errno;
  		warn("Failed to get capacity info");
  		close(fd);
  		return (error);
  	}
 +	if (status == MFI_STAT_NO_HW_PRESENT) {
 +		printf("mfi%d: No battery present\n", mfi_unit);
 +		close(fd);
 +		return (0);
 +	}
 +	show_capacity = (status == MFI_STAT_OK);
  
  	if (mfi_dcmd_command(fd, MFI_DCMD_BBU_GET_DESIGN_INFO, &design,
  	    sizeof(design), NULL, 0, NULL) < 0) {
 @@ -192,10 +193,14 @@ show_battery(int ac, char **av)
  	printf("                Model: %s\n", design.device_name);
  	printf("            Chemistry: %s\n", design.device_chemistry);
  	printf("      Design Capacity: %d mAh\n", design.design_capacity);
 -	printf(" Full Charge Capacity: %d mAh\n", cap.full_charge_capacity);
 -	printf("     Current Capacity: %d mAh\n", cap.remaining_capacity);
 -	printf("        Charge Cycles: %d\n", cap.cycle_count);
 -	printf("       Current Charge: %d%%\n", cap.relative_charge);
 +	if (show_capacity) {
 +		printf(" Full Charge Capacity: %d mAh\n",
 +		    cap.full_charge_capacity);
 +		printf("     Current Capacity: %d mAh\n",
 +		    cap.remaining_capacity);
 +		printf("        Charge Cycles: %d\n", cap.cycle_count);
 +		printf("       Current Charge: %d%%\n", cap.relative_charge);
 +	}
  	printf("       Design Voltage: %d mV\n", design.design_voltage);
  	printf("      Current Voltage: %d mV\n", stat.voltage);
  	printf("          Temperature: %d C\n", stat.temperature);
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 
>Unformatted:
