From lorne@cons.org.nz  Sun Feb  9 05:19:09 2014
Return-Path: <lorne@cons.org.nz>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1])
	(using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
	(No client certificate requested)
	by hub.freebsd.org (Postfix) with ESMTPS id D71F169C
	for <FreeBSD-gnats-submit@freebsd.org>; Sun,  9 Feb 2014 05:19:08 +0000 (UTC)
Received: from smtp4.clear.net.nz (smtp4.clear.net.nz [203.97.37.64])
	by mx1.freebsd.org (Postfix) with ESMTP id 9886714F0
	for <FreeBSD-gnats-submit@freebsd.org>; Sun,  9 Feb 2014 05:19:08 +0000 (UTC)
Received: from mxin2-orange.clear.net.nz
 (lb2-srcnat.clear.net.nz [203.97.32.237])
 by smtp4.clear.net.nz (CLEAR Net Mail)
 with ESMTP id <0N0P00I28Q2O8L20@smtp4.clear.net.nz> for
 FreeBSD-gnats-submit@freebsd.org; Sun, 09 Feb 2014 18:04:01 +1300 (NZDT)
Received: from 202-78-141-194.cable.telstraclear.net (HELO mail.cons.org.nz)
 ([202.78.141.194]) by smtpin2.clear.net.nz with ESMTP; Sun,
 09 Feb 2014 18:03:59 +1300
Received: from northind.cons.org.nz
 (northind.cons.org.nz [IPv6:2001:4428:27c:1::25])
	by mail.cons.org.nz (Postfix) with ESMTP id B205127F110	for
 <FreeBSD-gnats-submit@freebsd.org>; Sun, 09 Feb 2014 18:03:58 +1300 (NZDT)
Received: by northind.cons.org.nz (Postfix, from userid 1001)	id 446181B8; Sun,
 09 Feb 2014 18:03:55 +1300 (NZDT)
Message-Id: <20140209050355.446181B8@northind.cons.org.nz>
Date: Sun, 09 Feb 2014 18:03:55 +1300 (NZDT)
From: Andrew Childs <lorne@cons.org.nz>
Reply-To: Andrew Childs <lorne@cons.org.nz>
To: FreeBSD-gnats-submit@freebsd.org
Cc:
Subject: zpool history hangs (infinite loop)
X-Send-Pr-Version: 3.114
X-GNATS-Notify:

>Number:         186574
>Category:       kern
>Synopsis:       [zfs] zpool history hangs (infinite loop)
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    delphij
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Sun Feb 09 05:20:00 UTC 2014
>Closed-Date:    Mon Apr 28 06:31:59 UTC 2014
>Last-Modified:  Mon Apr 28 06:31:59 UTC 2014
>Originator:     Andrew Childs
>Release:        FreeBSD 10.0-RELEASE amd64
>Organization:
>Environment:
System: FreeBSD northind.cons.org.nz 10.0-RELEASE FreeBSD 10.0-RELEASE #0 r260789: Thu Jan 16 22:34:59 UTC 2014 root@snap.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64


	
>Description:

Running "zpool history zroot" on my FreeBSD 10-RELEASE machine results
in zpool entering an infinite loop.

>How-To-Repeat:

Run "zpool history" on an affected pool.

>Fix:

This appears to be caused by having a history record larger than
libzfs's HIS_BUF_LEN.

zpool_get_history reads HIS_BUF_LEN sized chunks and parses entire
records at a time. When an entire record isn't contained within a
single chunk, zpool_get_history correctly concludes the result was
truncated. The next read is aligned such that any truncated record
will be at the beginning of the buffer. zpool_get_history then loops
with the assumption this will read the entire record. When a record
larger than HIS_BUF_LEN is encountered, it will always be truncated,
even if it's at the start of the buffer, and zpool_get_history will
loop indefinitely.

With a warning patched in:

  History for 'zroot':
  HIS_BUF_LEN (0x20000) too small to fit record of length 0x239c8

With a dynamically increasing buffer the history is successfully
printed.

I don't believe the particular record is malformed. The slightly
redacted output of running strings over the data is here:
https://gist.github.com/thefloweringash/1fd434541c05fc091b10

Patch attached to introduce a dynamically sized buffer. This solves
the problem for me. However I don't know how HIS_BUF_LEN was chosen,
and could be patching a symptom.

--- libzfs_pool-dynamic-history-buffer begins here ---
--- libzfs_pool.c.orig	2014-02-09 16:25:17.210736940 +1300
+++ libzfs_pool.c	2014-02-09 17:50:52.585383154 +1300
@@ -3745,14 +3745,19 @@
 int
 zpool_get_history(zpool_handle_t *zhp, nvlist_t **nvhisp)
 {
-	char buf[HIS_BUF_LEN];
+	char *buf;
+	uint64_t buflen = HIS_BUF_LEN;
 	uint64_t off = 0;
 	nvlist_t **records = NULL;
 	uint_t numrecords = 0;
 	int err, i;
 
+	buf = malloc(buflen);
+	if (!buf)
+		return (1);
+
 	do {
-		uint64_t bytes_read = sizeof (buf);
+		uint64_t bytes_read = buflen;
 		uint64_t leftover;
 
 		if ((err = get_history(zhp, buf, &off, &bytes_read)) != 0)
@@ -3765,6 +3770,26 @@
 		if ((err = zpool_history_unpack(buf, bytes_read,
 		    &leftover, &records, &numrecords)) != 0)
 			break;
+
+		if (leftover == bytes_read) {
+			// No progress was made. Check if we're
+			// attempting to read a record longer than our
+			// buffer.
+
+			uint64_t reclen;
+			for (i = 0, reclen = 0; i < sizeof (reclen); i++)
+				reclen += (uint64_t)(((uchar_t *)buf)[i]) << (8*i);
+
+			while (buflen < sizeof(reclen) + reclen)
+				buflen <<= 1;
+
+			buf = realloc(buf, buflen);
+			if (!buf) {
+				err = 1;
+				break;
+			}
+		}
+
 		off -= leftover;
 
 		/* CONSTCOND */
@@ -3779,6 +3804,8 @@
 		nvlist_free(records[i]);
 	free(records);
 
+	free(buf);
+
 	return (err);
 }
 
--- libzfs_pool-dynamic-history-buffer ends here ---


>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->freebsd-fs 
Responsible-Changed-By: linimon 
Responsible-Changed-When: Mon Feb 10 04:26:16 UTC 2014 
Responsible-Changed-Why:  
Over to maintainer(s). 

http://www.freebsd.org/cgi/query-pr.cgi?pr=186574 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/186574: commit references a PR
Date: Mon, 14 Apr 2014 18:38:18 +0000 (UTC)

 Author: delphij
 Date: Mon Apr 14 18:38:14 2014
 New Revision: 264467
 URL: http://svnweb.freebsd.org/changeset/base/264467
 
 Log:
   Take into account when zpool history block grows exceeding 128KB in zpool(8)
   and zdb(8) by growing the buffer on demand with a cap of 1GB (specified in
   spa_history_create_obj()).
   
   PR:		bin/186574
   Submitted by:	Andrew Childs <lorne cons org nz> (with changes)
   MFC after:	2 weeks
 
 Modified:
   head/cddl/contrib/opensolaris/cmd/zdb/zdb.c
   head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_pool.c
 
 Modified: head/cddl/contrib/opensolaris/cmd/zdb/zdb.c
 ==============================================================================
 --- head/cddl/contrib/opensolaris/cmd/zdb/zdb.c	Mon Apr 14 18:14:09 2014	(r264466)
 +++ head/cddl/contrib/opensolaris/cmd/zdb/zdb.c	Mon Apr 14 18:38:14 2014	(r264467)
 @@ -929,11 +929,16 @@ dump_dtl(vdev_t *vd, int indent)
  		dump_dtl(vd->vdev_child[c], indent + 4);
  }
  
 +/* from spa_history.c: spa_history_create_obj() */
 +#define	HIS_BUF_LEN_DEF	(128 << 10)
 +#define	HIS_BUF_LEN_MAX	(1 << 30)
 +
  static void
  dump_history(spa_t *spa)
  {
  	nvlist_t **events = NULL;
 -	char buf[SPA_MAXBLOCKSIZE];
 +	char *buf = NULL;
 +	uint64_t bufsize = HIS_BUF_LEN_DEF;
  	uint64_t resid, len, off = 0;
  	uint_t num = 0;
  	int error;
 @@ -942,8 +947,11 @@ dump_history(spa_t *spa)
  	char tbuf[30];
  	char internalstr[MAXPATHLEN];
  
 +	if ((buf = malloc(bufsize)) == NULL)
 +		(void) fprintf(stderr, "Unable to read history: "
 +		    "out of memory\n");
  	do {
 -		len = sizeof (buf);
 +		len = bufsize;
  
  		if ((error = spa_history_get(spa, &off, &len, buf)) != 0) {
  			(void) fprintf(stderr, "Unable to read history: "
 @@ -953,9 +961,26 @@ dump_history(spa_t *spa)
  
  		if (zpool_history_unpack(buf, len, &resid, &events, &num) != 0)
  			break;
 -
  		off -= resid;
 +
 +		/*
 +		 * If the history block is too big, double the buffer
 +		 * size and try again.
 +		 */
 +		if (resid == len) {
 +			free(buf);
 +			buf = NULL;
 +
 +			bufsize <<= 1;
 +			if ((bufsize >= HIS_BUF_LEN_MAX) ||
 +			    ((buf = malloc(bufsize)) == NULL)) {
 +				(void) fprintf(stderr, "Unable to read history: "
 +				    "out of memory\n");
 +				return;
 +			}
 +		}
  	} while (len != 0);
 +	free(buf);
  
  	(void) printf("\nHistory:\n");
  	for (int i = 0; i < num; i++) {
 
 Modified: head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_pool.c
 ==============================================================================
 --- head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_pool.c	Mon Apr 14 18:14:09 2014	(r264466)
 +++ head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_pool.c	Mon Apr 14 18:38:14 2014	(r264467)
 @@ -3744,7 +3744,9 @@ zpool_history_unpack(char *buf, uint64_t
  	return (0);
  }
  
 -#define	HIS_BUF_LEN	(128*1024)
 +/* from spa_history.c: spa_history_create_obj() */
 +#define	HIS_BUF_LEN_DEF	(128 << 10)
 +#define	HIS_BUF_LEN_MAX	(1 << 30)
  
  /*
   * Retrieve the command history of a pool.
 @@ -3752,21 +3754,24 @@ zpool_history_unpack(char *buf, uint64_t
  int
  zpool_get_history(zpool_handle_t *zhp, nvlist_t **nvhisp)
  {
 -	char buf[HIS_BUF_LEN];
 +	char *buf = NULL;
 +	uint64_t bufsize = HIS_BUF_LEN_DEF;
  	uint64_t off = 0;
  	nvlist_t **records = NULL;
  	uint_t numrecords = 0;
  	int err, i;
  
 +	if ((buf = malloc(bufsize)) == NULL)
 +		return (ENOMEM);
  	do {
 -		uint64_t bytes_read = sizeof (buf);
 +		uint64_t bytes_read = bufsize;
  		uint64_t leftover;
  
  		if ((err = get_history(zhp, buf, &off, &bytes_read)) != 0)
  			break;
  
  		/* if nothing else was read in, we're at EOF, just return */
 -		if (!bytes_read)
 +		if (bytes_read == 0)
  			break;
  
  		if ((err = zpool_history_unpack(buf, bytes_read,
 @@ -3774,8 +3779,25 @@ zpool_get_history(zpool_handle_t *zhp, n
  			break;
  		off -= leftover;
  
 +		/*
 +		 * If the history block is too big, double the buffer
 +		 * size and try again.
 +		 */
 +		if (leftover == bytes_read) {
 +			free(buf);
 +			buf = NULL;
 +
 +			bufsize <<= 1;
 +			if ((bufsize >= HIS_BUF_LEN_MAX) ||
 +			    ((buf = malloc(bufsize)) == NULL)) {
 +				err = ENOMEM;
 +				break;
 +			}
 +		}
 +
  		/* CONSTCOND */
  	} while (1);
 +	free(buf);
  
  	if (!err) {
  		verify(nvlist_alloc(nvhisp, NV_UNIQUE_NAME, 0) == 0);
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/186574: commit references a PR
Date: Mon, 28 Apr 2014 06:12:18 +0000 (UTC)

 Author: delphij
 Date: Mon Apr 28 06:12:15 2014
 New Revision: 265041
 URL: http://svnweb.freebsd.org/changeset/base/265041
 
 Log:
   MFC r264467:
   
   Take into account when zpool history block grows exceeding 128KB in zpool(8)
   and zdb(8) by growing the buffer on demand with a cap of 1GB (specified in
   spa_history_create_obj()).
   
   PR:		bin/186574
   Submitted by:	Andrew Childs <lorne cons org nz> (with changes)
 
 Modified:
   stable/8/cddl/contrib/opensolaris/cmd/zdb/zdb.c
   stable/8/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_pool.c
 Directory Properties:
   stable/8/cddl/contrib/opensolaris/   (props changed)
   stable/8/cddl/contrib/opensolaris/lib/libzfs/   (props changed)
 
 Modified: stable/8/cddl/contrib/opensolaris/cmd/zdb/zdb.c
 ==============================================================================
 --- stable/8/cddl/contrib/opensolaris/cmd/zdb/zdb.c	Mon Apr 28 06:11:44 2014	(r265040)
 +++ stable/8/cddl/contrib/opensolaris/cmd/zdb/zdb.c	Mon Apr 28 06:12:15 2014	(r265041)
 @@ -929,11 +929,16 @@ dump_dtl(vdev_t *vd, int indent)
  		dump_dtl(vd->vdev_child[c], indent + 4);
  }
  
 +/* from spa_history.c: spa_history_create_obj() */
 +#define	HIS_BUF_LEN_DEF	(128 << 10)
 +#define	HIS_BUF_LEN_MAX	(1 << 30)
 +
  static void
  dump_history(spa_t *spa)
  {
  	nvlist_t **events = NULL;
 -	char buf[SPA_MAXBLOCKSIZE];
 +	char *buf = NULL;
 +	uint64_t bufsize = HIS_BUF_LEN_DEF;
  	uint64_t resid, len, off = 0;
  	uint_t num = 0;
  	int error;
 @@ -942,8 +947,11 @@ dump_history(spa_t *spa)
  	char tbuf[30];
  	char internalstr[MAXPATHLEN];
  
 +	if ((buf = malloc(bufsize)) == NULL)
 +		(void) fprintf(stderr, "Unable to read history: "
 +		    "out of memory\n");
  	do {
 -		len = sizeof (buf);
 +		len = bufsize;
  
  		if ((error = spa_history_get(spa, &off, &len, buf)) != 0) {
  			(void) fprintf(stderr, "Unable to read history: "
 @@ -953,9 +961,26 @@ dump_history(spa_t *spa)
  
  		if (zpool_history_unpack(buf, len, &resid, &events, &num) != 0)
  			break;
 -
  		off -= resid;
 +
 +		/*
 +		 * If the history block is too big, double the buffer
 +		 * size and try again.
 +		 */
 +		if (resid == len) {
 +			free(buf);
 +			buf = NULL;
 +
 +			bufsize <<= 1;
 +			if ((bufsize >= HIS_BUF_LEN_MAX) ||
 +			    ((buf = malloc(bufsize)) == NULL)) {
 +				(void) fprintf(stderr, "Unable to read history: "
 +				    "out of memory\n");
 +				return;
 +			}
 +		}
  	} while (len != 0);
 +	free(buf);
  
  	(void) printf("\nHistory:\n");
  	for (int i = 0; i < num; i++) {
 
 Modified: stable/8/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_pool.c
 ==============================================================================
 --- stable/8/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_pool.c	Mon Apr 28 06:11:44 2014	(r265040)
 +++ stable/8/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_pool.c	Mon Apr 28 06:12:15 2014	(r265041)
 @@ -3736,7 +3736,9 @@ zpool_history_unpack(char *buf, uint64_t
  	return (0);
  }
  
 -#define	HIS_BUF_LEN	(128*1024)
 +/* from spa_history.c: spa_history_create_obj() */
 +#define	HIS_BUF_LEN_DEF	(128 << 10)
 +#define	HIS_BUF_LEN_MAX	(1 << 30)
  
  /*
   * Retrieve the command history of a pool.
 @@ -3744,21 +3746,24 @@ zpool_history_unpack(char *buf, uint64_t
  int
  zpool_get_history(zpool_handle_t *zhp, nvlist_t **nvhisp)
  {
 -	char buf[HIS_BUF_LEN];
 +	char *buf = NULL;
 +	uint64_t bufsize = HIS_BUF_LEN_DEF;
  	uint64_t off = 0;
  	nvlist_t **records = NULL;
  	uint_t numrecords = 0;
  	int err, i;
  
 +	if ((buf = malloc(bufsize)) == NULL)
 +		return (ENOMEM);
  	do {
 -		uint64_t bytes_read = sizeof (buf);
 +		uint64_t bytes_read = bufsize;
  		uint64_t leftover;
  
  		if ((err = get_history(zhp, buf, &off, &bytes_read)) != 0)
  			break;
  
  		/* if nothing else was read in, we're at EOF, just return */
 -		if (!bytes_read)
 +		if (bytes_read == 0)
  			break;
  
  		if ((err = zpool_history_unpack(buf, bytes_read,
 @@ -3766,8 +3771,25 @@ zpool_get_history(zpool_handle_t *zhp, n
  			break;
  		off -= leftover;
  
 +		/*
 +		 * If the history block is too big, double the buffer
 +		 * size and try again.
 +		 */
 +		if (leftover == bytes_read) {
 +			free(buf);
 +			buf = NULL;
 +
 +			bufsize <<= 1;
 +			if ((bufsize >= HIS_BUF_LEN_MAX) ||
 +			    ((buf = malloc(bufsize)) == NULL)) {
 +				err = ENOMEM;
 +				break;
 +			}
 +		}
 +
  		/* CONSTCOND */
  	} while (1);
 +	free(buf);
  
  	if (!err) {
  		verify(nvlist_alloc(nvhisp, NV_UNIQUE_NAME, 0) == 0);
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/186574: commit references a PR
Date: Mon, 28 Apr 2014 06:11:07 +0000 (UTC)

 Author: delphij
 Date: Mon Apr 28 06:11:03 2014
 New Revision: 265039
 URL: http://svnweb.freebsd.org/changeset/base/265039
 
 Log:
   MFC r264467:
   
   Take into account when zpool history block grows exceeding 128KB in zpool(8)
   and zdb(8) by growing the buffer on demand with a cap of 1GB (specified in
   spa_history_create_obj()).
   
   PR:		bin/186574
   Submitted by:	Andrew Childs <lorne cons org nz> (with changes)
 
 Modified:
   stable/10/cddl/contrib/opensolaris/cmd/zdb/zdb.c
   stable/10/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_pool.c
 Directory Properties:
   stable/10/   (props changed)
 
 Modified: stable/10/cddl/contrib/opensolaris/cmd/zdb/zdb.c
 ==============================================================================
 --- stable/10/cddl/contrib/opensolaris/cmd/zdb/zdb.c	Mon Apr 28 05:39:20 2014	(r265038)
 +++ stable/10/cddl/contrib/opensolaris/cmd/zdb/zdb.c	Mon Apr 28 06:11:03 2014	(r265039)
 @@ -929,11 +929,16 @@ dump_dtl(vdev_t *vd, int indent)
  		dump_dtl(vd->vdev_child[c], indent + 4);
  }
  
 +/* from spa_history.c: spa_history_create_obj() */
 +#define	HIS_BUF_LEN_DEF	(128 << 10)
 +#define	HIS_BUF_LEN_MAX	(1 << 30)
 +
  static void
  dump_history(spa_t *spa)
  {
  	nvlist_t **events = NULL;
 -	char buf[SPA_MAXBLOCKSIZE];
 +	char *buf = NULL;
 +	uint64_t bufsize = HIS_BUF_LEN_DEF;
  	uint64_t resid, len, off = 0;
  	uint_t num = 0;
  	int error;
 @@ -942,8 +947,11 @@ dump_history(spa_t *spa)
  	char tbuf[30];
  	char internalstr[MAXPATHLEN];
  
 +	if ((buf = malloc(bufsize)) == NULL)
 +		(void) fprintf(stderr, "Unable to read history: "
 +		    "out of memory\n");
  	do {
 -		len = sizeof (buf);
 +		len = bufsize;
  
  		if ((error = spa_history_get(spa, &off, &len, buf)) != 0) {
  			(void) fprintf(stderr, "Unable to read history: "
 @@ -953,9 +961,26 @@ dump_history(spa_t *spa)
  
  		if (zpool_history_unpack(buf, len, &resid, &events, &num) != 0)
  			break;
 -
  		off -= resid;
 +
 +		/*
 +		 * If the history block is too big, double the buffer
 +		 * size and try again.
 +		 */
 +		if (resid == len) {
 +			free(buf);
 +			buf = NULL;
 +
 +			bufsize <<= 1;
 +			if ((bufsize >= HIS_BUF_LEN_MAX) ||
 +			    ((buf = malloc(bufsize)) == NULL)) {
 +				(void) fprintf(stderr, "Unable to read history: "
 +				    "out of memory\n");
 +				return;
 +			}
 +		}
  	} while (len != 0);
 +	free(buf);
  
  	(void) printf("\nHistory:\n");
  	for (int i = 0; i < num; i++) {
 
 Modified: stable/10/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_pool.c
 ==============================================================================
 --- stable/10/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_pool.c	Mon Apr 28 05:39:20 2014	(r265038)
 +++ stable/10/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_pool.c	Mon Apr 28 06:11:03 2014	(r265039)
 @@ -3744,7 +3744,9 @@ zpool_history_unpack(char *buf, uint64_t
  	return (0);
  }
  
 -#define	HIS_BUF_LEN	(128*1024)
 +/* from spa_history.c: spa_history_create_obj() */
 +#define	HIS_BUF_LEN_DEF	(128 << 10)
 +#define	HIS_BUF_LEN_MAX	(1 << 30)
  
  /*
   * Retrieve the command history of a pool.
 @@ -3752,21 +3754,24 @@ zpool_history_unpack(char *buf, uint64_t
  int
  zpool_get_history(zpool_handle_t *zhp, nvlist_t **nvhisp)
  {
 -	char buf[HIS_BUF_LEN];
 +	char *buf = NULL;
 +	uint64_t bufsize = HIS_BUF_LEN_DEF;
  	uint64_t off = 0;
  	nvlist_t **records = NULL;
  	uint_t numrecords = 0;
  	int err, i;
  
 +	if ((buf = malloc(bufsize)) == NULL)
 +		return (ENOMEM);
  	do {
 -		uint64_t bytes_read = sizeof (buf);
 +		uint64_t bytes_read = bufsize;
  		uint64_t leftover;
  
  		if ((err = get_history(zhp, buf, &off, &bytes_read)) != 0)
  			break;
  
  		/* if nothing else was read in, we're at EOF, just return */
 -		if (!bytes_read)
 +		if (bytes_read == 0)
  			break;
  
  		if ((err = zpool_history_unpack(buf, bytes_read,
 @@ -3774,8 +3779,25 @@ zpool_get_history(zpool_handle_t *zhp, n
  			break;
  		off -= leftover;
  
 +		/*
 +		 * If the history block is too big, double the buffer
 +		 * size and try again.
 +		 */
 +		if (leftover == bytes_read) {
 +			free(buf);
 +			buf = NULL;
 +
 +			bufsize <<= 1;
 +			if ((bufsize >= HIS_BUF_LEN_MAX) ||
 +			    ((buf = malloc(bufsize)) == NULL)) {
 +				err = ENOMEM;
 +				break;
 +			}
 +		}
 +
  		/* CONSTCOND */
  	} while (1);
 +	free(buf);
  
  	if (!err) {
  		verify(nvlist_alloc(nvhisp, NV_UNIQUE_NAME, 0) == 0);
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/186574: commit references a PR
Date: Mon, 28 Apr 2014 06:11:47 +0000 (UTC)

 Author: delphij
 Date: Mon Apr 28 06:11:44 2014
 New Revision: 265040
 URL: http://svnweb.freebsd.org/changeset/base/265040
 
 Log:
   MFC r264467:
   
   Take into account when zpool history block grows exceeding 128KB in zpool(8)
   and zdb(8) by growing the buffer on demand with a cap of 1GB (specified in
   spa_history_create_obj()).
   
   PR:		bin/186574
   Submitted by:	Andrew Childs <lorne cons org nz> (with changes)
 
 Modified:
   stable/9/cddl/contrib/opensolaris/cmd/zdb/zdb.c
   stable/9/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_pool.c
 Directory Properties:
   stable/9/cddl/contrib/opensolaris/   (props changed)
   stable/9/cddl/contrib/opensolaris/lib/libzfs/   (props changed)
 
 Modified: stable/9/cddl/contrib/opensolaris/cmd/zdb/zdb.c
 ==============================================================================
 --- stable/9/cddl/contrib/opensolaris/cmd/zdb/zdb.c	Mon Apr 28 06:11:03 2014	(r265039)
 +++ stable/9/cddl/contrib/opensolaris/cmd/zdb/zdb.c	Mon Apr 28 06:11:44 2014	(r265040)
 @@ -929,11 +929,16 @@ dump_dtl(vdev_t *vd, int indent)
  		dump_dtl(vd->vdev_child[c], indent + 4);
  }
  
 +/* from spa_history.c: spa_history_create_obj() */
 +#define	HIS_BUF_LEN_DEF	(128 << 10)
 +#define	HIS_BUF_LEN_MAX	(1 << 30)
 +
  static void
  dump_history(spa_t *spa)
  {
  	nvlist_t **events = NULL;
 -	char buf[SPA_MAXBLOCKSIZE];
 +	char *buf = NULL;
 +	uint64_t bufsize = HIS_BUF_LEN_DEF;
  	uint64_t resid, len, off = 0;
  	uint_t num = 0;
  	int error;
 @@ -942,8 +947,11 @@ dump_history(spa_t *spa)
  	char tbuf[30];
  	char internalstr[MAXPATHLEN];
  
 +	if ((buf = malloc(bufsize)) == NULL)
 +		(void) fprintf(stderr, "Unable to read history: "
 +		    "out of memory\n");
  	do {
 -		len = sizeof (buf);
 +		len = bufsize;
  
  		if ((error = spa_history_get(spa, &off, &len, buf)) != 0) {
  			(void) fprintf(stderr, "Unable to read history: "
 @@ -953,9 +961,26 @@ dump_history(spa_t *spa)
  
  		if (zpool_history_unpack(buf, len, &resid, &events, &num) != 0)
  			break;
 -
  		off -= resid;
 +
 +		/*
 +		 * If the history block is too big, double the buffer
 +		 * size and try again.
 +		 */
 +		if (resid == len) {
 +			free(buf);
 +			buf = NULL;
 +
 +			bufsize <<= 1;
 +			if ((bufsize >= HIS_BUF_LEN_MAX) ||
 +			    ((buf = malloc(bufsize)) == NULL)) {
 +				(void) fprintf(stderr, "Unable to read history: "
 +				    "out of memory\n");
 +				return;
 +			}
 +		}
  	} while (len != 0);
 +	free(buf);
  
  	(void) printf("\nHistory:\n");
  	for (int i = 0; i < num; i++) {
 
 Modified: stable/9/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_pool.c
 ==============================================================================
 --- stable/9/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_pool.c	Mon Apr 28 06:11:03 2014	(r265039)
 +++ stable/9/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_pool.c	Mon Apr 28 06:11:44 2014	(r265040)
 @@ -3736,7 +3736,9 @@ zpool_history_unpack(char *buf, uint64_t
  	return (0);
  }
  
 -#define	HIS_BUF_LEN	(128*1024)
 +/* from spa_history.c: spa_history_create_obj() */
 +#define	HIS_BUF_LEN_DEF	(128 << 10)
 +#define	HIS_BUF_LEN_MAX	(1 << 30)
  
  /*
   * Retrieve the command history of a pool.
 @@ -3744,21 +3746,24 @@ zpool_history_unpack(char *buf, uint64_t
  int
  zpool_get_history(zpool_handle_t *zhp, nvlist_t **nvhisp)
  {
 -	char buf[HIS_BUF_LEN];
 +	char *buf = NULL;
 +	uint64_t bufsize = HIS_BUF_LEN_DEF;
  	uint64_t off = 0;
  	nvlist_t **records = NULL;
  	uint_t numrecords = 0;
  	int err, i;
  
 +	if ((buf = malloc(bufsize)) == NULL)
 +		return (ENOMEM);
  	do {
 -		uint64_t bytes_read = sizeof (buf);
 +		uint64_t bytes_read = bufsize;
  		uint64_t leftover;
  
  		if ((err = get_history(zhp, buf, &off, &bytes_read)) != 0)
  			break;
  
  		/* if nothing else was read in, we're at EOF, just return */
 -		if (!bytes_read)
 +		if (bytes_read == 0)
  			break;
  
  		if ((err = zpool_history_unpack(buf, bytes_read,
 @@ -3766,8 +3771,25 @@ zpool_get_history(zpool_handle_t *zhp, n
  			break;
  		off -= leftover;
  
 +		/*
 +		 * If the history block is too big, double the buffer
 +		 * size and try again.
 +		 */
 +		if (leftover == bytes_read) {
 +			free(buf);
 +			buf = NULL;
 +
 +			bufsize <<= 1;
 +			if ((bufsize >= HIS_BUF_LEN_MAX) ||
 +			    ((buf = malloc(bufsize)) == NULL)) {
 +				err = ENOMEM;
 +				break;
 +			}
 +		}
 +
  		/* CONSTCOND */
  	} while (1);
 +	free(buf);
  
  	if (!err) {
  		verify(nvlist_alloc(nvhisp, NV_UNIQUE_NAME, 0) == 0);
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 
State-Changed-From-To: open->closed 
State-Changed-By: delphij 
State-Changed-When: Mon Apr 28 06:31:27 UTC 2014 
State-Changed-Why:  
A slightly changed version of patch committed and merged. 


Responsible-Changed-From-To: freebsd-fs->delphij 
Responsible-Changed-By: delphij 
Responsible-Changed-When: Mon Apr 28 06:31:27 UTC 2014 
Responsible-Changed-Why:  
Take.  Thanks for your submission! 

http://www.freebsd.org/cgi/query-pr.cgi?pr=186574 
>Unformatted:
