From nobody@FreeBSD.org  Mon Jan  8 19:40:22 2007
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52])
	by hub.freebsd.org (Postfix) with ESMTP id 6819A16A412
	for <freebsd-gnats-submit@FreeBSD.org>; Mon,  8 Jan 2007 19:40:22 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (www.freebsd.org [69.147.83.33])
	by mx1.freebsd.org (Postfix) with ESMTP id 5AA9013C469
	for <freebsd-gnats-submit@FreeBSD.org>; Mon,  8 Jan 2007 19:40:22 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by www.freebsd.org (8.13.1/8.13.1) with ESMTP id l08JeLWf003857
	for <freebsd-gnats-submit@FreeBSD.org>; Mon, 8 Jan 2007 19:40:21 GMT
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.13.1/8.13.1/Submit) id l08JeLlZ003856;
	Mon, 8 Jan 2007 19:40:21 GMT
	(envelope-from nobody)
Message-Id: <200701081940.l08JeLlZ003856@www.freebsd.org>
Date: Mon, 8 Jan 2007 19:40:21 GMT
From: Guy Helmer<ghelmer@palisadesys.com>
To: freebsd-gnats-submit@FreeBSD.org
Subject: newfs -O 1 doesn't create consistent filesystems
X-Send-Pr-Version: www-3.0

>Number:         107692
>Category:       bin
>Synopsis:       newfs(8): newfs -O 1 doesn't create consistent filesystems
>Confidential:   no
>Severity:       serious
>Priority:       low
>Responsible:    mckusick
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Mon Jan 08 19:50:10 GMT 2007
>Closed-Date:    Tue Nov 23 01:36:24 UTC 2010
>Last-Modified:  Tue Nov 23 01:40:09 UTC 2010
>Originator:     Guy Helmer
>Release:        FreeBSD 6.2 (prerelease)
>Organization:
Palisade Systems, Inc.
>Environment:
FreeBSD opteron6.internal.palisadesys.com 6.1-STABLE FreeBSD 6.1-STABLE #7: Wed Jul 19 08:11:08 CDT 2006     root@opteron6.internal.palisadesys.com:/usr/src/sys/amd64/compile/SMP  amd64

>Description:
Large (60GB) filesystems created using "newfs -U -O 1 -b 65536 -f 8192"
show incorrect results from "df" for free and used space when mounted
immediately after creation.    fsck on the new filesystem (before ever
mounting it once) gives a "SUMMARY INFORMATION BAD" error in phase 5.

This error hasn't occurred in any runs of fsck immediately after
"newfs -U -b 65536 -f 8192" (leaving out the "-O 1" option).

>How-To-Repeat:
"newfs -U -O 1 -b 65536 -f 8192 /dev/da0s1e" (assuming da0s1e is a large
enough partition to trigger the problem) and then "fsck /dev/da0s1e".

>Fix:

>Release-Note:
>Audit-Trail:

From: Bruce Cran <bruce.cran@gmail.com>
To: bug-followup@FreeBSD.org, ghelmer@palisadesys.com
Cc:  
Subject: Re: bin/107692: newfs(8): newfs -O 1 doesn't create consistent 
	filesystems
Date: Sun, 7 Jun 2009 05:58:11 +0100

 This error doesn't occur when passing a block size of 32768 to newfs.
 Debugging fsck_ffs, the following mismatch is seen with the
 inconsistent filesystem:
 
 Starting program: /sbin/fsck_ffs /dev/md0
 ** /dev/md0
 ** Last Mounted on
 ** Phase 1 - Check Blocks and Sizes
 ** Phase 2 - Check Pathnames
 ** Phase 3 - Check Connectivity
 ** Phase 4 - Check Reference Counts
 ** Phase 5 - Check Cyl groups
 
 Breakpoint 1, pass5 () at /usr/src/sbin/fsck_ffs/pass5.c:352
 352		if (cursnapshot == 0 &&
 (gdb) print cstotal
 $1 = {cs_ndir = 2, cs_nbfree = 1631816, cs_nifree = 3333116, cs_nffree = 21,
   cs_numclusters = 0, cs_spare = {0, 0, 0}}
 (gdb) print fs->fs_cstotal
 $2 = {cs_ndir = 0, cs_nbfree = 1631818, cs_nifree = 3333118, cs_nffree = 7,
   cs_numclusters = 0, cs_spare = {0, 0, 0}}
Responsible-Changed-From-To: freebsd-bugs->freebsd-fs 
Responsible-Changed-By: brucec 
Responsible-Changed-When: Sat Sep 11 14:56:34 UTC 2010 
Responsible-Changed-Why:  
Over to maintainer(s). 

http://www.freebsd.org/cgi/query-pr.cgi?pr=107692 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: bin/107692: commit references a PR
Date: Fri, 24 Sep 2010 19:09:02 +0000 (UTC)

 Author: mckusick
 Date: Fri Sep 24 19:08:56 2010
 New Revision: 213119
 URL: http://svn.freebsd.org/changeset/base/213119
 
 Log:
   Reported problem:
   Large (60GB) filesystems created using "newfs -U -O 1 -b 65536 -f 8192"
   show incorrect results from "df" for free and used space when mounted
   immediately after creation. fsck on the new filesystem (before ever
   mounting it once) gives a "SUMMARY INFORMATION BAD" error in phase 5.
   
   This error hasn't occurred in any runs of fsck immediately after
   "newfs -U -b 65536 -f 8192" (leaving out the "-O 1" option).
   
   Solution:
   The default UFS1 superblock is located at offset 8K in the filesystem
   partition; the default UFS2 superblock is located at offset 64K in
   the filesystem partition. For UFS1 filesystems with a blocksize of
   64K, the first alternate superblock resides at 64K which is the the
   location used for the default UFS2 superblock. By default, the
   system first checks for a valid superblock at the default location
   for a UFS2 filoesystem. For a UFS1 filesystem with a blocksize of
   64K, there is a valid UFS1 superblock at this location.  Thus, even
   though it is expected to be a backup superblock, the system will
   use it as its default superblock. So, we have to ensure that all the
   statistcs on usage are correct in this first alternate superblock
   as it is the superblock that will actually be used.
   
   While tracking down this problem, another limitation of UFS1 became
   evident. For UFS1, the number of inodes per cylinder group is stored
   in an int16_t. Thus the maximum number of inodes per cylinder group
   is limited to 2^15 - 1. This limit can easily be exceeded for block
   sizes of 32K and above. Thus when building UFS1 filesystems, newfs
   must limit the number of inodes per cylinder group to 2^15 - 1.
   
   Reported by: Guy Helmer<ghelmer@palisadesys.com>
   Followup by: Bruce Cran <brucec@freebsd.org>
   PR:          107692
   MFC after:   4 weeks
 
 Modified:
   head/sbin/newfs/mkfs.c
 
 Modified: head/sbin/newfs/mkfs.c
 ==============================================================================
 --- head/sbin/newfs/mkfs.c	Fri Sep 24 19:07:14 2010	(r213118)
 +++ head/sbin/newfs/mkfs.c	Fri Sep 24 19:08:56 2010	(r213119)
 @@ -376,16 +376,20 @@ restart:
  	 * Start packing more blocks into the cylinder group until
  	 * it cannot grow any larger, the number of cylinder groups
  	 * drops below MINCYLGRPS, or we reach the size requested.
 +	 * For UFS1 inodes per cylinder group are stored in an int16_t
 +	 * so fs_ipg is limited to 2^15 - 1.
  	 */
  	for ( ; sblock.fs_fpg < maxblkspercg; sblock.fs_fpg += sblock.fs_frag) {
  		sblock.fs_ipg = roundup(howmany(sblock.fs_fpg, fragsperinode),
  		    INOPB(&sblock));
 -		if (sblock.fs_size / sblock.fs_fpg < MINCYLGRPS)
 -			break;
 -		if (CGSIZE(&sblock) < (unsigned long)sblock.fs_bsize)
 -			continue;
 -		if (CGSIZE(&sblock) == (unsigned long)sblock.fs_bsize)
 -			break;
 +		if (Oflag > 1 || (Oflag == 1 && sblock.fs_ipg <= 0x7fff)) {
 +			if (sblock.fs_size / sblock.fs_fpg < MINCYLGRPS)
 +				break;
 +			if (CGSIZE(&sblock) < (unsigned long)sblock.fs_bsize)
 +				continue;
 +			if (CGSIZE(&sblock) == (unsigned long)sblock.fs_bsize)
 +				break;
 +		}
  		sblock.fs_fpg -= sblock.fs_frag;
  		sblock.fs_ipg = roundup(howmany(sblock.fs_fpg, fragsperinode),
  		    INOPB(&sblock));
 @@ -584,8 +588,20 @@ restart:
  		printf("** Exiting on Xflag 3\n");
  		exit(0);
  	}
 -	if (!Nflag)
 +	if (!Nflag) {
  		do_sbwrite(&disk);
 +		/*
 +		 * For UFS1 filesystems with a blocksize of 64K, the first
 +		 * alternate superblock resides at the location used for
 +		 * the default UFS2 superblock. As there is a valid
 +		 * superblock at this location, the boot code will use
 +		 * it as its first choice. Thus we have to ensure that
 +		 * all of its statistcs on usage are correct.
 +		 */
 +		if (Oflag == 1 && sblock.fs_bsize == 65536)
 +			wtfs(fsbtodb(&sblock, cgsblock(&sblock, 0)),
 +			    sblock.fs_bsize, (char *)&sblock);
 +	}
  	for (i = 0; i < sblock.fs_cssize; i += sblock.fs_bsize)
  		wtfs(fsbtodb(&sblock, sblock.fs_csaddr + numfrags(&sblock, i)),
  			sblock.fs_cssize - i < sblock.fs_bsize ?
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 
State-Changed-From-To: open->patched 
State-Changed-By: mckusick 
State-Changed-When: Fri Sep 24 19:25:32 UTC 2010 
State-Changed-Why:  
I am taking responsibility for this bug. 

A fix has been applied to head. If no problems arise it will be 
MFC'ed to 7-stable and 8-stable in 4 weeks time. 


Responsible-Changed-From-To: freebsd-fs->mckusick 
Responsible-Changed-By: mckusick 
Responsible-Changed-When: Fri Sep 24 19:25:32 UTC 2010 
Responsible-Changed-Why:  
I am taking responsibility for this bug. 

A fix has been applied to head. If no problems arise it will be 
MFC'ed to 7-stable and 8-stable in 4 weeks time. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=107692 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: bin/107692: commit references a PR
Date: Tue, 23 Nov 2010 01:24:34 +0000 (UTC)

 Author: mckusick
 Date: Tue Nov 23 01:24:27 2010
 New Revision: 215728
 URL: http://svn.freebsd.org/changeset/base/215728
 
 Log:
   MFC of 213119
   
   Reported problem:
   Large (60GB) filesystems created using "newfs -U -O 1 -b 65536 -f 8192"
   show incorrect results from "df" for free and used space when mounted
   immediately after creation. fsck on the new filesystem (before ever
   mounting it once) gives a "SUMMARY INFORMATION BAD" error in phase 5.
   
   This error hasn't occurred in any runs of fsck immediately after
   "newfs -U -b 65536 -f 8192" (leaving out the "-O 1" option).
   
   Solution:
   The default UFS1 superblock is located at offset 8K in the filesystem
   partition; the default UFS2 superblock is located at offset 64K in
   the filesystem partition. For UFS1 filesystems with a blocksize of
   64K, the first alternate superblock resides at 64K which is the the
   location used for the default UFS2 superblock. By default, the
   system first checks for a valid superblock at the default location
   for a UFS2 filoesystem. For a UFS1 filesystem with a blocksize of
   64K, there is a valid UFS1 superblock at this location. Thus, even
   though it is expected to be a backup superblock, the system will
   use it as its default superblock. So, we have to ensure that all the
   statistcs on usage are correct in this first alternate superblock
   as it is the superblock that will actually be used.
   
   While tracking down this problem, another limitation of UFS1 became
   evident. For UFS1, the number of inodes per cylinder group is stored
   in an int16_t. Thus the maximum number of inodes per cylinder group
   is limited to 2^15 - 1. This limit can easily be exceeded for block
   sizes of 32K and above. Thus when building UFS1 filesystems, newfs
   must limit the number of inodes per cylinder group to 2^15 - 1.
   
   Reported by: Guy Helmer<ghelmer@palisadesys.com>
   Followup by: Bruce Cran <brucec@freebsd.org>
   PR: 107692
 
 Modified:
   stable/8/sbin/newfs/mkfs.c
 Directory Properties:
   stable/8/sbin/newfs/   (props changed)
 
 Modified: stable/8/sbin/newfs/mkfs.c
 ==============================================================================
 --- stable/8/sbin/newfs/mkfs.c	Mon Nov 22 23:49:06 2010	(r215727)
 +++ stable/8/sbin/newfs/mkfs.c	Tue Nov 23 01:24:27 2010	(r215728)
 @@ -376,16 +376,20 @@ restart:
  	 * Start packing more blocks into the cylinder group until
  	 * it cannot grow any larger, the number of cylinder groups
  	 * drops below MINCYLGRPS, or we reach the size requested.
 +	 * For UFS1 inodes per cylinder group are stored in an int16_t
 +	 * so fs_ipg is limited to 2^15 - 1.
  	 */
  	for ( ; sblock.fs_fpg < maxblkspercg; sblock.fs_fpg += sblock.fs_frag) {
  		sblock.fs_ipg = roundup(howmany(sblock.fs_fpg, fragsperinode),
  		    INOPB(&sblock));
 -		if (sblock.fs_size / sblock.fs_fpg < MINCYLGRPS)
 -			break;
 -		if (CGSIZE(&sblock) < (unsigned long)sblock.fs_bsize)
 -			continue;
 -		if (CGSIZE(&sblock) == (unsigned long)sblock.fs_bsize)
 -			break;
 +		if (Oflag > 1 || (Oflag == 1 && sblock.fs_ipg <= 0x7fff)) {
 +			if (sblock.fs_size / sblock.fs_fpg < MINCYLGRPS)
 +				break;
 +			if (CGSIZE(&sblock) < (unsigned long)sblock.fs_bsize)
 +				continue;
 +			if (CGSIZE(&sblock) == (unsigned long)sblock.fs_bsize)
 +				break;
 +		}
  		sblock.fs_fpg -= sblock.fs_frag;
  		sblock.fs_ipg = roundup(howmany(sblock.fs_fpg, fragsperinode),
  		    INOPB(&sblock));
 @@ -584,8 +588,20 @@ restart:
  		printf("** Exiting on Xflag 3\n");
  		exit(0);
  	}
 -	if (!Nflag)
 +	if (!Nflag) {
  		do_sbwrite(&disk);
 +		/*
 +		 * For UFS1 filesystems with a blocksize of 64K, the first
 +		 * alternate superblock resides at the location used for
 +		 * the default UFS2 superblock. As there is a valid
 +		 * superblock at this location, the boot code will use
 +		 * it as its first choice. Thus we have to ensure that
 +		 * all of its statistcs on usage are correct.
 +		 */
 +		if (Oflag == 1 && sblock.fs_bsize == 65536)
 +			wtfs(fsbtodb(&sblock, cgsblock(&sblock, 0)),
 +			    sblock.fs_bsize, (char *)&sblock);
 +	}
  	for (i = 0; i < sblock.fs_cssize; i += sblock.fs_bsize)
  		wtfs(fsbtodb(&sblock, sblock.fs_csaddr + numfrags(&sblock, i)),
  			sblock.fs_cssize - i < sblock.fs_bsize ?
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 
State-Changed-From-To: patched->closed 
State-Changed-By: mckusick 
State-Changed-When: Tue Nov 23 01:34:52 UTC 2010 
State-Changed-Why:  
The fix has been MFC'ed to 7-stable and 8-stable. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=107692 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: bin/107692: commit references a PR
Date: Tue, 23 Nov 2010 01:32:51 +0000 (UTC)

 Author: mckusick
 Date: Tue Nov 23 01:32:44 2010
 New Revision: 215729
 URL: http://svn.freebsd.org/changeset/base/215729
 
 Log:
   MFC of 213119
   
   Reported problem:
   Large (60GB) filesystems created using "newfs -U -O 1 -b 65536 -f 8192"
   show incorrect results from "df" for free and used space when mounted
   immediately after creation. fsck on the new filesystem (before ever
   mounting it once) gives a "SUMMARY INFORMATION BAD" error in phase 5.
   
   This error hasn't occurred in any runs of fsck immediately after
   "newfs -U -b 65536 -f 8192" (leaving out the "-O 1" option).
   
   Solution:
   The default UFS1 superblock is located at offset 8K in the filesystem
   partition; the default UFS2 superblock is located at offset 64K in
   the filesystem partition. For UFS1 filesystems with a blocksize of
   64K, the first alternate superblock resides at 64K which is the the
   location used for the default UFS2 superblock. By default, the
   system first checks for a valid superblock at the default location
   for a UFS2 filoesystem. For a UFS1 filesystem with a blocksize of
   64K, there is a valid UFS1 superblock at this location. Thus, even
   though it is expected to be a backup superblock, the system will
   use it as its default superblock. So, we have to ensure that all the
   statistcs on usage are correct in this first alternate superblock
   as it is the superblock that will actually be used.
   
   While tracking down this problem, another limitation of UFS1 became
   evident. For UFS1, the number of inodes per cylinder group is stored
   in an int16_t. Thus the maximum number of inodes per cylinder group
   is limited to 2^15 - 1. This limit can easily be exceeded for block
   sizes of 32K and above. Thus when building UFS1 filesystems, newfs
   must limit the number of inodes per cylinder group to 2^15 - 1.
   
   Reported by: Guy Helmer<ghelmer@palisadesys.com>
   Followup by: Bruce Cran <brucec@freebsd.org>
   PR: 107692
 
 Modified:
   stable/7/sbin/newfs/mkfs.c
 Directory Properties:
   stable/7/sbin/newfs/   (props changed)
 
 Modified: stable/7/sbin/newfs/mkfs.c
 ==============================================================================
 --- stable/7/sbin/newfs/mkfs.c	Tue Nov 23 01:24:27 2010	(r215728)
 +++ stable/7/sbin/newfs/mkfs.c	Tue Nov 23 01:32:44 2010	(r215729)
 @@ -367,16 +367,20 @@ restart:
  	 * Start packing more blocks into the cylinder group until
  	 * it cannot grow any larger, the number of cylinder groups
  	 * drops below MINCYLGRPS, or we reach the size requested.
 +	 * For UFS1 inodes per cylinder group are stored in an int16_t
 +	 * so fs_ipg is limited to 2^15 - 1.
  	 */
  	for ( ; sblock.fs_fpg < maxblkspercg; sblock.fs_fpg += sblock.fs_frag) {
  		sblock.fs_ipg = roundup(howmany(sblock.fs_fpg, fragsperinode),
  		    INOPB(&sblock));
 -		if (sblock.fs_size / sblock.fs_fpg < MINCYLGRPS)
 -			break;
 -		if (CGSIZE(&sblock) < (unsigned long)sblock.fs_bsize)
 -			continue;
 -		if (CGSIZE(&sblock) == (unsigned long)sblock.fs_bsize)
 -			break;
 +		if (Oflag > 1 || (Oflag == 1 && sblock.fs_ipg <= 0x7fff)) {
 +			if (sblock.fs_size / sblock.fs_fpg < MINCYLGRPS)
 +				break;
 +			if (CGSIZE(&sblock) < (unsigned long)sblock.fs_bsize)
 +				continue;
 +			if (CGSIZE(&sblock) == (unsigned long)sblock.fs_bsize)
 +				break;
 +		}
  		sblock.fs_fpg -= sblock.fs_frag;
  		sblock.fs_ipg = roundup(howmany(sblock.fs_fpg, fragsperinode),
  		    INOPB(&sblock));
 @@ -568,8 +572,20 @@ restart:
  		printf("** Exiting on Eflag 3\n");
  		exit(0);
  	}
 -	if (!Nflag)
 +	if (!Nflag) {
  		sbwrite(&disk, 0);
 +		/*
 +		 * For UFS1 filesystems with a blocksize of 64K, the first
 +		 * alternate superblock resides at the location used for
 +		 * the default UFS2 superblock. As there is a valid
 +		 * superblock at this location, the boot code will use
 +		 * it as its first choice. Thus we have to ensure that
 +		 * all of its statistcs on usage are correct.
 +		 */
 +		if (Oflag == 1 && sblock.fs_bsize == 65536)
 +			wtfs(fsbtodb(&sblock, cgsblock(&sblock, 0)),
 +			    sblock.fs_bsize, (char *)&sblock);
 +	}
  	for (i = 0; i < sblock.fs_cssize; i += sblock.fs_bsize)
  		wtfs(fsbtodb(&sblock, sblock.fs_csaddr + numfrags(&sblock, i)),
  			sblock.fs_cssize - i < sblock.fs_bsize ?
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 
>Unformatted:
