From mla_strick@att.net  Thu Dec 24 23:19:18 2009
Return-Path: <mla_strick@att.net>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 7BB1F106566B
	for <FreeBSD-gnats-submit@freebsd.org>; Thu, 24 Dec 2009 23:19:18 +0000 (UTC)
	(envelope-from mla_strick@att.net)
Received: from smtp128.sbc.mail.sp1.yahoo.com (smtp128.sbc.mail.sp1.yahoo.com [69.147.65.187])
	by mx1.freebsd.org (Postfix) with SMTP id 61C908FC16
	for <FreeBSD-gnats-submit@freebsd.org>; Thu, 24 Dec 2009 23:19:17 +0000 (UTC)
Received: (qmail 85807 invoked from network); 24 Dec 2009 23:19:17 -0000
Received: from adsl-69-228-80-86.dsl.pltn13.pacbell.net (mla_strick@69.228.80.86 with login)
        by smtp128.sbc.mail.sp1.yahoo.com with SMTP; 24 Dec 2009 15:19:16 -0800 PST
Received: from mist.nodomain (localhost [127.0.0.1])
	by mist.nodomain (8.14.3/8.14.3) with ESMTP id nBONJG03020260;
	Thu, 24 Dec 2009 15:19:16 -0800 (PST)
	(envelope-from mla@mist.nodomain)
Received: (from dan@localhost)
	by mist.nodomain (8.14.3/8.14.3/Submit) id nBONJFjB020259;
	Thu, 24 Dec 2009 15:19:16 -0800 (PST)
	(envelope-from mla)
Message-Id: <200912242319.nBONJFjB020259@mist.nodomain>
Date: Thu, 24 Dec 2009 15:19:16 -0800 (PST)
From: Dan Strick <mla_strick@att.net>
Reply-To: Dan Strick <mla_strick@att.net>
To: FreeBSD-gnats-submit@freebsd.org
Cc: mla@mist.nodomain
Subject: fsck cannot repair file system in which it finds an error
X-Send-Pr-Version: 3.113
X-GNATS-Notify:

>Number:         141992
>Category:       kern
>Synopsis:       [ufs] fsck cannot repair file system in which it finds an error [regression]
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    mckusick
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Thu Dec 24 23:20:02 UTC 2009
>Closed-Date:    Wed Feb 10 21:08:32 UTC 2010
>Last-Modified:  Wed Feb 10 21:08:32 UTC 2010
>Originator:     Dan Strick <mla_strick@att.net>
>Release:        FreeBSD 8.0-RELEASE i386
>Organization:
none
>Environment:
System: FreeBSD mist 8.0-RELEASE FreeBSD 8.0-RELEASE #5: Mon Dec 7 18:59:52 PST 2009 root@mist:/usr/src/sys/i386/compile/MIST_DEBUG i386
>Description:
When I upgraded from FreeBSD release 6.1 to release 8.0 I discovered
that fsck complained about a problem in one of my old file systems
and claimed to have fixed the problem but did not really fix it.

The fsck on release 6.1 finds no problem at all in the file system.
I wonder if perhaps fsck could not fix the problem because there
was no real problem and fsck was confused by some difference in
file systems created by an older release of FreeBSD.
>How-To-Repeat:
This is a sample fsck dialogue under FreeBSD release 8.0.  I do not claim
that the file system meets all release 8.0 standards for UFS file systems
(since it was created under an old FreeBSD release) but I do claim that
it is a bug for fsck to have claimed to have fixed a problem and then to
find the problem again when the fsck is repeated.

mist# fsck /dev/ad4s4a
** /dev/ad4s4a
** Last Mounted on /fs/u4
** Phase 1 - Check Blocks and Sizes
CYLINDER GROUP 48: BAD MAGIC NUMBER
UNEXPECTED SOFT UPDATE INCONSISTENCY

REBUILD CYLINDER GROUP? [yn] y

** Phase 2 - Check Pathnames
** Phase 3 - Check Connectivity
** Phase 4 - Check Reference Counts
** Phase 5 - Check Cyl groups
SUMMARY INFORMATION BAD
SALVAGE? [yn] y

BLK(S) MISSING IN BIT MAPS
SALVAGE? [yn] y

46090 files, 2542520 used, 1766975 free (22607 frags, 218046 blocks, 0.5% fragmentation)

***** FILE SYSTEM IS CLEAN *****

***** FILE SYSTEM WAS MODIFIED *****
mist# !!
fsck /dev/ad4s4a
** /dev/ad4s4a
** Last Mounted on /fs/u4
** Phase 1 - Check Blocks and Sizes
CYLINDER GROUP 48: BAD MAGIC NUMBER
UNEXPECTED SOFT UPDATE INCONSISTENCY

REBUILD CYLINDER GROUP? [yn] y

** Phase 2 - Check Pathnames
** Phase 3 - Check Connectivity
** Phase 4 - Check Reference Counts
** Phase 5 - Check Cyl groups
SUMMARY INFORMATION BAD
SALVAGE? [yn] y

BLK(S) MISSING IN BIT MAPS
SALVAGE? [yn] y

46090 files, 2542520 used, 1766975 free (22607 frags, 218046 blocks, 0.5% fragmentation)

***** FILE SYSTEM IS CLEAN *****

***** FILE SYSTEM WAS MODIFIED *****
mist#
>Fix:
(workaround:)

1) Dump the file system.
2) Newfs the file system under FreeBSD release 8.0.
3) Restore the file system.
>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->freebsd-fs 
Responsible-Changed-By: linimon 
Responsible-Changed-When: Fri Dec 25 02:59:51 UTC 2009 
Responsible-Changed-Why:  
Over to maintainer(s). 

http://www.freebsd.org/cgi/query-pr.cgi?pr=141992 
Responsible-Changed-From-To: freebsd-fs->mckusick 
Responsible-Changed-By: mckusick 
Responsible-Changed-When: Tue Dec 29 18:47:02 UTC 2009 
Responsible-Changed-Why:  
I'll take this one. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=141992 

From: Kirk McKusick <mckusick@mckusick.com>
To: Dan Strick <mla_strick@att.net>
Cc: bug-followup@freebsd.org, freebsd-fs@freebsd.org
Subject: kern/141992: fsck cannot repair file system in which it finds an error
Date: Tue, 29 Dec 2009 10:43:11 -0800

 I am investigating your bug report about your problems with fsck
 in going from 6.1 to 8.0. There have been only insignificant
 changes in the filesystem during that time, so in theory you should
 not have any problems going back and forth between these two
 versions of FreeBSD. The fsck program has been changed to look
 more critically at the cylinder group maps, so some errors that
 would not previously have been detected will now be found. That 
 said though, it should be able to completely fix those errors
 which it appears that it is not able to do.
 
 To help me track down your problem, please supply me with the following
 information:
 
 1) Do you have more than one 6.1 filesystem that you are running on
    this system. If so, is this the only filesystem having trouble in
    this way, or are more than one of them exhibiting similar problems?
 
 2) Are you getting any disk errors reported for this drive in
    /var/log/messages? It really looks like there is a bad sector in
    cylinder group 48 and so even though fsck can calculate a correct
    one, it is not able to write it out. Even if errors are not being
    reported, I am inclined to write a small test program that attempts
    to read and write the affected sectors to see if the changes stick.
 
 3) Please provide me with a `dumpfs /dev/ad4s4a'. I don't need the
    whole (huge) thing. Just the start up through cylinder group 0,
    then the output for cylinder groups 47, 48, and 49. I am looking
    for any obvious anomolies in cylinder group 48 versus its neighbors.
 
 Hopefully we can sort out what is going wrong here.
 
 	Kirk McKusick

From: Dan Strick <mla_strick@att.net>
To: mckusick@mckusick.com
Cc: bug-followup@freebsd.org, freebsd-fs@freebsd.org, mla@mist.nodomain
Subject: Re: kern/141992: fsck cannot repair file system in which it finds an error
Date: Wed, 30 Dec 2009 15:00:16 -0800 (PST)

 > From mckusick@mckusick.com Wed Dec 30 09:45:41 2009
 > To: Dan Strick <mla_strick@att.net>
 > Subject: Re: kern/141992: fsck cannot repair file system in ...
 > Date: Wed, 30 Dec 2009 09:31:07 -0800
 >
 > Thanks.The thing that immediately jumps out at me is that the filesystem
 > is UFS1 format (which would make it 4.X vintage unless you specifically
 > requested UFS1 when you built it). The new code to fix cylinder groups
 > in fsck can only fix UFS2 filesystems. So the problem is that it is
 > unhappy with cylinder group 48, then tries to fix it and finds that
 > it cannot do anything. At a minimum it should just not do its check
 > for bad cylinder groups in UFS1 filesystems.
 
 The file system was probably created by FreeBSD 4.10, the first version
 of FreeBSD that would run on my machine.
 
 Perhaps when the new fsck finds a bad UFS1 cylinder group and cannot
 fix it, the new fsck should simply explain its limitation and recommend
 copying the UFS1 file system contents to a new UFS2 file system.
 
 > Is there a chance that I can either (1) get a copy of the mdconfig
 > disk image so I can work out a solution here, or (2) get a login
 > on your machine so I can poke at things there.
 >
 > 	Kirk McKusick
 
 I have prepared a new file system by deleting all files in the old file
 system, creating a huge file containing only zeros, and deleting the
 huge file.  The new file system gzips nicely and still has the bad
 cylinder group.  The new file system is about 16 MB compressed (almost
 9 GB uncompressed).  How should I send it to you?
 
 Dan Strick

From: Kirk McKusick <mckusick@mckusick.com>
To: Dan Strick <mla_strick@att.net>
Cc: freebsd-fs@freebsd.org, bug-followup@freebsd.org, mla@mist.nodomain
Subject: Re: kern/141992: fsck cannot repair file system in which it finds an error 
Date: Wed, 30 Dec 2009 16:11:41 -0800

 > Date: Wed, 30 Dec 2009 15:00:16 -0800 (PST)
 > From: Dan Strick <mla_strick@att.net>
 > To: mckusick@mckusick.com
 > In-Reply-To: <200912301731.nBUHV720089853@chez.mckusick.com>
 > Cc: freebsd-fs@freebsd.org, bug-followup@freebsd.org, mla@mist.nodomain
 > Subject: Re: kern/141992: fsck cannot repair file system in which it finds
 > 	an error
 > X-BeenThere: freebsd-fs@freebsd.org
 > 
 > > From mckusick@mckusick.com Wed Dec 30 09:45:41 2009
 > > To: Dan Strick <mla_strick@att.net>
 > > Subject: Re: kern/141992: fsck cannot repair file system in ...
 > > Date: Wed, 30 Dec 2009 09:31:07 -0800
 > >
 > > Thanks.The thing that immediately jumps out at me is that the filesystem
 > > is UFS1 format (which would make it 4.X vintage unless you specifically
 > > requested UFS1 when you built it). The new code to fix cylinder groups
 > > in fsck can only fix UFS2 filesystems. So the problem is that it is
 > > unhappy with cylinder group 48, then tries to fix it and finds that
 > > it cannot do anything. At a minimum it should just not do its check
 > > for bad cylinder groups in UFS1 filesystems.
 > 
 > The file system was probably created by FreeBSD 4.10, the first version
 > of FreeBSD that would run on my machine.
 
 Yes, quite believeable.
 
 > Perhaps when the new fsck finds a bad UFS1 cylinder group and cannot
 > fix it, the new fsck should simply explain its limitation and recommend
 > copying the UFS1 file system contents to a new UFS2 file system.
 
 That is what I will probably do unless an easy fix for UFS1 presents
 itself.
 
 > > Is there a chance that I can either (1) get a copy of the mdconfig
 > > disk image so I can work out a solution here, or (2) get a login
 > > on your machine so I can poke at things there.
 > >
 > > 	Kirk McKusick
 > 
 > I have prepared a new file system by deleting all files in the old file
 > system, creating a huge file containing only zeros, and deleting the
 > huge file.  The new file system gzips nicely and still has the bad
 > cylinder group.  The new file system is about 16 MB compressed (almost
 > 9 GB uncompressed).  How should I send it to you?
 > 
 > Dan Strick
 > _______________________________________________
 > freebsd-fs@freebsd.org mailing list
 > http://lists.freebsd.org/mailman/listinfo/freebsd-fs
 > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org"
 
 Excellent. I wish all folks that reported bugs could provide such
 help as you have :-)
 
 Do you have a web server on which you could put it from which I could
 do a download? Or an ftp server? If not, I'll create a guest account
 for you on my machine and let you download to that.
 
 	~Kirk

From: Dan Strick <mla_strick@att.net>
To: mckusick@mckusick.com
Cc: bug-followup@freebsd.org, freebsd-fs@freebsd.org, mla@mist.nodomain
Subject: Re: kern/141992: fsck cannot repair file system in which it finds an error
Date: Wed, 30 Dec 2009 17:34:15 -0800 (PST)

 > Do you have a web server on which you could put it from which I could
 > do a download? Or an ftp server? If not, I'll create a guest account
 > for you on my machine and let you download to that.
 
 I don't have a web or ftp server.  I am pretty sure my ISP does not
 provide me with a place to store a web page.  I should be able to
 anonymous ftp or do scp to a guest account.  Send instructions.
 
 Thanks,
 	Dan Strick
State-Changed-From-To: open->patched 
State-Changed-By: mckusick 
State-Changed-When: Thu Jan 7 00:23:19 UTC 2010 
State-Changed-Why:  
A fix has been found and is being tested. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=141992 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/141992: commit references a PR
Date: Thu,  7 Jan 2010 00:21:58 +0000 (UTC)

 Author: mckusick
 Date: Thu Jan  7 00:17:36 2010
 New Revision: 201700
 URL: http://svn.freebsd.org/changeset/base/201700
 
 Log:
   This corrects a bug that manifested itself as identifying the last
   cylinder group of a UFS1 filesystem as bad. The error was in the check
   and not in the cylinder group itself. So even though fsck fixed the
   cylinder group correctly, it was still endlessly reported as bad.
   
   PR:		141992
   MFC after:	2 weeks
   Reported by:	Dan Strick
 
 Modified:
   head/sbin/fsck_ffs/fsutil.c
 
 Modified: head/sbin/fsck_ffs/fsutil.c
 ==============================================================================
 --- head/sbin/fsck_ffs/fsutil.c	Thu Jan  7 00:04:29 2010	(r201699)
 +++ head/sbin/fsck_ffs/fsutil.c	Thu Jan  7 00:17:36 2010	(r201700)
 @@ -436,7 +436,7 @@ check_cgmagic(int cg, struct cg *cgp)
  	    ((sblock.fs_magic == FS_UFS1_MAGIC &&
  	      cgp->cg_old_niblk == sblock.fs_ipg &&
  	      cgp->cg_ndblk <= sblock.fs_fpg &&
 -	      cgp->cg_old_ncyl == sblock.fs_old_cpg) ||
 +	      cgp->cg_old_ncyl <= sblock.fs_old_cpg) ||
  	     (sblock.fs_magic == FS_UFS2_MAGIC &&
  	      cgp->cg_niblk == sblock.fs_ipg &&
  	      cgp->cg_ndblk <= sblock.fs_fpg &&
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 

From: Kirk McKusick <mckusick@mckusick.com>
To: Dan Strick <mla_strick@att.net>
Cc: freebsd-fs@freebsd.org, bug-followup@freebsd.org
Subject: Re: kern/141992: fsck cannot repair file system in which it finds an error 
Date: Wed, 06 Jan 2010 16:21:52 -0800

 So, I have (finally) gotten a chance to try out the disk image that
 you sent me using fsck_ffs from a FreeBSD 8.0 system. Here is the
 correction report and patch that I made:
 
 Author: mckusick
 Date: Thu Jan  7 00:17:36 2010
 New Revision: 201700
 URL: http://svn.freebsd.org/changeset/base/201700
 
 Log:
   This corrects a bug that manifested itself as identifying the last
   cylinder group of a UFS1 filesystem as bad. The error was in the check
   and not in the cylinder group itself. So even though fsck fixed the
   cylinder group correctly, it was still endlessly reported as bad.
   
   PR:		141992
   MFC after:	2 weeks
   Reported by:	Dan Strick
 
 Modified:
   head/sbin/fsck_ffs/fsutil.c
 
 Modified: head/sbin/fsck_ffs/fsutil.c
 ==============================================================================
 --- head/sbin/fsck_ffs/fsutil.c	Thu Jan  7 00:04:29 2010	(r201699)
 +++ head/sbin/fsck_ffs/fsutil.c	Thu Jan  7 00:17:36 2010	(r201700)
 @@ -436,7 +436,7 @@ check_cgmagic(int cg, struct cg *cgp)
  	    ((sblock.fs_magic == FS_UFS1_MAGIC &&
  	      cgp->cg_old_niblk == sblock.fs_ipg &&
  	      cgp->cg_ndblk <= sblock.fs_fpg &&
 -	      cgp->cg_old_ncyl == sblock.fs_old_cpg) ||
 +	      cgp->cg_old_ncyl <= sblock.fs_old_cpg) ||
  	     (sblock.fs_magic == FS_UFS2_MAGIC &&
  	      cgp->cg_niblk == sblock.fs_ipg &&
  	      cgp->cg_ndblk <= sblock.fs_fpg &&
 
 Please apply this patch to your fsck_ffs and confirm for me that it
 corrects your problem. Thanks for all your help.
 
 	Kirk McKusick

From: Dan Strick <mla_strick@att.net>
To: mckusick@mckusick.com
Cc: bug-followup@freebsd.org, freebsd-fs@freebsd.org, mla@mist.nodomain
Subject: Re: kern/141992: fsck cannot repair file system in which it finds an error
Date: Thu, 7 Jan 2010 01:12:28 -0800 (PST)

 > From mckusick@mckusick.com Thu Jan  7 00:55:35 2010
 > To: Dan Strick <mla_strick@att.net>
 > Subject: Re: kern/141992: fsck cannot repair file system in ...
 >
 	...
 >
 > Please apply this patch to your fsck_ffs and confirm for me that it
 > corrects your problem. Thanks for all your help.
 
 This patch does indeed make the problem I had with that particular
 file system go away.
 
 Thanks,
 	Dan Strick

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/141992: commit references a PR
Date: Wed, 10 Feb 2010 20:35:38 +0000 (UTC)

 Author: mckusick
 Date: Wed Feb 10 20:35:20 2010
 New Revision: 203765
 URL: http://svn.freebsd.org/changeset/base/203765
 
 Log:
   MFC of r201700 | mckusick | 2010-01-06
   
   This corrects a bug that manifested itself as identifying the last
   cylinder group of a UFS1 filesystem as bad. The error was in the check
   and not in the cylinder group itself. So even though fsck fixed the
   cylinder group correctly, it was still endlessly reported as bad.
   
   This bug first appeared in 8.0 so does not apply to earlier releases.
   
   PR:             141992
   Reported by:    Dan Strick
 
 Modified:
   stable/8/sbin/fsck_ffs/fsutil.c
 Directory Properties:
   stable/8/sbin/fsck_ffs/   (props changed)
 
 Modified: stable/8/sbin/fsck_ffs/fsutil.c
 ==============================================================================
 --- stable/8/sbin/fsck_ffs/fsutil.c	Wed Feb 10 20:17:46 2010	(r203764)
 +++ stable/8/sbin/fsck_ffs/fsutil.c	Wed Feb 10 20:35:20 2010	(r203765)
 @@ -436,7 +436,7 @@ check_cgmagic(int cg, struct cg *cgp)
  	    ((sblock.fs_magic == FS_UFS1_MAGIC &&
  	      cgp->cg_old_niblk == sblock.fs_ipg &&
  	      cgp->cg_ndblk <= sblock.fs_fpg &&
 -	      cgp->cg_old_ncyl == sblock.fs_old_cpg) ||
 +	      cgp->cg_old_ncyl <= sblock.fs_old_cpg) ||
  	     (sblock.fs_magic == FS_UFS2_MAGIC &&
  	      cgp->cg_niblk == sblock.fs_ipg &&
  	      cgp->cg_ndblk <= sblock.fs_fpg &&
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 
State-Changed-From-To: patched->closed 
State-Changed-By: mckusick 
State-Changed-When: Wed Feb 10 21:07:17 UTC 2010 
State-Changed-Why:  
The fix has been confirmed and MFC'ed to 8.0. 
It did not appear in releases before 8.0, 
so no further MFC's are needed. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=141992 
>Unformatted:
