From nobody@FreeBSD.org  Sat Oct 31 09:01:47 2009
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 937E4106568D
	for <freebsd-gnats-submit@FreeBSD.org>; Sat, 31 Oct 2009 09:01:47 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (www.freebsd.org [IPv6:2001:4f8:fff6::21])
	by mx1.freebsd.org (Postfix) with ESMTP id 781F28FC0C
	for <freebsd-gnats-submit@FreeBSD.org>; Sat, 31 Oct 2009 09:01:47 +0000 (UTC)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by www.freebsd.org (8.14.3/8.14.3) with ESMTP id n9V91lmA021932
	for <freebsd-gnats-submit@FreeBSD.org>; Sat, 31 Oct 2009 09:01:47 GMT
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.14.3/8.14.3/Submit) id n9V91lCx021931;
	Sat, 31 Oct 2009 09:01:47 GMT
	(envelope-from nobody)
Message-Id: <200910310901.n9V91lCx021931@www.freebsd.org>
Date: Sat, 31 Oct 2009 09:01:47 GMT
From: Dominic Fandrey <kamikaze@bsdforen.de>
To: freebsd-gnats-submit@FreeBSD.org
Subject: msdosfs write and fsck destroy filesystem integrity
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         140134
>Category:       kern
>Synopsis:       [msdosfs] write and fsck destroy filesystem integrity
>Confidential:   no
>Severity:       critical
>Priority:       medium
>Responsible:    freebsd-fs
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Sat Oct 31 09:10:01 UTC 2009
>Closed-Date:    Mon Jul 11 09:38:21 UTC 2011
>Last-Modified:  Mon Jul 11 09:38:21 UTC 2011
>Originator:     Dominic Fandrey
>Release:        RELENG_8
>Organization:
private
>Environment:
FreeBSD mobileKamikaze.norad 8.0-RC2 FreeBSD 8.0-RC2 #0: Sat Oct 31 02:49:30 CET 2009     root@mobileKamikaze.norad:/usr/obj/HP6510b-8/amd64/usr/src/sys/HP6510b-8  amd64
>Description:
Write operations on msdosfs causes all kinds of brokenness (e.g.: http://docs.freebsd.org/cgi/mid.cgi?4AE95C94.4040000 ). Lost clusters and the like.

fsck_msdosfs causes cross-linked files (at least according to Windows Vista chkdsk).

I can provide you with a broken image that was created as described in the following section.

Note that even a freshly created msdosfs file system is already broken.
>How-To-Repeat:
kamikaze@mobileKamikaze> dd bs=1m count=1024 < /dev/zero > test.msdosfs
1024+0 records in
1024+0 records out
1073741824 bytes transferred in 39.292063 secs (27327194 bytes/sec)
kamikaze@mobileKamikaze> mdconfig -f test.msdosfs
md0
kamikaze@mobileKamikaze> newfs_msdos -F32 md0
newfs_msdos: Cannot get number of sectors per track, Operation not supported
newfs_msdos: Cannot get number of heads, Operation not supported
newfs_msdos: trim 8 sectors to adjust to a multiple of 63
/dev/md0: 2096064 sectors in 65502 FAT32 clusters (16384 bytes/cluster)
bps=512 spc=32 res=32 nft=2 mid=0xf0 spt=63 hds=255 hid=0 bsec=2097144 bspf=512 rdcl=2 infs=1 bkbs=2
kamikaze@mobileKamikaze> fsck_msdosfs -y /dev/md0
** /dev/md0
** Phase 1 - Read and Compare FATs
** Phase 2 - Check Cluster Chains
** Phase 3 - Checking Directories
** Phase 4 - Checking for Lost Files
Free space in FSInfo block (-1) not correct (65501)
Fix? yes
Next free cluster in FSInfo block (2) not free
Fix? yes
1 files, 1048016 free (65501 clusters)
kamikaze@mobileKamikaze> mount_msdosfs /dev/md0 tmp/
kamikaze@mobileKamikaze> cd tmp/
kamikaze@mobileKamikaze> cp -rv ~/music/ogg/Kamikaze/Dio\ -\ 1983\ -\ Holy\ Diver ./
/home/kamikaze/music/ogg/Kamikaze/Dio - 1983 - Holy Diver -> ./Dio - 1983 - Holy Diver
/home/kamikaze/music/ogg/Kamikaze/Dio - 1983 - Holy Diver/cover.jpg -> ./Dio - 1983 - Holy Diver/cover.jpg
/home/kamikaze/music/ogg/Kamikaze/Dio - 1983 - Holy Diver/02 - Dio - Holy Diver.ogg -> ./Dio - 1983 - Holy Diver/02 - Dio - Holy Diver.ogg
/home/kamikaze/music/ogg/Kamikaze/Dio - 1983 - Holy Diver/03 - Dio - Gypsy.ogg -> ./Dio - 1983 - Holy Diver/03 - Dio - Gypsy.ogg
/home/kamikaze/music/ogg/Kamikaze/Dio - 1983 - Holy Diver/04 - Dio - Caught In The Middle.ogg -> ./Dio - 1983 - Holy Diver/04 - Dio - Caught In The Middle.ogg
/home/kamikaze/music/ogg/Kamikaze/Dio - 1983 - Holy Diver/05 - Dio - Don't Talk To Strangers.ogg -> ./Dio - 1983 - Holy Diver/05 - Dio - Don't Talk To Strangers.ogg
/home/kamikaze/music/ogg/Kamikaze/Dio - 1983 - Holy Diver/06 - Dio - Straight Through The Heart.ogg -> ./Dio - 1983 - Holy Diver/06 - Dio - Straight Through The Heart.ogg
/home/kamikaze/music/ogg/Kamikaze/Dio - 1983 - Holy Diver/07 - Dio - Invisible.ogg -> ./Dio - 1983 - Holy Diver/07 - Dio - Invisible.ogg
/home/kamikaze/music/ogg/Kamikaze/Dio - 1983 - Holy Diver/08 - Dio - Rainbow In The Dark.ogg -> ./Dio - 1983 - Holy Diver/08 - Dio - Rainbow In The Dark.ogg
/home/kamikaze/music/ogg/Kamikaze/Dio - 1983 - Holy Diver/09 - Dio - Shame On The Night.ogg -> ./Dio - 1983 - Holy Diver/09 - Dio - Shame On The Night.ogg
/home/kamikaze/music/ogg/Kamikaze/Dio - 1983 - Holy Diver/10 - Dio - Interview Forming Own Band (Bonus Track).ogg -> ./Dio - 1983 - Holy Diver/10 - Dio - Interview Forming Own Band (Bonus Track).ogg
/home/kamikaze/music/ogg/Kamikaze/Dio - 1983 - Holy Diver/11 - Dio - Interview Vinnie Appice Leaving Sabbath (Bonus Track).ogg -> ./Dio - 1983 - Holy Diver/11 - Dio - Interview Vinnie Appice Leaving Sabbath (Bonus Track).ogg
/home/kamikaze/music/ogg/Kamikaze/Dio - 1983 - Holy Diver/12 - Dio - Interview Seeking Out A Guitarist (Bonus Track).ogg -> ./Dio - 1983 - Holy Diver/12 - Dio - Interview Seeking Out A Guitarist (Bonus Track).ogg
cp: ./Dio - 1983 - Holy Diver/13 - Dio - Interview Working In London (Bonus Track).ogg: Invalid argument
/home/kamikaze/music/ogg/Kamikaze/Dio - 1983 - Holy Diver/14 - Dio - Interview Release Timing Of Holy Diver (Bonus Track).ogg -> ./Dio - 1983 - Holy Diver/14 - Dio - Interview Release Timing Of Holy Diver (Bonus Track).ogg
cp: ./Dio - 1983 - Holy Diver/15 - Dio - Interview Writing While With Sabbath (Bonus Track).ogg: No space left on device
cp: ./Dio - 1983 - Holy Diver/16 - Dio - Interview Album Concept (Bonus Track).ogg: No space left on device
cp: ./Dio - 1983 - Holy Diver/17 - Dio - Interview Producing The Album Himself (Bonus Track).ogg: No space left on device
cp: ./Dio - 1983 - Holy Diver/18 - Dio - Interview Remembering The Tracks (Bonus Track).ogg: No space left on device
cp: ./Dio - 1983 - Holy Diver/19 - Dio - Interview Artwork.ogg: No space left on device
cp: ./Dio - 1983 - Holy Diver/01 - Dio - Stand Up And Shout.ogg: No space left on device
kamikaze@mobileKamikaze> ls
Dio - 1983 - Holy Diver
kamikaze@mobileKamikaze> df -h                             0 /home/kamikaze/tmp
Filesystem                                             Size    Used   Avail Capacity  Mounted on
/dev/ufs/2root                                         496M    287M    169M    63%    /
devfs                                                  1.0K    1.0K      0B   100%    /dev
/dev/ufs/2tmp                                          1.9G     14K    1.8G     0%    /tmp
/dev/ufs/2usr                                           38G     25G     10G    71%    /usr
/dev/ufs/2var                                          3.9G    2.4G    1.2G    67%    /var
/usr/home/root                                          38G     25G     10G    71%    /root
/dev/fuse0                                              49G     43G    6.5G    87%    /var/run/automounter.mnt/ntfs/2vault
/dev/md0                                               1.0G    1.0G   -160K   100%    /usr/home/kamikaze/tmp
kamikaze@mobileKamikaze> du -hd 0 Dio\ -\ 1983\ -\ Holy\ Diver/*
  0B	Dio - 1983 - Holy Diver/01 - Dio - Stand Up And Shout.ogg
6.8M	Dio - 1983 - Holy Diver/02 - Dio - Holy Diver.ogg
4.5M	Dio - 1983 - Holy Diver/03 - Dio - Gypsy.ogg
5.3M	Dio - 1983 - Holy Diver/04 - Dio - Caught In The Middle.ogg
6.3M	Dio - 1983 - Holy Diver/05 - Dio - Don't Talk To Strangers.ogg
5.6M	Dio - 1983 - Holy Diver/06 - Dio - Straight Through The Heart.ogg
6.5M	Dio - 1983 - Holy Diver/07 - Dio - Invisible.ogg
5.2M	Dio - 1983 - Holy Diver/08 - Dio - Rainbow In The Dark.ogg
6.5M	Dio - 1983 - Holy Diver/09 - Dio - Shame On The Night.ogg
4.8M	Dio - 1983 - Holy Diver/10 - Dio - Interview Forming Own Band (Bonus Track).ogg
2.5M	Dio - 1983 - Holy Diver/11 - Dio - Interview Vinnie Appice Leaving Sabbath (Bonus Track).ogg
3.5M	Dio - 1983 - Holy Diver/12 - Dio - Interview Seeking Out A Guitarist (Bonus Track).ogg
965M	Dio - 1983 - Holy Diver/13 - Dio - Interview Working In London (Bonus Track).ogg
864K	Dio - 1983 - Holy Diver/14 - Dio - Interview Release Timing Of Holy Diver (Bonus Track).ogg
  0B	Dio - 1983 - Holy Diver/15 - Dio - Interview Writing While With Sabbath (Bonus Track).ogg
  0B	Dio - 1983 - Holy Diver/16 - Dio - Interview Album Concept (Bonus Track).ogg
  0B	Dio - 1983 - Holy Diver/17 - Dio - Interview Producing The Album Himself (Bonus Track).ogg
  0B	Dio - 1983 - Holy Diver/18 - Dio - Interview Remembering The Tracks (Bonus Track).ogg
  0B	Dio - 1983 - Holy Diver/19 - Dio - Interview Artwork.ogg
128K	Dio - 1983 - Holy Diver/cover.jpg


Just for comparison on an ufs2 file system:
kamikaze@mobileKamikaze> cp -rv ~/music/ogg/Kamikaze/Dio\ -\ 1983\ -\ Holy\ Diver ./
/home/kamikaze/music/ogg/Kamikaze/Dio - 1983 - Holy Diver -> ./Dio - 1983 - Holy Diver
/home/kamikaze/music/ogg/Kamikaze/Dio - 1983 - Holy Diver/cover.jpg -> ./Dio - 1983 - Holy Diver/cover.jpg
/home/kamikaze/music/ogg/Kamikaze/Dio - 1983 - Holy Diver/02 - Dio - Holy Diver.ogg -> ./Dio - 1983 - Holy Diver/02 - Dio - Holy Diver.ogg
/home/kamikaze/music/ogg/Kamikaze/Dio - 1983 - Holy Diver/03 - Dio - Gypsy.ogg -> ./Dio - 1983 - Holy Diver/03 - Dio - Gypsy.ogg
/home/kamikaze/music/ogg/Kamikaze/Dio - 1983 - Holy Diver/04 - Dio - Caught In The Middle.ogg -> ./Dio - 1983 - Holy Diver/04 - Dio - Caught In The Middle.ogg
/home/kamikaze/music/ogg/Kamikaze/Dio - 1983 - Holy Diver/05 - Dio - Don't Talk To Strangers.ogg -> ./Dio - 1983 - Holy Diver/05 - Dio - Don't Talk To Strangers.ogg
/home/kamikaze/music/ogg/Kamikaze/Dio - 1983 - Holy Diver/06 - Dio - Straight Through The Heart.ogg -> ./Dio - 1983 - Holy Diver/06 - Dio - Straight Through The Heart.ogg
/home/kamikaze/music/ogg/Kamikaze/Dio - 1983 - Holy Diver/07 - Dio - Invisible.ogg -> ./Dio - 1983 - Holy Diver/07 - Dio - Invisible.ogg
/home/kamikaze/music/ogg/Kamikaze/Dio - 1983 - Holy Diver/08 - Dio - Rainbow In The Dark.ogg -> ./Dio - 1983 - Holy Diver/08 - Dio - Rainbow In The Dark.ogg
/home/kamikaze/music/ogg/Kamikaze/Dio - 1983 - Holy Diver/09 - Dio - Shame On The Night.ogg -> ./Dio - 1983 - Holy Diver/09 - Dio - Shame On The Night.ogg
/home/kamikaze/music/ogg/Kamikaze/Dio - 1983 - Holy Diver/10 - Dio - Interview Forming Own Band (Bonus Track).ogg -> ./Dio - 1983 - Holy Diver/10 - Dio - Interview Forming Own Band (Bonus Track).ogg
/home/kamikaze/music/ogg/Kamikaze/Dio - 1983 - Holy Diver/11 - Dio - Interview Vinnie Appice Leaving Sabbath (Bonus Track).ogg -> ./Dio - 1983 - Holy Diver/11 - Dio - Interview Vinnie Appice Leaving Sabbath (Bonus Track).ogg
/home/kamikaze/music/ogg/Kamikaze/Dio - 1983 - Holy Diver/12 - Dio - Interview Seeking Out A Guitarist (Bonus Track).ogg -> ./Dio - 1983 - Holy Diver/12 - Dio - Interview Seeking Out A Guitarist (Bonus Track).ogg
/home/kamikaze/music/ogg/Kamikaze/Dio - 1983 - Holy Diver/13 - Dio - Interview Working In London (Bonus Track).ogg -> ./Dio - 1983 - Holy Diver/13 - Dio - Interview Working In London (Bonus Track).ogg
/home/kamikaze/music/ogg/Kamikaze/Dio - 1983 - Holy Diver/14 - Dio - Interview Release Timing Of Holy Diver (Bonus Track).ogg -> ./Dio - 1983 - Holy Diver/14 - Dio - Interview Release Timing Of Holy Diver (Bonus Track).ogg
/home/kamikaze/music/ogg/Kamikaze/Dio - 1983 - Holy Diver/15 - Dio - Interview Writing While With Sabbath (Bonus Track).ogg -> ./Dio - 1983 - Holy Diver/15 - Dio - Interview Writing While With Sabbath (Bonus Track).ogg
/home/kamikaze/music/ogg/Kamikaze/Dio - 1983 - Holy Diver/16 - Dio - Interview Album Concept (Bonus Track).ogg -> ./Dio - 1983 - Holy Diver/16 - Dio - Interview Album Concept (Bonus Track).ogg
/home/kamikaze/music/ogg/Kamikaze/Dio - 1983 - Holy Diver/17 - Dio - Interview Producing The Album Himself (Bonus Track).ogg -> ./Dio - 1983 - Holy Diver/17 - Dio - Interview Producing The Album Himself (Bonus Track).ogg
/home/kamikaze/music/ogg/Kamikaze/Dio - 1983 - Holy Diver/18 - Dio - Interview Remembering The Tracks (Bonus Track).ogg -> ./Dio - 1983 - Holy Diver/18 - Dio - Interview Remembering The Tracks (Bonus Track).ogg
/home/kamikaze/music/ogg/Kamikaze/Dio - 1983 - Holy Diver/19 - Dio - Interview Artwork.ogg -> ./Dio - 1983 - Holy Diver/19 - Dio - Interview Artwork.ogg
/home/kamikaze/music/ogg/Kamikaze/Dio - 1983 - Holy Diver/01 - Dio - Stand Up And Shout.ogg -> ./Dio - 1983 - Holy Diver/01 - Dio - Stand Up And Shout.ogg
kamikaze@mobileKamikaze> du -hd0 Dio\ -\ 1983\ -\ Holy\ Diver/*
4.1M	Dio - 1983 - Holy Diver/01 - Dio - Stand Up And Shout.ogg
6.8M	Dio - 1983 - Holy Diver/02 - Dio - Holy Diver.ogg
4.5M	Dio - 1983 - Holy Diver/03 - Dio - Gypsy.ogg
5.3M	Dio - 1983 - Holy Diver/04 - Dio - Caught In The Middle.ogg
6.3M	Dio - 1983 - Holy Diver/05 - Dio - Don't Talk To Strangers.ogg
5.6M	Dio - 1983 - Holy Diver/06 - Dio - Straight Through The Heart.ogg
6.5M	Dio - 1983 - Holy Diver/07 - Dio - Invisible.ogg
5.2M	Dio - 1983 - Holy Diver/08 - Dio - Rainbow In The Dark.ogg
6.6M	Dio - 1983 - Holy Diver/09 - Dio - Shame On The Night.ogg
4.8M	Dio - 1983 - Holy Diver/10 - Dio - Interview Forming Own Band (Bonus Track).ogg
2.5M	Dio - 1983 - Holy Diver/11 - Dio - Interview Vinnie Appice Leaving Sabbath (Bonus Track).ogg
3.5M	Dio - 1983 - Holy Diver/12 - Dio - Interview Seeking Out A Guitarist (Bonus Track).ogg
528K	Dio - 1983 - Holy Diver/13 - Dio - Interview Working In London (Bonus Track).ogg
880K	Dio - 1983 - Holy Diver/14 - Dio - Interview Release Timing Of Holy Diver (Bonus Track).ogg
992K	Dio - 1983 - Holy Diver/15 - Dio - Interview Writing While With Sabbath (Bonus Track).ogg
2.3M	Dio - 1983 - Holy Diver/16 - Dio - Interview Album Concept (Bonus Track).ogg
2.9M	Dio - 1983 - Holy Diver/17 - Dio - Interview Producing The Album Himself (Bonus Track).ogg
 17M	Dio - 1983 - Holy Diver/18 - Dio - Interview Remembering The Tracks (Bonus Track).ogg
1.6M	Dio - 1983 - Holy Diver/19 - Dio - Interview Artwork.ogg
124K	Dio - 1983 - Holy Diver/cover.jpg

>Fix:


>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-amd64->freebsd-fs 
Responsible-Changed-By: gavin 
Responsible-Changed-When: Sat Oct 31 12:57:34 UTC 2009 
Responsible-Changed-Why:  
Over to maintainer(s). 

http://www.freebsd.org/cgi/query-pr.cgi?pr=140134 

From: Dominic Fandrey <kamikaze@bsdforen.de>
To: bug-followup@FreeBSD.org, kamikaze@bsdforen.de
Cc:  
Subject: Re: kern/140134: [msdosfs] write and fsck destroy filesystem integrity
Date: Sun, 08 Nov 2009 12:18:53 +0100

 I just rebuilt world and kernel with RELENG_8 from ~3 hours ago.
 
 Afterwards I repeated the How-To-Repeat procedure and the
 problems shown therein didn't appear.
 
 In the next step I put my Cowon S9 (audio/video player with
 16GB fat32 partition) on the line.
 
 I mounted the device and created a file with touch.
 
 fsck_msdosfs said the drive was fine.
 Windows chkdsk reported the drive was fine.
 
 So the next attempt was to remove the empty file and
 copy a JPG ~2.5MB onto the drive.
 
 fsck_msdosfs reported the following:
 > fsck_msdosfs -n /dev/msdosfs/VALHALLA 
 ** /dev/msdosfs/VALHALLA
 ** Phase 1 - Read and Compare FATs
 ** Phase 2 - Check Cluster Chains
 ** Phase 3 - Checking Directories
 ** Phase 4 - Checking for Lost Files
 Lost cluster chain at cluster 4
 49 Cluster(s) lost
 Reconnect? no
 Clear? no
 Lost cluster chain at cluster 53
 37 Cluster(s) lost
 Reconnect? no
 Clear? no
 Lost cluster chain at cluster 90
 9 Cluster(s) lost
 Reconnect? no
 Clear? no
 Lost cluster chain at cluster 99
 9 Cluster(s) lost
 Reconnect? no
 Clear? no
 Lost cluster chain at cluster 108
 1126 Cluster(s) lost
 Reconnect? no
 Clear? no
 Lost cluster chain at cluster 1234
 9 Cluster(s) lost
 Reconnect? no
 Clear? no
 Lost cluster chain at cluster 1244
 5 Cluster(s) lost
 Reconnect? no
 Clear? no
 Lost cluster chain at cluster 1248
 49 Cluster(s) lost
 Reconnect? no
 Clear? no
 Lost cluster chain at cluster 1302
 282 Cluster(s) lost
 Reconnect? no
 Clear? no
 Lost cluster chain at cluster 1600
 5 Cluster(s) lost
 Reconnect? no
 Clear? no
 Lost cluster chain at cluster 1605
 59 Cluster(s) lost
 Reconnect? no
 Clear? no
 Lost cluster chain at cluster 449483
 1 Cluster(s) lost
 Reconnect? no
 Clear? no
 Free space in FSInfo block (-1) not correct (577592)
 Fix? no
 Next free cluster in FSInfo block (2) not free
 Fix? no
 2130 files, 426432 free (577592 clusters)
 
 
 Kinda panicked I ran Windows chkdsk and it reported the drive
 was just fine fine.
 
 fsck_msdosfs still reports the same errors. I conclude that
 writing to msdosfs seems to be fine, now, however fsck_msdosfs
 is a dangerous threat to fat32 file systems.

From: Dominic Fandrey <kamikaze@bsdforen.de>
To: bug-followup@FreeBSD.org, kamikaze@bsdforen.de
Cc:  
Subject: Re: kern/140134: [msdosfs] write and fsck destroy filesystem integrity
Date: Tue, 05 Jul 2011 11:38:24 +0200

 I request closing.
 
 This hasn't happened in a very long time and a lot of code has
 changed since.
 
 Regards
 
 -- 
 A: Because it fouls the order in which people normally read text.
 Q: Why is top-posting such a bad thing?
 A: Top-posting.
 Q: What is the most annoying thing on usenet and in e-mail? 
State-Changed-From-To: open->closed 
State-Changed-By: ae 
State-Changed-When: Mon Jul 11 09:35:23 UTC 2011 
State-Changed-Why:  
The submitter has requested close this PR. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=140134 
>Unformatted:
