From olli@lurza.secnetix.de  Tue Nov 28 17:37:43 2006
Return-Path: <olli@lurza.secnetix.de>
Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52])
	by hub.freebsd.org (Postfix) with ESMTP id 188AA16A49E
	for <FreeBSD-gnats-submit@freebsd.org>; Tue, 28 Nov 2006 17:37:43 +0000 (UTC)
	(envelope-from olli@lurza.secnetix.de)
Received: from lurza.secnetix.de (lurza.secnetix.de [83.120.8.8])
	by mx1.FreeBSD.org (Postfix) with ESMTP id 2625C43C9E
	for <FreeBSD-gnats-submit@freebsd.org>; Tue, 28 Nov 2006 17:37:37 +0000 (GMT)
	(envelope-from olli@lurza.secnetix.de)
Received: from lurza.secnetix.de (spslgh@localhost [127.0.0.1])
	by lurza.secnetix.de (8.13.4/8.13.4) with ESMTP id kASHbMhc015502;
	Tue, 28 Nov 2006 18:37:27 +0100 (CET)
	(envelope-from oliver.fromme@secnetix.de)
Received: (from olli@localhost)
	by lurza.secnetix.de (8.13.4/8.13.1/Submit) id kASHbM2V015501;
	Tue, 28 Nov 2006 18:37:22 +0100 (CET)
	(envelope-from olli)
Message-Id: <200611281737.kASHbM2V015501@lurza.secnetix.de>
Date: Tue, 28 Nov 2006 18:37:22 +0100 (CET)
From: Oliver Fromme <olli@secnetix.de>
Reply-To: Oliver Fromme <olli@secnetix.de>
To: FreeBSD-gnats-submit@freebsd.org
Cc: Oliver Fromme <olli@secnetix.de>
Subject: Make MSDOSFS_LARGE a mount option
X-Send-Pr-Version: 3.113
X-GNATS-Notify:

>Number:         105964
>Category:       kern
>Synopsis:       [msdosfs] Make MSDOSFS_LARGE a mount option
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    rodrigc
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          change-request
>Submitter-Id:   current-users
>Arrival-Date:   Tue Nov 28 17:40:17 GMT 2006
>Closed-Date:    Sun Jul 08 15:52:11 GMT 2007
>Last-Modified:  Sun Jul 08 15:52:11 GMT 2007
>Originator:     Oliver Fromme
>Release:        n/a
>Organization:
secnetix GmbH & Co. KG
		http://www.secnetix.de/bsd
>Environment:
   n/a
>Description:

   This problem has been discussed on the -stable mailing list,
   an Craig Rodrigues <rodrigc@crodrigues.org> asked me to
   submit a PR for this issue because he's interested to pick
   it up.  So here we go.

   The FAT file system format doesn't support file ID numbers
   (UFS/FFS calls them "inode numbers").  Therefore MSDOSFS has
   to create such numbers somehow.  Currently there are two
   hacks for that purpose, with different drawbacks:

   -1-  (Default)  Use the directory entry offset of the file
        as the file ID number.  Assume that the hole media is
        divided into blocks the size of a directory entry
        (32 bytes), and use that "block number" for the file
        ID.  Since file ID numbers (a.k.a. inodes) are 32 bit,
        that algorithm will overflow above 32 * 2^32 = 128 GB.
        If you try to mount a FAT file system larger than
        128 GB, it will fail and print "disk too big, sorry".

   -2-  (With MSDOSFS_LARGE in the kernel)  Maintain a table
        that dynamically maps 64bit offsets (that are computed
        like above) to 32bit ID numbers.  This works for FAT
        file systems of any size > 128 GB (the code falls back
        to algorithm 1 for file systems < 128 GB).
        Two drawbacks:

        -A- If a large number of files is accessed, the table
            will grow very big and consume much kernel memory.
            It is possible that the machine panics when it
            runs out of kernel memory.
        -B- Since, the mapping is dynamic, file ID numbers may
            be different when the file system is unmounted and
            re-mounted.  That will break NFS exports, because
            NFS assumes that file ID numbers (which are used
            for NFS handles) are constant.

        It should be noted that those drawbacks only apply if
        the file system is > 128 GB.  For smaller file systems
        the code will automatically use the simpler algorithm
        described first.  This is controlled by the flag
        MSDOSFS_LARGEFS (different from MSDOSFS_LARGE!).

>How-To-Repeat:

   Try to mount FAT file systems of various sizes and encounter
   the situations mentioned above.

   For testing and experimenting, you can easily use a md(4)
   device and newfs_msdos(8) to create a 160 GB FAT disk:

   # truncate -s 160000000000 testfat.img
   # mdconfig -a -t vnode -f testfat.img
   md1
   # fdisk -BI /dev/md1
   ******* Working on device /dev/md1 *******
   # newfs_msdos -s 312496317 -c 128 -h 254 -u 63 /dev/md1s1 orb
   /dev/md1s1: 312458112 sectors in 2441079 FAT32 clusters (65536 bytes/cluster)
   bps=512 spc=128 res=32 nft=2 mid=0xf0 spt=63 hds=254 hid=0 bsec=312496317 bspf=19071 rdcl=2 infs=1 bkbs=2
   # mount -t msdos -o ro /dev/md1s1 /mnt
   mount_msdosfs: /dev/md1s1: Invalid argument
   # dmesg | tail -1
   mountmsdosfs(): disk too big, sorry

   (Note:  The newfs_msdos command is not very fast.  It will
   take a few seconds.)

>Fix:

   Unfortunately, there is no real fix known for the problem.

   However, the problem is made worse by the fact that you
   have to recompile your kernel and reboot in order to
   enable the second hack (kernel option MSDOSFS_LARGE).

   That aspect of the problem could be fixed by making
   it a mount option instead of a kernel compile option,
   essentially converting the #ifdef's to regular if's.

   It has been considered to even enable the MSDOSFS_LARGE
   code by default.  However, because of the drawbacks (i.e.
   possibility of a panic because of kernel memory usage, and
   inability to NFS-export the file system) it should only be
   used if specifically requested by the user.

>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->trhodes 
Responsible-Changed-By: remko 
Responsible-Changed-When: Fri Dec 29 20:38:28 UTC 2006 
Responsible-Changed-Why:  
assign to tom, he cares about msdosfs 

http://www.freebsd.org/cgi/query-pr.cgi?pr=105964 
Responsible-Changed-From-To: trhodes->rodrigc 
Responsible-Changed-By: rodrigc 
Responsible-Changed-When: Mon Jan 29 02:20:55 UTC 2007 
Responsible-Changed-Why:  
Taking. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=105964 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/105964: commit references a PR
Date: Tue, 30 Jan 2007 03:12:02 +0000 (UTC)

 rodrigc     2007-01-30 03:11:45 UTC
 
   FreeBSD src repository
 
   Modified files:
     sys/conf             files 
     sys/fs/msdosfs       msdosfs_vfsops.c msdosfs_vnops.c 
   Log:
   Add a "-o large" mount option for msdosfs.  Convert compile-time checks for
   #ifdef MSDOSFS_LARGE to run-time checks to see if "-o large" was specified.
   
   Test case provided by Oliver Fromme:
     truncate -s 200G test.img
     mdconfig -a -t vnode -f test.img -u 9
     newfs_msdos -s 419430400 -n 1 /dev/md9 zip250
     mount -t msdosfs /dev/md9 /mnt    # should fail
     mount -t msdosfs -o large /dev/md9 /mnt   # should succeed
   
   PR:             105964
   Requested by:   Oliver Fromme <olli lurza secnetix de>
   Tested by:      trhodes
   MFC after:      2 weeks
   
   Revision  Changes    Path
   1.1173    +1 -1      src/sys/conf/files
   1.157     +34 -19    src/sys/fs/msdosfs/msdosfs_vfsops.c
   1.167     +20 -17    src/sys/fs/msdosfs/msdosfs_vnops.c
 _______________________________________________
 cvs-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/cvs-all
 To unsubscribe, send any mail to "cvs-all-unsubscribe@freebsd.org"
 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/105964: commit references a PR
Date: Tue, 30 Jan 2007 05:01:23 +0000 (UTC)

 rodrigc     2007-01-30 05:01:06 UTC
 
   FreeBSD src repository
 
   Modified files:
     sys/conf             NOTES options 
   Log:
   Remove MSDOSFS_LARGE compile time option.  It has been converted
   to a run time "-o large" mount option.
   
   PR:             105964
   MFC after:      2 weeks
   
   Revision  Changes    Path
   1.1408    +0 -12     src/sys/conf/NOTES
   1.574     +0 -3      src/sys/conf/options
 _______________________________________________
 cvs-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/cvs-all
 To unsubscribe, send any mail to "cvs-all-unsubscribe@freebsd.org"
 
State-Changed-From-To: open->patched 
State-Changed-By: linimon 
State-Changed-When: Mon May 21 15:21:34 UTC 2007 
State-Changed-Why:  
Needs MFC. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=105964 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/105964: commit references a PR
Date: Sun,  8 Jul 2007 15:30:37 +0000 (UTC)

 rodrigc     2007-07-08 15:30:28 UTC
 
   FreeBSD src repository
 
   Modified files:        (Branch: RELENG_6)
     sys/fs/msdosfs       msdosfs_vfsops.c msdosfs_vnops.c 
     sys/modules/msdosfs  Makefile 
     sys/conf             NOTES files options 
   Log:
   MFC: Convert MSDOSFS_LARGE compile time option to a "-o large" mount option.
   
   PR:     105964
   
   Revision     Changes    Path
   1.1325.2.36  +0 -12     src/sys/conf/NOTES
   1.1031.2.64  +1 -1      src/sys/conf/files
   1.510.2.21   +0 -3      src/sys/conf/options
   1.144.2.11   +35 -21    src/sys/fs/msdosfs/msdosfs_vfsops.c
   1.160.2.4    +20 -19    src/sys/fs/msdosfs/msdosfs_vnops.c
   1.22.2.1     +3 -3      src/sys/modules/msdosfs/Makefile
 _______________________________________________
 cvs-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/cvs-all
 To unsubscribe, send any mail to "cvs-all-unsubscribe@freebsd.org"
 
State-Changed-From-To: patched->closed 
State-Changed-By: rodrigc 
State-Changed-When: Sun Jul 8 15:51:58 UTC 2007 
State-Changed-Why:  


http://www.freebsd.org/cgi/query-pr.cgi?pr=105964 
>Unformatted:
