From nobody@FreeBSD.org  Thu Dec 14 06:30:16 2006
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52])
	by hub.freebsd.org (Postfix) with ESMTP id 4284816A407
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 14 Dec 2006 06:30:16 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (www.freebsd.org [69.147.83.33])
	by mx1.FreeBSD.org (Postfix) with ESMTP id A2F7343CA1
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 14 Dec 2006 06:28:42 +0000 (GMT)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by www.freebsd.org (8.13.1/8.13.1) with ESMTP id kBE6UFEN020980
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 14 Dec 2006 06:30:15 GMT
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.13.1/8.13.1/Submit) id kBE6UFU2020979;
	Thu, 14 Dec 2006 06:30:15 GMT
	(envelope-from nobody)
Message-Id: <200612140630.kBE6UFU2020979@www.freebsd.org>
Date: Thu, 14 Dec 2006 06:30:15 GMT
From: Axel Gonzalez<loox@e-shell.net>
To: freebsd-gnats-submit@FreeBSD.org
Subject: [PATCH] vn_stat() fails with files > 2Gb on msdosfs (non 386)
X-Send-Pr-Version: www-3.0

>Number:         106703
>Category:       kern
>Synopsis:       [msdosfs] [patch] vn_stat() fails with files > 2Gb on msdosfs (non 386)
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    rodrigc
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Thu Dec 14 06:40:12 GMT 2006
>Closed-Date:    Mon Jan 29 02:20:15 GMT 2007
>Last-Modified:  Mon Jan 29 02:20:15 GMT 2007
>Originator:     Axel Gonzalez
>Release:        6.2-PRERELEASE
>Organization:
>Environment:
FreeBSD moonlight 6.2-PRERELEASE FreeBSD 6.2-PRERELEASE #4: Wed Dec 13 23:29:13 CST 2006     loox@moonlight:/usr/obj/usr/src/sys/LXAMD64  amd64
>Description:
When you copy/create a file > 2Gb on a msdosfs, it is created correctly, but vn_stat() fails on the file.

Since the function is used on several userland programs (ls, rm), it seems like the file is not on the FS.

$ ls
ls: DVD.iso: Value too large to be stored in data type.


This is caused by an uncasted macro, instead of reporting the corrected size, it reports a negative one (or a very large one)

This problem is only on non-386 systems (which use alternate macro)
>How-To-Repeat:
Copy (create) a file > 2Gb on a msdosfs

try to ls (rm, stat, etc)

$ cp /ufs_disk/DVD.iso /msdosfs_disk
$ ls
ls: DVD.iso: Value too large to be stored in data type

>Fix:
Apply the attached patch, recompile

sys_fs_msdosfs_bpb.h.patch



Patch attached with submission follows:

--- bpb.h.orig	Wed Dec 13 22:58:50 2006
+++ bpb.h	Wed Dec 13 22:59:39 2006
@@ -105,7 +105,8 @@
 #define	putulong(p, v)	(*((u_int32_t *)(p)) = (v))
 #else
 #define getushort(x)	(((u_int8_t *)(x))[0] + (((u_int8_t *)(x))[1] << 8))
-#define getulong(x)	(((u_int8_t *)(x))[0] + (((u_int8_t *)(x))[1] << 8) \
+#define getulong(x)	(u_int32_t)(((u_int8_t *)(x))[0] \
+		         + (((u_int8_t *)(x))[1] << 8) \
 			 + (((u_int8_t *)(x))[2] << 16)	\
 			 + (((u_int8_t *)(x))[3] << 24))
 #define putushort(p, v)	(((u_int8_t *)(p))[0] = (v),	\

>Release-Note:
>Audit-Trail:

From: Remko Lodder <remko@elvandar.org>
To: Axel Gonzalez <loox@e-shell.net>
Cc: freebsd-gnats-submit@FreeBSD.org, trhodes@FreeBSD.org
Subject: Re: kern/106703: [PATCH] vn_stat() fails with files > 2Gb on msdosfs (non 386)
Date: Thu, 14 Dec 2006 08:06:53 +0100

 On Thu, Dec 14, 2006 at 06:30:15AM +0000, Axel Gonzalez wrote:
 > 
 > >Description:
 > When you copy/create a file > 2Gb on a msdosfs, it is created correctly, but vn_stat() fails on the file.
 > 
 > Since the function is used on several userland programs (ls, rm), it seems like the file is not on the FS.
 > 
 
 Hello,
 
 	first of all thanks for the report to help make FreeBSD better!
 
 	One thing that crosses my mind is that (in my believing) the maximum
 	size of a file under msdos(fs) was 2gb, which could explain this
 	"problem" you are seeing. If that is still accurate, I don't think
 	we should patch the file you mention, but refuse to accept files
 	larger then 2gb since they will then not be visible at all on the
 	disk when msdos itself is being used (or some other OS that reads
 	out the msdosfs).
 
 	I copied in Tom Rhodes for more clarification (he maintains msdosfs).
 
 	Again thanks for the report and taking the time to mention this!
 
 	Cheers,
 	remko
 -- 
 Kind regards,
 
      Remko Lodder               ** remko@elvandar.org
      FreeBSD                    ** remko@FreeBSD.org
 
      /* Quis custodiet ipsos custodes */

From: Tom Rhodes <trhodes@FreeBSD.org>
To: Remko Lodder <remko@elvandar.org>
Cc: loox@e-shell.net, freebsd-gnats-submit@FreeBSD.org
Subject: Re: kern/106703: [PATCH] vn_stat() fails with files > 2Gb on
 msdosfs (non 386)
Date: Thu, 14 Dec 2006 19:48:48 -0500

 On Thu, 14 Dec 2006 08:06:53 +0100
 Remko Lodder <remko@elvandar.org> wrote:
 
 > On Thu, Dec 14, 2006 at 06:30:15AM +0000, Axel Gonzalez wrote:
 > > 
 > > >Description:
 > > When you copy/create a file > 2Gb on a msdosfs, it is created correctly, but vn_stat() fails on the file.
 > > 
 > > Since the function is used on several userland programs (ls, rm), it seems like the file is not on the FS.
 > > 
 > 
 > Hello,
 > 
 > 	first of all thanks for the report to help make FreeBSD better!
 > 
 > 	One thing that crosses my mind is that (in my believing) the maximum
 > 	size of a file under msdos(fs) was 2gb, which could explain this
 > 	"problem" you are seeing. If that is still accurate, I don't think
 > 	we should patch the file you mention, but refuse to accept files
 > 	larger then 2gb since they will then not be visible at all on the
 > 	disk when msdos itself is being used (or some other OS that reads
 > 	out the msdosfs).
 > 
 > 	I copied in Tom Rhodes for more clarification (he maintains msdosfs).
 > 
 > 	Again thanks for the report and taking the time to mention this!
 > 
 > 	Cheers,
 > 	remko
 
 Hmm, that is an interesting problem, and I'm digging the fix.
 FAT32 should handle file sizes up to (2^32)-1 bytes (one byte
 fewer than four gigabytes.  So we should properly handle this
 for all FAT32 file systems.
 
 Heh, I don't want maintainership of MSDOSFS, can I give it
 back or sell it to someone?  It's rarely used, I've only touched
 it a few times.  :P
 
 -- 
 Tom Rhodes

From: Axel Gonzalez <loox@e-shell.net>
To: Tom Rhodes <trhodes@freebsd.org>
Cc: Remko Lodder <remko@elvandar.org>,
 freebsd-gnats-submit@freebsd.org
Subject: Re: kern/106703: [PATCH] vn_stat() fails with files > 2Gb on msdosfs (non 386)
Date: Thu, 14 Dec 2006 20:21:16 -0600

 > > 	One thing that crosses my mind is that (in my believing) the maximum
 > > 	size of a file under msdos(fs) was 2gb, which could explain this
 > > 	"problem" you are seeing. If that is still accurate,
 
 msdosfs_vnops.c:#define DOS_FILESIZE_MAX        0xffffffff
 (4294967295) = 2^32 -1
 
 cp works ok, after patching my kernel, the file is correct (md5 match).
 
 
 
 >
 > Hmm, that is an interesting problem, and I'm digging the fix.
 > FAT32 should handle file sizes up to (2^32)-1 bytes (one byte
 > fewer than four gigabytes.  So we should properly handle this
 > for all FAT32 file systems.
 
 I traced the problem down to the getulong() macro (non 386 version).
 
 Problem is not with handling of the file, clusters, dirs. Just that 
 msdosfs_getattr() reports an incorrect (signed) size, and it makes vn_stat() 
 fail.
 
 If it helps, more info:
 
 MAX_OFF:9223372036854775807
 (max size of any file)
 
 size of the file:
 Correct (ufs): 
 3015487488 0xB3BCB000
 
 msdosfs_getattr():
 reported by w/o patch: 
 18446744072430071808 0xFFFFFFFEB3BCB000
 (this is why it returns EOVERFLOW)
 
 reported with patch:
 3015487488 0xB3BCB000

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/106703: commit references a PR
Date: Tue, 19 Dec 2006 01:56:15 +0000 (UTC)

 rodrigc     2006-12-19 01:55:45 UTC
 
   FreeBSD src repository
 
   Modified files:
     sys/fs/msdosfs       bpb.h 
   Log:
   Fix get_ulong() macro on AMD64 (or any little-endian 64-bit platform).
   This bug caused vn_stat() to fail on files larger than 2gb on msdosfs
   filesystems on AMD64.
   
   PR:             106703
   Tested by:      Axel Gonzalez <loox e-shell net>
   MFC after:      3 days
   
   Revision  Changes    Path
   1.12      +1 -5      src/sys/fs/msdosfs/bpb.h
 _______________________________________________
 cvs-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/cvs-all
 To unsubscribe, send any mail to "cvs-all-unsubscribe@freebsd.org"
 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/106703: commit references a PR
Date: Tue, 19 Dec 2006 02:32:15 +0000 (UTC)

 rodrigc     2006-12-19 02:31:58 UTC
 
   FreeBSD src repository
 
   Modified files:
     sys/fs/msdosfs       bpb.h 
   Log:
   For big-endian version of getulong() macro, cast result to u_int32_t.
   This macro was written expecting a 32-bit unsigned long, and
   doesn't work properly on 64-bit systems.  This bug caused vn_stat()
   to return incorrect values for files larger than 2gb on msdosfs filesystems
   on 64-bit systems.
   
   PR:             106703
   Submitted by:   Axel Gonzalez <loox e-shell net>
   MFC after:      3 days
   
   Revision  Changes    Path
   1.13      +1 -1      src/sys/fs/msdosfs/bpb.h
 _______________________________________________
 cvs-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/cvs-all
 To unsubscribe, send any mail to "cvs-all-unsubscribe@freebsd.org"
 
Responsible-Changed-From-To: freebsd-bugs->trhodes 
Responsible-Changed-By: remko 
Responsible-Changed-When: Fri Dec 29 20:45:08 UTC 2006 
Responsible-Changed-Why:  
assign to tom 

http://www.freebsd.org/cgi/query-pr.cgi?pr=106703 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/106703: commit references a PR
Date: Fri,  5 Jan 2007 05:50:53 +0000 (UTC)

 rodrigc     2007-01-05 05:50:36 UTC
 
   FreeBSD src repository
 
   Modified files:        (Branch: RELENG_6)
     sys/fs/msdosfs       bpb.h 
   Log:
   MFC 1.14, 1.15:
   marcel      2006-12-21 05:40:46 UTC
   
     Unbreak 64-bit little-endian systems that do require alignment.
     The fix involves using le16dec(), le32dec(), le16enc() and
     le32enc(). This eliminates invalid casts and duplicated logic.
   
   PR:             106703
   Tested by:      Axel Gonzalez <loox e-shell net>
   
   Revision  Changes    Path
   1.11.2.1  +5 -26     src/sys/fs/msdosfs/bpb.h
 _______________________________________________
 cvs-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/cvs-all
 To unsubscribe, send any mail to "cvs-all-unsubscribe@freebsd.org"
 
State-Changed-From-To: open->closed 
State-Changed-By: rodrigc 
State-Changed-When: Mon Jan 29 02:19:36 UTC 2007 
State-Changed-Why:  
Fix committed to HEAD and RELENG_6. 


Responsible-Changed-From-To: trhodes->rodrigc 
Responsible-Changed-By: rodrigc 
Responsible-Changed-When: Mon Jan 29 02:19:36 UTC 2007 
Responsible-Changed-Why:  
Fix committed to HEAD and RELENG_6. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=106703 
>Unformatted:
