From gordon@hammy.burditt.org  Tue Dec  9 12:55:53 2003
Return-Path: <gordon@hammy.burditt.org>
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id B0DF616A4CE
	for <FreeBSD-gnats-submit@freebsd.org>; Tue,  9 Dec 2003 12:55:53 -0800 (PST)
Received: from hammy.burditt.org (hammy.burditt.org [206.138.224.67])
	by mx1.FreeBSD.org (Postfix) with ESMTP id 8E31443D21
	for <FreeBSD-gnats-submit@freebsd.org>; Tue,  9 Dec 2003 12:55:52 -0800 (PST)
	(envelope-from gordon@hammy.burditt.org)
Received: from gordon by hammy.burditt.org with local (Exim 4.24; FreeBSD)
	id 1ATotZ-0003vD-FP; Tue, 09 Dec 2003 14:55:49 -0600
Message-Id: <E1ATotZ-0003vD-FP@hammy.burditt.org>
Date: Tue, 09 Dec 2003 14:55:49 -0600
From: Gordon Burditt <gordonb@airmail.net>
Sender: Gordon Burditt <gordon@hammy.burditt.org>
Reply-To: Gordon Burditt <gordonb@airmail.net>
To: FreeBSD-gnats-submit@freebsd.org
Cc: gordonb@airmail.net
Subject: UDF filesystem appends garbage to files
X-Send-Pr-Version: 3.113
X-GNATS-Notify:

>Number:         60089
>Category:       kern
>Synopsis:       [udf] UDF filesystem appends garbage to files
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    scottl
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Tue Dec 09 13:00:38 PST 2003
>Closed-Date:    Sun Dec 04 21:18:34 UTC 2011
>Last-Modified:  Sun Dec 04 21:18:34 UTC 2011
>Originator:     Gordon Burditt
>Release:        FreeBSD 5.1-RELEASE i386
>Organization:
>Environment:
System: FreeBSD hammy.burditt.org 4.9-STABLE FreeBSD 4.9-STABLE #7: Sat Nov 22 14:13:11 CST 2003 gordon@hammy.burditt.org:/scratch5/i386-obj/usr/src/sys/HAMMY i386

	(Reporting this from a FreeBSD-4.9 system even though the
	DVD drive is installed on a 5.1-RELEASE system and the
	bug is observed on 5.1-RELEASE.  FreeBSD 4.9-RELEASE does
	not have mount_udf).

	

	mkisofs 2.0.3
	FreeBSD 5.1-RELEASE
	Generic Cheap DVD+/-R+/-RW recorder (reproducible without this)

>Description:
	
	If you build a UDF filesystem image of compressed files 
	(using mkisofs), then mount it (either burning it to DVD and mounting
	it, or mounting the image with mdconfig/mount), different programs
	will see the file length differently (either correct, or apparently
	rounded up to the next 64k in size).  In my case I used compressed
	filesystem dumps.

	ls -l says the original var.dump.gz and the one in the UDF 
	filesystem are the same length.
	cmp says the files are identical (not EOF on one of the files).
	wc says the UDF filesystem file is longer.
		6909   40703 1900544 /mnt/book/var.dump.gz     (UDF filesystem)
		6905   40385 1863645 /backups/dumps/0/book/var.dump.gz
 
	gzip -tv says there is trailing garbage at the end of the UDF file, 
		but the checksum is OK.
	hd shows extra stuff at the end of the UDF file.
	md5 says that the checksums are different.

	Not all files have their sizes rounded up to the next 64k, in
	particular not small files.  Files with the observed problem
	range from 326KB to almost 396MB.  Files without the problem
	range from 109 bytes to 6.5KB.  As it happens, there were no files
	between 6.5KB and 326KB.  I do not know for sure that this
	is strictly a file size issue.

	Whether or not mkisofs is producing a corrupted UDF filesystem,
	it does seem strange that some programs see the file as one size
	and others see it as another size.

>How-To-Repeat:
	
	Build a directory tree of large (e.g. 2MB to 400MB) gzipped files.
	Using mkisofs 2.0, build a UDF filesystem image from these files.
	Either (a) Use mdconfig and mount the image, or (b) burn the
	image to a CD-R/DVD-R and mount it.  Run gzip -tv on the
	gzipped files.  Note message about "decompression OK, trailing
	garbage ignored".  Verify that this message does not occur on
	the original files used to build the UDF filesystem image.
	
>Fix:

	not known.

	

						Gordon L. Burditt
>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->scottl 
Responsible-Changed-By: rwatson 
Responsible-Changed-When: Tue Dec 9 13:43:40 PST 2003 
Responsible-Changed-Why:  
Scott wrote the UDF file system support, hand this PR over to him. 


http://www.freebsd.org/cgi/query-pr.cgi?pr=60089 

From: gordonb@airmail.net
To: freebsd-gnats-submit@freebsd.org, gordonb@airmail.net
Cc:  
Subject: Re: kern/60089: UDF filesystem appends garbage to files
Date: Tue, 9 Dec 2003 20:44:30 -0600 (CST)

 This problem is also present on FreeBSD-CURRENT, built Dec.  3
 evening and cvsupped earlier the same day.  Also, it's on different
 hardware from the tests in the original report.
 
 Files larger than 65536 seem to have their size rounded up to the
 next higher multiple of 65536.
 
 I can reproduce the problem with this script.  Put it in an empty
 directory and run it as root.  (Make sure mdconfig and mount can
 be found with $PATH.  Also, you need mkisofs installed.)  Should
 run with under 5MB free space.  It *SHOULD* produce no differences
 in the 'wc' output, but it does.
 
 
 #!/bin/sh
 
 I=`pwd`
 
 rm -rf files
 # Make some files, compress them
 mkdir files
 for x in 60 61 62 63 64 65 66 67 68 69 70 80 90 95 100 110 120
 do
 	dd if=/dev/random of=files/$x bs=1k count=$x
 	gzip -9 files/$x
 done
 # Get length according to wc.
 cd files
 wc *.gz > $I/wc.orig
 cd ..
 # Build the udf image, and mount it
 umount $I/udf > /dev/null 2>&1
 mkisofs -udf -o udf.image files
 unit=`mdconfig -a -t vnode  -f udf.image`
 mkdir udf
 mount -t udf /dev/$unit $I/udf
 # Get the UDF image version of the length
 cd udf
 wc *.gz > $I/wc.udf
 cd ..
 # See what's different
 diff wc.orig wc.udf
 # Clean up for next run
 umount $I/udf
 mdconfig -d -u $unit
 exit
 
 
 Relevant part of output (compression of random data will yield
 slightly different sizes):
 
 1,3c1,3
 <      390    2382  102442 100.gz
 <      475    2621  112682 110.gz
 <      455    2764  122922 120.gz
 ---
 >      509    3001  131072 100.gz
 >      534    2980  131072 110.gz
 >      486    2906  131072 120.gz
 8,18c8,18
 <      215    1501   65567 64.gz
 <      258    1574   66596 65.gz
 <      260    1577   67620 66.gz
 <      267    1568   68644 67.gz
 <      269    1572   69668 68.gz
 <      281    1632   70692 69.gz
 <      276    1595   71716 70.gz
 <      326    1879   81956 80.gz
 <      364    2026   92196 90.gz
 <      403    2204   97316 95.gz
 <     5215   30666 1342045 total
 ---
 >      463    2996  131072 64.gz
 >      499    3040  131072 65.gz
 >      497    2960  131072 66.gz
 >      501    2963  131072 67.gz
 >      509    2934  131072 68.gz
 >      520    2967  131072 69.gz
 >      512    2918  131072 70.gz
 >      488    2916  131072 80.gz
 >      513    2823  131072 90.gz
 >      403    2220  131072 95.gz
 >     7410   43395 1955964 total
 
 						Gordon L. Burditt

From: Scott Long <scott_long@adaptec.com>
To: freebsd-gnats-submit@FreeBSD.org, gordonb@airmail.net
Cc:  
Subject: Re: kern/60089: UDF filesystem appends garbage to files
Date: Tue, 09 Dec 2003 16:10:29 -0700

 Can you provide a reasonably sized image that I can review?  I'm not
 aware of any problems with a similar data mix created from DirectCD
 or Nero.
 
 Scott
 
State-Changed-From-To: open->closed 
State-Changed-By: scottl 
State-Changed-When: Sun Dec 4 21:13:29 UTC 2011 
State-Changed-Why:  
s was fixed through a series of commits in 2007-2009. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=60089 
>Unformatted:
