From nge@cs.hmc.edu  Fri Jan 20 06:34:40 2006
Return-Path: <nge@cs.hmc.edu>
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id 8633716A41F
	for <freebsd-gnats-submit@freebsd.org>; Fri, 20 Jan 2006 06:34:40 +0000 (GMT)
	(envelope-from nge@cs.hmc.edu)
Received: from turing.cs.hmc.edu (turing.cs.hmc.edu [134.173.42.99])
	by mx1.FreeBSD.org (Postfix) with ESMTP id 516D543D45
	for <freebsd-gnats-submit@freebsd.org>; Fri, 20 Jan 2006 06:34:40 +0000 (GMT)
	(envelope-from nge@cs.hmc.edu)
Received: by turing.cs.hmc.edu (Postfix, from userid 26983)
	id AB87D53227; Thu, 19 Jan 2006 22:34:39 -0800 (PST)
Received: from localhost (localhost [127.0.0.1])
	by turing.cs.hmc.edu (Postfix) with ESMTP id 95EB45A8D5
	for <freebsd-gnats-submit@freebsd.org>; Thu, 19 Jan 2006 22:34:39 -0800 (PST)
Message-Id: <Pine.GSO.4.63.0601192232560.13186@turing>
Date: Thu, 19 Jan 2006 22:34:39 -0800 (PST)
From: Nate Eldredge <nge@cs.hmc.edu>
To: freebsd-gnats-submit@freebsd.org
Subject: mmap/cp fails on small file in UDF

>Number:         92040
>Category:       kern
>Synopsis:       [udf] mmap/cp fails on small file in UDF
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    scottl
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Fri Jan 20 06:40:04 GMT 2006
>Closed-Date:    Sun Jan 07 00:05:07 GMT 2007
>Last-Modified:  Sun Jan 07 00:05:07 GMT 2007
>Originator:     Nate Eldredge
>Release:        FreeBSD 6.0-RELEASE amd64
>Organization:
>Environment:
System: FreeBSD vulcan.lan 6.0-RELEASE FreeBSD 6.0-RELEASE #0: Wed Dec 14 20:08:57 PST 2005 nate@vulcan.lan:/usr/obj/usr/src/sys/VULCAN amd64



>Description:
If you try to copy a small file from a UDF filesystem, cp fails with EINVAL.

The underlying cause is that cp attempts to copy small files using mmap,
and mmap fails in this case.  I don't know whether cp is calling it wrong,
or mmap is simply unimplemented for udf, or if there is some other reason
for the failure.

In any case, it is probably a good idea for cp to fall back to read() if
mmap doesn't work.  It might also be useful to be able to explicitly
tell it to use read().  For example, I don't know what the mmap approach
will do in the presence of I/O errors.  I'm worried that it might silently
make a corrupt copy of the file.

This is on amd64.

Perhaps this bug should be cross-filed in category bin, if that is possible.
I don't know how to do that.

>How-To-Repeat:
nate@vulcan:~/bugs/udf$ mkdir dir
nate@vulcan:~/bugs/udf$ cat >dir/hello
hello world
nate@vulcan:~/bugs/udf$ mkisofs -o img -udf dir/
...
vulcan#	mdconfig -a -t vnode -f img
md0
vulcan#	mount_udf /dev/md0 /mnt/md0
vulcan#	ls -l /mnt/md0
total 0
-r--r--r--  1 root  wheel  12 Jan 20 22:10 hello
vulcan#	cat /mnt/md0/hello
hello world
vulcan#	cp /mnt/md0/hello .
cp: /mnt/md0/hello: Invalid argument

Script done on Thu Jan 19 22:11:55 2006
>Fix:
>Release-Note:
>Audit-Trail:

From: Giorgos Keramidas <keramida@freebsd.org>
To: Nate Eldredge <nge@cs.hmc.edu>
Cc: bug-followup@FreeBSD.org
Subject: Re: kern/92040: mmap/cp fails on small file in UDF
Date: Mon, 23 Jan 2006 05:15:47 +0200

 On 2006-01-19 22:34, Nate Eldredge <nge@cs.hmc.edu> wrote:
 > If you try to copy a small file from a UDF filesystem, cp fails with EINVAL.
 >
 > The underlying cause is that cp attempts to copy small files using mmap,
 > and mmap fails in this case.  I don't know whether cp is calling it wrong,
 > or mmap is simply unimplemented for udf, or if there is some other reason
 > for the failure.
 >
 > In any case, it is probably a good idea for cp to fall back to read() if
 > mmap doesn't work.  It might also be useful to be able to explicitly
 > tell it to use read().  For example, I don't know what the mmap approach
 > will do in the presence of I/O errors.  I'm worried that it might silently
 > make a corrupt copy of the file.
 >
 > This is on amd64.
 >
 > Perhaps this bug should be cross-filed in category bin, if that is possible.
 > I don't know how to do that.
 >
 > >How-To-Repeat:
 > nate@vulcan:~/bugs/udf$ mkdir dir
 > nate@vulcan:~/bugs/udf$ cat >dir/hello
 > hello world
 > nate@vulcan:~/bugs/udf$ mkisofs -o img -udf dir/
 > ...
 > vulcan#	mdconfig -a -t vnode -f img
 > md0
 > vulcan#	mount_udf /dev/md0 /mnt/md0
 > vulcan#	ls -l /mnt/md0
 > total 0
 > -r--r--r--  1 root  wheel  12 Jan 20 22:10 hello
 > vulcan#	cat /mnt/md0/hello
 > hello world
 > vulcan#	cp /mnt/md0/hello .
 > cp: /mnt/md0/hello: Invalid argument
 
 The following patch adds a `retry' variable in utils.c, which is set by
 default to true.  If mmap() succeeds, then `retry' is reset to false, to
 avoid copying some blocks with mmap() and others with read/write.
 
 It seems to work fine here...
 
     $ mount | grep /mnt
     /dev/md1 on /mnt (udf, local, read-only)
     $ ls -l /mnt
     total 0
     -r--r--r--  1 root  wheel  - 12 Jan 24 05:03 hello
     $ ./cp /mnt/hello .
     $ ls -ld hello
     -r--r--r--  1 keramida  keramida  - 12 Jan 23 05:12 hello
     $ cmp /mnt/hello hello ; echo $?
     0
     $
 
 Removing the need for rval = 1 when mmap() fails, means we can revert
 the condition of the following if statement and reduce the indentation
 level too:
 
 %%%
 Index: utils.c
 ===================================================================
 RCS file: /home/ncvs/src/bin/cp/utils.c,v
 retrieving revision 1.46
 diff -u -r1.46 utils.c
 --- utils.c	5 Sep 2005 04:36:08 -0000	1.46
 +++ utils.c	23 Jan 2006 03:09:33 -0000
 @@ -61,7 +61,7 @@
  {
  	static char buf[MAXBSIZE];
  	struct stat *fs;
 -	int ch, checkch, from_fd, rcount, rval, to_fd;
 +	int ch, checkch, from_fd, rcount, retry, rval, to_fd;
  	ssize_t wcount;
  	size_t wresid;
  	size_t wtotal;
 @@ -125,6 +125,7 @@
  	}
  
  	rval = 0;
 +	retry = 1;		/* Retry copying if mmap() fails. */
  
  	/*
  	 * Mmap and write if less than 8M (the limit is so we don't totally
 @@ -133,41 +134,37 @@
  	 */
  #ifdef VM_AND_BUFFER_CACHE_SYNCHRONIZED
  	if (S_ISREG(fs->st_mode) && fs->st_size > 0 &&
 -	    fs->st_size <= 8 * 1048576) {
 -		if ((p = mmap(NULL, (size_t)fs->st_size, PROT_READ,
 -		    MAP_SHARED, from_fd, (off_t)0)) == MAP_FAILED) {
 +	    fs->st_size <= 8 * 1048576 &&
 +	    (p = mmap(NULL, (size_t)fs->st_size, PROT_READ,
 +	    MAP_SHARED, from_fd, (off_t)0)) != MAP_FAILED) {
 +		retry = 0;
 +		wtotal = 0;
 +		for (bufp = p, wresid = fs->st_size; ;
 +		    bufp += wcount, wresid -= (size_t)wcount) {
 +			wcount = write(to_fd, bufp, wresid);
 +			wtotal += wcount;
 +			if (info) {
 +				info = 0;
 +				(void)fprintf(stderr,
 +					"%s -> %s %3d%%\n",
 +					entp->fts_path, to.p_path,
 +					cp_pct(wtotal, fs->st_size));
 +			}
 +			if (wcount >= (ssize_t)wresid || wcount <= 0)
 +				break;
 +		}
 +		if (wcount != (ssize_t)wresid) {
 +			warn("%s", to.p_path);
 +			rval = 1;
 +		}
 +		/* Some systems don't unmap on close(2). */
 +		if (munmap(p, fs->st_size) < 0) {
  			warn("%s", entp->fts_path);
  			rval = 1;
 -		} else {
 -			wtotal = 0;
 -			for (bufp = p, wresid = fs->st_size; ;
 -			    bufp += wcount, wresid -= (size_t)wcount) {
 -				wcount = write(to_fd, bufp, wresid);
 -				wtotal += wcount;
 -				if (info) {
 -					info = 0;
 -					(void)fprintf(stderr,
 -						"%s -> %s %3d%%\n",
 -						entp->fts_path, to.p_path,
 -						cp_pct(wtotal, fs->st_size));
 -						
 -				}
 -				if (wcount >= (ssize_t)wresid || wcount <= 0)
 -					break;
 -			}
 -			if (wcount != (ssize_t)wresid) {
 -				warn("%s", to.p_path);
 -				rval = 1;
 -			}
 -			/* Some systems don't unmap on close(2). */
 -			if (munmap(p, fs->st_size) < 0) {
 -				warn("%s", entp->fts_path);
 -				rval = 1;
 -			}
  		}
 -	} else
 +	}
  #endif
 -	{
 +	if (retry != 0) {
  		wtotal = 0;
  		while ((rcount = read(from_fd, buf, MAXBSIZE)) > 0) {
  			for (bufp = buf, wresid = rcount; ;
 %%%
Responsible-Changed-From-To: freebsd-bugs->scottl 
Responsible-Changed-By: kris 
Responsible-Changed-When: Fri Jan 27 20:14:34 UTC 2006 
Responsible-Changed-Why:  
Assign to maintainer 

http://www.freebsd.org/cgi/query-pr.cgi?pr=92040 

From: Pav Lucistnik <pav@FreeBSD.org>
To: bug-followup@FreeBSD.org, nge@cs.hmc.edu
Cc:  
Subject: Re: kern/92040: [udf] mmap/cp fails on small file in UDF
Date: Sat, 23 Dec 2006 02:09:28 +0100

 Hey Scott and folks,
 
 a patch to fix the described problem. Looks like it was missing
 vp->v_object thing. Adapted it from cd9660_vnops.c rev 1.107.
 Tested in RELENG_6, works! Fixed a random corruption on reading big
 files, too. Should just work on HEAD too.
 
 Index: udf_vnops.c
 ===================================================================
 RCS file: /home/ncvs/src/sys/fs/udf/udf_vnops.c,v
 retrieving revision 1.58.2.1
 diff -a -u -r1.58.2.1 udf_vnops.c
 --- udf_vnops.c	12 Mar 2006 21:50:02 -0000	1.58.2.1
 +++ udf_vnops.c	23 Dec 2006 01:04:20 -0000
 @@ -57,6 +57,7 @@
  
  static vop_access_t	udf_access;
  static vop_getattr_t	udf_getattr;
 +static vop_open_t	udf_open;
  static vop_ioctl_t	udf_ioctl;
  static vop_pathconf_t	udf_pathconf;
  static vop_read_t	udf_read;
 @@ -80,6 +81,7 @@
  	.vop_getattr =		udf_getattr,
  	.vop_ioctl =		udf_ioctl,
  	.vop_lookup =		vfs_cache_lookup,
 +	.vop_open =		udf_open,
  	.vop_pathconf =		udf_pathconf,
  	.vop_read =		udf_read,
  	.vop_readdir =		udf_readdir,
 @@ -159,6 +161,16 @@
  	    a_mode, a->a_cred, NULL));
  }
  
 +static int
 +udf_open(struct vop_open_args *ap) {
 +	struct udf_node *np = VTON(ap->a_vp);
 +	off_t fsize;
 +
 +	fsize = le64toh(np->fentry->inf_len);
 +	vnode_create_vobject_off(ap->a_vp, fsize, ap->a_td);
 +	return 0;
 +}
 +
  static int mon_lens[2][12] = {
  	{31, 28, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31},
  	{31, 29, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31}
 
 
 -- 
 Pav Lucistnik <pav@oook.cz>
               <pav@FreeBSD.org>
 
 Can't sing. Can't dance. Can handle a sword a little.
State-Changed-From-To: open->patched 
State-Changed-By: pav 
State-Changed-When: Sat Dec 23 18:50:46 UTC 2006 
State-Changed-Why:  
Fixed HEAD head, will MFC. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=92040 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/92040: commit references a PR
Date: Sat, 23 Dec 2006 18:53:38 +0000 (UTC)

 pav         2006-12-23 18:53:22 UTC
 
   FreeBSD src repository (doc,ports committer)
 
   Modified files:
     sys/fs/udf           udf_vnops.c 
   Log:
   Call vnode_create_vobject() in VOP_OPEN.  Makes mmap work on UDF filesystem.
   
   PR:             kern/92040
   Approved by:    scottl
   MFC after:      1 week
   
   Revision  Changes    Path
   1.62      +12 -0     src/sys/fs/udf/udf_vnops.c
 _______________________________________________
 cvs-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/cvs-all
 To unsubscribe, send any mail to "cvs-all-unsubscribe@freebsd.org"
 
State-Changed-From-To: patched->closed 
State-Changed-By: pav 
State-Changed-When: Sun Jan 7 00:03:52 UTC 2007 
State-Changed-Why:  
MFCed to RELENG_6 

http://www.freebsd.org/cgi/query-pr.cgi?pr=92040 
>Unformatted:
