From avg@topspin.kiev.ua  Mon Feb  7 20:07:52 2005
Return-Path: <avg@topspin.kiev.ua>
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id A471116A4CE
	for <FreeBSD-gnats-submit@freebsd.org>; Mon,  7 Feb 2005 20:07:52 +0000 (GMT)
Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140])
	by mx1.FreeBSD.org (Postfix) with ESMTP id E249A43D41
	for <FreeBSD-gnats-submit@freebsd.org>; Mon,  7 Feb 2005 20:07:50 +0000 (GMT)
	(envelope-from avg@topspin.kiev.ua)
Received: from oddity.topspin.kiev.ua (oddity.topspin.kiev.ua [212.40.38.87])
	by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id WAA09633
	for <FreeBSD-gnats-submit@freebsd.org>; Mon, 07 Feb 2005 22:07:49 +0200 (EET)
	(envelope-from avg@topspin.kiev.ua)
Received: from oddity.topspin.kiev.ua (localhost [127.0.0.1])
	by oddity.topspin.kiev.ua (8.12.10/8.12.10) with ESMTP id j17K7mVY016740
	for <FreeBSD-gnats-submit@freebsd.org>; Mon, 7 Feb 2005 22:07:48 +0200 (EET)
	(envelope-from avg@oddity.topspin.kiev.ua)
Received: (from avg@localhost)
	by oddity.topspin.kiev.ua (8.12.10/8.12.10/Submit) id j17K7m2R016739;
	Mon, 7 Feb 2005 22:07:48 +0200 (EET)
	(envelope-from avg)
Message-Id: <200502072007.j17K7m2R016739@oddity.topspin.kiev.ua>
Date: Mon, 7 Feb 2005 22:07:48 +0200 (EET)
From: Andriy Gapon <avg@icyb.net.ua>
To: FreeBSD-gnats-submit@freebsd.org
Cc:
Subject: corrupted data is read from UDF filesystem if read starts at non-aligned offset
X-Send-Pr-Version: 3.113
X-GNATS-Notify:

>Number:         77234
>Category:       kern
>Synopsis:       [udf] [patch] corrupted data is read from UDF filesystem if read starts at non-aligned offset
>Confidential:   no
>Severity:       critical
>Priority:       high
>Responsible:    remko
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Mon Feb 07 20:10:24 GMT 2005
>Closed-Date:    Tue Jun 26 07:01:59 GMT 2007
>Last-Modified:  Tue Jun 26 07:10:04 GMT 2007
>Originator:     Andriy Gapon
>Release:        FreeBSD 5.2.1-RELEASE-p13 i386
>Organization:
>Environment:
System: FreeBSD 5.2.1-RELEASE-p13 386
sys/fs/udf/udf_vnops.c is patched to revision 1.50


	
>Description:
it seems that udf_readatoffset() function does not properly handle
certain offset, size combinations:
offset%bsize != 0
and
size in ](bsize-offset%bsize) + n*bsize, bsize + n*bsize]
where n = 0, 1, 2, ... 

or with graphic illustration:
  current offset
  v
  |--------|        - data that we want to read
  |******|&|        - '*' are for good data, '&' for junk in memory
|--------|--------| - data on disk
|^^^^^^^^|          - this is what would be read with current code
|^^^^^^^^|^^^^^^^^| - this is what should be read
^        ^        ^
sector (or logical block) boundaries

this happens because calculations of number of sectors to read do not
take into account the fact that an additional sector may be needed because
of current offset alignment.

>How-To-Repeat:
mount udf disk and perform something similar to large enough file:
tmp$ dd if=/mnt/dvdrom/oddity/export.level-7.2005-02-01.dump.gz of=ttt bs=2000
21+1 records in
21+1 records out
42472 bytes transferred in 0.226800 secs (187266 bytes/sec)
tmp$ diff /mnt/dvdrom/oddity/export.level-7.2005-02-01.dump.gz ttt
Binary files /mnt/dvdrom/oddity/export.level-7.2005-02-01.dump.gz and ttt differ
tmp$ dd if=/mnt/dvdrom/oddity/export.level-7.2005-02-01.dump.gz of=ttt bs=2048
20+1 records in
20+1 records out
42472 bytes transferred in 0.038471 secs (1104001 bytes/sec)
tmp$ diff /mnt/dvdrom/oddity/export.level-7.2005-02-01.dump.gz ttt
tmp$ 

file should be the same in both cases

>Fix:

number of requested bytes passed to udf_readlblks should be increased 
by (offset & udfmp->bmask) bytes.
For the same reasons max_size calculated in udf_bmap_internal() should be
decreased by the same value, so that we do not read beyond extent end.

>Release-Note:
>Audit-Trail:

From: Andriy Gapon <avg@icyb.net.ua>
To: FreeBSD-gnats-submit@FreeBSD.org, freebsd-bugs@FreeBSD.org
Cc:  
Subject: Re: kern/77234: corrupted data is read from UDF filesystem if read
 starts at non-aligned offset
Date: Mon, 7 Feb 2005 23:31:30 +0200 (EET)

 If meaning of max_size is interpreted as maximum number of contiguous 
 bytes that can be read starting from a given offset rather than starting 
 from a beginning of a calculated sector number then a patch could be like 
 the following. (Please note that currently max_size contains number of 
 bytes in an extent to which current offset belongs, which is total 
 nonsense since any code that calls udf_bmap_internal() has no notion of 
 extents)
 
 --- udf_vnops.c.orig	Mon Feb  7 22:59:34 2005
 +++ udf_vnops.c	Mon Feb  7 23:18:06 2005
 @@ -1107,19 +1107,21 @@
   		*size = max_size;
   	*size = min(*size, MAXBSIZE);
 
 -	if ((error = udf_readlblks(udfmp, sector, *size, bp))) {
 +	if ((error = udf_readlblks(udfmp, sector, *size + (offset & udfmp->bmask), bp))) {
   		printf("warning: udf_readlblks returned error %d\n", error);
   		return (error);
   	}
 
   	bp1 = *bp;
 -	*data = (uint8_t *)&bp1->b_data[offset % udfmp->bsize];
 +	*data = (uint8_t *)&bp1->b_data[offset & udfmp->bmask];
   	return (0);
   }
 
   /*
    * Translate a file offset into a logical block and then into a physical
    * block.
 + * max_size - maximum number of bytes that can be read starting from given
 + * offset, not beginning of calculated sector number
    */
   static int
   udf_bmap_internal(struct udf_node *node, off_t offset, daddr_t *sector, uint32_t *max_size)
 @@ -1172,7 +1174,7 @@
   		lsector = (offset  >> udfmp->bshift) +
   		    ((struct short_ad *)(icb))->pos;
 
 -		*max_size = GETICBLEN(short_ad, icb);
 +		*max_size = icblen - offset;
 
   		break;
   	case 1:
 @@ -1196,7 +1198,7 @@
   		lsector = (offset >> udfmp->bshift) +
   		    ((struct long_ad *)(icb))->loc.lb_num;
 
 -		*max_size = GETICBLEN(long_ad, icb);
 +		*max_size = icblen - offset;
 
   		break;
   	case 3:

From: Andriy Gapon <avg@icyb.net.ua>
To: freebsd-gnats-submit@FreeBSD.org
Cc:  
Subject: Re: kern/77234: corrupted data is read from UDF filesystem if read
 starts at non-aligned offset
Date: Mon, 28 Mar 2005 18:53:00 +0300

 This is a multi-part message in MIME format.
 --------------020302040802060308050405
 Content-Type: text/plain; charset=KOI8-U
 Content-Transfer-Encoding: 7bit
 
 
 updated patch
 
 -- 
 Andriy Gapon
 
 --------------020302040802060308050405
 Content-Type: text/plain;
  name="offset.patch"
 Content-Transfer-Encoding: 7bit
 Content-Disposition: inline;
  filename="offset.patch"
 
 --- sys/fs/udf/udf_vnops.c.orig	Thu Mar 17 15:08:39 2005
 +++ sys/fs/udf/udf_vnops.c	Thu Mar 17 15:13:41 2005
 @@ -1091,23 +1097,25 @@
  		*size = max_size;
  	*size = min(*size, MAXBSIZE);
  
 -	if ((error = udf_readlblks(udfmp, sector, *size, bp))) {
 +	if ((error = udf_readlblks(udfmp, sector, *size + (offset & udfmp->bmask), bp))) {
  		printf("warning: udf_readlblks returned error %d\n", error);
  		/* note: *bp may be non-NULL */
  		return (error);
  	}
  
  	bp1 = *bp;
 -	*data = (uint8_t *)&bp1->b_data[offset % udfmp->bsize];
 +	*data = (uint8_t *)&bp1->b_data[offset & udfmp->bmask];
  	return (0);
  }
  
  /*
   * Translate a file offset into a logical block and then into a physical
   * block.
 + * max_size - maximum number of bytes that can be read starting from given
 + * offset, rather than beginning of calculated sector number
   */
  static int
  udf_bmap_internal(struct udf_node *node, off_t offset, daddr_t *sector, uint32_t *max_size)
  {
  	struct udf_mnt *udfmp;
  	struct file_entry *fentry;
 @@ -1158,7 +1166,7 @@
  		lsector = (offset  >> udfmp->bshift) +
  		    ((struct short_ad *)(icb))->pos;
  
 -		*max_size = GETICBLEN(short_ad, icb);
 +		*max_size = icblen - offset;
  
  		break;
  	case 1:
 @@ -1183,7 +1191,7 @@
  		lsector = (offset >> udfmp->bshift) +
  		    le32toh(((struct long_ad *)(icb))->loc.lb_num);
  
 -		*max_size = GETICBLEN(long_ad, icb);
 +		*max_size = icblen - offset;
  
  		break;
  	case 3:
 
 --------------020302040802060308050405--

From: Andriy Gapon <avg@icyb.net.ua>
To: FreeBSD-gnats-submit@FreeBSD.org, freebsd-bugs@FreeBSD.org
Cc:  
Subject: Re: kern/77234: corrupted data is read from UDF filesystem if read
 starts at non-aligned offset
Date: Mon, 7 Feb 2005 23:31:30 +0200 (EET)

 If meaning of max_size is interpreted as maximum number of contiguous 
 bytes that can be read starting from a given offset rather than starting 
 from a beginning of a calculated sector number then a patch could be like 
 the following. (Please note that currently max_size contains number of 
 bytes in an extent to which current offset belongs, which is total 
 nonsense since any code that calls udf_bmap_internal() has no notion of 
 extents)
 
 --- udf_vnops.c.orig	Mon Feb  7 22:59:34 2005
 +++ udf_vnops.c	Mon Feb  7 23:18:06 2005
 @@ -1107,19 +1107,21 @@
   		*size = max_size;
   	*size = min(*size, MAXBSIZE);
 
 -	if ((error = udf_readlblks(udfmp, sector, *size, bp))) {
 +	if ((error = udf_readlblks(udfmp, sector, *size + (offset & udfmp->bmask), bp))) {
   		printf("warning: udf_readlblks returned error %d\n", error);
   		return (error);
   	}
 
   	bp1 = *bp;
 -	*data = (uint8_t *)&bp1->b_data[offset % udfmp->bsize];
 +	*data = (uint8_t *)&bp1->b_data[offset & udfmp->bmask];
   	return (0);
   }
 
   /*
    * Translate a file offset into a logical block and then into a physical
    * block.
 + * max_size - maximum number of bytes that can be read starting from given
 + * offset, not beginning of calculated sector number
    */
   static int
   udf_bmap_internal(struct udf_node *node, off_t offset, daddr_t *sector, uint32_t *max_size)
 @@ -1172,7 +1174,7 @@
   		lsector = (offset  >> udfmp->bshift) +
   		    ((struct short_ad *)(icb))->pos;
 
 -		*max_size = GETICBLEN(short_ad, icb);
 +		*max_size = icblen - offset;
 
   		break;
   	case 1:
 @@ -1196,7 +1198,7 @@
   		lsector = (offset >> udfmp->bshift) +
   		    ((struct long_ad *)(icb))->loc.lb_num;
 
 -		*max_size = GETICBLEN(long_ad, icb);
 +		*max_size = icblen - offset;
 
   		break;
   	case 3:
 _______________________________________________
 freebsd-bugs@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/freebsd-bugs
 To unsubscribe, send any mail to "freebsd-bugs-unsubscribe@freebsd.org"
 
State-Changed-From-To: open->feedback 
State-Changed-By: netchild 
State-Changed-When: Sun Jan 8 16:25:15 UTC 2006 
State-Changed-Why:  
Is this still a problem with a recent 6.x or -current? 

http://www.freebsd.org/cgi/query-pr.cgi?pr=77234 
State-Changed-From-To: feedback->open 
State-Changed-By: linimon 
State-Changed-When: Sun Jan 8 17:54:03 UTC 2006 
State-Changed-Why:  
Feedback received. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=77234 

From: Andriy Gapon <avg@icyb.net.ua>
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/77234: [udf] [patch] corrupted data is read from UDF filesystem
 if read starts at non-aligned offset
Date: Wed, 24 May 2006 15:49:10 +0300

 This is a multi-part message in MIME format.
 --------------040007090500020000090100
 Content-Type: text/plain; charset=KOI8-U
 Content-Transfer-Encoding: 7bit
 
 
 This problem is still present in 6.1.
 Updated patch is attached.
 
 -- 
 Andriy Gapon
 
 --------------040007090500020000090100
 Content-Type: text/x-patch;
  name="offset.patch"
 Content-Transfer-Encoding: 7bit
 Content-Disposition: inline;
  filename="offset.patch"
 
 --- sys/fs/udf/udf_vnops.c.orig	Thu Mar 17 15:08:39 2005
 +++ sys/fs/udf/udf_vnops.c	Thu Mar 17 15:13:41 2005
 @@ -1091,19 +1097,21 @@
  		*size = max_size;
  	*size = min(*size, MAXBSIZE);
  
 -	if ((error = udf_readlblks(udfmp, sector, *size, bp))) {
 +	if ((error = udf_readlblks(udfmp, sector, *size + (offset & udfmp->bmask), bp))) {
  		printf("warning: udf_readlblks returned error %d\n", error);
  		/* note: *bp may be non-NULL */
  		return (error);
  	}
  
  	bp1 = *bp;
 -	*data = (uint8_t *)&bp1->b_data[offset % udfmp->bsize];
 +	*data = (uint8_t *)&bp1->b_data[offset & udfmp->bmask];
  	return (0);
  }
  
  /*
   * Translate a file offset into a logical block and then into a physical
   * block.
 + * max_size - maximum number of bytes that can be read starting from given
 + * offset, rather than beginning of calculated sector number
   */
  static int
 @@ -1158,7 +1166,7 @@
  		lsector = (offset  >> udfmp->bshift) +
  		    ((struct short_ad *)(icb))->pos;
  
 -		*max_size = GETICBLEN(short_ad, icb);
 +		*max_size = icblen - offset;
  
  		break;
  	case 1:
 @@ -1183,7 +1191,7 @@
  		lsector = (offset >> udfmp->bshift) +
  		    le32toh(((struct long_ad *)(icb))->loc.lb_num);
  
 -		*max_size = GETICBLEN(long_ad, icb);
 +		*max_size = icblen - offset;
  
  		break;
  	case 3:
 
 --------------040007090500020000090100--
Responsible-Changed-From-To: freebsd-bugs->remko 
Responsible-Changed-By: remko 
Responsible-Changed-When: Sun Mar 11 09:22:04 UTC 2007 
Responsible-Changed-Why:  
I will look at this 

http://www.freebsd.org/cgi/query-pr.cgi?pr=77234 
State-Changed-From-To: open->patched 
State-Changed-By: remko 
State-Changed-When: Mon Jun 11 20:15:37 UTC 2007 
State-Changed-Why:  
Patched in -CURRENT, MFC in a little. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=77234 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/77234: commit references a PR
Date: Mon, 11 Jun 2007 20:14:56 +0000 (UTC)

 remko       2007-06-11 20:14:44 UTC
 
   FreeBSD src repository
 
   Modified files:
     sys/fs/udf           udf_vnops.c 
   Log:
   Correct corrupt read when the read starts at a non-aligned offset.
   
   PR:             kern/77234
   MFC After:      1 week
   Approved by:    imp (mentor)
   Requested by:   many many people
   Submitted by:   Andriy Gapon <avg at icyb dot net dot ua>
   
   Revision  Changes    Path
   1.66      +6 -4      src/sys/fs/udf/udf_vnops.c
 _______________________________________________
 cvs-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/cvs-all
 To unsubscribe, send any mail to "cvs-all-unsubscribe@freebsd.org"
 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/77234: commit references a PR
Date: Tue, 26 Jun 2007 06:59:31 +0000 (UTC)

 remko       2007-06-26 06:59:24 UTC
 
   FreeBSD src repository
 
   Modified files:        (Branch: RELENG_6)
     sys/fs/udf           udf_vnops.c 
   Log:
   MFC v 1.66 udf_vnops.c
   
           Correct corrupt read when the read starts at a non-aligned offset.
   
           PR: kern/77234
           Approved by: imp (mentor)
           Requested by: many many people
           Submitted by: Andriy Gapon <avg at icyb dot net dot ua>
   
   Approved by:    imp (implicit, mentor)
   
   Revision  Changes    Path
   1.58.2.4  +6 -4      src/sys/fs/udf/udf_vnops.c
 _______________________________________________
 cvs-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/cvs-all
 To unsubscribe, send any mail to "cvs-all-unsubscribe@freebsd.org"
 
State-Changed-From-To: patched->closed 
State-Changed-By: remko 
State-Changed-When: Tue Jun 26 07:01:55 UTC 2007 
State-Changed-Why:  
The fix had been imported into -CURRENT and a few secs ago into RELENG_5 
and RELENG_6, thanks for the submission and the patience! 

http://www.freebsd.org/cgi/query-pr.cgi?pr=77234 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/77234: commit references a PR
Date: Tue, 26 Jun 2007 07:00:47 +0000 (UTC)

 remko       2007-06-26 07:00:39 UTC
 
   FreeBSD src repository
 
   Modified files:        (Branch: RELENG_5)
     sys/fs/udf           udf_vnops.c 
   Log:
   MFC v 1.66 udf_vnops.c
   
           Correct corrupt read when the read starts at a non-aligned offset.
   
           PR: kern/77234
           Approved by: imp (mentor)
           Requested by: many many people
           Submitted by: Andriy Gapon <avg at icyb dot net dot ua>
   
   Approved by:    imp (implicit, mentor)
   
   Revision  Changes    Path
   1.37.2.3  +6 -4      src/sys/fs/udf/udf_vnops.c
 _______________________________________________
 cvs-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/cvs-all
 To unsubscribe, send any mail to "cvs-all-unsubscribe@freebsd.org"
 
>Unformatted:
