From marcolz@stack.nl  Mon Mar 27 04:57:58 2000
Return-Path: <marcolz@stack.nl>
Received: from mailhost.stack.nl (vaak.stack.nl [131.155.140.140])
	by hub.freebsd.org (Postfix) with ESMTP id E8A5537BAEF
	for <FreeBSD-gnats-submit@freebsd.org>; Mon, 27 Mar 2000 04:57:49 -0800 (PST)
	(envelope-from marcolz@stack.nl)
Received: from toad.stack.nl (toad.stack.nl [131.155.140.135])
	by mailhost.stack.nl (Postfix) with ESMTP id 372635D006
	for <FreeBSD-gnats-submit@freebsd.org>; Mon, 27 Mar 2000 14:57:43 +0200 (CEST)
Received: by toad.stack.nl (Postfix, from userid 333)
	id 7FF719713; Mon, 27 Mar 2000 14:57:39 +0200 (CEST)
Message-Id: <20000327125739.7FF719713@toad.stack.nl>
Date: Mon, 27 Mar 2000 14:57:39 +0200 (CEST)
From: marcolz@stack.nl
Reply-To: marcolz@stack.nl
To: FreeBSD-gnats-submit@freebsd.org
Subject: pax cannot read all tar files created by tar.
X-Send-Pr-Version: 3.2

>Number:         17619
>Category:       bin
>Synopsis:       pax cannot read all tar files created by tar.
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    freebsd-bugs
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Mon Mar 27 05:00:01 PST 2000
>Closed-Date:    Sat Jul 12 18:09:50 PDT 2003
>Last-Modified:  Sat Jul 12 18:09:50 PDT 2003
>Originator:     Marc Olzheim
>Release:        FreeBSD 3.4-RELEASE i386
>Organization:
M.C.G.V. Stack
>Environment:
>Description:

	When trying to read a tar file created by GNU-tar or Solaris's tar,
	pax asks for another archive. It seems to misinterpret the tar-header.
	Some sizes of tar's will be handled just right, others don't.

>How-To-Repeat:

	prompt> dd if=/dev/zero of=foo bs=1759830 count=1
	prompt> tar cf bar.tar foo
	prompt> pax -vf bar.tar
	-rw-r--r--  1 marcolz          wheel    1759830 Mar 27 14:52 foo
	pax: End of archive volume 1 reached
	pax: ustar vol 1, 1 files, 1761280 bytes read, 0 bytes written.

	ATTENTION! pax archive volume change required.
	Ready for archive volume: 2
	Input archive name or "." to quit pax.
	

>Fix:
	
	A work around for this specific example is to do:
	prompt> { cat bar.tar ; dd if=/dev/zero bs=512 count=1 ; } | pax -v

	But in general it seems like a good idea to just add a complete
	<blocksize> block of zeroes, i.e. 10k, so:
	prompt> { cat bar.tar ; dd if=/dev/zero bs=10k count=1 ; } | pax -v


>Release-Note:
>Audit-Trail:

From: Sheldon Hearn <sheldonh@uunet.co.za>
To: marcolz@stack.nl
Cc: FreeBSD-gnats-submit@FreeBSD.ORG
Subject: Re: bin/17619: pax cannot read all tar files created by tar. 
Date: Wed, 29 Mar 2000 12:41:38 +0200

 On Mon, 27 Mar 2000 14:57:39 +0200, marcolz@stack.nl wrote:
 
 > 	When trying to read a tar file created by GNU-tar or Solaris's tar,
 > 	pax asks for another archive. It seems to misinterpret the tar-header.
 > 	Some sizes of tar's will be handled just right, others don't.
 
 Unfortunately, pax doesn't look like one of those parts of FreeBSD that
 anyone takes an active interest in.  This may not be fixed any time soon
 without patches. :-(
 
 Ciao,
 Sheldon.
 

From: Marc Olzheim <marcolz@stack.nl>
To: Sheldon Hearn <sheldonh@uunet.co.za>
Cc: marcolz@stack.nl, FreeBSD-gnats-submit@FreeBSD.ORG
Subject: Re: bin/17619: pax cannot read all tar files created by tar.
Date: Wed, 29 Mar 2000 13:08:09 +0200

 > 
 > 
 > On Mon, 27 Mar 2000 14:57:39 +0200, marcolz@stack.nl wrote:
 > 
 > > 	When trying to read a tar file created by GNU-tar or Solaris's tar,
 > > 	pax asks for another archive. It seems to misinterpret the tar-header.
 > > 	Some sizes of tar's will be handled just right, others don't.
 > 
 > Unfortunately, pax doesn't look like one of those parts of FreeBSD that
 > anyone takes an active interest in.  This may not be fixed any time soon
 > without patches. :-(
 
 OK, then I'll look into it as soon as I've got the time.
 
 Zlo
 

From: Anatoly Vorobey <mellon@pobox.com>
To: Sheldon Hearn <sheldonh@uunet.co.za>, marcolz@stack.nl
Cc: FreeBSD-gnats-submit@FreeBSD.ORG
Subject: Re: bin/17619: pax cannot read all tar files created by tar.
Date: Sun, 2 Apr 2000 10:13:31 +0000

 On Wed, Mar 29, 2000 at 02:50:03AM -0800, Sheldon Hearn wrote:
 > The following reply was made to PR bin/17619; it has been noted by GNATS.
 > 
 > From: Sheldon Hearn <sheldonh@uunet.co.za>
 > To: marcolz@stack.nl
 > Cc: FreeBSD-gnats-submit@FreeBSD.ORG
 > Subject: Re: bin/17619: pax cannot read all tar files created by tar. 
 > Date: Wed, 29 Mar 2000 12:41:38 +0200
 > 
 >  On Mon, 27 Mar 2000 14:57:39 +0200, marcolz@stack.nl wrote:
 >  
 >  > 	When trying to read a tar file created by GNU-tar or Solaris's tar,
 >  > 	pax asks for another archive. It seems to misinterpret the tar-header.
 >  > 	Some sizes of tar's will be handled just right, others don't.
 
 I looked into this. pax thinks tar archives should end with at least
 2 blocks of zero (i.e. 1024 zeroed bytes *after* the file ends). 
 In the example provided, GNU tar creates only one such block, and 
 pax thinks it must read another one which is not there, so it asks
 for the next volume. I don't know which one of them is right. In fact,
 I tried to look into GNU tar's code for creating archives, and couldn't
 understand where it creates even one block and what governs its
 behavior. The code there is extremely unreadable.
 
 pax can be made happy by this:
 
 --- tar.h.orig  Sun Apr  2 10:10:11 2000
 +++ tar.h       Sun Apr  2 10:10:19 2000
 @@ -44,7 +44,7 @@
  #define CHK_LEN                8               /* length of checksum field */
  #define TNMSZ          100             /* size of name field */
  #ifdef _PAX_
 -#define NULLCNT                2               /* number of null blocks in trailer */
 +#define NULLCNT                1               /* number of null blocks in trailer */
  #define CHK_OFFSET     148             /* start of checksum field */
  #define BLNKSUM                256L            /* sum of checksum field using ' ' */
  #endif /* _PAX_ */
 
 This shouldn't create any adverse effects, and appears to work on
 all archives created by GNU tar; however, pax will also add one
 trailer block rather than two to its archives after the patch. 
 The relevant code which uses this constant is in the tar_trail()
 function in tar.c .
 
 -- 
 Anatoly Vorobey,
 mellon@pobox.com http://pobox.com/~mellon/
 "Angels can fly because they take themselves lightly" - G.K.Chesterton
 

From: Anatoly Vorobey <mellon@pobox.com>
To: Sheldon Hearn <sheldonh@uunet.co.za>, marcolz@stack.nl
Cc: FreeBSD-gnats-submit@FreeBSD.ORG
Subject: Re: bin/17619: pax cannot read all tar files created by tar.
Date: Sun, 2 Apr 2000 10:38:22 +0000

 On Sun, Apr 02, 2000 at 12:20:02AM -0800, Anatoly Vorobey wrote:
 
 >  I looked into this. pax thinks tar archives should end with at least
 >  2 blocks of zero (i.e. 1024 zeroed bytes *after* the file ends). 
 >  In the example provided, GNU tar creates only one such block, and 
 >  pax thinks it must read another one which is not there, so it asks
 >  for the next volume. 
 
 I forgot to demonstrate this on the example given my Marc:
 
 prompt> dd if=/dev/zero of=foo bs=1759830 count=1
 prompt> tar cf bar.tar foo
 prompt> ls -l bar.tar
 -rw-r--r--  1 mellon  wheel  1761280 Apr  2 07:19 bar.tar
 
 Now 1759830 is padded to the 512 bytes boundary when written out
 which results in 1760256 bytes; together with 512 bytes of the header
 and 512 bytes of one trailer block, 512 + 1760256 + 512 = 1761280.
 
 FWIW, the page at www.paranoia.com/~vax/tar_format.html , currently
 uavailable but cached at
 http://www.google.com/search?q=cache:www.paranoia.com/~vax/tar_format.html ,
 says that there must be two trailer blocks. 
 
 -- 
 Anatoly Vorobey,
 mellon@pobox.com http://pobox.com/~mellon/
 "Angels can fly because they take themselves lightly" - G.K.Chesterton
 

From: Anatoly Vorobey <mellon@pobox.com>
To: Marc Olzheim <marcolz@stack.nl>
Cc: freebsd-gnats-submit@freebsd.org
Subject: Re: bin/17619: pax cannot read all tar files created by tar.
Date: Thu, 6 Apr 2000 08:46:40 +0000

 On Mon, Apr 03, 2000 at 10:20:54AM +0200, Marc Olzheim wrote:
 > > Bah. So what should be done in this case (assuming something needs
 > > to be done?). I can patch pax to accept one-block archives and
 > > yet produce correct archives - maybe that's the way to go? Given that 
 > > the GNU tar maintainer had more than 8 years to think about it, he's
 > > obviously emotionally attached to one-trailing-block files and won't
 > > let them go ;)
 > 
 > That seems like a good idea to me.
 
 See the attached patch. Tested, works fine. Maybe someone shall
 review/commit it?
 
 Index: tar.c
 ===================================================================
 RCS file: /freebsd/cvs/src/bin/pax/tar.c,v
 retrieving revision 1.13
 diff -u -r1.13 tar.c
 --- tar.c	1999/08/27 23:14:47	1.13
 +++ tar.c	2000/04/06 08:36:55
 @@ -86,7 +86,7 @@
  tar_endwr()
  #endif
  {
 -	return(wr_skip((off_t)(NULLCNT*BLKMULT)));
 +	return(wr_skip((off_t)(NULL_PUT*BLKMULT)));
  }
  
  /*
 @@ -104,7 +104,7 @@
  tar_endrd()
  #endif
  {
 -	return((off_t)(NULLCNT*BLKMULT));
 +	return((off_t)(NULL_PUT*BLKMULT));
  }
  
  /*
 @@ -152,8 +152,14 @@
  	 * NOT try to id a trailer during resync mode. During resync mode we
  	 * might as well throw this block out since a valid header can NEVER be
  	 * a block of all 0 (we must have a valid file name).
 +         *
 +         * A bug in GNU tar causes it to sometimes produce trailers with 
 +         * just one zero block. To handle this, we will put NULL_PUT
 +         * blocks in our archives, but will expect NULL_EXPECT on reads.
 +         * If there are actually two zero blocks, the second one will be
 +         * skipped on the next next_head() call.   
  	 */
 -	if (!in_resync && (++*cnt >= NULLCNT))
 +	if (!in_resync && (++*cnt >= NULL_EXPECT))
  		return(0);
  	return(1);
  }
 Index: tar.h
 ===================================================================
 RCS file: /freebsd/cvs/src/bin/pax/tar.h,v
 retrieving revision 1.6
 diff -u -r1.6 tar.h
 --- tar.h	1999/08/27 23:14:47	1.6
 +++ tar.h	2000/04/06 08:32:28
 @@ -44,7 +44,16 @@
  #define CHK_LEN		8		/* length of checksum field */
  #define TNMSZ		100		/* size of name field */
  #ifdef _PAX_
 -#define NULLCNT		2		/* number of null blocks in trailer */
 +
 +/* 
 + * The USTAR format requires two null blocks to trail every file, yet
 + * GNU tar will sometimes produce tar files with only one. Hence the 
 + * double standard.
 + */
 +
 +#define NULL_PUT	2	        /* num of blocks to put in trailer */
 +#define NULL_EXPECT	1               /* num of blocks to expect there */
 +	
  #define CHK_OFFSET	148		/* start of checksum field */
  #define BLNKSUM		256L		/* sum of checksum field using ' ' */
  #endif /* _PAX_ */
 
 -- 
 Anatoly Vorobey,
 mellon@pobox.com http://pobox.com/~mellon/
 "Angels can fly because they take themselves lightly" - G.K.Chesterton
 

From: Marc Olzheim <marcolz@stack.nl>
To: Anatoly Vorobey <mellon@pobox.com>
Cc: Marc Olzheim <marcolz@stack.nl>, freebsd-gnats-submit@freebsd.org
Subject: Re: bin/17619: pax cannot read all tar files created by tar.
Date: Sun, 23 Jun 2002 18:09:21 +0200

 > See the attached patch. Tested, works fine. Maybe someone shall
 > review/commit it?
 
 Hmmm, the patch has worked fine for met for quite some time now. Is
 anyone still maintaining pax ?
 
 OpenBSD has fixed it in a different way, somehow by the way...
 
 Zlo
State-Changed-From-To: open->closed 
State-Changed-By: kris 
State-Changed-When: Sat Jul 12 18:09:09 PDT 2003 
State-Changed-Why:  
This problem appears to be resolved. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=17619 
>Unformatted:
