From scheidell@secnap.net  Mon Jul 31 20:40:09 2006
Return-Path: <scheidell@secnap.net>
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id 2056F16A4DD
	for <FreeBSD-gnats-submit@freebsd.org>; Mon, 31 Jul 2006 20:40:09 +0000 (UTC)
	(envelope-from scheidell@secnap.net)
Received: from scanner.secnap.net (scanner.secnap.net [204.89.241.64])
	by mx1.FreeBSD.org (Postfix) with ESMTP id C83D243D49
	for <FreeBSD-gnats-submit@freebsd.org>; Mon, 31 Jul 2006 20:40:08 +0000 (GMT)
	(envelope-from scheidell@secnap.net)
Received: by scanner.secnap.net (Postfix, from userid 1001)
	id CBE71137BB7; Mon, 31 Jul 2006 16:40:07 -0400 (EDT)
Message-Id: <20060731204007.CBE71137BB7@scanner.secnap.net>
Date: Mon, 31 Jul 2006 16:40:07 -0400 (EDT)
From: Michael Scheidell <scheidell@secnap.net>
To: FreeBSD-gnats-submit@freebsd.org
Subject: gzip -l signed value error
X-Send-Pr-Version: 3.113
X-GNATS-Notify:

>Number:         101123
>Category:       bin
>Synopsis:       gzip -l signed value error
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    delphij
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Mon Jul 31 20:40:21 GMT 2006
>Closed-Date:    Tue Mar 20 14:24:07 GMT 2007
>Last-Modified:  Tue Mar 20 14:24:07 GMT 2007
>Originator:     Michael Scheidell
>Release:        FreeBSD
>Organization:
SECNAP Network Security
>Environment:
System: FreeBSD 0.mail.spammertrap.net 5.5-RELEASE FreeBSD 5.5-RELEASE #2: Fri May 26 21:06:57 EDT 2006     admin@0.mail.spammertrap.net:/usr/obj/usr/src/sys/HACKERTRAP_850  i386
>Description:
	run gzip -l on big file.
	POC file is 80GB, compressed down to 396MB
>How-To-Repeat:
	run gzip -l on big file
>Fix:
	I ASSUME some long needs to be a long long unsigned.


>Release-Note:
>Audit-Trail:

From: Michael Scheidell <scheidell@secnap.net>
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: bin/101123: gzip -l signed value error
Date: Mon, 31 Jul 2006 16:59:49 -0400

 Additional notes: gzip -l thinks its -1604378624
 
 zcat | wc -c sees real size at 80GB
 
 gzip -l /var/livefs/var/ht-850-5.5.gz
 compressed  uncompr. ratio uncompressed_name
 393954905 -1604378624   0.0% /var/livefs/var/ht-850-5.5
 
 
 0# zcat /var/livefs/var/ht-850-5.5.gz | wc -c
  80000000000
 
 near line 1300 in /usr/src/gnu/usr.bin/gzip
 
 I see some %9lu and %9ld
 above value is 12 digits long.
 
      
     if (first_time && method >= 0) {
         first_time = 0;
         if (verbose)  {
             printf("method  crc     date  time  ");
         }
         if (!quiet) {
             printf("compressed  uncompr. ratio uncompressed_name\n");
         }
     } else if (method < 0) {
         if (total_in <= 0 || total_out <= 0) return;
         if (verbose) {
             printf("                            %9lu %9lu ",
                    total_in, total_out);
         } else if (!quiet) {
             printf("%9ld %9ld ", total_in, total_out);
 
 -- 
 Michael Scheidell, CTO
 SECNAP Network Security / www.secnap.com
 scheidell@secnap.net  / 1+561-999-5000, x 1131
 

From: Michael Scheidell <scheidell@secnap.net>
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: bin/101123: gzip -l signed value error
Date: Tue, 01 Aug 2006 11:08:32 -0400

 newer gzip is BETTER, but not right:
 
 this on MAC OSX 10.4:
 
 gzip -V
 gzip 1.3.5
 (2002-09-30)
 
 
 gzip -l ht-850-5.5.gz
          compressed        uncompressed  ratio uncompressed_name
           393954905          2690588672  85.4% ht-850-5.5
 
 (uncompressed should be 80000000000ULL)
 
 this is the gzip being used on freebsd 5.5:
 
 gzip -V
 gzip 1.2.4 (18 Aug 93)
 
 I was able to edit gzip.c to get the same 26905888672 as in newer gzip, 
 but didn't save patches due to it still being wrong.
 (by changing some of the formats to llu)
 
 -- 
 Michael Scheidell, CTO
 SECNAP Network Security / www.secnap.com
 scheidell@secnap.net  / 1+561-999-5000, x 1131
 

From: Michael Scheidell <scheidell@secnap.net>
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: bin/101123: gzip -l signed value error
Date: Tue, 01 Aug 2006 16:39:17 -0400

 last entry: problem is twofold
 #1, freebsd base using old version of gzip (ports version is better)
 #2, design flaw in gzip won't record input file size > 0xffffffff
 
 Structure of zip header only allows 4 bytes for isize
 http://www.gzip.org/zlib/rfc-gzip.html
 
 ports version (gzip 1.3.5) attempts to correctly show size as
 
 ./gzip -l /var/livefs/var/ht-850-5.5.gz
          compressed        uncompressed  ratio uncompressed_name
           393954905          2690588672  85.4% /var/livefs/var/ht-850-5.5
 
 in order to be able to record the 80000000000LLU size of the input file, 
 I would need 5 bytes:
 12A05F2000.
 When gzip wrote the header, it dropped the 5th byte: 12, leaving A05F2000
 A05F2000 converted back to decimal is: 2690588672, so the ports version 
 of gzip is doing the best it can.
 Suggestion: can we upgrade the base gzip to 1.3.5 or 5.6 or 6.2? or add 
 a GZIP_OVERWRITE_BASE option to the ports version?
 Other than that, I see no way to fix gzip, but at least it can process a 
 4294967295 file in 1.3.5
 
 -- 
 Michael Scheidell, CTO
 SECNAP Network Security / www.secnap.com
 scheidell@secnap.net  / 1+561-999-5000, x 1131
 
Responsible-Changed-From-To: freebsd-bugs->delphij 
Responsible-Changed-By: delphij 
Responsible-Changed-When: Fri Jan 26 15:08:18 UTC 2007 
Responsible-Changed-Why:  
Take because I have just replaced the GNU gzip with a BSD licensed 
one so claim all gzip related PRs. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=101123 
State-Changed-From-To: open->feedback 
State-Changed-By: delphij 
State-Changed-When: Wed Jan 31 06:20:59 UTC 2007 
State-Changed-Why:  
Dear submitter, 

I think the newly added bsdgzip has resolved your problem 
with a different solution (at least for my test case). 

Would you please download the archive available as: 
http://people.freebsd.org/~delphij/gzip-r6/gzip.tbz 

Extract it under /usr/src/usr.bin/ 

and do 

cd /usr/src/usr.bin/gzip && make clean obj depend all install 

to see if this has resolved your problem? 

Thanks in advance! 

http://www.freebsd.org/cgi/query-pr.cgi?pr=101123 
State-Changed-From-To: feedback->closed 
State-Changed-By: delphij 
State-Changed-When: Tue Mar 20 14:23:33 UTC 2007 
State-Changed-Why:  
Feedback timed out and my own test case shows that this is 
no longer a problem for recent FreeBSD versions. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=101123 
>Unformatted:
