From nobody@FreeBSD.org  Wed Jun 29 18:22:23 2005
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id AB25016A41C
	for <freebsd-gnats-submit@FreeBSD.org>; Wed, 29 Jun 2005 18:22:23 +0000 (GMT)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (www.freebsd.org [216.136.204.117])
	by mx1.FreeBSD.org (Postfix) with ESMTP id 8F81043D48
	for <freebsd-gnats-submit@FreeBSD.org>; Wed, 29 Jun 2005 18:22:23 +0000 (GMT)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by www.freebsd.org (8.13.1/8.13.1) with ESMTP id j5TIMN3H024653
	for <freebsd-gnats-submit@FreeBSD.org>; Wed, 29 Jun 2005 18:22:23 GMT
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.13.1/8.13.1/Submit) id j5TIMNIQ024652;
	Wed, 29 Jun 2005 18:22:23 GMT
	(envelope-from nobody)
Message-Id: <200506291822.j5TIMNIQ024652@www.freebsd.org>
Date: Wed, 29 Jun 2005 18:22:23 GMT
From: Michael Scheidell <scheidell@secnap.net>
To: freebsd-gnats-submit@FreeBSD.org
Subject: ongoing problems with fetch
X-Send-Pr-Version: www-2.3

>Number:         82787
>Category:       bin
>Synopsis:       ongoing problems with fetch
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    des
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Wed Jun 29 18:30:21 GMT 2005
>Closed-Date:    Sun Jul 03 18:57:44 GMT 2005
>Last-Modified:  Mon Jul  4 09:20:08 GMT 2005
>Originator:     Michael Scheidell
>Release:        4.11
>Organization:
SECNAP
>Environment:
FreeBSD hackertrap 4.11-RELEASE-p9 FreeBSD 4.11-RELEASE-p9 #0: Tue Jun 28 22:32:47 EDT 2005     admin@hackertrap:/usr/obj/usr/src/sys/HACKERTRAP_305  i386

>Description:
Fetch fails too ofter for it to be our problem.
Starting with FBSD 4.10, FETCH has been a problem.
certain sites REFUSE fetch http://url (but allow curl -O and wget) and it doesn't matter if its with passive on or off.

trying to install port zip23, fetch doesn't see to understand how to log on anonmyously.

wget works great, curl -O works great.

see transcripts: see the 'Not loged on'.
tried fetch by hand, same problem.
ftp'd to uunet, user ftp, bassword blah@blah.com, worked fine.

curl -O works fine, wget works fine.

=> zip23.tar.gz doesn't seem to exist in /usr/ports/distfiles/.
=> Attempting to fetch from ftp://ftp.uu.net/pub/archiving/zip/src/.
fetch: ftp://ftp.uu.net/pub/archiving/zip/src/zip23.tar.gz: Not logged in
=> Attempting to fetch from ftp://ftp.funet.fi/pub/TeX/CTAN/tools/zip/info-zip/src/.
fetch: ftp://ftp.funet.fi/pub/TeX/CTAN/tools/zip/info-zip/src/zip23.tar.gz: File unavailable (e.g., file not found, no access)
=> Attempting to fetch from ftp://ctan.unsw.edu.au/tex-archive/tools/zip/info-zip/src/.
fetch: ftp://ctan.unsw.edu.au/tex-archive/tools/zip/info-zip/src/zip23.tar.gz: File unavailable (e.g., file not found, no access)
=> Attempting to fetch from ftp://ftp.tex.ac.uk/tex-archive/tools/zip/info-zip/src/.
fetch: ftp://ftp.tex.ac.uk/tex-archive/tools/zip/info-zip/src/zip23.tar.gz: File unavailable (e.g., file not found, no access)
=> Attempting to fetch from ftp://ftp.kddlabs.co.jp/CTAN/tools/zip/info-zip/src/.
fetch: ftp://ftp.kddlabs.co.jp/CTAN/tools/zip/info-zip/src/zip23.tar.gz: File unavailable (e.g., file not found, no access)
===>    Verifying install for zip in /usr/ports/archivers/zip
=> zip23.tar.gz doesn't seem to exist in /usr/ports/distfiles/.
=> Attempting to fetch from ftp://ftp.uu.net/pub/archiving/zip/src/.
fetch: ftp://ftp.uu.net/pub/archiving/zip/src/zip23.tar.gz: Not logged in
=> Attempting to fetch from ftp://ftp.funet.fi/pub/TeX/CTAN/tools/zip/info-zip/src/.
^Cfetch: transfer interrupted

hackertrap# wget ftp://ftp.uu.net/pub/archiving/zip/src/zip23.tar.gz
--13:34:16--  ftp://ftp.uu.net/pub/archiving/zip/src/zip23.tar.gz
           => `zip23.tar.gz'
Resolving ftp.uu.net... 192.48.96.9
Connecting to ftp.uu.net|192.48.96.9|:21... connected.
Logging in as anonymous ... Logged in!
==> SYST ... done.    ==> PWD ... done.
==> TYPE I ... done.  ==> CWD /pub/archiving/zip/src ... done.
==> PASV ... done.    ==> RETR zip23.tar.gz ... done.
Length: 723,283 (706K) (unauthoritative)

100%[========================================================================================>] 723,283      176.19K/s    ETA 00:00

13:34:21 (166.05 KB/s) - `zip23.tar.gz' saved [723283]

>How-To-Repeat:
   use   fetch.
>Fix:
      use curl or wget
>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->des 
Responsible-Changed-By: kris 
Responsible-Changed-When: Sat Jul 2 18:17:09 GMT 2005 
Responsible-Changed-Why:  
Assign to fetch maintainer 

http://www.freebsd.org/cgi/query-pr.cgi?pr=82787 

From: des@des.no (=?iso-8859-1?q?Dag-Erling_Sm=F8rgrav?=)
To: Michael Scheidell <scheidell@secnap.net>
Cc: freebsd-gnats-submit@FreeBSD.org
Subject: Re: bin/82787: ongoing problems with fetch
Date: Sun, 03 Jul 2005 12:36:16 +0200

 Michael Scheidell <scheidell@secnap.net> writes:
 > =3D> Attempting to fetch from ftp://ftp.uu.net/pub/archiving/zip/src/.
 > fetch: ftp://ftp.uu.net/pub/archiving/zip/src/zip23.tar.gz: Not logged in
 
 des@xps ~% fetch -dvvv ftp://ftp.uu.net/pub/archiving/zip/src/zip23.tar.gz
 scheme:   [ftp]
 user:     []
 password: []
 host:     [ftp.uu.net]
 port:     [0]
 document: [/pub/archiving/zip/src/zip23.tar.gz]
 ---> ftp.uu.net:21
 looking up ftp.uu.net
 connecting to ftp.uu.net:21
 <<< 220 FTP server ready.
 >>> USER anonymous
 <<< 331 Guest login ok, send your complete e-mail address as password.
 >>> PASS des@xps.des.no
 [...]
 <<< 230 Guest login ok, access restrictions apply.
 >>> TYPE I
 <<< 200 Type set to I.
 >>> CWD /pub/archiving/zip/src
 <<< 250 CWD command successful.
 >>> SIZE zip23.tar.gz
 <<< 213 723283
 size: [723283]
 >>> MDTM zip23.tar.gz
 <<< 213 20010628230921
 last modified: [2001-06-28 23:09:21]
 setting passive mode
 >>> PASV
 <<< 227 Entering Passive Mode (192,48,96,9,130,32)
 opening data connection
 initiating transfer
 >>> RETR zip23.tar.gz
 <<< 150 Opening BINARY mode data connection for zip23.tar.gz (723283 bytes).
 remote size / mtime: 723283 / 993769761
 zip23.tar.gz                                  100% of  706 kB  262 kBps
 Waiting for final status
 <<< 226 Transfer complete.
 
 Are you using a proxy?  Are you behind a firewall / NAT with
 transparent FTP proxy?  What happens when you run the above command?
 
 DES
 --=20
 Dag-Erling Sm=F8rgrav - des@des.no
 

From: "Michael Scheidell" <scheidell@secnap.net>
To: =?iso-8859-1?Q?Dag-Erling_Sm=F8rgrav?= <des@des.no>
Cc: <freebsd-gnats-submit@FreeBSD.org>
Subject: RE: bin/82787: ongoing problems with fetch
Date: Sun, 3 Jul 2005 07:22:37 -0400

 No proxy, behind a stateful firewall, and wget and curl -O work fine.
 
 In fact, I 'fixed' the problem by doing this:
 
 rm /usr/bin/fetch
 
 Replace with:
 #!/bin/sh
 /usr/local/bin/wget $*
 
 Everything works. Now.
 
 > ftp://ftp.uu.net/pub/archiving/zip/src/zip23.t> ar.gz
 > scheme:  =20
 > [ftp]
 > user:     []
 > password: []
 > host:    =20
 > [ftp.uu.net]
 
 So, problem is fetch us using a null user and password for you also, =
 right?
 
 Seems fetch is broken.

From: des@des.no (=?iso-8859-1?q?Dag-Erling_Sm=F8rgrav?=)
To: "Michael Scheidell" <scheidell@secnap.net>
Cc: <freebsd-gnats-submit@FreeBSD.org>
Subject: Re: bin/82787: ongoing problems with fetch
Date: Sun, 03 Jul 2005 14:32:37 +0200

 "Michael Scheidell" <scheidell@secnap.net> writes:
 > No proxy, behind a stateful firewall, and wget and curl -O work fine.
 
 What kind of stateful firewall?
 
 >> ftp://ftp.uu.net/pub/archiving/zip/src/zip23.t> ar.gz
 >> scheme:=20=20=20
 >> [ftp]
 >> user:     []
 >> password: []
 >> host:=20=20=20=20=20
 >> [ftp.uu.net]
 >
 > So, problem is fetch us using a null user and password for you also, righ=
 t?
 
 No, it isn't.  Read the entire log.
 
 > Seems fetch is broken.
 
 No, it seems your setup is broken and you are blaming fetch for it.
 
 DES
 --=20
 Dag-Erling Sm=F8rgrav - des@des.no
 

From: "Michael Scheidell" <scheidell@secnap.net>
To: <freebsd-gnats-submit@FreeBSD.org>
Cc:  
Subject: RE: bin/82787: ongoing problems with fetch
Date: Sun, 3 Jul 2005 13:14:22 -0400

 If anyone else wants to discuss this out of the public, I would be more
 than willing to continue this conversation.
 
 Fetch is intermittent.  Works sometimes, on some sites, won't work on
 sourceforge, won't work if someone does a 301 or 302 redirect.
 
 Substituting wget or curl -O as a shell works fine.
 
 As I said, if anyone else wants to fix it, I am more than wiling to
 discuss this offline.
 
 I have reported this several times since 4.10 came out.
 
 If I don't replace fetch, the build scripts we use fails 100% of the
 time, not always in the same place, not always on an ftp fetch, but
 sometimes on an http fetch.
 
 By substituting fetch with a shell that calls wget or curl -O it works
 100% of the time.
 
 How can that be anything but a problem with fetch?
 
 
 
State-Changed-From-To: open->closed 
State-Changed-By: des 
State-Changed-When: Sun Jul 3 18:57:43 GMT 2005 
State-Changed-Why:  
Submitter refuses to cooperate 

http://www.freebsd.org/cgi/query-pr.cgi?pr=82787 

From: "Michael Scheidell" <scheidell@secnap.net>
To: "Dag-Erling Smorgrav" <des@FreeBSD.org>
Cc: <FreeBSD-gnats-submit@FreeBSD.org>
Subject: RE: bin/82787: ongoing problems with fetch
Date: Mon, 4 Jul 2005 04:43:46 -0400

 No, its fixed.  You refused to even consider that fetch could be broken.
 
 
 Call it what it is.  You refused to even consider the posibility that
 there is a problem with fetch.
 
 Not sure why that means I refused to cooperate.
 
 This fixes the problem that FreeBSD 4.10 introduced into fetch.
 rm /usr/bin/fetch
 ln -s /usr/local/bin/wget /usr/bin/fetch
 
 
 
 > -----Original Message-----
 > From: Dag-Erling Smorgrav [mailto:des@FreeBSD.org]=20
 > Sent: Sunday, July 03, 2005 2:58 PM
 > To: Michael Scheidell; des@FreeBSD.org; des@FreeBSD.org
 > Subject: Re: bin/82787: ongoing problems with fetch
 >=20
 >=20
 > Synopsis: ongoing problems with fetch
 >=20
 > State-Changed-From-To: open->closed
 > State-Changed-By: des
 > State-Changed-When: Sun Jul 3 18:57:43 GMT 2005
 > State-Changed-Why:=20
 > Submitter refuses to cooperate
 >=20
 http://www.freebsd.org/cgi/query-pr.cgi?pr=3D82787
 

From: des@des.no (=?iso-8859-1?q?Dag-Erling_Sm=F8rgrav?=)
To: "Michael Scheidell" <scheidell@secnap.net>
Cc: <FreeBSD-gnats-submit@FreeBSD.org>
Subject: Re: bin/82787: ongoing problems with fetch
Date: Mon, 04 Jul 2005 11:15:03 +0200

 "Michael Scheidell" <scheidell@secnap.net> writes:
 > No, its fixed.  You refused to even consider that fetch could be broken.
 
 There is no evidence that it is.  Thousands of other people use it
 every day without a hitch.  We even have machines which do nothing all
 day but download distfiles, both to populate mirrors and to detect
 lame master sites (see <URL:http://people.freebsd.org/~fenner/>)
 
 > Not sure why that means I refused to cooperate.
 
 I asked you some very specific questions which you refused to answer:
 
  - what kind of firewall do you have in front of you?
 
  - what happens when you run the following command:
 
 fetch -dvvv ftp://ftp.uu.net/pub/archiving/zip/src/zip23.tar.gz
 
 DES
 --=20
 Dag-Erling Sm=F8rgrav - des@des.no
 
>Unformatted:
