From nobody@FreeBSD.org  Thu Aug 29 12:51:11 2002
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id F19FB37B401
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 29 Aug 2002 12:51:10 -0700 (PDT)
Received: from www.freebsd.org (www.FreeBSD.org [216.136.204.117])
	by mx1.FreeBSD.org (Postfix) with ESMTP id 90BD743E86
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 29 Aug 2002 12:51:03 -0700 (PDT)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by www.freebsd.org (8.12.4/8.12.4) with ESMTP id g7TJn3OT026752
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 29 Aug 2002 12:49:03 -0700 (PDT)
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.12.4/8.12.4/Submit) id g7TJn3J2026751;
	Thu, 29 Aug 2002 12:49:03 -0700 (PDT)
Message-Id: <200208291949.g7TJn3J2026751@www.freebsd.org>
Date: Thu, 29 Aug 2002 12:49:03 -0700 (PDT)
From: Ted Mittelstaedt <tedm@toybox.placo.com>
To: freebsd-gnats-submit@FreeBSD.org
Subject: ftp and fetch client needs an active command line option now
X-Send-Pr-Version: www-1.0

>Number:         42172
>Category:       misc
>Synopsis:       ftp and fetch client needs an active command line option now
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    des
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Thu Aug 29 13:00:15 PDT 2002
>Closed-Date:    Sat Sep 20 07:09:33 PDT 2003
>Last-Modified:  Sat Sep 20 07:09:33 PDT 2003
>Originator:     Ted Mittelstaedt
>Release:        4.6-RELEASE
>Organization:
Cool Dudes Inc.
>Environment:
N/A
>Description:
      Several FreeBSD versions ago the ftp client in FreeBSD was 
modified to default to Passive mode, rather than active mode.  This was
a good thing because so many FTP sites these days are behind
restrictive firewalls that don't permit active FTP.
      However, the problem is that nobody thought about the -p
option.  According to the manual, -p means "start in passive mode"
This is now unnecessary.  Instead, we need an option, such as -c,
that makes FTP start in active mode.  I'd assume that we want to
leave -p in there just to keep anyone's scripts from breaking, even
though it now does nothing.
      Now, I know that most people are going to be saying "so what,
all FTP servers support passive mode" but au-contraire, that is not
true.  I ran across one that ONLY supports ACTIVE mode FTP - it is
a network storage device called a "Snap!" server, manufactured by
Quantum.  The lack of an active switch makes it rather difficult to 
write scripts that deal with this device.
>How-To-Repeat:
      
>Fix:
      
>Release-Note:
>Audit-Trail:

From: Ian Dowse <iedowse@maths.tcd.ie>
To: Ted Mittelstaedt <tedm@toybox.placo.com>
Cc: freebsd-gnats-submit@FreeBSD.org
Subject: Re: misc/42172: ftp and fetch client needs an active command line option now 
Date: Thu, 29 Aug 2002 22:17:51 +0100

 In message <200208291949.g7TJn3J2026751@www.freebsd.org>, Ted Mittelstaedt writ
 es:
 >      Now, I know that most people are going to be saying "so what,
 >all FTP servers support passive mode" but au-contraire, that is not
 >true.  I ran across one that ONLY supports ACTIVE mode FTP - it is
 >a network storage device called a "Snap!" server, manufactured by
 >Quantum.  The lack of an active switch makes it rather difficult to 
 >write scripts that deal with this device.
 
 It seems that setting the environment variable FTP_PASSIVE_MODE to
 "NO" should do what you want. Does that work?
 
 Ian

From: Ted Mittelstaedt <tedm@toybox.placo.com>
To: Ian Dowse <iedowse@maths.tcd.ie>
Cc: Ted Mittelstaedt <tedm@toybox.placo.com>,
	freebsd-gnats-submit@FreeBSD.org
Subject: Re: misc/42172: ftp and fetch client needs an active command line option now
Date: Thu, 29 Aug 2002 17:05:04 -0700 (PDT)

 Quoting Ian Dowse <iedowse@maths.tcd.ie>:
 
 > In message <200208291949.g7TJn3J2026751@www.freebsd.org>, Ted
 > Mittelstaedt writ
 > es:
 > >      Now, I know that most people are going to be saying "so what,
 > >all FTP servers support passive mode" but au-contraire, that is not
 > >true.  I ran across one that ONLY supports ACTIVE mode FTP - it is
 > >a network storage device called a "Snap!" server, manufactured by
 > >Quantum.  The lack of an active switch makes it rather difficult to 
 > >write scripts that deal with this device.
 > 
 > It seems that setting the environment variable FTP_PASSIVE_MODE to
 > "NO" should do what you want. Does that work?
 > 
 > Ian
 > 
 
 It does work with FTP however it doesen't appear to work with the fetch
 program.  (perhaps that should be a second PR, fetch not responding to that 
 variable?) Also this workaround doesen't address the changes that need
 to be made in the man page for ftp.
 
 Thanks, though!
 Ted

From: Peter Pentchev <roam@ringlet.net>
To: Ted Mittelstaedt <tedm@toybox.placo.com>
Cc: bug-followup@FreeBSD.org
Subject: Re: misc/42172: ftp and fetch client needs an active command line option now
Date: Fri, 30 Aug 2002 09:59:57 +0300

 On Thu, Aug 29, 2002 at 05:10:04PM -0700, Ted Mittelstaedt wrote:
 >  Quoting Ian Dowse <iedowse@maths.tcd.ie>:
 >  
 >  > In message <200208291949.g7TJn3J2026751@www.freebsd.org>, Ted
 >  > Mittelstaedt writ
 >  > es:
 >  > >      Now, I know that most people are going to be saying "so what,
 >  > >all FTP servers support passive mode" but au-contraire, that is not
 >  > >true.  I ran across one that ONLY supports ACTIVE mode FTP - it is
 >  > >a network storage device called a "Snap!" server, manufactured by
 >  > >Quantum.  The lack of an active switch makes it rather difficult to 
 >  > >write scripts that deal with this device.
 >  > 
 >  > It seems that setting the environment variable FTP_PASSIVE_MODE to
 >  > "NO" should do what you want. Does that work?
 >  > 
 >  > Ian
 >  > 
 >  
 >  It does work with FTP however it doesen't appear to work with the fetch
 >  program.  (perhaps that should be a second PR, fetch not responding to that 
 >  variable?) Also this workaround doesen't address the changes that need
 >  to be made in the man page for ftp.
 
 Actually, here, on 4.6-STABLE as of yesterday, it seems to work with
 fetch(1), too:
 
 [roam@straylight:p6 ~/tmp/ff]$ fetch -v ftp://ftp.online.bg/pub/channels
 looking up ftp.online.bg
 connecting to ftp.online.bg:21
 setting passive mode
 opening data connection
 initiating transfer
 remote size / mtime: 3211 / 1010590214
 Receiving channels (3211 bytes): 100%
 3211 bytes transferred in 2.3 seconds (1.38 kBps)
 [roam@straylight:p6 ~/tmp/ff]$ env FTP_PASSIVE_MODE=NO fetch -v ftp://ftp.online.bg/pub/channels
 looking up ftp.online.bg
 connecting to ftp.online.bg:21
 binding data socket
 initiating transfer
 local size / mtime: 3211 / 1010590214
 remote size / mtime: 3211 / 1010590214
 Receiving channels (3211 bytes): 100%
 3211 bytes transferred in 1.4 seconds (2.31 kBps)
 [roam@straylight:p6 ~/tmp/ff]$
 
 Note the 'setting passive mode' in the first invocation and the lack
 thereof in the second one.  A 'fetch -v -v' would demonstrate it even
 better - fetch(1) does indeed obey "FTP_PASSIVE_MODE=NO".
 
 G'luck,
 Peter
 
 -- 
 Peter Pentchev	roam@ringlet.net	roam@FreeBSD.org
 PGP key:	http://people.FreeBSD.org/~roam/roam.key.asc
 Key fingerprint	FDBA FD79 C26F 3C51 C95E  DF9E ED18 B68D 1619 4553
 Hey, out there - is it *you* reading me, or is it someone else?

From: "Ted Mittelstaedt" <tedm@toybox.placo.com>
To: "Peter Pentchev" <roam@ringlet.net>
Cc: <bug-followup@FreeBSD.org>
Subject: RE: misc/42172: ftp and fetch client needs an active command line option now
Date: Fri, 30 Aug 2002 01:51:25 -0700

 >-----Original Message-----
 >From: Peter Pentchev [mailto:roam@ringlet.net]
 >Sent: Friday, August 30, 2002 12:00 AM
 >To: Ted Mittelstaedt
 >
 >Actually, here, on 4.6-STABLE as of yesterday, it seems to work with
 >fetch(1), too:
 >
 
 Well, in digging deeper I think that there's something else going on
 here that is making fetch different than a get from FTP.  Perhaps this
 ought to be written up in a different PR - but I don't know which one
 (fetch or the ftp client) is violating the spec.
 
 Fetch does see the variable setting, true, (of course this isn't
 documented in the fetch man page and should be) but the transfer is
 still failing - it just fails differently.
 
 I've included 2 debugging runs here, the first is fetch with the
 FTP_PASSIVE_MODE=NO set into the environment - this one is failing.  The
 second is me doing it manually with the ftp client - that succeeds.  This
 is 4.5-RELEASE.
 
 I believe the reason that fetch is failing from the Snap server is because
 it is issuing a MDTM for the file _first_, before actually getting the
 file.  The ftp client issues the MDTM _second_, after getting the file.
 In an active FTP session I think you have to close and reopen the
 data channel each time - perhaps issuing another port command - and
 the successive RETR and MDTM on the same channel isn't allowed in
 active mode FTP but permitted in passive mode?
 
 --------first run------------------
 
 $ fetch -v -v ftp://tedm:XXXX@mrssnapper.ipinc.net/tedm/trs
 ---> mrssnapper.ipinc.net:21
 looking up mrssnapper.ipinc.net
 connecting to mrssnapper.ipinc.net:21
 <<< 220 Service ready for new user.
 >>> USER tedm
 <<< 331 User name okay, need password.
 >>> PASS XXXX
 <<< 230 User logged in, proceed.
 >>> TYPE I
 <<< 200 Command okay.
 >>> CWD /tedm
 <<< 250 Requested file action okay, completed.
 >>> SIZE trs
 <<< 213  48
 size: [48]
 >>> MDTM trs
 <<< 213 20020820132731
 last modified: [2002-08-20 13:27:31]
 binding data socket
 >>> PORT 205,139,102,3,192,15
 <<< 200 Command okay.
 initiating transfer
 >>> RETR trs
 <<< 125 Data connection already open; transfer starting.
 fetch: trs: Data connection already open; transfer starting
 $
 
 ----------end of first run-------------
 
 --------second run----------------------
 
 $ ftp -n -v
 ftp> debug
 Debugging on (debug=1).
 ftp> passive
 Passive mode off.
 ftp> open mrssnapper.ipinc.net
 Connected to mrssnapper.ipinc.net.
 220 Service ready for new user.
 ---> SYST
 530 Not logged in.
 ftp> user tedm
 ---> USER tedm
 331 User name okay, need password.
 Password:
 ---> PASS XXXX
 230 User logged in, proceed.
 ftp> bin
 ---> TYPE I
 200 Command okay.
 ftp> cd /tedm
 ---> CWD /tedm
 250 Requested file action okay, completed.
 ftp> get trs
 local: trs remote: trs
 ---> SIZE trs
 213  48
 ---> PORT 205,139,102,3,192,19
 200 Command okay.
 ---> RETR trs
 125 Data connection already open; transfer starting.
 100%     48       00:00 ETA
 226 Closing data connection. Requested file action successful.
 48 bytes received in 0.02 seconds (3.06 KB/s)
 ---> MDTM trs
 213 20020820132731
 ftp>
 
 -----------end of second run--------------------- 
 
 Ted            
Responsible-Changed-From-To: freebsd-bugs->des 
Responsible-Changed-By: kris 
Responsible-Changed-When: Sat Jul 12 18:18:29 PDT 2003 
Responsible-Changed-Why:  
Assign to fetch maintainer 

http://www.freebsd.org/cgi/query-pr.cgi?pr=42172 
State-Changed-From-To: open->patched 
State-Changed-By: des 
State-Changed-When: Tue Aug 19 04:45:09 PDT 2003 
State-Changed-Why:  
Fixed in -CURRENT, awaiting MFC (hopefully in time for 4.9) 

http://www.freebsd.org/cgi/query-pr.cgi?pr=42172 

From: des@des.no (Dag-Erling =?iso-8859-1?q?Sm=F8rgrav?=)
To: freebsd-gnats-submit@freebsd.org
Cc:  
Subject: Re: misc/42172
Date: Tue, 19 Aug 2003 13:42:03 +0200

 This has nothing to do with the MDTM command (which doesn't use a
 separate data channel); libfetch just didn't expect a 125 response to
 RETR in active mode.  I'll fix that in a minute.
 
 DES
 --=20
 Dag-Erling Sm=F8rgrav - des@des.no

From: "Ted Mittelstaedt" <tedm@toybox.placo.com>
To: <freebsd-gnats-submit@FreeBSD.org>,
	"Ted Mittelstaedt" <tedm@toybox.placo.com>
Cc: <des@FreeBSD.org>
Subject: Re: misc/42172: ftp and fetch client needs an active command line option now
Date: Sun, 14 Sep 2003 15:25:01 -0700

 I have another violator on the FTP standard for you - Microsoft.
 (as usual, what do you expect I guess)
 
 It seems that while ftp.microsoft.com supports both Passive and
 Active mode FTP, the Windows server it runs on apparently does
 not understand the PASV command.
 
 Here's debug output from a _windows_ FTP program that shows the
 error: (just for laughs)
 
 connecting to 207.46.133.140:21
 Connected to 207.46.133.140 port 21
 220 Microsoft FTP Service
 USER anonymous
 331 Anonymous access allowed, send identity (e-mail name) as password.
 PASS (hidden)
 230-This is FTP.Microsoft.Com
 230 Anonymous user logged in.
 PWD
 257 "/" is current directory.
 SYST
 215 Windows_NT
 Host type (S): Microsoft NT
 PASV
 227 Entering Passive Mode (207,46,133,140,102,62).
 connecting to 207.46.133.140:26174
 - -
 connecting to 207.46.133.140:26174
 ! Connection failed 207.46.133.140 - connection timed out
 ! connect: error 0
 PORT 192,168,1,20,9,170
 200 PORT command successful.
 LIST
 150 Opening ASCII mode data connection for /bin/ls.
 Received 809 bytes in 0.2 secs, (36.84 Kbps), transfer succeeded
 226 Transfer complete.
 
 The problem is even worse with fetch - because instead of just
 aborting the command and continuing on, fetch just hangs forever.
 This is on FreeBSD 4.8.  I assume that the fix that's being
 put into libfetch is going to take care of this?
 
 Try for example plugging the following URL into fetch:
 
 ftp://ftp.microsoft.com/Products/frontpage/fp50.freebsd.tar.Z
 
 and you will see it hang.  The same thing happens with using
 the regular ftp program.  Now, use the regular ftp program and
 issue the command "passive" to switch off passive mode, before
 initiating the transfer - works fine then.
 
 Kind of sucks because this URL is used by the Front Page port.
 
 (Probably not many other ports in FreeBSD use Microsoft stuff.
 I leave it to guess why they aren't using Win2K for this server)
 
 Ted
 
 
 

From: des@des.no (Dag-Erling =?iso-8859-1?q?Sm=F8rgrav?=)
To: "Ted Mittelstaedt" <tedm@toybox.placo.com>
Cc: freebsd-gnats-submit@freebsd.org
Subject: Re: misc/42172: ftp and fetch client needs an active command line
 option now
Date: Mon, 15 Sep 2003 01:17:19 +0200

 "Ted Mittelstaedt" <tedm@toybox.placo.com> writes:
 >  It seems that while ftp.microsoft.com supports both Passive and
 >  Active mode FTP, the Windows server it runs on apparently does
 >  not understand the PASV command.
 >  [...]
 >  Try for example plugging the following URL into fetch:
 >=20=20
 >  ftp://ftp.microsoft.com/Products/frontpage/fp50.freebsd.tar.Z
 >=20=20
 >  and you will see it hang.
 
 Works fine here.  Are you sure you're not stuck behind a broken
 firewall or a content-scanning proxy?
 
 DES
 --=20
 Dag-Erling Sm=F8rgrav - des@des.no

From: "Ted Mittelstaedt" <tedm@toybox.placo.com>
To: <freebsd-gnats-submit@FreeBSD.org>
Cc:  
Subject: RE: misc/42172: ftp and fetch client needs an active command line option now
Date: Mon, 15 Sep 2003 01:05:11 -0700

 There is no proxy or firewall.  You aren't clear on what versions
 your running, this is on 4.8-RELEASE out-of-the-box, not 4.8-patched
 or some such.  Are you sure you didn't fix it already in your code?
 
 Ted
 
 
State-Changed-From-To: patched->closed 
State-Changed-By: des 
State-Changed-When: Sat Sep 20 07:09:32 PDT 2003 
State-Changed-Why:  
Originator reports that the problem has gone away. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=42172 
>Unformatted:
