From clkao@genius.cirx.org  Sat Dec 12 22:42:23 1998
Received: from genius.cirx.org (r00t.m1.ntu.edu.tw [140.112.240.59])
          by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id WAA20212
          for <FreeBSD-gnats-submit@freebsd.org>; Sat, 12 Dec 1998 22:41:58 -0800 (PST)
          (envelope-from clkao@genius.cirx.org)
Received: (from root@localhost)
	by genius.cirx.org (8.9.1/8.9.1) id OAA20036;
	Sun, 13 Dec 1998 14:43:54 +0800 (CST)
	(envelope-from clkao)
Message-Id: <199812130643.OAA20036@genius.cirx.org>
Date: Sun, 13 Dec 1998 14:43:54 +0800 (CST)
From: clkao@CirX.ORG
Reply-To: clkao@CirX.ORG
To: FreeBSD-gnats-submit@freebsd.org
Subject: [PATCH] propose adding `direct' option in fetch(3)
X-Send-Pr-Version: 3.2

>Number:         9064
>Category:       bin
>Synopsis:       [PATCH] propose adding `direct' option in fetch(3)
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    des
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          change-request
>Submitter-Id:   current-users
>Arrival-Date:   Sat Dec 12 22:50:00 PST 1998
>Closed-Date:    Wed Jul 19 02:22:15 PDT 2000
>Last-Modified:  Wed Jul 19 02:23:12 PDT 2000
>Originator:     Chia-liang Kao
>Release:        FreeBSD 3.0-CURRENT i386
>Organization:
CirX
>Environment:

FreeBSD genius.cirx.org 3.0-CURRENT FreeBSD 3.0-CURRENT #0: Mon Dec  7 12:16:11 CST 1998     clkao@genius.cirx.org:/usr/src/sys/compile/GENIUS  i386

>Description:

When `make fetch', we usually add a very nearby site in
MASTER_SITE_BACKUP, and then some of the primary distfile-mirroring sites.

The problem is that, I'd like those nearby site being fetched directly, rather
than via proxies, while the actually far sites(like ftp.freebsd.org) go through
proxies.

>How-To-Repeat:

>Fix:
	
Here is a patch adding a `-d' option to fetch(3). the `d' stands for `direct'
(it's quite hard to find an unused option character in fetch! :p)

you can modify your MASTER_SITE_BACKUP like this:

MASTER_SITE_BACKUP?=	\
	"-d ftp://oio/pub/freebsd/distfiles/${DIST_SUBDIR}/" \
	"-d ftp://freebsd.ntu.edu.tw/pub/FreeBSD/distfiles/${DIST_SUBDIR}/" \
	"-d ftp://bsd.sinica.edu.tw/pub/FreeBSD/distfiles/${DIST_SUBDIR}/" \
	ftp://ftp.freebsd.org/pub/FreeBSD/distfiles/${DIST_SUBDIR}/


cvs diff: Diffing .
Index: fetch.1
===================================================================
RCS file: /home/ncvs/src/usr.bin/fetch/fetch.1,v
retrieving revision 1.26
diff -u -r1.26 fetch.1
--- fetch.1	1998/12/08 13:00:48	1.26
+++ fetch.1	1998/12/13 06:30:13
@@ -61,6 +61,8 @@
 The file to retrieve is in directory
 .Ar dir
 on the remote host.
+.It Fl d
+Directly fetch from the host; skip PROTO_PROXY environment variables.
 .It Fl f Ar file
 The file to retrieve is named
 .Ar file
Index: fetch.h
===================================================================
RCS file: /home/ncvs/src/usr.bin/fetch/fetch.h,v
retrieving revision 1.7
diff -u -r1.7 fetch.h
--- fetch.h	1998/12/08 13:00:49	1.7
+++ fetch.h	1998/12/13 06:20:02
@@ -53,6 +53,7 @@
 	int fs_use_connect;	/* -t option */
 	off_t fs_expectedsize;	/* -S option */
 	int fs_reportsize;	/* -s option */
+	int fs_skip_proxy;	/* -d option */
 	time_t fs_modtime;
 	void *fs_proto;
 	int (*fs_retrieve)(struct fetch_state *);
Index: main.c
===================================================================
RCS file: /home/ncvs/src/usr.bin/fetch/main.c,v
retrieving revision 1.49
diff -u -r1.49 main.c
--- main.c	1998/12/08 13:00:49	1.49
+++ main.c	1998/12/13 06:24:46
@@ -52,7 +52,7 @@
 usage()
 {
 	fprintf(stderr, "%s\n%s\n", 
-		"usage: fetch [-DHILMNPRTVablmnpqrstv] [-o outputfile] [-S bytes]",
+		"usage: fetch [-DHILMNPRTVabdlmnpqrstv] [-o outputfile] [-S bytes]",
 		"             [-f file -h host [-c dir] | URL]");
 	exit(EX_USAGE);
 }
@@ -68,14 +68,13 @@
     int error, rv;
     unsigned long l;
 
-    init_schemes();
     fs = clean_fetch_state;
     fs.fs_verbose = 1;
     fs.fs_reportsize = 0;
     fs.fs_expectedsize = -1;
     change_to_dir = file_to_get = hostname = 0;
 
-    while ((c = getopt(argc, argv, "abc:D:f:h:HIlLmMnNo:pPqRrS:stT:vV:")) != -1) {
+    while ((c = getopt(argc, argv, "abc:dD:f:h:HIlLmMnNo:pPqRrS:stT:vV:")) != -1) {
 	    switch (c) {
 	    case 'D': case 'H': case 'I': case 'L': case 'N': case 'V': 
 		    break;	/* ncftp compatibility */
@@ -92,6 +91,10 @@
 		    change_to_dir = optarg;
 		    break;
 
+	    case 'd':
+		    fs.fs_skip_proxy = 1;
+		    break;
+
 	    case 'f':
 		    file_to_get = optarg;
 		    break;
@@ -174,6 +177,8 @@
     }
 
     clean_fetch_state = fs; /* preserve option settings */
+    if(!fs.fs_skip_proxy)
+	init_schemes();
 
     if (argv[optind] && (hostname || change_to_dir || file_to_get)) {
 	    warnx("cannot use -h, -c, or -f with a URI argument");
>Release-Note:
>Audit-Trail:

From: Nick Hibma <nick.hibma@jrc.it>
To: freebsd-gnats-submit@freebsd.org, clkao@CirX.ORG
Cc:  
Subject: Re: bin/9064: [PATCH] propose adding `direct' option in fetch(3)
Date: Tue, 13 Jul 1999 15:33:36 +0200 (MET DST)

 http://www.freebsd.org/cgi/query-pr.cgi?pr=9064
 
    > When `make fetch', we usually add a very nearby site in
    > MASTER_SITE_BACKUP, and then some of the primary distfile-mirroring
    > sites.
 
    > The problem is that, I'd like those nearby site being fetched
    > directly, rather than via proxies, while the actually far sites(like
    > ftp.freebsd.org) go through proxies.
 
 Although I can see the use of the functionality, I have the
 feeling that we are into feature creep here.
 
 This functionality belongs in the proxy library, not in fetch, 
 so it is available for all programs, IMHO.
 
 Any opinions?
 
 Nick
 -- 
 ISIS/STA, T.P.270, Joint Research Centre, 21020 Ispra, Italy
 
 
State-Changed-From-To: open->analyzed 
State-Changed-By: n_hibma 
State-Changed-When: Tue Jul 13 07:02:47 PDT 1999 
State-Changed-Why:  
Had a look at it and submitted an opinion on it. 
. 


Responsible-Changed-From-To: freebsd-bugs->n_hibma 
Responsible-Changed-By: n_hibma 
Responsible-Changed-When: Tue Jul 13 07:02:47 PDT 1999 
Responsible-Changed-Why:  
Remember to take care of it in the next few weeks. 

From: Nick Hibma <nick.hibma@jrc.it>
To: Dag-Erling Smorgrav <des@flood.ping.uio.no>
Cc: Chia-liang Kao <clkao@CirX.ORG>, freebsd-gnats-submit@freebsd.org
Subject: Re: bin/9064: [PATCH] propose adding `direct' option in fetch(3)
Date: Tue, 13 Jul 1999 17:16:53 +0200 (MET DST)

  > > > > The idea of disabling the proxy for close sites does make sense. Would  
  > > > > it be an idea to implement it in the fetch (3) library?
  > > > ...and how do you propose we determine how close a site is?
  > > As stated in the PR, by adding the -d flag to the URL.
  > 
  > Oh, well, OK then.
 
 This is what a proxy.pac for netscape looks like at our site.
 
         function FindProxyForURL(url, host)
         {
             if   ( isPlainHostName(host) ||
                    dnsDomainIs(host, ".gelos.org")  ||
                    dnsDomainIs(host, ".ioccg.org")  ||
                    dnsDomainIs(host, ".ceo.org") 
                  ) 
                         return "DIRECT";
 
            return "PROXY 139.191.7.10:80; DIRECT";
         }
 
 
 Maybe instead of using a flag to specify exactly which ones should be
 fetched directly we should use something like
 
   export HTTP_PROXY="http://mda00.jrc.it:80/"
   export HTTP_PROXY_DIRECT=".gelos.org:.ioccg.org:.ceo.org";
 
 making the list of hosts to contact directly a colon separated list of
 domain- or hostnames.
 
 Any chance you, Chia-liang Kao, could have a look at how to implement
 this in libfetch (man 3 fetch) ? I guess Dag-Erling (DES) would be
 willing to review and commit the patch. 
 
 Cheers,
 
 Nick
 
 
State-Changed-From-To: analyzed->feedback 
State-Changed-By: des 
State-Changed-When: Thu Jun 29 07:29:49 PDT 2000 
State-Changed-Why:  
The new fetch(1) in -CURRENT supports the -d option. 


Responsible-Changed-From-To: n_hibma->des 
Responsible-Changed-By: des 
Responsible-Changed-When: Thu Jun 29 07:29:49 PDT 2000 
Responsible-Changed-Why:  
fetch(1) is mine. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=9064 
State-Changed-From-To: feedback->closed 
State-Changed-By: des 
State-Changed-When: Wed Jul 19 02:22:15 PDT 2000 
State-Changed-Why:  
fetch(1) supports the -d option in -CURRENT and -STABLE. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=9064 
>Unformatted:
