From rse@en1.engelschall.com  Thu Aug 17 04:31:56 2000
Return-Path: <rse@en1.engelschall.com>
Received: from slarti.muc.de (slarti.muc.de [193.149.48.10])
	by hub.freebsd.org (Postfix) with SMTP id A7DB437B634
	for <FreeBSD-gnats-submit@freebsd.org>; Thu, 17 Aug 2000 04:31:55 -0700 (PDT)
Received: (qmail 26872 invoked by uid 66); 17 Aug 2000 11:40:00 -0000
Received: from en by slarti with UUCP; Thu Aug 17 11:40:00 2000 -0000
Received: by en1.engelschall.com (Sendmail 8.11.0+)
	id e7HBVYk56997; Thu, 17 Aug 2000 13:31:34 +0200 (CEST)
Message-Id: <200008171131.e7HBVYk56997@en1.engelschall.com>
Date: Thu, 17 Aug 2000 13:31:34 +0200 (CEST)
From: "Ralf S. Engelschall" <rse@en1.engelschall.com>
Reply-To: rse@engelschall.com
To: FreeBSD-gnats-submit@freebsd.org
Subject: make SORTED_MASTER_SITES_CMD variable overridable
X-Send-Pr-Version: 3.2

>Number:         20678
>Category:       ports
>Synopsis:       make SORTED_MASTER_SITES_CMD variable overridable
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    portmgr
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          change-request
>Submitter-Id:   current-users
>Arrival-Date:   Thu Aug 17 04:40:01 PDT 2000
>Closed-Date:    Mon Feb 24 12:00:14 PST 2003
>Last-Modified:  Mon Feb 24 12:00:14 PST 2003
>Originator:     Ralf S. Engelschall
>Release:        FreeBSD 4.1-STABLE i386
>Organization:
Engelschall, Germany.
>Environment:

FreeBSD en1.engelschall.com 4.1-STABLE FreeBSD 4.1-STABLE #0: Sun Aug 13
15:33:17 CEST 2000     rse@en1.engelschall.com:/usr/obj/usr/src/sys/EN1  i386

>Description:

With our recently introduced sorting of the master sites, it is no longer
possible to extend/override the master sites on a *per-distfile basis*.
Previously MASTER_SITE_OVERRIDE could be used for this purpose by using things
like this in /etc/make.conf:

MASTER_SITE_OVERRIDE=`/path/to/script $$$${file}`

I used this since years to run /path/to/script with the currently to be
fetched distfile. If this script determined (how it does this is outside of
this scope) that the distfile already exists somewhere locally, the ports
system first tried it to fetch from there. Keep in mind that a script has to
be used here, because the files are stored not in a single directory (for me
they are stored in a very large software archive with thousands of subdirs).
So something like MASTER_SITE_OVERRIDE=/path/to/local/stuff is not sufficient.

But this no longer works with the current bsd.port.mk stuff after the
introduced master site sorting. But if one is again able to override
SORTED_MASTER_SITES_CMD in /etc/make.conf one can achieve the old effect with:

SORTED_MASTER_SITES_CMD=\
    /path/to/script $$file; \
    echo '${MASTER_SITES}' | ${AWK} '${MASTER_SORT_AWK}'

Voila! All which is required to again have such local flexibility is that we
are nice and allow the SORTED_MASTER_SITES_CMD variable to be overridden from
/etc/make.conf. The following patch does it. It is harmless but very useful as
I tried to explain.

I would highly appreciate if we could apply this patch to bsd.port.mk soon.

>How-To-Repeat:

Without editing a ports Makefile, try to force the port to first try to fetch
the distfile from a location which was determined on-the-fly by a script.

>Fix:

This patch allows one to override SORTED_MASTER_SITES_CMD and
this way one is again able to achieve the effect.

Index: bsd.port.mk
===================================================================
RCS file: /home/ncvs/ports/Mk/bsd.port.mk,v
retrieving revision 1.345
diff -u -d -r1.345 bsd.port.mk
--- bsd.port.mk	2000/08/03 09:28:57	1.345
+++ bsd.port.mk	2000/08/08 12:17:15
@@ -1115,7 +1115,7 @@
 MASTER_SORT_AWK+= /${srt:S^/^\\/^g}/ { good["${srt}"] = good["${srt}"] " " $$0 ; next; } 
 .endfor
 MASTER_SORT_AWK+= { rest = rest " " $$0; } END { n=split(gl, gla); for(i=1;i<=n;i++) { print good[gla[i]]; } print rest; }
-SORTED_MASTER_SITES_CMD= echo '${MASTER_SITES}' | ${AWK} '${MASTER_SORT_AWK}'
+SORTED_MASTER_SITES_CMD?= echo '${MASTER_SITES}' | ${AWK} '${MASTER_SORT_AWK}'
 
 DISTFILES?=		${DISTNAME}${EXTRACT_SUFX}
 ALLFILES?=	${DISTFILES} ${PATCHFILES}



>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-ports->asami  
Responsible-Changed-By: dannyboy 
Responsible-Changed-When: Sat Aug 19 10:20:35 PDT 2000 
Responsible-Changed-Why:  
Over to PW. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=20678 

From: "Ralf S. Engelschall" <rse@engelschall.com>
To: FreeBSD-gnats-submit@freebsd.org
Cc: asami@freebsd.org
Subject: Re: ports/20678: make SORTED_MASTER_SITES_CMD variable overridable
Date: Fri, 25 Aug 2000 21:57:42 +0200

 In article <vqcn1i538et.fsf@silvia.hip.berkeley.edu> you wrote:
 
 >  * With our recently introduced sorting of the master sites, it is no longer
 >  * possible to extend/override the master sites on a *per-distfile basis*.
 >  * Previously MASTER_SITE_OVERRIDE could be used for this purpose by using things
 >  * like this in /etc/make.conf:
 >  * 
 >  * MASTER_SITE_OVERRIDE=`/path/to/script $$$${file}`
 >  * 
 >  * I used this since years to run /path/to/script with the currently to be
 >  * fetched distfile. If this script determined (how it does this is outside of
 >  * this scope) that the distfile already exists somewhere locally, the ports
 >  * system first tried it to fetch from there. Keep in mind that a script has to
 >  * be used here, because the files are stored not in a single directory (for me
 >  * they are stored in a very large software archive with thousands of subdirs).
 >  * So something like MASTER_SITE_OVERRIDE=/path/to/local/stuff is not sufficient.
 > 
 > Hmm.  Actually, the current implementation of SORTED_MASTER_SITES is
 > broken in that it completely ignores the meaning of
 > MASTER_SITE_OVERRIDE, putting it into the list to be sorted.
 > 
 > Will something like the below work?  I wasn't sure how to do a double
 > command substitution ($() inside $()'s) so there are new targets,
 > master-sites and patch-sites, which print out the desired list of
 > master and patch sites.  I also made the patch sites sortable.
 > [...]
 
 Sorry for the delay. I've seen that you've now already committed your new
 stuff.  The whole master site sorting works fine with your patch, of course.
 Even the use of MASTER_SITE_OVERRIDE works now again as expected. 
 
 But unfortunately the old functionality of *per distfile* overrides is still
 not possible, because the $file variable is no longer reachable - caused by
 the additional make targets you introduced. So in order to close the PR#20678
 we still have to patch a little bit more ;)
 
 Whatever we do the goal is that a command in a /etc/make.conf's
 MASTER_SITE_OVERRIDE sees the $file in the distfile fetching targets.  I've
 two possible patches for you: 
 
 1. Either we pass the stuff via DISTFILE=$${file} to the master-site
    and patch-site targets and this way even would allow ``make master-sites
    DISTFILE=foo-1.2.3.tar.gz'' to check an oen MASTER_SITE_OVERRIDE script:
 
 Index: bsd.port.mk
 ===================================================================
 RCS file: /home/ncvs/ports/Mk/bsd.port.mk,v
 retrieving revision 1.347
 diff -u -d -r1.347 bsd.port.mk
 --- bsd.port.mk	2000/08/25 10:17:39	1.347
 +++ bsd.port.mk	2000/08/25 19:49:05
 @@ -1120,8 +1120,8 @@
  MASTER_SORT_AWK+= /${srt:S^/^\\/^g}/ { good["${srt}"] = good["${srt}"] " " $$0 ; next; } 
  .endfor
  MASTER_SORT_AWK+= { rest = rest " " $$0; } END { n=split(gl, gla); for(i=1;i<=n;i++) { print good[gla[i]]; } print rest; }
 -SORTED_MASTER_SITES_CMD=	cd ${.CURDIR} && ${MAKE} ${__softMAKEFLAGS} master-sites
 -SORTED_PATCH_SITES_CMD=		cd ${.CURDIR} && ${MAKE} ${__softMAKEFLAGS} patch-sites
 +SORTED_MASTER_SITES_CMD=	cd ${.CURDIR} && ${MAKE} ${__softMAKEFLAGS} master-sites DISTFILE=$${file}
 +SORTED_PATCH_SITES_CMD=		cd ${.CURDIR} && ${MAKE} ${__softMAKEFLAGS} patch-sites  DISTFILE=$${file}
  
  master-sites:
  	@echo ${_MASTER_SITE_OVERRIDE} `echo '${MASTER_SITES}' | ${AWK} '${MASTER_SORT_AWK}'` ${_MASTER_SITE_BACKUP}
 
 2. Or we at least allow a /etc/make.conf to override the SORTED_XXX_SITES_CMD
    (with a value which then includes ...DISTFILE=$${file}...):
 
 Index: bsd.port.mk
 ===================================================================
 RCS file: /home/ncvs/ports/Mk/bsd.port.mk,v
 retrieving revision 1.347
 diff -u -d -r1.347 bsd.port.mk
 --- bsd.port.mk	2000/08/25 10:17:39	1.347
 +++ bsd.port.mk	2000/08/25 19:50:12
 @@ -1120,8 +1120,8 @@
  MASTER_SORT_AWK+= /${srt:S^/^\\/^g}/ { good["${srt}"] = good["${srt}"] " " $$0 ; next; } 
  .endfor
  MASTER_SORT_AWK+= { rest = rest " " $$0; } END { n=split(gl, gla); for(i=1;i<=n;i++) { print good[gla[i]]; } print rest; }
 -SORTED_MASTER_SITES_CMD=	cd ${.CURDIR} && ${MAKE} ${__softMAKEFLAGS} master-sites
 -SORTED_PATCH_SITES_CMD=		cd ${.CURDIR} && ${MAKE} ${__softMAKEFLAGS} patch-sites
 +SORTED_MASTER_SITES_CMD?=	cd ${.CURDIR} && ${MAKE} ${__softMAKEFLAGS} master-sites
 +SORTED_PATCH_SITES_CMD?=		cd ${.CURDIR} && ${MAKE} ${__softMAKEFLAGS} patch-sites
  
  master-sites:
  	@echo ${_MASTER_SITE_OVERRIDE} `echo '${MASTER_SITES}' | ${AWK} '${MASTER_SORT_AWK}'` ${_MASTER_SITE_BACKUP}
 
 I personally would appreciate patch 1, but I can life with patch 2, too.
 Important is just that a script in MASTER_SITE_OVERRIDE can see the particular
 distfile which should be fetched. 
 
 Thanks for your effort, Satoshi.
 
 Yours,
                                        Ralf S. Engelschall
                                        rse@engelschall.com
                                        www.engelschall.com
 
Responsible-Changed-From-To: asami->portmgr 
Responsible-Changed-By: will 
Responsible-Changed-When: Wed Apr 4 01:14:53 PDT 2001 
Responsible-Changed-Why:  
Over to new maintainer. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=20678 

From: Kris Kennaway <kris@obsecurity.org>
To: freebsd-gnats-submit@FreeBSD.org, rse@engelschall.com
Cc:  
Subject: Re: ports/20678
Date: Sun, 23 Feb 2003 14:56:03 -0800

 --f2QGlHpHGjS2mn6Y
 Content-Type: text/plain; charset=us-ascii
 Content-Disposition: inline
 
 Is this still an issue?
 
 Kris
 
 --f2QGlHpHGjS2mn6Y
 Content-Type: application/pgp-signature
 Content-Disposition: inline
 
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v1.2.1 (FreeBSD)
 
 iD8DBQE+WVGDWry0BWjoQKURAuVTAKCK46upzps5XzkhnrB3mKfGJ67LgQCffGVo
 ORGCMp0DRaB6VHisT7fxz9U=
 =QfS1
 -----END PGP SIGNATURE-----
 
 --f2QGlHpHGjS2mn6Y--

From: "Ralf S. Engelschall" <rse@engelschall.com>
To: freebsd-gnats-submit@FreeBSD.org
Cc:  
Subject: Re: ports/20678
Date: Mon, 24 Feb 2003 15:33:51 +0100

 On Sun, Feb 23, 2003, Kris Kennaway wrote:
 
 > Is this still an issue?
 
 No, the PR can be closed. Thanks.
 
                                        Ralf S. Engelschall
                                        rse@engelschall.com
                                        www.engelschall.com
 
State-Changed-From-To: open->closed 
State-Changed-By: kris 
State-Changed-When: Mon Feb 24 12:00:01 PST 2003 
State-Changed-Why:  
Submitter confirms patch is no longer necessary 

http://www.freebsd.org/cgi/query-pr.cgi?pr=20678 
>Unformatted:
