From eikemeier@fillmore-labs.com  Mon Oct  6 10:53:18 2003
Return-Path: <eikemeier@fillmore-labs.com>
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id 631CE16A4B3
	for <FreeBSD-gnats-submit@FreeBSD.org>; Mon,  6 Oct 2003 10:53:18 -0700 (PDT)
Received: from mx2.fillmore-labs.com (lima.fillmore-labs.com [62.138.193.83])
	by mx1.FreeBSD.org (Postfix) with ESMTP id 8862643FEA
	for <FreeBSD-gnats-submit@FreeBSD.org>; Mon,  6 Oct 2003 10:53:17 -0700 (PDT)
	(envelope-from eikemeier@fillmore-labs.com)
Received: from pd958a39d.dip.t-dialin.net
	([217.88.163.157] helo=fillmore-labs.com ident=srvzpyipy6mlgh3j)
	by mx2.fillmore-labs.com with asmtp (TLSv1:AES256-SHA:256)
	(Exim 4.24; FreeBSD 4.9)
	id 1A6ZXn-0003kN-V0
	for FreeBSD-gnats-submit@FreeBSD.org; Mon, 06 Oct 2003 19:53:16 +0200
Message-Id: <3F81AC08.2030004@fillmore-labs.com>
Date: Mon, 06 Oct 2003 19:53:12 +0200
From: Oliver Eikemeier <eikemeier@fillmore-labs.com>
To: FreeBSD-gnats-submit@FreeBSD.org
Subject: [PATCH] bsd.port.mk: LIB_DEPENDS should not use libraries in ${LOCALBASE}/lib/compat/pkg

>Number:         57665
>Category:       ports
>Synopsis:       [PATCH] bsd.port.mk: LIB_DEPENDS should not use libraries in ${LOCALBASE}/lib/compat/pkg
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    portmgr
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Mon Oct 06 11:00:25 PDT 2003
>Closed-Date:    Fri Nov 07 13:03:34 PST 2003
>Last-Modified:  Fri Nov 07 13:03:34 PST 2003
>Originator:     Oliver Eikemeier
>Release:        FreeBSD 5.1-CURRENT i386
>Organization:
Fillmore Labs - http://www.fillmore-labs.com
>Environment:
System: FreeBSD nuuk.fillmore-labs.com 5.1-CURRENT

>Description:

When using port sysutils/portupgrade to manage port installation, it is possible
for stale libraries to end up in ${LOCALBASE}/lib/compat/pkg. Even though
portupgrade does its best to clean them up, they can persist when the library
changes its version number (libintl.so.4 => libintl.so.5), or the upgrade process
is interrupted by ctrl-C.

ldconfig -r (and bsd.port.mk) finds these libraries, assuming the whole port
needed for LIB_DEPENDS (including headers) is installed, which it isn't.

This patch aborts building if a needed library is found only in
${LOCALBASE}/lib/compat/pkg.

The initial bug report came from Thomas Fritz <tf@slash10.com>, thanks for the
help in tracking it down.

>How-To-Repeat:

# portinstall net/openldap21-client
[...]

# pkg_deinstall -P 'openldap-*'
[...]
--->  Preserving /usr/local/lib/libldap.so.2 as /usr/local/lib/compat/pkg/libldap.so.2
[...]

# portinstall net/openldap21-server
[...]
===>   openldap-server-2.1.22_2 depends on shared library: ldap.2 - found
[...]

# pkg_glob 'openldap-*'
Stale dependency: openldap-server-2.1.22_2 --> openldap-client-2.1.22 -- manually run 'pkgdb -F' to fix, or specify -O to force.

The usage of `pkg_deinstall -P' may seem artificial, but this is exactly the
command used by portupgrade if used without the `-u' option.

>Fix:

--- bsd.port.mk.patch begins here ---
--- bsd.port.mk.orig	3 Sep 2003 19:50:24 -0000
+++ bsd.port.mk	6 Oct 2003 16:51:09 -0000
@@ -1009,6 +1009,8 @@
 PREFIX?=		${LOCALBASE}
 .endif
 
+PKGCOMPATDIR?=		${LOCALBASE}/lib/compat/pkg
+
 .if !defined(PERL_LEVEL) && defined(PERL_VERSION)
 perl_major=		${PERL_VERSION:C|^([1-9]+).*|\1|}
 _perl_minor=	00${PERL_VERSION:C|^([1-9]+)\.([0-9]+).*|\2|}
@@ -3894,9 +3896,9 @@
 	@for i in ${LIB_DEPENDS}; do \
 		lib=$${i%%:*}; \
 		case $$lib in \
-			*.*.*)	pattern=$$lib ;;\
+			*.*.*)	pattern="`${ECHO_CMD} $$lib | ${SED} -e 's/\./\\./g'`" ;;\
 			*.*)	pattern="$${lib%%.*}\.$${lib#*.}" ;;\
-			*)		pattern=$$lib ;;\
+			*)		pattern="$$lib" ;;\
 		esac; \
 		dir=$${i#*:}; \
 		target=$${i##*:}; \
@@ -3941,6 +3943,11 @@
 					${FALSE}; \
 				fi; \
 			fi; \
+		fi; \
+		if ${LDCONFIG} -r | ${GREP} -wE -e "-l$$pattern" | ${HEAD} -n 1 | grep -qwF -e "${PKGCOMPATDIR}"; then \
+			${ECHO_MSG} "Error: stale library \"$$lib\" found in ${PKGCOMPATDIR}."; \
+			${ECHO_MSG} "Please clean out ${PKGCOMPATDIR} manually."; \
+			${FALSE}; \
 		fi; \
 	done
 .endif
--- bsd.port.mk.patch ends here ---

>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-ports-bugs->portmgr  
Responsible-Changed-By: krion 
Responsible-Changed-When: Mon Oct 6 11:04:55 PDT 2003 
Responsible-Changed-Why:  
Portmgr territory 

http://www.freebsd.org/cgi/query-pr.cgi?pr=57665 

From: Joe Marcus Clarke <marcus@marcuscom.com>
To: freebsd-gnats-submit@FreeBSD.org, eikemeier@fillmore-labs.com
Cc:  
Subject: Re: ports/57665: [PATCH] bsd.port.mk: LIB_DEPENDS should not use
	libraries in ${LOCALBASE}/lib/compat/pkg
Date: Mon, 06 Oct 2003 14:10:06 -0400

 --=-8kF0sWuGesBMHmgVMPZA
 Content-Type: text/plain
 Content-Transfer-Encoding: quoted-printable
 
 This looks like good stuff, Oliver.  However, do we need to make it
 fatal when we find a compat lib?  Why can't we just ignore it, and
 install the dependent port anyway?
 
 Joe
 
 --=20
 PGP Key : http://www.marcuscom.com/pgp.asc
 
 
 
 --=-8kF0sWuGesBMHmgVMPZA
 Content-Type: application/pgp-signature; name=signature.asc
 Content-Description: This is a digitally signed message part
 
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v1.2.3 (FreeBSD)
 
 iD8DBQA/ga/9b2iPiv4Uz4cRAp97AJwKXTNBDFmVzLnOR5kQi0qRS9BpcgCgmFNB
 fKUgqLSDIDCIKDVqSNQ4m7M=
 =IuB2
 -----END PGP SIGNATURE-----
 
 --=-8kF0sWuGesBMHmgVMPZA--
 

From: Oliver Eikemeier <eikemeier@fillmore-labs.com>
To: Joe Marcus Clarke <marcus@marcuscom.com>
Cc: freebsd-gnats-submit@FreeBSD.org
Subject: Re: ports/57665: [PATCH] bsd.port.mk: LIB_DEPENDS should not use
 libraries in ${LOCALBASE}/lib/compat/pkg
Date: Mon, 06 Oct 2003 20:35:01 +0200

 Joe Marcus Clarke wrote:
 
 > This looks like good stuff, Oliver.  However, do we need to make it
 > fatal when we find a compat lib?  Why can't we just ignore it, and
 > install the dependent port anyway?
 
 I leave this to portmgr@ to decide, my reasoning is:
 
 - I feel uneasy if things are wrong, yet work 'somehow'...
 
 - Most compilations will fail anyway, because the headers are missing
   (openldap works, because it compiles with included headers, not
    installed ones)
 
 - Maybe the compiling does not fail, but the configure script does
   not detect the library (missing headers), so the port is silently
   compiled WITHOUT_whatever
 
 - The build process records a bogus dependency (pkg_glob -> "Stale dependency")
 
 - It is far more difficult to track errors later than to let the
   build fail.
 
 But if you think it is better to continue the build and just
 issue a warning: Go ahead. Maybe it is a good idea to tell the user
 to install the port manually?
 

From: Joe Marcus Clarke <marcus@marcuscom.com>
To: Oliver Eikemeier <eikemeier@fillmore-labs.com>
Cc: freebsd-gnats-submit@FreeBSD.org
Subject: Re: ports/57665: [PATCH] bsd.port.mk: LIB_DEPENDS should not use
	libraries in ${LOCALBASE}/lib/compat/pkg
Date: Mon, 06 Oct 2003 14:56:44 -0400

 --=-1XUDRwvdHZLX3bpAVz7X
 Content-Type: text/plain
 Content-Transfer-Encoding: quoted-printable
 
 On Mon, 2003-10-06 at 14:35, Oliver Eikemeier wrote:
 > Joe Marcus Clarke wrote:
 >=20
 > > This looks like good stuff, Oliver.  However, do we need to make it
 > > fatal when we find a compat lib?  Why can't we just ignore it, and
 > > install the dependent port anyway?
 >=20
 > I leave this to portmgr@ to decide, my reasoning is:
 >=20
 > - I feel uneasy if things are wrong, yet work 'somehow'...
 >=20
 > - Most compilations will fail anyway, because the headers are missing
 >   (openldap works, because it compiles with included headers, not
 >    installed ones)
 
 I'm not saying install the current port, but rather pretend the library
 isn't on the system at all, then do the dependency install like normal.=20
 For example, in the OpenLDAP case, libldap.so.2 is found in
 PKGCOMPATDIR, but ignored, so openldap-client is installed anyway.
 
 >=20
 > - Maybe the compiling does not fail, but the configure script does
 >   not detect the library (missing headers), so the port is silently
 >   compiled WITHOUT_whatever
 
 See above.
 
 >=20
 > - The build process records a bogus dependency (pkg_glob -> "Stale depend=
 ency")
 >=20
 > - It is far more difficult to track errors later than to let the
 >   build fail.
 
 I think we're talking about two different things.  Does my paragraph
 above clarify things for you?  Maybe I'm confused.
 
 Joe
 
 >=20
 > But if you think it is better to continue the build and just
 > issue a warning: Go ahead. Maybe it is a good idea to tell the user
 > to install the port manually?
 --=20
 PGP Key : http://www.marcuscom.com/pgp.asc
 
 
 
 --=-1XUDRwvdHZLX3bpAVz7X
 Content-Type: application/pgp-signature; name=signature.asc
 Content-Description: This is a digitally signed message part
 
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v1.2.3 (FreeBSD)
 
 iD8DBQA/gbrsb2iPiv4Uz4cRAk6dAJ9tmR/GAsRl/hy0M/8245XbRmOpXACfRoPV
 47UvzDWkeWB12/vg3cmGqlc=
 =lrSz
 -----END PGP SIGNATURE-----
 
 --=-1XUDRwvdHZLX3bpAVz7X--
 

From: Oliver Eikemeier <eikemeier@fillmore-labs.com>
To: Joe Marcus Clarke <marcus@marcuscom.com>
Cc: freebsd-gnats-submit@FreeBSD.org
Subject: Re: ports/57665: [PATCH] bsd.port.mk: LIB_DEPENDS should not use
 libraries in ${LOCALBASE}/lib/compat/pkg
Date: Mon, 06 Oct 2003 21:30:47 +0200

 Joe Marcus Clarke wrote:
 
 > On Mon, 2003-10-06 at 14:35, Oliver Eikemeier wrote:
 > 
 >>Joe Marcus Clarke wrote:
 >>
 >>>[...]  However, do we need to make it
 >>>fatal when we find a compat lib?  Why can't we just ignore it, and
 >>>install the dependent port anyway?
 >>
 >>- Most compilations will fail anyway, because the headers are missing
 >>  (openldap works, because it compiles with included headers, not
 >>   installed ones)
 > 
 > I'm not saying install the current port, but rather pretend the library
 > isn't on the system at all, then do the dependency install like normal. 
 > For example, in the OpenLDAP case, libldap.so.2 is found in
 > PKGCOMPATDIR, but ignored, so openldap-client is installed anyway.
 
 Ah, now I get it. Something like:
 
 --- bsd.port.mk.patch begins here ---
 --- bsd.port.mk.orig	3 Sep 2003 19:50:24 -0000
 +++ bsd.port.mk	6 Oct 2003 19:26:20 -0000
 @@ -1009,6 +1009,8 @@
  PREFIX?=		${LOCALBASE}
  .endif
  
 +PKGCOMPATDIR?=		${LOCALBASE}/lib/compat/pkg
 +
  .if !defined(PERL_LEVEL) && defined(PERL_VERSION)
  perl_major=		${PERL_VERSION:C|^([1-9]+).*|\1|}
  _perl_minor=	00${PERL_VERSION:C|^([1-9]+)\.([0-9]+).*|\2|}
 @@ -3894,9 +3896,9 @@
  	@for i in ${LIB_DEPENDS}; do \
  		lib=$${i%%:*}; \
  		case $$lib in \
 -			*.*.*)	pattern=$$lib ;;\
 +			*.*.*)	pattern="`${ECHO_CMD} $$lib | ${SED} -e 's/\./\\./g'`" ;;\
  			*.*)	pattern="$${lib%%.*}\.$${lib#*.}" ;;\
 -			*)		pattern=$$lib ;;\
 +			*)		pattern="$$lib" ;;\
  		esac; \
  		dir=$${i#*:}; \
  		target=$${i##*:}; \
 @@ -3907,7 +3909,7 @@
  			dir=$${dir%%:*}; \
  		fi; \
  		${ECHO_MSG} -n "===>   ${PKGNAME} depends on shared library: $$lib"; \
 -		if ${LDCONFIG} -r | ${GREP} -qwE -e "-l$$pattern"; then \
 +		if ${LDCONFIG} -r | ${GREP} -vwF -e "${PKGCOMPATDIR}" | ${GREP} -qwE -e "-l$$pattern"; then \
  			${ECHO_MSG} " - found"; \
  			if [ ${_DEPEND_ALWAYS} = 1 ]; then \
  				${ECHO_MSG} "       (but building it anyway)"; \
 @@ -3936,11 +3938,16 @@
  					(cd $$dir; ${MAKE} -DINSTALLS_DEPENDS $$target $$depends_args) ; \
  				fi ; \
  				${ECHO_MSG} "===>   Returning to build of ${PKGNAME}"; \
 -				if ! ${LDCONFIG} -r | ${GREP} -qwE -e "-l$$pattern"; then \
 +				if ! ${LDCONFIG} -r | ${GREP} -vwF -e "${PKGCOMPATDIR}" | ${GREP} -qwE -e "-l$$pattern"; then \
  					${ECHO_MSG} "Error: shared library \"$$lib\" does not exist"; \
  					${FALSE}; \
  				fi; \
  			fi; \
 +		fi; \
 +		if ${LDCONFIG} -r | ${GREP} -wE -e "-l$$pattern" | ${HEAD} -n 1 | grep -qwF -e "${PKGCOMPATDIR}"; then \
 +			${ECHO_MSG} "Error: stale library \"$$lib\" found in ${PKGCOMPATDIR}."; \
 +			${ECHO_MSG} "Please clean out ${PKGCOMPATDIR} manually or use \"portsclean -iL\"."; \
 +			${FALSE}; \
  		fi; \
  	done
  .endif
 --- bsd.port.mk.patch ends here ---
 
 Which fails only if the library from the port is shadowed by the one in PKGCOMPATDIR?
 
 

From: Joe Marcus Clarke <marcus@marcuscom.com>
To: Oliver Eikemeier <eikemeier@fillmore-labs.com>
Cc: freebsd-gnats-submit@FreeBSD.org
Subject: Re: ports/57665: [PATCH] bsd.port.mk: LIB_DEPENDS should not use
	libraries in ${LOCALBASE}/lib/compat/pkg
Date: Mon, 06 Oct 2003 15:38:27 -0400

 --=-uvw86WJyHvDC31Q8i7R5
 Content-Type: text/plain
 Content-Transfer-Encoding: quoted-printable
 
 On Mon, 2003-10-06 at 15:30, Oliver Eikemeier wrote:
 > Joe Marcus Clarke wrote:
 >=20
 > > On Mon, 2003-10-06 at 14:35, Oliver Eikemeier wrote:
 > >=20
 > >>Joe Marcus Clarke wrote:
 > >>
 > >>>[...]  However, do we need to make it
 > >>>fatal when we find a compat lib?  Why can't we just ignore it, and
 > >>>install the dependent port anyway?
 > >>
 > >>- Most compilations will fail anyway, because the headers are missing
 > >>  (openldap works, because it compiles with included headers, not
 > >>   installed ones)
 > >=20
 > > I'm not saying install the current port, but rather pretend the library
 > > isn't on the system at all, then do the dependency install like normal.=
 =20
 > > For example, in the OpenLDAP case, libldap.so.2 is found in
 > > PKGCOMPATDIR, but ignored, so openldap-client is installed anyway.
 >=20
 > Ah, now I get it. Something like:
 
 Exactly!  Except I don't think you need the last block as it [hopefully]
 should never be reached.  I guess you're just being cautious?
 
 Joe
 
 >=20
 > --- bsd.port.mk.patch begins here ---
 > --- bsd.port.mk.orig	3 Sep 2003 19:50:24 -0000
 > +++ bsd.port.mk	6 Oct 2003 19:26:20 -0000
 > @@ -1009,6 +1009,8 @@
 >  PREFIX?=3D		${LOCALBASE}
 >  .endif
 > =20
 > +PKGCOMPATDIR?=3D		${LOCALBASE}/lib/compat/pkg
 > +
 >  .if !defined(PERL_LEVEL) && defined(PERL_VERSION)
 >  perl_major=3D		${PERL_VERSION:C|^([1-9]+).*|\1|}
 >  _perl_minor=3D	00${PERL_VERSION:C|^([1-9]+)\.([0-9]+).*|\2|}
 > @@ -3894,9 +3896,9 @@
 >  	@for i in ${LIB_DEPENDS}; do \
 >  		lib=3D$${i%%:*}; \
 >  		case $$lib in \
 > -			*.*.*)	pattern=3D$$lib ;;\
 > +			*.*.*)	pattern=3D"`${ECHO_CMD} $$lib | ${SED} -e 's/\./\\./g'`" ;;\
 >  			*.*)	pattern=3D"$${lib%%.*}\.$${lib#*.}" ;;\
 > -			*)		pattern=3D$$lib ;;\
 > +			*)		pattern=3D"$$lib" ;;\
 >  		esac; \
 >  		dir=3D$${i#*:}; \
 >  		target=3D$${i##*:}; \
 > @@ -3907,7 +3909,7 @@
 >  			dir=3D$${dir%%:*}; \
 >  		fi; \
 >  		${ECHO_MSG} -n "=3D=3D=3D>   ${PKGNAME} depends on shared library: $$l=
 ib"; \
 > -		if ${LDCONFIG} -r | ${GREP} -qwE -e "-l$$pattern"; then \
 > +		if ${LDCONFIG} -r | ${GREP} -vwF -e "${PKGCOMPATDIR}" | ${GREP} -qwE -=
 e "-l$$pattern"; then \
 >  			${ECHO_MSG} " - found"; \
 >  			if [ ${_DEPEND_ALWAYS} =3D 1 ]; then \
 >  				${ECHO_MSG} "       (but building it anyway)"; \
 > @@ -3936,11 +3938,16 @@
 >  					(cd $$dir; ${MAKE} -DINSTALLS_DEPENDS $$target $$depends_args) ; \
 >  				fi ; \
 >  				${ECHO_MSG} "=3D=3D=3D>   Returning to build of ${PKGNAME}"; \
 > -				if ! ${LDCONFIG} -r | ${GREP} -qwE -e "-l$$pattern"; then \
 > +				if ! ${LDCONFIG} -r | ${GREP} -vwF -e "${PKGCOMPATDIR}" | ${GREP} -q=
 wE -e "-l$$pattern"; then \
 >  					${ECHO_MSG} "Error: shared library \"$$lib\" does not exist"; \
 >  					${FALSE}; \
 >  				fi; \
 >  			fi; \
 > +		fi; \
 > +		if ${LDCONFIG} -r | ${GREP} -wE -e "-l$$pattern" | ${HEAD} -n 1 | grep=
  -qwF -e "${PKGCOMPATDIR}"; then \
 > +			${ECHO_MSG} "Error: stale library \"$$lib\" found in ${PKGCOMPATDIR}.=
 "; \
 > +			${ECHO_MSG} "Please clean out ${PKGCOMPATDIR} manually or use \"ports=
 clean -iL\"."; \
 > +			${FALSE}; \
 >  		fi; \
 >  	done
 >  .endif
 > --- bsd.port.mk.patch ends here ---
 >=20
 > Which fails only if the library from the port is shadowed by the one in P=
 KGCOMPATDIR?
 >=20
 --=20
 PGP Key : http://www.marcuscom.com/pgp.asc
 
 
 
 --=-uvw86WJyHvDC31Q8i7R5
 Content-Type: application/pgp-signature; name=signature.asc
 Content-Description: This is a digitally signed message part
 
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v1.2.3 (FreeBSD)
 
 iD8DBQA/gcSzb2iPiv4Uz4cRAq9aAJsEXvIeXsDRhq/ucLQlPYzjWYci+gCfTYDc
 eDfUSBySWiO9/g5i6znt77I=
 =R7hc
 -----END PGP SIGNATURE-----
 
 --=-uvw86WJyHvDC31Q8i7R5--
 

From: Oliver Eikemeier <eikemeier@fillmore-labs.com>
To: Joe Marcus Clarke <marcus@marcuscom.com>
Cc: freebsd-gnats-submit@FreeBSD.org
Subject: Re: ports/57665: [PATCH] bsd.port.mk: LIB_DEPENDS should not use
 libraries in ${LOCALBASE}/lib/compat/pkg
Date: Mon, 06 Oct 2003 22:10:42 +0200

 Joe Marcus Clarke wrote:
 
 > [...]
 > 
 > Exactly!  Except I don't think you need the last block as it [hopefully]
 > should never be reached.  I guess you're just being cautious?
 
 Yepp. Delete it if you like.
 

From: Oliver Eikemeier <eikemeier@fillmore-labs.com>
To: freebsd-gnats-submit@FreeBSD.org
Cc:  
Subject: Re: ports/57665: [PATCH] bsd.port.mk: LIB_DEPENDS should not use
 libraries in ${LOCALBASE}/lib/compat/pkg
Date: Wed, 08 Oct 2003 16:14:29 +0200

 -			*.*.*)	pattern=$$lib ;;\
 +			*.*.*)	pattern="`${ECHO_CMD} $$lib | ${SED} -e 's/\./\\./g'`" ;;\
 
 should be
 
 -			*.*.*)	pattern=$$lib ;;\
 +			*.*.*)	pattern="`${ECHO_CMD} $$lib | ${SED} -e 's/\./\\\\./g'`" ;;\
 
 otherwise it is a noop.
 
 
State-Changed-From-To: open->analyzed 
State-Changed-By: marcus 
State-Changed-When: Sat Nov 1 12:31:32 PST 2003 
State-Changed-Why:  
This is currently being tested on bento. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=57665 
State-Changed-From-To: analyzed->closed 
State-Changed-By: marcus 
State-Changed-When: Fri Nov 7 13:03:23 PST 2003 
State-Changed-Why:  
Committed, thanks! 

http://www.freebsd.org/cgi/query-pr.cgi?pr=57665 
>Unformatted:
