From horcicka@www.freebsd.cz  Mon May 12 02:08:40 2003
Return-Path: <horcicka@www.freebsd.cz>
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id D433337B401
	for <FreeBSD-gnats-submit@freebsd.org>; Mon, 12 May 2003 02:08:40 -0700 (PDT)
Received: from www.freebsd.cz (www.freebsd.cz [195.113.19.84])
	by mx1.FreeBSD.org (Postfix) with ESMTP id E8A4143FDF
	for <FreeBSD-gnats-submit@freebsd.org>; Mon, 12 May 2003 02:08:39 -0700 (PDT)
	(envelope-from horcicka@www.freebsd.cz)
Received: from www.freebsd.cz (localhost [127.0.0.1])
	by www.freebsd.cz (8.12.6/8.12.6) with ESMTP id h4C98QZB016251
	for <FreeBSD-gnats-submit@freebsd.org>; Mon, 12 May 2003 11:08:26 +0200 (CEST)
	(envelope-from horcicka@www.freebsd.cz)
Received: (from horcicka@localhost)
	by www.freebsd.cz (8.12.6/8.12.6/Submit) id h4C986r9016250;
	Mon, 12 May 2003 11:08:06 +0200 (CEST)
Message-Id: <200305120908.h4C986r9016250@www.freebsd.cz>
Date: Mon, 12 May 2003 11:08:06 +0200 (CEST)
From: Martin Horcicka <horcicka@FreeBSD.cz>
To: FreeBSD-gnats-submit@freebsd.org
Subject: +COMMENT files in all newly made packages contain quoted spaces
X-Send-Pr-Version: 3.113
X-GNATS-Notify:

>Number:         52097
>Category:       ports
>Synopsis:       +COMMENT files in all newly made packages contain quoted spaces
>Confidential:   no
>Severity:       non-critical
>Priority:       high
>Responsible:    portmgr
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Mon May 12 02:10:08 PDT 2003
>Closed-Date:    Mon Jun 09 10:28:31 PDT 2003
>Last-Modified:  Mon Jun 09 10:28:31 PDT 2003
>Originator:     Martin Horcicka
>Release:        FreeBSD 4.7-RELEASE i386
>Organization:
>Environment:
System: FreeBSD dual.ms.mff.cuni.cz 4.7-RELEASE FreeBSD 4.7-RELEASE #0: Tue Feb 4 19:34:42 CET 2003 dan@dzungle.ms.mff.cuni.cz:/usr/src/sys/compile/DUAL i386

	
>Description:
Due to recent change of ports/Mk/bsd.port.mk, new locally made packages
contain +COMMENT files with quoted shell meta-characters (including spaces)
in their content. The problematic part of bsd.port.mk (revision 1.448) is
line:

PKG_ARGS=               -v -c "-${COMMENT:Q}" -d ${DESCR} -f ${TMPPLIST} -p ${PREFIX} -P "${MAKE} package-depends | ${GREP} -v -E ${PKG_IGNORE_DEPENDS} | sort -u" ${EXTRA_PKG_ARGS}

	
>How-To-Repeat:
1. make package
2. check +COMMENT file in new package

	
>Fix:
	
>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-ports-bugs->portmgr 
Responsible-Changed-By: petef 
Responsible-Changed-When: Thu May 15 21:55:59 CDT 2003 
Responsible-Changed-Why:  
Over to maintainer. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=52097 

From: Mario Sergio Fujikawa Ferreira <lioux@FreeBSD.org>
To: FreeBSD-gnats-submit@FreeBSD.ORG
Cc:  
Subject: Re: ports/52097 - better sanity checks on package information
Date: Mon, 9 Jun 2003 14:07:42 -0300

 --rS8CxjVDS/+yyDmU
 Content-Type: multipart/mixed; boundary="1yeeQ81UyVL57Vl7"
 Content-Disposition: inline
 
 
 --1yeeQ81UyVL57Vl7
 Content-Type: text/plain; charset=us-ascii
 Content-Disposition: inline
 Content-Transfer-Encoding: quoted-printable
 
 	This is for better documenting the solution to this problem.
 
 	I did better digging at the problem. The problem lies
 between pkg_create and pkg_info. My previous solution using a
 trailing ${ECHO_CMD} to add a newline really doesn't work for all
 cases. Moreover, we should really attack the cause of the problem
 rather than try to provide workarounds for it.
 
 	1) Problem
 
 Original report from Martin Horcicka <horcicka@FreeBSD.cz>
 
 revision 1.448 of ports/Mk/bsd.port.mk has brought an error into "make
 package" function - +COMMENT files contain all shell metacharacters (includ=
 ing
 spaces!) in quoted form - e.g. (after make package):
 
 $ tar xvzf stunnel-4.04_2.tgz +COMMENT
 +COMMENT
 $ cat +COMMENT
 A\ character\ set\ conversion\ library$
 
 This was documented in PR ports/52097.
 
 	The problem only occured with generated packages.
 Ports installed from ports worked just fine but we got escaped
 metacharacters with packages. Therefore, the problem lied in package
 creation. I tracked the problem down. The following patch fixed.
 Which was the same conclusion provided by Martin Horcicka.
 
 --- bsd.port.mk.orig	Wed May 21 20:30:19 2003
 +++ bsd.port.mk	Wed May 21 20:31:10 2003
 @@ -1609,7 +1609,7 @@
  DISABLE_CONFLICTS=3D     YES
  .endif
  .if !defined(PKG_ARGS)
 -PKG_ARGS=3D		-v -c "-${COMMENT:Q}" -d ${DESCR} -f ${TMPPLIST} -p ${PREFIX}=
  -P "`${MAKE} package-depends | ${GREP} -v -E ${PKG_IGNORE_DEPENDS} | sort =
 -u`" ${EXTRA_PKG_ARGS}
 +PKG_ARGS=3D		-v -c -${COMMENT:Q} -d ${DESCR} -f ${TMPPLIST} -p ${PREFIX} -=
 P "`${MAKE} package-depends | ${GREP} -v -E ${PKG_IGNORE_DEPENDS} | sort -u=
 `" ${EXTRA_PKG_ARGS}
  .if exists(${PKGINSTALL})
  PKG_ARGS+=3D		-i ${PKGINSTALL}
  .endif
 
 	However, this surfaced a long standing bug in pkg_install
 tools.
 
 	2) Problem Analysis
 
 	pkg_create incorrectly does not add trailing
 \n when it receives either COMMENT or DESCR from the command line.
 When you install a port, you get both +COMMENT and +DESCR files
 with trailing \n. However, +COMMENT does not contain a trailing \n
 when it is installed from a package due to this behavior of pkg_create.
 
 	pkg_info trusts that it will always receive trailing \n
 instead of rolling its own.
 
 	Therefore, when using pkg_info to gather information
 =66rom ports, we would miss \n in several instances. For instance,
 
 hugelatex-1.0       LaTeX format files with large limits
 ickle-0.3.2         ICQ2000 protocol implementationiconv-2.0_3         Char=
 set conversion library and utilities
 id3lib-3.8.2        Library for manipulating ID3v1/v1.1 and ID3v2 tags
 
 hugelatex, iconv and id3lib were installed from ports while ickle
 was installed from packages.
 
 	3) Solution
 
 	The following patches fix problems in both pkg_create
 and pkg_info. Patches are relative to src/usr.sbin/pkg_install from
 -CURRENT. They work in -STABLE as well.
 
 - pkg_create : we will add a trailing \n to COMMENT and DESCR
 provided in the command lines. Thus, mimicking the exact behavior
 as if this information was read from files.
 
 	* patches
 		+ patch-lib::lib.h
 		+ patch-lib::str.c
 
 - pkg_info : add trailing \n if none is found (fixes not only COMMENT
 but any other instance if such a fix should be necessary) when invoked
 as
 
 $ pkg_info
 
 or
 
 $ pkg_info some_package_name
 
 	* patches
 		+ patch-info::show.c
 			- patch to show_file() taken from NetBSD repo
 			- patch to show_index() taken from OpenBSD repo
 
 	4) Solution Analysis
 
 	Whilst it can be argued that pkg_create should trust
 its input and not do any form of transformation with it, there is
 no good reason why we should not add protections there to mirror
 the same behavior for command line provided information as we get
 with file provided information. This only affects COMMENT and DESCR.
 Therefore, I advocate we fix pkg_create so that /var/db/pkg files
 are exactly the same whether they are installed from ports or
 packages.
 
 	Furthermore, pkg_info should never trust its input to provide
 proper formatting when printing. It should properly add trailing
 \n if necessary. We never know what lies in the future. This special
 knowledge that files are required to have trailing \n for proper
 pkg_info functionality was not known. I learned this while researching
 this problem. This knowledge will be lost easily. We should not
 have to know such subtleties. The tools should be robust enough to
 just work if the solution is easy enough.
 
 	5) Conclusion
 
 	Fixing either pkg_create or pkg_info should be enough
 to achieve the desired results. Patching pkg_create should be
 preferred due to the cleaness of the solution. Nonetheless, NetBSD
 pursued a solution by patching pkg_info so it could be preferred
 to maintain a shared codebase.
 
 	Nevertheless, adding both solutions will add robustness to
 the pkg_install base tools. Thus, I advise we apply both of them
 with re@ approval. I will commit these fixes if both portmgr@ and
 re@ agree to it. Of course, this should be tried at bento first for
 cross verification.
 
 	Regards,
 
 --=20
 Mario S F Ferreira - DF - Brazil - "I guess this is a signature."
 FreeBSD Committer | FreeBSD-KDE Team | CS Developer
 feature, n: a documented bug | bug, n: an undocumented feature
 
 --1yeeQ81UyVL57Vl7
 Content-Type: text/plain; charset=us-ascii
 Content-Disposition: attachment; filename="patch-info::show.c"
 Content-Transfer-Encoding: quoted-printable
 
 --- info/show.c.orig	Thu Apr 17 06:55:43 2003
 +++ info/show.c	Wed May 21 21:53:00 2003
 @@ -39,12 +39,16 @@
      if (!Quiet)
  	printf("%s%s", InfoPrefix, title);
      fp =3D fopen(fname, "r");
 -    if (!fp)
 +    if (fp =3D=3D (FILE *) NULL)
  	printf("ERROR: show_file: Can't open '%s' for reading!\n", fname);
      else {
 +	int append_nl =3D 0;
  	while ((n =3D fread(line, 1, 1024, fp)) !=3D 0)
  	    fwrite(line, 1, n, stdout);
  	fclose(fp);
 +	append_nl =3D (line[n - 1] !=3D '\n');	/* Do we have a trailing \n ? */
 +	if (append_nl)
 +	   printf("\n");
      }
      printf("\n");	/* just in case */
  }
 @@ -55,20 +59,25 @@
      FILE *fp;
      char line[MAXINDEXSIZE+2];
 =20
 +    strlcpy(line, "???\n", sizeof(line));
 +
      if (!Quiet)
          printf("%s%s", InfoPrefix, title);
      fp =3D fopen(fname, "r");
 -    if (!fp) {
 +    if (fp =3D=3D (FILE *) NULL) {
          warnx("show_file: can't open '%s' for reading", fname);
 -        return;
 -    }
 -    if(fgets(line, MAXINDEXSIZE+1, fp)) {
 -	if(line[MAXINDEXSIZE-1] !=3D '\n')
 -          line[MAXINDEXSIZE] =3D '\n';
 -	line[MAXINDEXSIZE+1] =3D 0;
 -	fputs(line, stdout);
 +    } else {
 +    	if(fgets(line, MAXINDEXSIZE + 1, fp)) {
 +		size_t line_length =3D strlen(line);
 +
 +		if (line[line_length - 1] !=3D '\n') {	/* Do we have a trailing \n ? */
 +			line[line_length] =3D '\n';	/* Add a trailing \n */
 +			line[line_length + 1] =3D '\0';	/* Terminate string */
 +		}
 +	}
 +	fclose(fp);
      }
 -    fclose(fp);
 +    fputs(line, stdout);
  }
 =20
  /* Show a packing list item type.  If showall is TRUE, show all */
 
 --1yeeQ81UyVL57Vl7
 Content-Type: text/plain; charset=us-ascii
 Content-Disposition: attachment; filename="patch-lib::lib.h"
 
 --- lib/lib.h.orig	Wed May 21 20:52:40 2003
 +++ lib/lib.h	Wed May 21 20:52:51 2003
 @@ -147,6 +147,7 @@
  /* String */
  char 		*get_dash_string(char **);
  char		*copy_string(const char *);
 +char		*copy_string_plus_newline(const char *);
  Boolean		suffix(const char *, const char *);
  void		nuke_suffix(char *);
  void		str_lowercase(char *);
 
 --1yeeQ81UyVL57Vl7
 Content-Type: text/plain; charset=us-ascii
 Content-Disposition: attachment; filename="patch-lib::str.c"
 Content-Transfer-Encoding: quoted-printable
 
 --- lib/str.c.orig	Wed May 21 20:35:01 2003
 +++ lib/str.c	Wed May 21 20:59:41 2003
 @@ -42,7 +42,7 @@
      char *s =3D *str;
 =20
      if (*s =3D=3D '-')
 -	*str =3D copy_string(s + 1);
 +	*str =3D copy_string_plus_newline(s + 1);
      else
  	*str =3D fileGetContents(s);
      return *str;
 @@ -53,6 +53,27 @@
  copy_string(const char *str)
  {
      return (str ? strdup(str) : NULL);
 +}
 +
 +/* Rather Obvious but adds a trailing \n newline */
 +char *
 +copy_string_plus_newline(const char *str)
 +{
 +    if (str =3D=3D NULL) {
 +	return (NULL);
 +    } else  {
 +	char *copy;
 +	size_t line_length;
 +
 +	line_length =3D strlen(str) + 2;
 +	if ((copy =3D malloc(line_length)) =3D=3D NULL)
 +		return (NULL);
 +	memcpy(copy, str, line_length - 2);
 +	copy[line_length - 2] =3D '\n';	/* Adds trailing \n */
 +	copy[line_length - 1] =3D '\0';
 +
 +	return (copy);
 +   }
  }
 =20
  /* Return TRUE if 'str' ends in suffix 'suff' */
 
 --1yeeQ81UyVL57Vl7--
 
 --rS8CxjVDS/+yyDmU
 Content-Type: application/pgp-signature
 Content-Disposition: inline
 
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v1.2.2 (FreeBSD)
 
 iD8DBQE+5L7z99ikl0dk/7oRAjB5AKCuemYk8yXvwTeVxIBnmYiC1ZyqtQCeKvP3
 d6SOwGmIcIVK92uMf+F29qs=
 =p1eF
 -----END PGP SIGNATURE-----
 
 --rS8CxjVDS/+yyDmU--
State-Changed-From-To: open->closed 
State-Changed-By: lioux 
State-Changed-When: Mon Jun 9 10:28:29 PDT 2003 
State-Changed-Why:  
A fix has been committed, thanks! 

http://www.freebsd.org/cgi/query-pr.cgi?pr=52097 
>Unformatted:
