From nobody@FreeBSD.org  Sat Sep 14 19:30:09 2013
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115])
	(using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
	(No client certificate requested)
	by hub.freebsd.org (Postfix) with ESMTP id D1955679
	for <freebsd-gnats-submit@FreeBSD.org>; Sat, 14 Sep 2013 19:30:09 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from oldred.freebsd.org (oldred.freebsd.org [8.8.178.121])
	(using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
	(No client certificate requested)
	by mx1.freebsd.org (Postfix) with ESMTPS id BEA1D2647
	for <freebsd-gnats-submit@FreeBSD.org>; Sat, 14 Sep 2013 19:30:09 +0000 (UTC)
Received: from oldred.freebsd.org ([127.0.1.6])
	by oldred.freebsd.org (8.14.5/8.14.7) with ESMTP id r8EJU9Mr087897
	for <freebsd-gnats-submit@FreeBSD.org>; Sat, 14 Sep 2013 19:30:09 GMT
	(envelope-from nobody@oldred.freebsd.org)
Received: (from nobody@localhost)
	by oldred.freebsd.org (8.14.5/8.14.5/Submit) id r8EJU9VS087871;
	Sat, 14 Sep 2013 19:30:09 GMT
	(envelope-from nobody)
Message-Id: <201309141930.r8EJU9VS087871@oldred.freebsd.org>
Date: Sat, 14 Sep 2013 19:30:09 GMT
From: Derek Schrock <dereks@lifeofadishwasher.com>
To: freebsd-gnats-submit@FreeBSD.org
Subject: [patch] Change kldxref fts_open ordering so it produces a consistent linker.hints between machines of the same architecture.
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         182098
>Category:       bin
>Synopsis:       [patch] Change kldxref fts_open ordering so it produces a consistent linker.hints between machines of the same architecture.
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    jilles
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          change-request
>Submitter-Id:   current-users
>Arrival-Date:   Sat Sep 14 19:40:00 UTC 2013
>Closed-Date:    Sun Jan 12 20:58:24 UTC 2014
>Last-Modified:  Sun Jan 12 20:58:24 UTC 2014
>Originator:     Derek Schrock
>Release:        9.1-RELEASE-p7
>Organization:
>Environment:
FreeBSD ircbsd 9.1-RELEASE-p7 FreeBSD 9.1-RELEASE-p7 #0: Mon Sep  9 21:34:37 UTC 2013     root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC  amd64
>Description:
Currently there an issue with linker.hints being different between machines that create binary update for freebsd-update.

http://www.freebsd.org/cgi/query-pr.cgi?pr=amd64/178247

Use freebsd-update to update:

$ freebsd-update fetch
Looking up update.FreeBSD.org mirrors... 5 mirrors found.
Fetching metadata signature for 9.1-RELEASE from update2.freebsd.org... done.
Fetching metadata index... done.
Inspecting system... done.
Preparing to download files... done.

The following files will be updated as part of updating to 9.1-RELEASE-p7:
/boot/kernel/linker.hints

$ freebsd-update install
Installing updates... done.

$ freebsd-update fetch
Looking up update.FreeBSD.org mirrors... 5 mirrors found.
Fetching metadata signature for 9.1-RELEASE from update5.freebsd.org... done.
Fetching metadata index... done.
Inspecting system... done.
Preparing to download files... done.

The following files will be updated as part of updating to 9.1-RELEASE-p7:
/boot/kernel/linker.hints


Looking at /boot/kernel on two different systems:

Host A /boot/kernel/:
$ sha256 /boot/kernel/* | sha256
6193fe0ea115ae745f8b99273906892d5a8714d7fd55f506f24064ef0396f1eb

Host B /boot/kernel:
$ sha256 /boot/kernel/* | sha256
6193fe0ea115ae745f8b99273906892d5a8714d7fd55f506f24064ef0396f1eb

'kldxref /boot/kernel/' will produce different linker.hints due to the way the fts_open() traverses the file system:

Host A:
$ kldxref /boot/kernel/
$ sha256 /boot/kernel/linker.hints 
SHA256 (/boot/kernel/linker.hints) = 3deeb3eb746b40b009f67238b924c32d1a6d0d586f127aded45bef0010345290

Host B:
$ kldxref /boot/kernel/
$ sha256 /boot/kernel/linker.hints
SHA256 (/boot/kernel/linker.hints) = ec6c0c80e745371f83faa100f1fe1a36062903c527d20dce7e0161ce26c82e54

If you change fts_open() to order the files in /boot/kernel/ the same linker.hints should be created between systems:

Host A:
$ ./patched.kldxref /boot/kernel/
$ sha256 /boot/kernel/linker.hints
SHA256 (/boot/kernel/linker.hints) = 2bda1a7568108dfb511d5156efce2d4df173dc8e91124e0c9300a5fef90218c8

$ sha256 /boot/kernel/linker.hints
SHA256 (/boot/kernel/linker.hints) = 2bda1a7568108dfb511d5156efce2d4df173dc8e91124e0c9300a5fef90218c8


>How-To-Repeat:
Find a system the produces a different linker.hints than another system's.

Host A:
$ kldxref /boot/kernel/
$ sha256 /boot/kernel/linker.hints 
SHA256 (/boot/kernel/linker.hints) = 3deeb3eb746b40b009f67238b924c32d1a6d0d586f127aded45bef0010345290

Host B:
$ kldxref /boot/kernel/
$ sha256 /boot/kernel/linker.hints
SHA256 (/boot/kernel/linker.hints) = ec6c0c80e745371f83faa100f1fe1a36062903c527d20dce7e0161ce26c82e54
>Fix:
patch kldxref.c to use fts_open() with a compare function to order the filenames.

(another possible fix would be to not include linker.hints in freebsd-update binary updates)

Patch attached with submission follows:

--- kldxref.c.old	2013-09-14 00:31:05.000000000 -0400
+++ kldxref.c	2013-09-14 14:53:53.000000000 -0400
@@ -275,6 +275,16 @@
 	exit(1);
 }
 
+int 
+compare(const FTSENT* const* a, const FTSENT* const* b)
+{
+    const char *pa = (const char *)((*a)->fts_name);
+    const char *pb = (const char *)((*b)->fts_name);
+    size_t as = (*a)->fts_namelen;
+    size_t bs = (*b)->fts_namelen;
+    return strncmp(pa, pb, as > bs ? bs : as);
+}
+
 int
 main(int argc, char *argv[])
 {
@@ -316,7 +326,7 @@
 		err(1, "%s", argv[0]);
 	}
 
-	ftsp = fts_open(argv, fts_options, 0);
+	ftsp = fts_open(argv, fts_options, compare);
 	if (ftsp == NULL)
 		exit(1);
 


>Release-Note:
>Audit-Trail:

From: Jilles Tjoelker <jilles@stack.nl>
To: bug-followup@FreeBSD.org, dereks@lifeofadishwasher.com
Cc:  
Subject: Re: bin/182098: [patch] Change kldxref fts_open ordering so it
 produces a consistent linker.hints between machines of the same
 architecture.
Date: Mon, 16 Sep 2013 00:39:58 +0200

 In PR bin/182098, you wrote:
 > Currently there an issue with linker.hints being different between
 > machines that create binary update for freebsd-update.
 
 > 'kldxref /boot/kernel/' will produce different linker.hints due to the
 > way the fts_open() traverses the file system:
 
 In the interest of reproducible builds, your patch seems a good idea. It
 seems unattractive to run kldxref /boot/kernel on every machine.
 
 The implementation of compare() seems unnecessarily complex though. In
 find -s, the fts_names are simply passed to strcoll() (here, strcmp()
 would be better). The trickery with the length may cause inconsistent
 results if one filename is a prefix of another (rare).
 
 This change may also expose a latent bug with kldxref -R: it does not
 work properly if a directory contains both files that need a mention in
 a hints file and subdirectories, and at least one such file appears
 after a subdirectory. Because your change alters the traversal order, it
 might break a use of kldxref -R that previously happened to work. You
 can make it work reliably by sorting FTS_D entries after other entries.
 
 -- 
 Jilles Tjoelker

From: Derek Schrock <dereks@lifeofadishwasher.com>
To: Jilles Tjoelker <jilles@stack.nl>
Cc: bug-followup@FreeBSD.org
Subject: Re: bin/182098: [patch] Change kldxref fts_open ordering so it
 produces a consistent linker.hints between machines of the same
 architecture.
Date: Tue, 24 Sep 2013 21:32:42 -0400

 --NzB8fVQJ5HfG6fxh
 Content-Type: text/plain; charset=us-ascii
 Content-Disposition: inline
 
 On Mon, Sep 16, 2013 at 12:39:58AM +0200, Jilles Tjoelker wrote:
 > In the interest of reproducible builds, your patch seems a good idea. It
 > seems unattractive to run kldxref /boot/kernel on every machine.
 > 
 > The implementation of compare() seems unnecessarily complex though. In
 > find -s, the fts_names are simply passed to strcoll() (here, strcmp()
 > would be better). The trickery with the length may cause inconsistent
 > results if one filename is a prefix of another (rare).
 
 Fair enough after reading more of the fts(3) man fts_name is always null terminated.
 
 > 
 > This change may also expose a latent bug with kldxref -R: it does not
 > work properly if a directory contains both files that need a mention in
 > a hints file and subdirectories, and at least one such file appears
 > after a subdirectory. Because your change alters the traversal order, it
 > might break a use of kldxref -R that previously happened to work. You
 > can make it work reliably by sorting FTS_D entries after other entries.
 
 Yep, after some additional testing with the patched kldxref it produced different linker.hints files. 
 
 The new patch uses fts_parent's fts_name (the directory's name).  First compare the parent's name, if the same compare the passed FTSENTs.
 
 --NzB8fVQJ5HfG6fxh
 Content-Type: text/plain; charset=us-ascii
 Content-Disposition: attachment; filename="patch.txt"
 
 --- kldxref.c.old	2013-09-24 21:06:07.000000000 -0400
 +++ kldxref.c	2013-09-24 21:31:39.000000000 -0400
 @@ -275,6 +275,19 @@
  	exit(1);
  }
  
 +int 
 +compare(const FTSENT* const* a, const FTSENT* const* b)
 +{
 +	char* a_parent = ((*a)->fts_parent)->fts_name;
 +	char* b_parent = ((*b)->fts_parent)->fts_name;
 +	int match = strcmp(a_parent, b_parent);
 +
 +	if (match == 0)
 +		match = strcmp((*a)->fts_name, (*b)->fts_name);
 +
 +	return match;
 +}
 +
  int
  main(int argc, char *argv[])
  {
 @@ -316,7 +329,7 @@
  		err(1, "%s", argv[0]);
  	}
  
 -	ftsp = fts_open(argv, fts_options, 0);
 +	ftsp = fts_open(argv, fts_options, compare);
  	if (ftsp == NULL)
  		exit(1);
  
 
 --NzB8fVQJ5HfG6fxh--

From: Jilles Tjoelker <jilles@stack.nl>
To: Derek Schrock <dereks@lifeofadishwasher.com>
Cc: bug-followup@FreeBSD.org
Subject: Re: bin/182098: [patch] Change kldxref fts_open ordering so it
 produces a consistent linker.hints between machines of the same
 architecture.
Date: Sat, 28 Sep 2013 20:39:53 +0200

 On Tue, Sep 24, 2013 at 09:32:42PM -0400, Derek Schrock wrote:
 > On Mon, Sep 16, 2013 at 12:39:58AM +0200, Jilles Tjoelker wrote:
 > > In the interest of reproducible builds, your patch seems a good idea. It
 > > seems unattractive to run kldxref /boot/kernel on every machine.
 > > 
 > > The implementation of compare() seems unnecessarily complex though. In
 > > find -s, the fts_names are simply passed to strcoll() (here, strcmp()
 > > would be better). The trickery with the length may cause inconsistent
 > > results if one filename is a prefix of another (rare).
 
 > Fair enough after reading more of the fts(3) man fts_name is always
 > null terminated.
 
 > > This change may also expose a latent bug with kldxref -R: it does not
 > > work properly if a directory contains both files that need a mention in
 > > a hints file and subdirectories, and at least one such file appears
 > > after a subdirectory. Because your change alters the traversal order, it
 > > might break a use of kldxref -R that previously happened to work. You
 > > can make it work reliably by sorting FTS_D entries after other entries.
 
 > Yep, after some additional testing with the patched kldxref it
 > produced different linker.hints files. 
 
 Hmm, that is unexpected. The sorting happens on entries of each
 directory, and find -s does the same.
 
 > The new patch uses fts_parent's fts_name (the directory's name).
 > First compare the parent's name, if the same compare the passed
 > FTSENTs.
 
 The comparison on the parent's name seems redundant due to the way
 fts(3) does the sorting.
 
 I tried my idea using FTS_D.
 
 Here is a script to test kldxref on this:
 
 #!/bin/sh
 
 kldxref=${1:-kldxref}
 kldxrefpath=$(command -v "$kldxref" 2>/dev/null) || {
 	printf "%s not found\n" "$kldxref"
 	exit
 }
 printf "Testing %s\n" "$kldxrefpath"
 tmpdir=$(mktemp -d -t kldxreftest) || exit
 cd "$tmpdir" || exit
 cp /boot/kernel/if_re.ko . || exit
 mkdir subdir1 || exit
 cp /boot/kernel/if_faith.ko . || exit
 cp /boot/kernel/zlib.ko . || exit
 cp /boot/kernel/unionfs.ko subdir1 || exit
 printf "Directory structure created in %s\n" "$tmpdir"
 "$kldxref" -R "$tmpdir"
 mainstrings=$(strings linker.hints)
 subdir1strings=$(strings subdir1/linker.hints)
 bad=
 for m in if_re.ko if_faith.ko zlib.ko; do
 	case $mainstrings in
 	*"$m"*) ;;
 	*) printf "%s is missing from main\n" "$m"; bad=1 ;;
 	esac
 	case $subdir1strings in
 	*"$m"*) printf "subdir1 wrongly has %s\n" "$m"; bad=1 ;;
 	esac
 done
 for m in unionfs.ko; do
 	case $subdir1strings in
 	*"$m"*) ;;
 	*) printf "%s is missing from subdir1\n" "$m"; bad=1 ;;
 	esac
 	case $mainstrings in
 	*"$m"*) printf "main wrongly has %s\n" "$m"; bad=1 ;;
 	esac
 done
 if [ -z "$bad" ]; then
 	printf "%s appears to be working correctly\n" "$kldxrefpath"
 fi
 
 My suggested patch:
 
 Index: usr.sbin/kldxref/kldxref.c
 ===================================================================
 --- usr.sbin/kldxref/kldxref.c	(revision 255570)
 +++ usr.sbin/kldxref/kldxref.c	(working copy)
 @@ -274,6 +274,16 @@
  	exit(1);
  }
  
 +int 
 +compare(const FTSENT* const* a, const FTSENT* const* b)
 +{
 +	if ((*a)->fts_info == FTS_D && (*b)->fts_info != FTS_D)
 +		return 1;
 +	if ((*a)->fts_info != FTS_D && (*b)->fts_info == FTS_D)
 +		return -1;
 +	return strcmp((*a)->fts_name, (*b)->fts_name);
 +}
 +
  int
  main(int argc, char *argv[])
  {
 @@ -315,7 +325,7 @@
  		err(1, "%s", argv[0]);
  	}
  
 -	ftsp = fts_open(argv, fts_options, 0);
 +	ftsp = fts_open(argv, fts_options, compare);
  	if (ftsp == NULL)
  		exit(1);
  
 
 -- 
 Jilles Tjoelker

From: Derek Schrock <dereks@lifeofadishwasher.com>
To: Jilles Tjoelker <jilles@stack.nl>
Cc: bug-followup@FreeBSD.org
Subject: Re: bin/182098: [patch] Change kldxref fts_open ordering so it
 produces a consistent linker.hints between machines of the same
 architecture.
Date: Sat, 28 Sep 2013 16:50:10 -0400

 Ok I see what you mean by using FTS_D.  Everything looks good to me, I
 compared the linker.hints generated by the two system from
 
 On Sat, Sep 28, 2013 at 08:39:53PM +0200, Jilles Tjoelker wrote:
 > On Tue, Sep 24, 2013 at 09:32:42PM -0400, Derek Schrock wrote:
 > > On Mon, Sep 16, 2013 at 12:39:58AM +0200, Jilles Tjoelker wrote:
 > > > In the interest of reproducible builds, your patch seems a good idea. It
 > > > seems unattractive to run kldxref /boot/kernel on every machine.
 > > > 
 > > > The implementation of compare() seems unnecessarily complex though. In
 > > > find -s, the fts_names are simply passed to strcoll() (here, strcmp()
 > > > would be better). The trickery with the length may cause inconsistent
 > > > results if one filename is a prefix of another (rare).
 > 
 > > Fair enough after reading more of the fts(3) man fts_name is always
 > > null terminated.
 > 
 > > > This change may also expose a latent bug with kldxref -R: it does not
 > > > work properly if a directory contains both files that need a mention in
 > > > a hints file and subdirectories, and at least one such file appears
 > > > after a subdirectory. Because your change alters the traversal order, it
 > > > might break a use of kldxref -R that previously happened to work. You
 > > > can make it work reliably by sorting FTS_D entries after other entries.
 > 
 > > Yep, after some additional testing with the patched kldxref it
 > > produced different linker.hints files. 
 > 
 > Hmm, that is unexpected. The sorting happens on entries of each
 > directory, and find -s does the same.
 > 
 > > The new patch uses fts_parent's fts_name (the directory's name).
 > > First compare the parent's name, if the same compare the passed
 > > FTSENTs.
 > 
 > The comparison on the parent's name seems redundant due to the way
 > fts(3) does the sorting.
 > 
 > I tried my idea using FTS_D.
 > 
 > Here is a script to test kldxref on this:
 > 
 > #!/bin/sh
 > 
 > kldxref=${1:-kldxref}
 > kldxrefpath=$(command -v "$kldxref" 2>/dev/null) || {
 > 	printf "%s not found\n" "$kldxref"
 > 	exit
 > }
 > printf "Testing %s\n" "$kldxrefpath"
 > tmpdir=$(mktemp -d -t kldxreftest) || exit
 > cd "$tmpdir" || exit
 > cp /boot/kernel/if_re.ko . || exit
 > mkdir subdir1 || exit
 > cp /boot/kernel/if_faith.ko . || exit
 > cp /boot/kernel/zlib.ko . || exit
 > cp /boot/kernel/unionfs.ko subdir1 || exit
 > printf "Directory structure created in %s\n" "$tmpdir"
 > "$kldxref" -R "$tmpdir"
 > mainstrings=$(strings linker.hints)
 > subdir1strings=$(strings subdir1/linker.hints)
 > bad=
 > for m in if_re.ko if_faith.ko zlib.ko; do
 > 	case $mainstrings in
 > 	*"$m"*) ;;
 > 	*) printf "%s is missing from main\n" "$m"; bad=1 ;;
 > 	esac
 > 	case $subdir1strings in
 > 	*"$m"*) printf "subdir1 wrongly has %s\n" "$m"; bad=1 ;;
 > 	esac
 > done
 > for m in unionfs.ko; do
 > 	case $subdir1strings in
 > 	*"$m"*) ;;
 > 	*) printf "%s is missing from subdir1\n" "$m"; bad=1 ;;
 > 	esac
 > 	case $mainstrings in
 > 	*"$m"*) printf "main wrongly has %s\n" "$m"; bad=1 ;;
 > 	esac
 > done
 > if [ -z "$bad" ]; then
 > 	printf "%s appears to be working correctly\n" "$kldxrefpath"
 > fi
 > 
 > My suggested patch:
 > 
 > Index: usr.sbin/kldxref/kldxref.c
 > ===================================================================
 > --- usr.sbin/kldxref/kldxref.c	(revision 255570)
 > +++ usr.sbin/kldxref/kldxref.c	(working copy)
 > @@ -274,6 +274,16 @@
 >  	exit(1);
 >  }
 >  
 > +int 
 > +compare(const FTSENT* const* a, const FTSENT* const* b)
 > +{
 > +	if ((*a)->fts_info == FTS_D && (*b)->fts_info != FTS_D)
 > +		return 1;
 > +	if ((*a)->fts_info != FTS_D && (*b)->fts_info == FTS_D)
 > +		return -1;
 > +	return strcmp((*a)->fts_name, (*b)->fts_name);
 > +}
 > +
 >  int
 >  main(int argc, char *argv[])
 >  {
 > @@ -315,7 +325,7 @@
 >  		err(1, "%s", argv[0]);
 >  	}
 >  
 > -	ftsp = fts_open(argv, fts_options, 0);
 > +	ftsp = fts_open(argv, fts_options, compare);
 >  	if (ftsp == NULL)
 >  		exit(1);
 >  
 > 
 > -- 
 > Jilles Tjoelker

From: Derek Schrock <dereks@lifeofadishwasher.com>
To: Jilles Tjoelker <jilles@stack.nl>
Cc: bug-followup@FreeBSD.org
Subject: Re: bin/182098: [patch] Change kldxref fts_open ordering so it
 produces a consistent linker.hints between machines of the same
 architecture.
Date: Sat, 28 Sep 2013 17:06:00 -0400

 Looks like I wanted to postpone the last message instead of sending it:
 
 Ok I see what you mean by using FTS_D.  Everything looks good to me, I
 compared the linker.hints generated by the two system from the initial 
 description.  They're the same.
 
 However, I slightly modified the testing script from your last comment
 and found three .ko files that don't seem to generate an entry in the 
 linker.hints files:
 
 Patched kldxref:
 $ /usr/src/usr.sbin/kldxref/kldxref -R /boot/
 
 $ dir="kernel"; strings="$(strings /boot/$dir/linker.hints)"; for f in /boot/"$dir"/*.ko ; do f="${f##*/}"; case "$strings" in *"$f"*) ;; *) echo "what? $f" ;; esac; done
 what? musb.ko
 what? scc.ko
 what? uss820dci.ko
 
 System kldxref:
 $ kldxref -R /boot/
 
 $ dir="kernel"; strings="$(strings /boot/$dir/linker.hints)"; for f in /boot/"$dir"/*.ko ; do f="${f##*/}"; case "$strings" in *"$f"*) ;; *) echo "what? $f" ;; esac; done
 what? musb.ko
 what? scc.ko
 what? uss820dci.ko
 
 Can we assume this isn't a problem with patch since the system kldxref 
 doesn't generate a linker.hints with strings that reference those ko 
 files?
 
 On Sat, Sep 28, 2013 at 04:50:10PM -0400, Derek Schrock wrote:
 > Ok I see what you mean by using FTS_D.  Everything looks good to me, I
 > compared the linker.hints generated by the two system from
 > 
 > On Sat, Sep 28, 2013 at 08:39:53PM +0200, Jilles Tjoelker wrote:
 > > On Tue, Sep 24, 2013 at 09:32:42PM -0400, Derek Schrock wrote:
 > > > On Mon, Sep 16, 2013 at 12:39:58AM +0200, Jilles Tjoelker wrote:
 > > > > In the interest of reproducible builds, your patch seems a good idea. It
 > > > > seems unattractive to run kldxref /boot/kernel on every machine.
 > > > > 
 > > > > The implementation of compare() seems unnecessarily complex though. In
 > > > > find -s, the fts_names are simply passed to strcoll() (here, strcmp()
 > > > > would be better). The trickery with the length may cause inconsistent
 > > > > results if one filename is a prefix of another (rare).
 > > 
 > > > Fair enough after reading more of the fts(3) man fts_name is always
 > > > null terminated.
 > > 
 > > > > This change may also expose a latent bug with kldxref -R: it does not
 > > > > work properly if a directory contains both files that need a mention in
 > > > > a hints file and subdirectories, and at least one such file appears
 > > > > after a subdirectory. Because your change alters the traversal order, it
 > > > > might break a use of kldxref -R that previously happened to work. You
 > > > > can make it work reliably by sorting FTS_D entries after other entries.
 > > 
 > > > Yep, after some additional testing with the patched kldxref it
 > > > produced different linker.hints files. 
 > > 
 > > Hmm, that is unexpected. The sorting happens on entries of each
 > > directory, and find -s does the same.
 > > 
 > > > The new patch uses fts_parent's fts_name (the directory's name).
 > > > First compare the parent's name, if the same compare the passed
 > > > FTSENTs.
 > > 
 > > The comparison on the parent's name seems redundant due to the way
 > > fts(3) does the sorting.
 > > 
 > > I tried my idea using FTS_D.
 > > 
 > > Here is a script to test kldxref on this:
 > > 
 > > #!/bin/sh
 > > 
 > > kldxref=${1:-kldxref}
 > > kldxrefpath=$(command -v "$kldxref" 2>/dev/null) || {
 > > 	printf "%s not found\n" "$kldxref"
 > > 	exit
 > > }
 > > printf "Testing %s\n" "$kldxrefpath"
 > > tmpdir=$(mktemp -d -t kldxreftest) || exit
 > > cd "$tmpdir" || exit
 > > cp /boot/kernel/if_re.ko . || exit
 > > mkdir subdir1 || exit
 > > cp /boot/kernel/if_faith.ko . || exit
 > > cp /boot/kernel/zlib.ko . || exit
 > > cp /boot/kernel/unionfs.ko subdir1 || exit
 > > printf "Directory structure created in %s\n" "$tmpdir"
 > > "$kldxref" -R "$tmpdir"
 > > mainstrings=$(strings linker.hints)
 > > subdir1strings=$(strings subdir1/linker.hints)
 > > bad=
 > > for m in if_re.ko if_faith.ko zlib.ko; do
 > > 	case $mainstrings in
 > > 	*"$m"*) ;;
 > > 	*) printf "%s is missing from main\n" "$m"; bad=1 ;;
 > > 	esac
 > > 	case $subdir1strings in
 > > 	*"$m"*) printf "subdir1 wrongly has %s\n" "$m"; bad=1 ;;
 > > 	esac
 > > done
 > > for m in unionfs.ko; do
 > > 	case $subdir1strings in
 > > 	*"$m"*) ;;
 > > 	*) printf "%s is missing from subdir1\n" "$m"; bad=1 ;;
 > > 	esac
 > > 	case $mainstrings in
 > > 	*"$m"*) printf "main wrongly has %s\n" "$m"; bad=1 ;;
 > > 	esac
 > > done
 > > if [ -z "$bad" ]; then
 > > 	printf "%s appears to be working correctly\n" "$kldxrefpath"
 > > fi
 > > 
 > > My suggested patch:
 > > 
 > > Index: usr.sbin/kldxref/kldxref.c
 > > ===================================================================
 > > --- usr.sbin/kldxref/kldxref.c	(revision 255570)
 > > +++ usr.sbin/kldxref/kldxref.c	(working copy)
 > > @@ -274,6 +274,16 @@
 > >  	exit(1);
 > >  }
 > >  
 > > +int 
 > > +compare(const FTSENT* const* a, const FTSENT* const* b)
 > > +{
 > > +	if ((*a)->fts_info == FTS_D && (*b)->fts_info != FTS_D)
 > > +		return 1;
 > > +	if ((*a)->fts_info != FTS_D && (*b)->fts_info == FTS_D)
 > > +		return -1;
 > > +	return strcmp((*a)->fts_name, (*b)->fts_name);
 > > +}
 > > +
 > >  int
 > >  main(int argc, char *argv[])
 > >  {
 > > @@ -315,7 +325,7 @@
 > >  		err(1, "%s", argv[0]);
 > >  	}
 > >  
 > > -	ftsp = fts_open(argv, fts_options, 0);
 > > +	ftsp = fts_open(argv, fts_options, compare);
 > >  	if (ftsp == NULL)
 > >  		exit(1);
 > >  
 > > 
 > > -- 
 > > Jilles Tjoelker

From: Jilles Tjoelker <jilles@stack.nl>
To: Derek Schrock <dereks@lifeofadishwasher.com>
Cc: bug-followup@FreeBSD.org
Subject: Re: bin/182098: [patch] Change kldxref fts_open ordering so it
 produces a consistent linker.hints between machines of the same
 architecture.
Date: Fri, 4 Oct 2013 18:19:21 +0200

 On Sat, Sep 28, 2013 at 05:06:00PM -0400, Derek Schrock wrote:
 > Looks like I wanted to postpone the last message instead of sending it:
 
 > Ok I see what you mean by using FTS_D.  Everything looks good to me, I
 > compared the linker.hints generated by the two system from the initial 
 > description.  They're the same.
 
 > However, I slightly modified the testing script from your last comment
 > and found three .ko files that don't seem to generate an entry in the 
 > linker.hints files:
 
 > Patched kldxref:
 > $ /usr/src/usr.sbin/kldxref/kldxref -R /boot/
 
 > $ dir="kernel"; strings="$(strings /boot/$dir/linker.hints)"; for f in /boot/"$dir"/*.ko ; do f="${f##*/}"; case "$strings" in *"$f"*) ;; *) echo "what? $f" ;; esac; done
 > what? musb.ko
 > what? scc.ko
 > what? uss820dci.ko
 
 > System kldxref:
 > $ kldxref -R /boot/
 
 > $ dir="kernel"; strings="$(strings /boot/$dir/linker.hints)"; for f in /boot/"$dir"/*.ko ; do f="${f##*/}"; case "$strings" in *"$f"*) ;; *) echo "what? $f" ;; esac; done
 > what? musb.ko
 > what? scc.ko
 > what? uss820dci.ko
 
 > Can we assume this isn't a problem with patch since the system kldxref 
 > doesn't generate a linker.hints with strings that reference those ko 
 > files?
 
 Yes, this seems not the fault of the patch. It is possible that these
 three modules do not actually work.
 
 -- 
 Jilles Tjoelker
State-Changed-From-To: open->patched 
State-Changed-By: jilles 
State-Changed-When: Fri Oct 4 21:28:32 UTC 2013 
State-Changed-Why:  
Committed to 10-current, thanks. 


Responsible-Changed-From-To: freebsd-bugs->jilles 
Responsible-Changed-By: jilles 
Responsible-Changed-When: Fri Oct 4 21:28:32 UTC 2013 
Responsible-Changed-Why:  
I did the commit. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=182098 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: bin/182098: commit references a PR
Date: Fri,  4 Oct 2013 21:26:02 +0000 (UTC)

 Author: jilles
 Date: Fri Oct  4 21:25:55 2013
 New Revision: 256060
 URL: http://svnweb.freebsd.org/changeset/base/256060
 
 Log:
   kldxref: Do not depend on the directory order.
   
   Sort the filenames to get a consistent result between machines of the same
   architecture.
   
   Also, sort FTS_D entries after other entries so kldxref -R works properly in
   the uncommon case that a directory contains both subdirectories and modules.
   Previously, this may have happened to work, depending on the order of files
   in the directory.
   
   PR:		bin/182098
   Submitted by:	Derek Schrock (original version)
   Tested by:	Derek Schrock
   Approved by:	re (delphij)
   MFC after:	1 week
 
 Modified:
   head/usr.sbin/kldxref/kldxref.c
 
 Modified: head/usr.sbin/kldxref/kldxref.c
 ==============================================================================
 --- head/usr.sbin/kldxref/kldxref.c	Fri Oct  4 19:57:47 2013	(r256059)
 +++ head/usr.sbin/kldxref/kldxref.c	Fri Oct  4 21:25:55 2013	(r256060)
 @@ -274,6 +274,16 @@ usage(void)
  	exit(1);
  }
  
 +int 
 +compare(const FTSENT *const *a, const FTSENT *const *b)
 +{
 +	if ((*a)->fts_info == FTS_D && (*b)->fts_info != FTS_D)
 +		return 1;
 +	if ((*a)->fts_info != FTS_D && (*b)->fts_info == FTS_D)
 +		return -1;
 +	return strcmp((*a)->fts_name, (*b)->fts_name);
 +}
 +
  int
  main(int argc, char *argv[])
  {
 @@ -315,7 +325,7 @@ main(int argc, char *argv[])
  		err(1, "%s", argv[0]);
  	}
  
 -	ftsp = fts_open(argv, fts_options, 0);
 +	ftsp = fts_open(argv, fts_options, compare);
  	if (ftsp == NULL)
  		exit(1);
  
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: bin/182098: commit references a PR
Date: Mon, 23 Dec 2013 22:38:48 +0000 (UTC)

 Author: jilles
 Date: Mon Dec 23 22:38:41 2013
 New Revision: 259799
 URL: http://svnweb.freebsd.org/changeset/base/259799
 
 Log:
   MFC r256060,r256650: kldxref: Do not depend on the directory order.
   
   Sort the filenames to get a consistent result between machines of the same
   architecture.
   
   Also, sort FTS_D entries after other entries so kldxref -R works properly in
   the uncommon case that a directory contains both subdirectories and modules.
   Previously, this may have happened to work, depending on the order of files
   in the directory.
   
   PR:		bin/182098
   Submitted by:	Derek Schrock (original version)
   Tested by:	Derek Schrock
 
 Modified:
   stable/9/usr.sbin/kldxref/kldxref.c
 Directory Properties:
   stable/9/usr.sbin/kldxref/   (props changed)
 
 Modified: stable/9/usr.sbin/kldxref/kldxref.c
 ==============================================================================
 --- stable/9/usr.sbin/kldxref/kldxref.c	Mon Dec 23 22:37:59 2013	(r259798)
 +++ stable/9/usr.sbin/kldxref/kldxref.c	Mon Dec 23 22:38:41 2013	(r259799)
 @@ -274,6 +274,16 @@ usage(void)
  	exit(1);
  }
  
 +static int
 +compare(const FTSENT *const *a, const FTSENT *const *b)
 +{
 +	if ((*a)->fts_info == FTS_D && (*b)->fts_info != FTS_D)
 +		return 1;
 +	if ((*a)->fts_info != FTS_D && (*b)->fts_info == FTS_D)
 +		return -1;
 +	return strcmp((*a)->fts_name, (*b)->fts_name);
 +}
 +
  int
  main(int argc, char *argv[])
  {
 @@ -315,7 +325,7 @@ main(int argc, char *argv[])
  		err(1, "%s", argv[0]);
  	}
  
 -	ftsp = fts_open(argv, fts_options, 0);
 +	ftsp = fts_open(argv, fts_options, compare);
  	if (ftsp == NULL)
  		exit(1);
  
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 
State-Changed-From-To: patched->closed 
State-Changed-By: jilles 
State-Changed-When: Sun Jan 12 20:57:27 UTC 2014 
State-Changed-Why:  
Fixed in stable/9, stable/10 and head. 
There will be no new 8.x release. 
Thanks for the report. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=182098 
>Unformatted:
