From uspoerlein@gmail.com  Sat Apr 26 14:23:42 2008
Return-Path: <uspoerlein@gmail.com>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id A2A921065670
	for <FreeBSD-gnats-submit@freebsd.org>; Sat, 26 Apr 2008 14:23:42 +0000 (UTC)
	(envelope-from uspoerlein@gmail.com)
Received: from fg-out-1718.google.com (fg-out-1718.google.com [72.14.220.154])
	by mx1.freebsd.org (Postfix) with ESMTP id 276068FC20
	for <FreeBSD-gnats-submit@freebsd.org>; Sat, 26 Apr 2008 14:23:41 +0000 (UTC)
	(envelope-from uspoerlein@gmail.com)
Received: by fg-out-1718.google.com with SMTP id 16so4597275fgg.35
        for <FreeBSD-gnats-submit@freebsd.org>; Sat, 26 Apr 2008 07:23:41 -0700 (PDT)
Received: by 10.86.99.9 with SMTP id w9mr3433624fgb.58.1209218266650;
        Sat, 26 Apr 2008 06:57:46 -0700 (PDT)
Received: from acme.spoerlein.net ( [217.172.44.86])
        by mx.google.com with ESMTPS id d13sm6007172fka.7.2008.04.26.06.57.45
        (version=TLSv1/SSLv3 cipher=OTHER);
        Sat, 26 Apr 2008 06:57:46 -0700 (PDT)
Received: from roadrunner.spoerlein.net (e180142204.adsl.alicedsl.de [85.180.142.204])
	by acme.spoerlein.net (8.14.2/8.14.2) with ESMTP id m3QDvh5S040688
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Sat, 26 Apr 2008 15:57:44 +0200 (CEST)
	(envelope-from uspoerlein@gmail.com)
Received: from roadrunner.spoerlein.net (localhost [127.0.0.1])
	by roadrunner.spoerlein.net (8.14.2/8.14.2) with ESMTP id m3QDveoE030041
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO);
	Sat, 26 Apr 2008 15:57:41 +0200 (CEST)
	(envelope-from uqs@roadrunner.spoerlein.net)
Received: (from uqs@localhost)
	by roadrunner.spoerlein.net (8.14.2/8.14.2/Submit) id m3QDveiI030040;
	Sat, 26 Apr 2008 15:57:40 +0200 (CEST)
	(envelope-from uqs)
Message-Id: <200804261357.m3QDveiI030040@roadrunner.spoerlein.net>
Date: Sat, 26 Apr 2008 15:57:40 +0200 (CEST)
From: Ulrich Spörlein <uspoerlein@gmail.com>
To: FreeBSD-gnats-submit@freebsd.org
Cc: stas@freebsd.org, pjd@freebsd.org, jw@innerewut.de
Subject: [PATCH] rdoc node numbering non-deterministic on ZFS volumes
X-Send-Pr-Version: 3.113
X-GNATS-Notify:

>Number:         123112
>Category:       ports
>Synopsis:       [PATCH] lang/ruby18 - rdoc node numbering non-deterministic on ZFS volumes
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    stas
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          change-request
>Submitter-Id:   current-users
>Arrival-Date:   Sat Apr 26 14:30:01 UTC 2008
>Closed-Date:    Mon Apr 06 19:52:43 UTC 2009
>Last-Modified:  Fri Nov 12 20:51:56 UTC 2010
>Originator:     Ulrich Spoerlein
>Release:        FreeBSD 7.0-STABLE i386
>Organization:
>Environment:
>Description:
I'm running /usr on ZFS and noticed that all pkg-plists from rdoc-using ports
are broken. rdoc is used to convert ruby's inline markup to external HTML files.
It runs recursively, but doesn't sort the output of readdir() *and* uses unique
numbers for every file.

This in effect will lead to "random" page numbering, depending on the output of
readdir()
>How-To-Repeat:
Simple test to demonstrate the ZFS readdir() sorting "problem"

UFS:
% while :; do rm -rf foo; mkdir -p foo/{a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z} && (ls -1f foo | md5); done | uniq
b7fdd99fac291c4bbf958d9aee731951
^C

ZFS:
% while :; do rm -rf foo; mkdir -p foo/{a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z} && (ls -1f foo | md5); done | uniq
fe4647c6a1bc36cd85ec44ab938e8489
83584ba58ea8abb2a174147b3e16efbb
517eff118b8009964cb4fb83087ed135
aefa552bbe162e3ff881c0fbf27bc350
359620a9f16172ce7e1cd9ad9d79ea2b
517eff118b8009964cb4fb83087ed135
a1f1827e51a455d7cbf83e7f1c2ed8d2
e2c202117483de9b7c8a6e3992e26470
^C

As you can see, directories are not necessarily returned in the order they have
been created, which can be true for UFS, too, but not when appending new
entries to an existing directory, AFAICS.

To see the specific problem with rdoc, install devel/ruby-gems with "make
install package" on a ZFS volume, the pkg_create will fail since the installed
rdoc node files don't match what's listed in pkg-plist.

>Fix:
This patch to ruby's rdoc seems to stabilize the rdoc numbering in my testing.

BE WARNED THOUGH: This will break lots of plists, as they are currently generated
unsorted but deterministically (for UFS; non-deterministically for ZFS!). After
the patch they will be generated sorted all the time.

The patch should be submitted upstream, I'm not sure about the style and conventions
in the ruby world, though.


Side note: There also seems to be an rdoc switch --one-file which will probably
generate only a single page. If we could implement this for all rubygems and
other rdoc using ports (by making it the default on FreeBSD?), this could solve
the problem, too (and cut down the pkg-plist size for a lot of ports).

--- rdoc.patch begins here ---
--- rdoc.rb.orig	2007-02-13 00:01:19.000000000 +0100
+++ rdoc.rb	2008-04-26 15:38:21.830669906 +0200
@@ -200,7 +200,7 @@
     # we may well contain subdirectories which must
     # be tested for .document files
     def list_files_in_directory(dir, options)
-      normalized_file_list(options, Dir.glob(File.join(dir, "*")), false, options.exclude)
+      normalized_file_list(options, Dir.glob(File.join(dir, "*")).sort, false, options.exclude)
     end
 
 
--- rdoc.patch ends here ---


>Release-Note:
>Audit-Trail:
Date: Sun, 27 Apr 2008 00:40:47 +0400
From: Stanislav Sedov <stas@FreeBSD.org>
 
 The best way would be to not put dynamically generated files in pkg-plist
 at all. I noticed that problem when working on ruby19 ports, then when
 building packages on zfs volumes. ruby19 rdoc seems to sort files by
 default.
 
 I plan to add a knob to bsd.ruby.mk to dynamically generate plist entries
 for rdoc files. It seems that these bits are pretty common for all ruby
 ports (both gems and non-gems). This way we'll be able to relatively
 easily convert all existent ports to not use static plists for rdoc
 files. I tried to ping some rubygem-* maintainers a while ago about
 this bug, but none replied unfortunately:-(
 
 -- 
 Stanislav Sedov
 ST4096-RIPE
Responsible-Changed-From-To: freebsd-ports-bugs->stas 
Responsible-Changed-By: edwin 
Responsible-Changed-When: Sun Apr 27 04:07:20 UTC 2008 
Responsible-Changed-Why:  
Over to maintainer (via the GNATS Auto Assign Tool) 

http://www.freebsd.org/cgi/query-pr.cgi?pr=123112 
State-Changed-From-To: open->closed 
State-Changed-By: stas 
State-Changed-When: Mon Apr 6 19:52:42 UTC 2009 
State-Changed-Why:  
RUBYGEM_AUTO_PLIST feature has been committed 

http://www.freebsd.org/cgi/query-pr.cgi?pr=123112 
>Unformatted:
