From nobody@FreeBSD.org  Thu Jun 14 01:22:45 2007
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52])
	by hub.freebsd.org (Postfix) with ESMTP id F35EC16A41F
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 14 Jun 2007 01:22:44 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (www.freebsd.org [69.147.83.33])
	by mx1.freebsd.org (Postfix) with ESMTP id E38A813C468
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 14 Jun 2007 01:22:44 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by www.freebsd.org (8.13.1/8.13.1) with ESMTP id l5E1MiWu009618
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 14 Jun 2007 01:22:44 GMT
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.13.1/8.13.1/Submit) id l5E1MiAQ009617;
	Thu, 14 Jun 2007 01:22:44 GMT
	(envelope-from nobody)
Message-Id: <200706140122.l5E1MiAQ009617@www.freebsd.org>
Date: Thu, 14 Jun 2007 01:22:44 GMT
From: Ed Ravin <eravin@panix.com>
To: freebsd-gnats-submit@FreeBSD.org
Subject: scandir(3) uses st_size of directory in unsupported manner
X-Send-Pr-Version: www-3.0

>Number:         113668
>Category:       kern
>Synopsis:       [libc] scandir(3) uses st_size of directory in unsupported manner
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    das
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Thu Jun 14 01:30:03 GMT 2007
>Closed-Date:    Tue May 06 05:46:48 UTC 2008
>Last-Modified:  Tue May  6 05:50:00 UTC 2008
>Originator:     Ed Ravin
>Release:        
>Organization:
Public Access Networks
>Environment:
>Description:
src/lib/libc/gen/scandir.c assumes that a directory's st_size is multiples
of 24.  This is against POSIX, which doesn't define what st_size means
for directories, and will cause problems when run against an NFS-exported
ZFS file system from a Solaris box, where st_size is the number of entries
in the directory.  For example, a 5-entry directory returns st_size of 5,
and then scandir does "arraysz= st_size / 24", and then does a malloc()
of arraysz bytes, then writes something into the memory (that wasn't)
allocated.

See also NetBSD bug PR/36464.  OpenBSD is similarly affected.
>How-To-Repeat:

>Fix:


>Release-Note:
>Audit-Trail:

From: David Schultz <das@FreeBSD.ORG>
To: Ed Ravin <eravin@panix.com>
Cc: freebsd-gnats-submit@FreeBSD.ORG
Subject: Re: misc/113668: scandir(3) uses st_size of directory in unsupported manner
Date: Sun, 13 Jan 2008 05:03:10 -0500

 In the original Unix, directories were just files. FreeBSD has
 always supported that philosophy; you can still `cat' a directory,
 and so it makes sense that st_size gives you the size in bytes.
 It's not scandir's fault that System V made directories more
 magical, then POSIX came along and had to cope with it, then the
 ZFS team exploited the underspecification in POSIX for their own
 nefarious purposes. :) That said, given that FreeBSD now has ZFS,
 and some of the ZFS utils probably assume the new ZFS rules for
 st_size, I guess we ought to do something!
 
 Note that this is just a performance problem, since scandir will
 realloc the array as necessary. However, it does the increases in
 increments of 10 entries, and it should probably realloc
 multiplicatively so the amortized cost of the reallocs is linear.
 I'm not sure what to do about the initial estimate, though.
 If we really want to support every old st_size value filesystem
 designers come up with, we need to worry about st_size being too
 large as well, so we basically shouldn't use it at all...
Responsible-Changed-From-To: freebsd-bugs->das 
Responsible-Changed-By: das 
Responsible-Changed-When: Sun Mar 16 19:07:35 UTC 2008 
Responsible-Changed-Why:  
over to me 

http://www.freebsd.org/cgi/query-pr.cgi?pr=113668 
State-Changed-From-To: open->patched 
State-Changed-By: das 
State-Changed-When: Sun Mar 16 19:08:58 UTC 2008 
State-Changed-Why:  
Fixed in HEAD, thanks! 
src/lib/libc/gen/scandir.c,v 1.8 

http://www.freebsd.org/cgi/query-pr.cgi?pr=113668 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/113668: commit references a PR
Date: Sun, 16 Mar 2008 19:08:59 +0000 (UTC)

 das         2008-03-16 19:08:53 UTC
 
   FreeBSD src repository
 
   Modified files:
     lib/libc/gen         scandir.c 
   Log:
   scandir(3) previously used st_size to obtain an initial estimate
   of the array length needed to store all the directory entries.
   Although BSD has historically guaranteed that st_size is the size
   of the directory file, POSIX does not, and more to the point, some
   recent filesystems such as ZFS use st_size to mean something else.
   
   The fix is to not stat the directory at all, set the initial
   array size to 32 entries, and realloc it in powers of 2 if that
   proves insufficient.
   
   PR:     113668
   
   Revision  Changes    Path
   1.9       +3 -13     src/lib/libc/gen/scandir.c
 _______________________________________________
 cvs-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/cvs-all
 To unsubscribe, send any mail to "cvs-all-unsubscribe@freebsd.org"
 
State-Changed-From-To: patched->closed 
State-Changed-By: das 
State-Changed-When: Tue May 6 05:46:28 UTC 2008 
State-Changed-Why:  
Merged to RELENG_7. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=113668 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/113668: commit references a PR
Date: Tue,  6 May 2008 05:46:05 +0000 (UTC)

 das         2008-05-06 05:45:58 UTC
 
   FreeBSD src repository
 
   Modified files:        (Branch: RELENG_7)
     lib/libc/gen         scandir.c 
   Log:
   MFC scandir.c,v 1.9
   
     scandir(3) previously used st_size to obtain an initial estimate
     of the array length needed to store all the directory entries.
     Although BSD has historically guaranteed that st_size is the size
     of the directory file, POSIX does not, and more to the point, some
     recent filesystems such as ZFS use st_size to mean something else.
   
     The fix is to not stat the directory at all, set the initial
     array size to 32 entries, and realloc it in powers of 2 if that
     proves insufficient.
   
     PR:   113668
   
   Revision  Changes    Path
   1.8.2.1   +3 -13     src/lib/libc/gen/scandir.c
 _______________________________________________
 cvs-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/cvs-all
 To unsubscribe, send any mail to "cvs-all-unsubscribe@freebsd.org"
 
>Unformatted:
