From nobody@FreeBSD.org  Wed Feb 20 03:45:09 2008
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 7170E16A402
	for <freebsd-gnats-submit@FreeBSD.org>; Wed, 20 Feb 2008 03:45:09 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (www.freebsd.org [IPv6:2001:4f8:fff6::21])
	by mx1.freebsd.org (Postfix) with ESMTP id 4CB8813C45E
	for <freebsd-gnats-submit@FreeBSD.org>; Wed, 20 Feb 2008 03:45:09 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by www.freebsd.org (8.14.2/8.14.2) with ESMTP id m1K3gbAo072448
	for <freebsd-gnats-submit@FreeBSD.org>; Wed, 20 Feb 2008 03:42:38 GMT
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.14.2/8.14.1/Submit) id m1K3gbY8072447;
	Wed, 20 Feb 2008 03:42:37 GMT
	(envelope-from nobody)
Message-Id: <200802200342.m1K3gbY8072447@www.freebsd.org>
Date: Wed, 20 Feb 2008 03:42:37 GMT
From: Yuri <yuri@tsoft.com>
To: freebsd-gnats-submit@FreeBSD.org
Subject: [procfs] 'stat' shows that all files have 0-length when they are actually not empty
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         120869
>Category:       kern
>Synopsis:       [procfs] 'stat' shows that all files have 0-length when they are actually not empty
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    freebsd-fs
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Wed Feb 20 03:50:00 UTC 2008
>Closed-Date:    Wed Feb 20 12:08:22 UTC 2008
>Last-Modified:  Thu Feb 21 14:10:01 UTC 2008
>Originator:     Yuri
>Release:        7.0-PRERELEASE FreeBSD 7.0-PRERELEASE #4: Tue Feb 19 08:18:11 PST 2008
>Organization:
>Environment:
>Description:
When I do 'ls -l /proc/*/*' I see that all files have 0-length.
And non-empty content can be seen for most of them, for example for
/proc/curproc/cmdline.

>How-To-Repeat:

>Fix:


>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->imp 
Responsible-Changed-By: remko 
Responsible-Changed-When: Wed Feb 20 06:41:12 UTC 2008 
Responsible-Changed-Why:  
Hi WArner, you touched procfs a couple of times (from the ident strings 
on the procfs files) can you have a look please? 

http://www.freebsd.org/cgi/query-pr.cgi?pr=120869 
Responsible-Changed-From-To: imp->freebsd-fs 
Responsible-Changed-By: remko 
Responsible-Changed-When: Wed Feb 20 07:42:08 UTC 2008 
Responsible-Changed-Why:  
OK Warner wasn't the right person to redirect to, forward over to the FS 
group 

http://www.freebsd.org/cgi/query-pr.cgi?pr=120869 
State-Changed-From-To: open->closed 
State-Changed-By: remko 
State-Changed-When: Wed Feb 20 12:08:22 UTC 2008 
State-Changed-Why:  
After a bit of discussion, this is not something which we are going to 
fix. It is not worth the hassle. If you think this should be different, 
we do welcome patches. Thanks fo rusing FreeBSD! 

http://www.freebsd.org/cgi/query-pr.cgi?pr=120869 

From: Robert Watson <rwatson@FreeBSD.org>
To: remko@FreeBSD.org
Cc: yuri@tsoft.com, freebsd-fs@FreeBSD.org, bug-followup@FreeBSD.org
Subject: Re: kern/120869: [procfs] 'stat' shows that all files have 0-length
 when they are actually not empty
Date: Wed, 20 Feb 2008 13:32:30 +0000 (GMT)

 On Wed, 20 Feb 2008, remko@FreeBSD.org wrote:
 
 > After a bit of discussion, this is not something which we are going to fix. 
 > It is not worth the hassle. If you think this should be different, we do 
 > welcome patches. Thanks fo rusing FreeBSD!
 
 Just as two data points here: Solaris attempts to provide coherent file sizes 
 in /proc (at least to the extent that tried a few for objects where it is 
 remotely possible), and the Linux 2.6.12 kernel I have on a box locally 
 basically doesn't.
 
 My view is that it's a synthetic file system with data that varies dynamically 
 at runtime, and that while it wouldn't hurt to produce file size information 
 that's correct, it's quite a bit of work to do so and that I wouldn't 
 prioritize it above other, more critical things that need to happen.  We 
 should certainly evaluate any patches that come in for possible inclusion, 
 assuming they don't muck up the internals of procfs too much; it's not clear 
 to me if they necessarily do so or not.
 
 Robert N M Watson
 Computer Laboratory
 University of Cambridge

From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= <des@des.no>
To: Robert Watson <rwatson@FreeBSD.org>
Cc: remko@FreeBSD.org,  freebsd-fs@FreeBSD.org,  yuri@tsoft.com,  bug-followup@FreeBSD.org
Subject: Re: kern/120869: [procfs] 'stat' shows that all files have 0-length when they are actually not empty
Date: Wed, 20 Feb 2008 17:53:52 +0100

 Robert Watson <rwatson@FreeBSD.org> writes:
 > Just as two data points here: Solaris attempts to provide coherent
 > file sizes in /proc (at least to the extent that tried a few for
 > objects where it is remotely possible), and the Linux 2.6.12 kernel I
 > have on a box locally basically doesn't.
 
 Correct; neither to more recent Linux kernels.  2.6.12 is ancient!  :)
 
 > My view is that it's a synthetic file system with data that varies
 > dynamically at runtime, and that while it wouldn't hurt to produce
 > file size information that's correct, it's quite a bit of work to do
 > so and that I wouldn't prioritize it above other, more critical things
 > that need to happen.
 
 Most of the time, it can't be done.
 
 Some of the files in /proc have a fixed size (/proc/$$/{,db,fp}regs for
 instance), but others have highly variable content, and there is no
 other way to figure out how large it is than to generate it.  Even then,
 it may change between the stat(2) call and the read(2) call.  A good
 example is /proc/$$/status, which among other things contains a textual
 representation of the process's user and system time in seconds and
 microseconds.  A process reading its own /proc/$$/status will get
 inconsistent results.
 
 As for /proc/$$/map, the simple act of malloc()ing a buffer for it may
 change its contents if jemalloc needs to mmap() a new chunk.
 
 Some files actually *don't* have any size, such as /proc/$$/ctl, which
 is write-only.
 
 > We should certainly evaluate any patches that come in for possible
 > inclusion, assuming they don't muck up the internals of procfs too
 > much; it's not clear to me if they necessarily do so or not.
 
 I'll be glad to review and commit patches, but procfs doesn't have any
 internals to speak of, all it does is provide content for pseudofs.
 
 DES
 --=20
 Dag-Erling Sm=C3=B8rgrav - des@des.no

From: Robert Watson <rwatson@FreeBSD.org>
To: Bruce Evans <brde@optusnet.com.au>
Cc: remko@freebsd.org, freebsd-fs@freebsd.org, yuri@tsoft.com, 
    bug-followup@freebsd.org
Subject: Re: kern/120869: [procfs] 'stat' shows that all files have 0-length
 when they are actually not empty
Date: Thu, 21 Feb 2008 09:36:04 +0000 (GMT)

 On Thu, 21 Feb 2008, Bruce Evans wrote:
 
 > On Wed, 20 Feb 2008, Robert Watson wrote:
 >
 >> On Wed, 20 Feb 2008, remko@FreeBSD.org wrote:
 >> 
 >>> After a bit of discussion, this is not something which we are going to 
 >>> fix. It is not worth the hassle. If you think this should be different, we 
 >>> do welcome patches. Thanks fo rusing FreeBSD!
 >> 
 >> Just as two data points here: Solaris attempts to provide coherent file 
 >> sizes in /proc (at least to the extent that tried a few for objects where 
 >> it is remotely possible), and the Linux 2.6.12 kernel I have on a box 
 >> locally basically doesn't.
 >> 
 >> My view is that it's a synthetic file system with data that varies 
 >> dynamically at runtime, and that while it wouldn't hurt to produce file 
 >> size information that's correct, it's quite a bit of work to do so and that 
 >> I wouldn't prioritize it above other, more critical things that need to 
 >> happen. We should certainly evaluate any patches that come in for possible 
 >> inclusion, assuming they don't muck up the internals of procfs too much; 
 >> it's not clear to me if they necessarily do so or not.
 >
 > The bug is mainly that stat() claims that the files are regular when they 
 > highly irregular (they are more like fifos).  This confuses naive 
 > applications into thinking that normal access methods for regular files 
 > actually work.
 
 I feel this way more generally about synthetic file systems with objects in 
 them that don't correspond with any of the standard file system objects that 
 applications known how to deal with.
 
 Robert N M Watson
 Computer Laboratory
 University of Cambridge

From: Bruce Evans <brde@optusnet.com.au>
To: Robert Watson <rwatson@FreeBSD.org>
Cc: Bruce Evans <brde@optusnet.com.au>, remko@FreeBSD.org,
        freebsd-fs@FreeBSD.org, yuri@tsoft.com, bug-followup@FreeBSD.org
Subject: Re: kern/120869: [procfs] 'stat' shows that all files have 0-length
 when they are actually not empty
Date: Thu, 21 Feb 2008 21:11:46 +1100 (EST)

 On Thu, 21 Feb 2008, Robert Watson wrote:
 
 > On Thu, 21 Feb 2008, Bruce Evans wrote:
 >>> [about files in procfs]
 >> The bug is mainly that stat() claims that the files are regular when they 
 >> highly irregular (they are more like fifos).  This confuses naive 
 >> applications into thinking that normal access methods for regular files 
 >> actually work.
 >
 > I feel this way more generally about synthetic file systems with objects in 
 > them that don't correspond with any of the standard file system objects that 
 > applications known how to deal with.
 
 Enough to break compatibility/portabiility by adding a new file type? :-)
 S_IFMT has 4 bits, so it could encode 16 file types, but it currently only
 encodes 8.  It looks like it once had only 3 bits but was expanded for
 fifos.  It was last changed in ~1993 in 4.4BSD to add whiteouts.
 
 Bruce

From: Robert Watson <rwatson@FreeBSD.org>
To: Bruce Evans <brde@optusnet.com.au>
Cc: remko@FreeBSD.org, freebsd-fs@FreeBSD.org, yuri@tsoft.com, 
    bug-followup@FreeBSD.org
Subject: Re: kern/120869: [procfs] 'stat' shows that all files have 0-length
 when they are actually not empty
Date: Thu, 21 Feb 2008 11:34:13 +0000 (GMT)

 On Thu, 21 Feb 2008, Bruce Evans wrote:
 
 > On Thu, 21 Feb 2008, Robert Watson wrote:
 >
 >> On Thu, 21 Feb 2008, Bruce Evans wrote:
 >>>> [about files in procfs]
 >>> The bug is mainly that stat() claims that the files are regular when they 
 >>> highly irregular (they are more like fifos).  This confuses naive 
 >>> applications into thinking that normal access methods for regular files 
 >>> actually work.
 >> 
 >> I feel this way more generally about synthetic file systems with objects in 
 >> them that don't correspond with any of the standard file system objects 
 >> that applications known how to deal with.
 >
 > Enough to break compatibility/portabiility by adding a new file type? :-) 
 > S_IFMT has 4 bits, so it could encode 16 file types, but it currently only 
 > encodes 8.  It looks like it once had only 3 bits but was expanded for 
 > fifos.  It was last changed in ~1993 in 4.4BSD to add whiteouts.
 
 Not that much :-).
 
 If someone hadn't been working to fix unionfs recently, I'd suggest GCing the 
 whiteout flag so that we did, in fact, have a spare bit.
 
 That said, work to reduce dependence on procfs seems to be gradually getting 
 done.  There are a few things left, such as ps -e and gcore that could use 
 some attention.  Once nice thing about /proc/$pid/mem is that you can use it 
 asynchronously with respect to the process and in a way that is 
 non-interfering with its debugging and signal delivery.  We almost want a 
 ptrace attachment mode that specifies in advance that no execution flow 
 interference will occur, just allowing memory access, register snapshots, etc, 
 rather than an attachment necessarily getting in the signal delivery path 
 (etc).
 
 Robert N M Watson
 Computer Laboratory
 University of Cambridge

From: Bruce Evans <brde@optusnet.com.au>
To: Robert Watson <rwatson@freebsd.org>
Cc: remko@freebsd.org, freebsd-fs@freebsd.org, yuri@tsoft.com,
        bug-followup@freebsd.org
Subject: Re: kern/120869: [procfs] 'stat' shows that all files have 0-length
 when they are actually not empty
Date: Thu, 21 Feb 2008 20:24:13 +1100 (EST)

 On Wed, 20 Feb 2008, Robert Watson wrote:
 
 > On Wed, 20 Feb 2008, remko@FreeBSD.org wrote:
 >
 >> After a bit of discussion, this is not something which we are going to fix. 
 >> It is not worth the hassle. If you think this should be different, we do 
 >> welcome patches. Thanks fo rusing FreeBSD!
 >
 > Just as two data points here: Solaris attempts to provide coherent file sizes 
 > in /proc (at least to the extent that tried a few for objects where it is 
 > remotely possible), and the Linux 2.6.12 kernel I have on a box locally 
 > basically doesn't.
 >
 > My view is that it's a synthetic file system with data that varies 
 > dynamically at runtime, and that while it wouldn't hurt to produce file size 
 > information that's correct, it's quite a bit of work to do so and that I 
 > wouldn't prioritize it above other, more critical things that need to happen. 
 > We should certainly evaluate any patches that come in for possible inclusion, 
 > assuming they don't muck up the internals of procfs too much; it's not clear 
 > to me if they necessarily do so or not.
 
 The bug is mainly that stat() claims that the files are regular when
 they highly irregular (they are more like fifos).  This confuses naive
 applications into thinking that normal access methods for regular files
 actually work.
 
 Bruce
>Unformatted:
