From hlh@cocoon.cercle.be  Sun Feb  1 05:16:34 2004
Return-Path: <hlh@cocoon.cercle.be>
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id 36EF816A4D9
	for <FreeBSD-gnats-submit@freebsd.org>; Sun,  1 Feb 2004 05:16:34 -0800 (PST)
Received: from cocoon.cercle.be (ppp-62-235-117-60.tiscali.be [62.235.117.60])
	by mx1.FreeBSD.org (Postfix) with ESMTP id 8AE5643D66
	for <FreeBSD-gnats-submit@freebsd.org>; Sun,  1 Feb 2004 05:15:53 -0800 (PST)
	(envelope-from hlh@cocoon.cercle.be)
Received: from norquay.cocoon.bel (localhost.cocoon.bel [127.0.0.1])
	by cocoon.cercle.be (8.12.11/8.12.11) with ESMTP id i11DEOFQ021979
	for <FreeBSD-gnats-submit@freebsd.org>; Sun, 1 Feb 2004 14:14:25 +0100 (CET)
Received: (from root@localhost)
	by norquay.cocoon.bel (8.12.11/8.12.11/Submit) id i11D5VuL021659;
	Sun, 1 Feb 2004 14:05:31 +0100 (CET)
Message-Id: <200402011305.i11D5VuL021659@norquay.cocoon.bel>
Date: Sun, 1 Feb 2004 14:05:31 +0100 (CET)
From: Henri Hennebert <hlh@cocoon.cercle.be>
Reply-To: Henri Hennebert <hlh@cocoon.cercle.be>
To: FreeBSD-gnats-submit@freebsd.org
Cc:
Subject: perl syswrite not writing the buffer under (heavy) disk load
X-Send-Pr-Version: 3.113
X-GNATS-Notify:

>Number:         62216
>Category:       kern
>Synopsis:       perl syswrite not writing the buffer under (heavy) disk load
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    remko
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Sun Feb 01 05:20:19 PST 2004
>Closed-Date:    Sun Mar 11 12:39:43 GMT 2007
>Last-Modified:  Sun Mar 11 12:39:43 GMT 2007
>Originator:     Henri Hennebert
>Release:        FreeBSD 5.2-RELEASE i386
>Organization:
CIGER sa
>Environment:
System: FreeBSD norquay.cocoon.bel 5.2-RELEASE FreeBSD 5.2-RELEASE #0: Sun Jan 11 15:03:32 CET 2004 hlh@norquay.cocoon.bel:/usr/obj/usr/src/sys/NORQUAY i386

Summary of my perl5 (revision 5.0 version 8 subversion 2) configuration:
  Platform:
    osname=freebsd, osvers=5.2-rc, archname=i386-freebsd
    uname='freebsd norquay.cocoon.bel 5.2-rc freebsd 5.2-rc #0: thu dec 11 08:00:25 cet 2003 hlh@norquay.cocoon.bel:usrobjusrsrcsysnorquay i386 '
    config_args='-sde -Dprefix=/usr/local -Darchlib=/usr/local/lib/perl5/5.8.2/mach -Dprivlib=/usr/local/lib/perl5/5.8.2 -Dman3dir=/usr/local/lib/perl5/5.8.2/man/man3 -Dman1dir=/usr/local/man/man1 -Dsitearch=/usr/local/lib/perl5/site_perl/5.8.2/mach -Dsitelib=/usr/local/lib/perl5/site_perl/5.8.2 -Dscriptdir=/usr/local/bin -Ui_malloc -Ui_iconv -Uinstallusrbinperl -Dcc=cc -Doptimize=-O -pipe -march=pentium3 -Duseshrplib -Dccflags=-DAPPLLIB_EXP="/usr/local/lib/perl5/5.8.2/BSDPAN" -Ud_dosuid -Ui_gdbm -Dusethreads=n -Dusemymalloc=y'
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=undef use5005threads=undef useithreads=undef usemultiplicity=undef
    useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=undef use64bitall=undef uselongdouble=undef
    usemymalloc=y, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-DAPPLLIB_EXP="/usr/local/lib/perl5/5.8.2/BSDPAN" -DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing -I/usr/local/include',
    optimize='-O -pipe -march=pentium3',
    cppflags='-DAPPLLIB_EXP="/usr/local/lib/perl5/5.8.2/BSDPAN" -DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing -I/usr/local/include'
    ccversion='', gccversion='3.3.3 [FreeBSD] 20031106', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=4, prototype=define
  Linker and Libraries:
    ld='cc', ldflags ='-Wl,-E  -L/usr/local/lib'
    libpth=/usr/lib /usr/local/lib
    libs=-lbind -lgdbm -lm -lcrypt -lutil -lc
    perllibs=-lbind -lm -lcrypt -lutil -lc
    libc=, so=so, useshrplib=true, libperl=libperl.so
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='  -Wl,-R/usr/local/lib/perl5/5.8.2/mach/CORE'
    cccdlflags='-DPIC -fPIC', lddlflags='-shared  -L/usr/local/lib'


Characteristics of this binary (from libperl):
  Compile-time options: USE_LARGE_FILES
  Built under freebsd
  Compiled at Dec 20 2003 13:33:01
  @INC:
    /usr/local/lib/perl5/site_perl/5.8.2/mach
    /usr/local/lib/perl5/site_perl/5.8.2
    /usr/local/lib/perl5/site_perl
    /usr/local/lib/perl5/5.8.2/BSDPAN
    /usr/local/lib/perl5/5.8.2/mach
    /usr/local/lib/perl5/5.8.2
    .

>Description:
When I install p5-Mail-Spamassassin, {site-perl}/Mail/Spamassassin.pm
and modules in {site-perl}/Mail/Spamassassin/ are truncated to 16384 bytes!
strange isn't it :-o . 

After digging I trace the culprit in
/usr/local/lib/perl5/5.8.2/ExtUtils/Install.pm. 

In fuction run_filter, I add 2 lines of debugging:

sub run_filter {
    my ($cmd, $src, $dest) = @_;
    local(*CMD, *SRC);
    open(CMD, "|$cmd >$dest") || die "Cannot fork: $!";
    open(SRC, $src)           || die "Cannot open $src: $!";
    my $buf;
    my $sz = 1024;
    while (my $len = sysread(SRC, $buf, $sz)) {
        print STDERR "$len=";        //DEBUG - print length read
        my $len_w = syswrite(CMD, $buf, $len);
        print STDERR "$len_w ";      //DEBUG - print length written
    }
    close SRC;
    close CMD or die "Filter command '$cmd' failed for $src";
}

If I run make
(in /usr/ports/mail/p5-Mail-SpamAssassin/work/Mail-SpamAssassin-2.61)
on a quiet machine, all is running well. My trace give me something like:

1024=1024 1024=1024 ... 390=390

Without the 2 `print STDERR', the files are truncated if make is run in
the same quiet condition. So the print modify the experiment (some
quantum theory here).

If I run make (after `make clean' and `perl Makefile.PL') during
a large copy from disk to disk, the trace give me:

1024=1024 10240=1024 ... 1024=1024 1024= 1024= 1024= ... 390= 
                                        ^     ^     ^        ^  
I conclude that syswrite do nothing, and don't return 0 as expected in
this case (maybe undef)!

For the record, I encounter this same problem in 5.1 some time ago but
don't study the problem at the time.

I make some google and mailing list searches and can't find any trace of
this problem elsewhere??? Maybe I am in a fold of spacetime ?

Any light is welcome.

Henri

>How-To-Repeat:
 
 See above.

>Fix:



>Release-Note:
>Audit-Trail:

From: Ian Dowse <iedowse@maths.tcd.ie>
To: freebsd-gnats-submit@freebsd.org
Cc:  
Subject: Re: kern/62216: perl syswrite not writing the buffer under (heavy) disk load 
Date: Fri, 06 Feb 2004 09:37:35 +0000

 Adding to the audit trail - it seems that EAGAIN/EWOULDBLOCK is
 being returned by a normal blocking write:
 
 In message <40234A7B.3090103@ciger.be>, Henri Hennebert writes:
 >Ian Dowse wrote:
 > > In message <200402011305.i11D5VuL021659@norquay.cocoon.bel>, Henri 
 >Hennebert wr
 > > ites:
 > >
 > >>   while (my $len = sysread(SRC, $buf, $sz)) {
 > >>       print STDERR "$len=";        //DEBUG - print length read
 > >>       my $len_w = syswrite(CMD, $buf, $len);
 > >>       print STDERR "$len_w ";      //DEBUG - print length written
 > >>   }
 > >
 > >
 > > syswrite returns the number of bytes written, or undef if the write
 > > failed, so a loop like the above needs to check the return value
 > 
 >I agree but how does it work so long for so many ???
 > 
 > > and handle it appropriately. What happens if you change it to
 > > something like this?
 > >
 > >       while (my $len = sysread(SRC, $buf, $sz)) {
 > >               my $wlen = 0;
 > >               while ($wlen < $len) {
 > >                       my $len_w = syswrite(CMD, $buf, $len - $wlen, 
 >$wlen);
 > >                       die "$cmd: $!\n" unless defined($len_w);
 > >                       $wlen += $len_w;
 > >               }
 > >       }
 > >
 > > It's the "die .. $!" that will help to debug the problem, since it
 > > will print out the error message associated with the write failure.
 > >
 >Here is the result of make :
 >===>  Configuring for p5-Mail-SpamAssassin-2.61_1
 >Checking if your kit is complete...
 Warning: the following files are missing in your kit:
 >        rules/local.cf
 >Please inform the author.
 >Writing Makefile for Mail::SpamAssassin
 >Makefile written by ExtUtils::MakeMaker 6.17
 >===>  Building for p5-Mail-SpamAssassin-2.61_1
 >/usr/local/bin/perl5.8.2 build/preprocessor -Mconditional -Mbytes
 >-DPERL_VERSION="5.8.2" -Mvars -DVERSION="2.61" -DPREFIX="/usr/local"
 ><lib/Mail/SpamAssassin/AutoWhitelist.pm 
 > >blib/lib/Mail/SpamAssassin/AutoWhitelist.pm
 >/usr/local/bin/perl5.8.2 build/preprocessor -Mconditional -Mbytes
 >-DPERL_VERSION="5.8.2" -Mvars -DVERSION="2.61" -DPREFIX="/usr/local": 
 >Resource
 >temporarily unavailable
 >*** Error code 35
 > 
 >Stop in
 >/var/spool/ports/mail/p5-Mail-SpamAssassin/work/Mail-SpamAssassin-2.61.
 >*** Error code 1
 > 
 >Stop in /var/spool/ports/mail/p5-Mail-SpamAssassin.
 > 
 > 
 >Henri
 > 
 >
 >
 >
State-Changed-From-To: open->feedback 
State-Changed-By: remko 
State-Changed-When: Sun Mar 11 12:37:17 UTC 2007 
State-Changed-Why:  
So, is this still the case on more recent FreeBSD versions? 
I have to admit that I do not see why this is a bug within 
FreeBSD, please feel free to elaborate this! 


Responsible-Changed-From-To: freebsd-bugs->remko 
Responsible-Changed-By: remko 
Responsible-Changed-When: Sun Mar 11 12:37:17 UTC 2007 
Responsible-Changed-Why:  
grab the pr 

http://www.freebsd.org/cgi/query-pr.cgi?pr=62216 
State-Changed-From-To: feedback->closed 
State-Changed-By: remko 
State-Changed-When: Sun Mar 11 12:39:42 UTC 2007 
State-Changed-Why:  
The email address is no longer reachable, close the ticket. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=62216 
>Unformatted:
