From nobody@FreeBSD.ORG  Wed Oct  4 16:28:25 2000
Return-Path: <nobody@FreeBSD.ORG>
Received: by hub.freebsd.org (Postfix, from userid 32767)
	id F064137B66F; Wed,  4 Oct 2000 16:28:24 -0700 (PDT)
Message-Id: <20001004232824.F064137B66F@hub.freebsd.org>
Date: Wed,  4 Oct 2000 16:28:24 -0700 (PDT)
From: reed@reedmedia.net
Sender: nobody@FreeBSD.ORG
To: freebsd-gnats-submit@FreeBSD.org
Subject: mailing list archiving or retrieval broken from "\nFrom lines"
X-Send-Pr-Version: www-1.0

>Number:         21762
>Category:       docs
>Synopsis:       mailing list archiving or retrieval broken from "\nFrom lines"
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    freebsd-doc
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          doc-bug
>Submitter-Id:   current-users
>Arrival-Date:   Wed Oct 04 16:30:00 PDT 2000
>Closed-Date:    Sun Oct 29 08:09:49 PST 2000
>Last-Modified:  Sun Oct 29 08:10:55 PST 2000
>Originator:     Jeremy C. Reed
>Release:        NA
>Organization:
>Environment:
NA
>Description:
The freebsd-questions archiving or retrieval routine is broken
if a blank line is followed by "From and a space". The mail is sent okay;
it is just not shown in its entirety via the getmsg.cgi routine.

Here was my test message:

http://docs.freebsd.org/cgi/getmsg.cgi?fetch=1319586+0+current/freebsd-questions

Since it doesn't all exists... here is the rest of it.
>How-To-Repeat:
Send an email to a freebsd mailing list that has a blank line followed
by a line starting with "From " (and followed by a space). And then check the
archive.
>Fix:
I don't know. I have not looked at the getmsg.cgi routine.

>Release-Note:
>Audit-Trail:

From: Nik Clayton <nik@freebsd.org>
To: reed@reedmedia.net
Cc: freebsd-gnats-submit@FreeBSD.org
Subject: Re: docs/21762: mailing list archiving or retrieval broken from "\nFrom lines"
Date: Sat, 28 Oct 2000 23:55:10 +0100

 On Wed, Oct 04, 2000 at 04:28:24PM -0700, reed@reedmedia.net wrote:
 > Here was my test message:
 > 
 > http://docs.freebsd.org/cgi/getmsg.cgi?fetch=1319586+0+current/freebsd-questions
 > 
 
 Looks OK to me.  What am I missing?
 
 N
 -- 
 Internet connection, $19.95 a month.  Computer, $799.95.  Modem, $149.95.
 Telephone line, $24.95 a month.  Software, free.  USENET transmission,
 hundreds if not thousands of dollars.  Thinking before posting, priceless.
 Somethings in life you can't buy.  For everything else, there's MasterCard.
   -- Graham Reed, in the Scary Devil Monastery
 

From: "Jeremy C. Reed" <reed@reedmedia.net>
To: Nik Clayton <nik@freebsd.org>
Cc: freebsd-gnats-submit@FreeBSD.org
Subject: Re: docs/21762: mailing list archiving or retrieval broken from
 "\nFrom lines"
Date: Sat, 28 Oct 2000 19:27:13 -0700 (PDT)

 On Sat, 28 Oct 2000, Nik Clayton wrote:
 
 > On Wed, Oct 04, 2000 at 04:28:24PM -0700, reed@reedmedia.net wrote:
 > > Here was my test message:
 > > 
 > > http://docs.freebsd.org/cgi/getmsg.cgi?fetch=1319586+0+current/freebsd-questions
 > > 
 > 
 > Looks OK to me.  What am I missing?
 
 The archive is missing the rest of my message. This wasn't a one time
 test -- I noticed this a few times before, then I figured out the problem
 and then I tested it to verify it.
 
 It is not okay. Please read the message at
 http://docs.freebsd.org/cgi/getmsg.cgi?fetch=1319586+0+archive/2000/freebsd-questions/20001008.freebsd-questions
 
 My last line says "And the next line will be blank, followed by "From"
 with a space after it." But then NOTHING is after it. Also my signature is
 gone. This was explained in my bug report.
 
 If you notice, it has problems with the common unix mailbox message
 separator "\nFrom ".
 
 The email is sent correctly (with the next lines and with my signature),
 just it appears that the archiving is broken. (As explained in my email
 and in the bug report # 21762).
 
 This bug is not critical. But if someone writes an email that has a blank
 line and then the next line starts with "From " (followed by a space),
 then the web-based archived message will not contain the remainder of the
 email. (I am not sure if the message is still available -- maybe just the
 getmsg.cgi tool is broken.) This little problem could make it hard for
 someone to research and utilize some archived messages.
 
 Hope this explains this. I'll be glad to help more. Of course, I have not
 tested this since Oct. 4.
 
 I guess I should just find the problem myself. When I have time, I'll
 search through the cvsweb to find the source to the getmsg.cgi program to
 see if I can find the issue. But maybe the problem is below or before that
 -- maybe the problem is with the tool (mailindex 1.0 ?) that archives the
 message.
 
    Jeremy C. Reed
    http://www.reedmedia.net/
 
 
 
 

From: Nik Clayton <nik@freebsd.org>
To: "Jeremy C. Reed" <reed@reedmedia.net>, postmaster@freebsd.org
Cc: Nik Clayton <nik@freebsd.org>, freebsd-gnats-submit@FreeBSD.org
Subject: Re: docs/21762: mailing list archiving or retrieval broken from "\nFrom lines"
Date: Sun, 29 Oct 2000 14:37:34 +0000

 I'm copying this to the Postmaster, 'cos I'm not sure how our mail archiving
 works.
 
 On Sat, Oct 28, 2000 at 07:27:13PM -0700, Jeremy C. Reed wrote:
 > On Sat, 28 Oct 2000, Nik Clayton wrote:
 > > On Wed, Oct 04, 2000 at 04:28:24PM -0700, reed@reedmedia.net wrote:
 > > > Here was my test message:
 > > > 
 > > > http://docs.freebsd.org/cgi/getmsg.cgi?fetch=1319586+0+current/freebsd-questions
 > > > 
 > > 
 > > Looks OK to me.  What am I missing?
 > 
 > The archive is missing the rest of my message. This wasn't a one time
 > test -- I noticed this a few times before, then I figured out the problem
 > and then I tested it to verify it.
 > 
 > It is not okay. Please read the message at
 > http://docs.freebsd.org/cgi/getmsg.cgi?fetch=1319586+0+archive/2000/freebsd-questions/20001008.freebsd-questions
 > 
 > My last line says "And the next line will be blank, followed by "From"
 > with a space after it." But then NOTHING is after it. Also my signature is
 > gone. This was explained in my bug report.
 > 
 > If you notice, it has problems with the common unix mailbox message
 > separator "\nFrom ".
 
 Ah.  Not exactly.  What's supposed to happen is that "\nFrom" in the body 
 of an e-mail should be replaced by "\n>From" for exactly this reason.
 
 However, it looks like this isn't happening when messages are archived.
 
 Postmaster, you can see this in
 
 freefall:/usr/local/www/db/text/2000/freebsd-questions/2001008.freebsd-questions
 
 just search for the message containing "will be blank" to see the unescaped
 "From " at the beginning of the line.
 
 getmsg.cgi (rightly, IMHO) assumes that \nFrom starts a new message.  It 
 could be hacked to work around this, but there will always be corner cases.
 
 If you can point me at the code that puts incoming e-mail in to the archive
 then I'll see if I can fix it.
 
 Thanks,
 
 N
 -- 
 Internet connection, $19.95 a month.  Computer, $799.95.  Modem, $149.95.
 Telephone line, $24.95 a month.  Software, free.  USENET transmission,
 hundreds if not thousands of dollars.  Thinking before posting, priceless.
 Somethings in life you can't buy.  For everything else, there's MasterCard.
   -- Graham Reed, in the Scary Devil Monastery
 
State-Changed-From-To: open->closed 
State-Changed-By: nik 
State-Changed-When: Sun Oct 29 08:09:49 PST 2000 
State-Changed-Why:  
jmb's hacked getmsg.cgi to look for "^From.*d{4}$", which should at 
least be more reliable (albeit not 100% foolproof). 

http://www.freebsd.org/cgi/query-pr.cgi?pr=21762 
>Unformatted:
 >From reed
 
 I believe that the archive will not reach this far down in the email.
 Check out http://docs.freebsd.org/mail/current/freebsd-questions.html to
 see.
 
    Jeremy C. Reed
    http://www.reedmedia.net/
