From nobody@FreeBSD.ORG Sun Nov 14 06:35:51 1999
Return-Path: <nobody@FreeBSD.ORG>
Received: by hub.freebsd.org (Postfix, from userid 32767)
	id E1EF11530B; Sun, 14 Nov 1999 06:35:33 -0800 (PST)
Message-Id: <19991114143533.E1EF11530B@hub.freebsd.org>
Date: Sun, 14 Nov 1999 06:35:33 -0800 (PST)
From: administrator@tbv.se
Sender: nobody@FreeBSD.ORG
To: freebsd-gnats-submit@freebsd.org
Subject: corrupted file attachments when mail passes a FreeBSD v.3.1 exchanger
X-Send-Pr-Version: www-1.0

>Number:         14873
>Category:       misc
>Synopsis:       corrupted file attachments when mail passes a FreeBSD v.3.1 exchanger
>Confidential:   no
>Severity:       non-critical
>Priority:       high
>Responsible:    grog
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Sun Nov 14 06:40:01 PST 1999
>Closed-Date:    Wed Dec 22 17:57:31 PST 1999
>Last-Modified:  Wed Dec 22 18:10:03 PST 1999
>Originator:     James Wilde
>Release:        2.2.5 and 3.1
>Organization:
TBV, Stockholm, Sweden
>Environment:
3.1-RELEASE FreeBSD 3.1-RELEASE #0:
>Description:
When mail client is configured for html certain file attachments (MS
Word, MS Excel) can become corrupted when they pass BSD mail
exchangers. I have been able to create this problem when sending mail
from Swedish Telias mail servers and when bouncing mail of our new
smtp server, smtp2.tbv.se.  The old (v.2.2.5) FreeBSD firewall does
not cause this problem.

>How-To-Repeat:
Configure client (in our case MS Outlook or Outlook Express) for html
format (as opposed to plain text format).  Attach a simple MS Word or
MS Excel file to a message.  Send the message via a (recent version)
BSD mail exchanger.  >Fix: Only known fix as yet is to configure
clients for plain text which is, unfortunately, not the default in
Outlook Express.  It is possible that something was overlooked in
configuring FreeBSD 3.1 in our new smtp server.  However, the problem
can also be caused by sending mail from Swedish Telias mail server
(Telia uses a BSD variety on their mail exchangers).

>Fix:
>Release-Note:
>Audit-Trail:

From: "James A Wilde" <james.wilde@telia.com>
To: <freebsd-gnats-submit@freebsd.org>, <administrator@tbv.se>
Cc:
Subject: Re: misc/14873: corrupted file attachments when mail passes a FreeBSD v.3.1 exchanger
Date: Wed, 24 Nov 1999 23:00:14 +0100

 Hello:

 I have now investigated this further and have had my findings
 verified by an independent UNIX consultant.

 I am now convinced that the problem lies not in the mail client
 software but in the operating system.  Somewhere between versions
 2.2.5 and 3.1 FreeBSD - and possibly other varieties of BSD UNIX
 became 'broken'.  The problem is reproducible and demonstrable.  It
 has not gone away in version 3.2.

 Since I have now taken our Y2K-safe mail exchanger out of function as
 a result of this problem and don't know in which version the problem
 first arose, Y2K-safe or pre-Y2K-safe, I really need a solution
 within the next 30 days and would like to ask you to upgrade the
 problem to critical.

 I repeat that you may have access to my experiments and I can arrange
 that you relay from our mail exchangers for the purposes of testing
 for reproducibility.

 Thanks for your attention.

 mvh/regards

 James Wilde
 administrator@tbv.se
 Mail and DNS administrator
 TBV Head Office
 Stockholm, Sweden


State-Changed-From-To: open->feedback
State-Changed-By: grog
State-Changed-When: Wed Nov 24 14:41:32 PST 1999
State-Changed-Why:
Awaiting feedback from submitter.


Responsible-Changed-From-To: freebsd-bugs->grog
Responsible-Changed-By: grog
Responsible-Changed-When: Wed Nov 24 14:41:32 PST 1999
Responsible-Changed-Why:
grog is looking at this PR.
State-Changed-From-To: feedback->closed 
State-Changed-By: grog 
State-Changed-When: Wed Dec 22 17:57:31 PST 1999 
State-Changed-Why:  
Problem has been resolved (sendmail, not FreeBSD) 

From: "James A Wilde" <james.wilde@telia.com>
To: <freebsd-gnats-submit@freebsd.org>, <administrator@tbv.se>
Cc: <grog@lemis.com>
Subject: Re: misc/14873: corrupted file attachments when mail passes a FreeBSD v.3.1 exchanger
Date: Mon, 29 Nov 1999 20:17:57 +0100

 Further investigation has revealed the following:
 
 The machine which shows no trace of the problem, stofw.tbv.se, runs =
 FreeBSD 2.2.5 with which the included=20
 Sendmail is version 8.9.1.  The affected machine, smtp2.tbv.se, runs =
 FreeBSD 3.1 which has Senmail 8.9.2.
 A third machine we have tested runs FreeBSD 3.2 and Sendmail 8.9.3.
 
 Compiling Sendmail 8.9.1 on the machine running FreeBSD 3.1 produces =
 clean output.  We have yet to take the
 reverse step of compiling Sendmail 8.9.2 or above on the machine running =
 FreeBSD 2.2.5 to see whether this
 causes the problem.
 
 In that case, it would appear reasonable to assume that the version of =
 Sendmail in use on the machine is important.
 
 It would further be interesting to see which version of Sendmail is =
 being used on the two systems which are not
 affected by the problem, and which use Sun Solaris as the UNIX operating =
 system.
 
 mvh/regards
 
 James Wilde
 
 
 
 

From: Greg Lehey <grog@mojave.sitaranetworks.com>
To: James A Wilde <james.wilde@telia.com>,
	freebsd-gnats-submit@freebsd.org, administrator@tbv.se
Cc:  
Subject: Re: misc/14873: corrupted file attachments when mail passes a FreeBSD v.3.1 exchanger
Date: Mon, 29 Nov 1999 16:27:47 -0500

 [Format recovered--see http://www.lemis.com/email/email-format.html]
 
 On Monday, 29 November 1999 at 20:17:57 +0100, James A Wilde wrote:
 > Further investigation has revealed the following:
 >
 > The machine which shows no trace of the problem, stofw.tbv.se, runs
 > FreeBSD 2.2.5 with which the included Sendmail is version 8.9.1.
 > The affected machine, smtp2.tbv.se, runs FreeBSD 3.1 which has
 > Senmail 8.9.2.  A third machine we have tested runs FreeBSD 3.2 and
 > Sendmail 8.9.3.
 
 And what were the results with this last machine?
 
 > Compiling Sendmail 8.9.1 on the machine running FreeBSD 3.1 produces
 > clean output.  We have yet to take the reverse step of compiling
 > Sendmail 8.9.2 or above on the machine running FreeBSD 2.2.5 to see
 > whether this causes the problem.
 >
 > In that case, it would appear reasonable to assume that the version
 > of Sendmail in use on the machine is important.
 
 Yes, looking at the stuff you sent me, it seemed so.  I haven't had
 time for a thorough examination, but I note that at one point the text
 // at the beginning of a line grows a prepended 'file:'.  Since this
 is an encoded file, the text // is random, and the 'file:' undoubtedly
 causes corruption.  I don't know if this is the only problem, though,
 but considering that sendmail does autoconversion, it's possible that
 this is a bug in sendmail.
 
 > It would further be interesting to see which version of Sendmail is
 > being used on the two systems which are not affected by the problem,
 > and which use Sun Solaris as the UNIX operating system.
 
 Indeed.  Keep me informed.
 
 Greg
 --
 When replying to this message, please take care not to mutilate the
 original text.
 For more information, see http://www.lemis.com/email.html
 Finger grog@lemis.com for PGP public key
 See complete headers for address and phone numbers
 
>Unformatted:

Greg Lehey, 24 November 1999

   Cleaned up problem report format.  The original message was in
   one-line-per-paragraph mode, the second in untranslated quoted
   printable.  Severity set to critical as per submitter request.

   You write:

   > Only known fix as yet is to configure clients for plain text
   > which is, unfortunately, not the default in Outlook Express.

   This puzzles me.  You can't attach MS Word or Excel files either as
   text/html or text/plain attachements; they *should* be
   application/msword or application/<mumble>.  They should not be
   application/octet-stream.

   You'll appreciate that we don't use Microsoft mailers, particularly
   due to their propensity to corrupt messages.  Could you please send
   me three copies of a short message which illustrates the problem:

   1.  To me directly at grog@lemis.com.
   2.  To me via a route which causes the corruption.
   3.  A locally saved copy, including all headers, for comparison.
       Send this from a FreeBSD machine.

Greg Lehey, 23 December 1999

From private mail:

> Sorry to keep you waiting here, James.  I've verified that something
> funny is going on, and it's almost certainly related to the version of
> sendmail, not the version of FreeBSD.  We're certainly interested in
> keeping sendmail working under FreeBSD, but you might find that the
> people at sendmail are as well, even if you're not a paying customer.

Correct!  I sent this to the people at sendmail.org and received a somewhat
gruff response.  It is a known bug, skilfully disguised as the 'illegal'
line length bug although this example is nothing to do with the line length.
There is, quite simply a line missing in the corrupted files:

Content-Transfer-Encoding: base64

Insert that line at the appropriate place in the corrupt message and the
file immediately becomes readable.  Take it out again and the file becomes
corrupt.
This apparently surfaced in sendmail 8.9.2.  Our old server with FreeBSD
2.2.5 had sendmail 8.9.1 and hence is okay.  Our new server, the one with
FreeBSD 3.1, had sendmail 8.9.2 as the accompanying package, and the third
server, the one mentioned at the top of this message, has FreeBSD 3.2 with
accompanying sendmail 8.9.3.

Sendmail have a patch for this at:

http://www.sendmail.org/~ca/email/smpatches.html

We have installed sendmail 8.9.1 on the v 3.1 server and taken it back into
production, and my colleague is experimenting with the patch on the v 3.2
server.
