From schweikh@ito.uni-stuttgart.de  Thu Jan  4 07:48:29 1996
Received: from ito.uni-stuttgart.de (hoesun.ito.uni-stuttgart.de [129.69.65.20])
          by freefall.freebsd.org (8.7.3/8.7.3) with SMTP id HAA11870
          for <FreeBSD-gnats-submit@freebsd.org>; Thu, 4 Jan 1996 07:48:17 -0800 (PST)
Received: from itosun.ito.uni-stuttgart.de by ito.uni-stuttgart.de (5.x/SMI-SVR4/BelWue-2.0)
	id AA15748; Thu, 4 Jan 1996 16:47:43 +0100
Received: by itosun.ito.uni-stuttgart.de (5.x/SVR4/BelWue-1.0.3)
	id AA14616; Thu, 4 Jan 1996 16:47:39 +0100
Message-Id: <9601041547.AA14616@itosun.ito.uni-stuttgart.de>
Date: Thu, 4 Jan 1996 16:47:39 +0100
From: schweikh@ito.uni-stuttgart.de
Reply-To: schweikh@ito.uni-stuttgart.de
To: FreeBSD-gnats-submit@freebsd.org
Subject: fmt strips 8bit characters (bad)
X-Send-Pr-Version: 3.2

>Number:         931
>Category:       bin
>Synopsis:       fmt strips 8bit characters
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    freebsd-bugs
>State:          closed
>Quarter:
>Keywords:
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Thu Jan  4 07:50:02 PST 1996
>Closed-Date:    Sat Jan 6 16:04:32 MET 1996
>Last-Modified:  Sat Jan  6 16:05:58 MET 1996
>Originator:     Jens Schweikhardt
>Release:        FreeBSD 2.1-STABLE i386
>Organization:
uni-stuttgart
>Environment:

	FreeBSD 2.1.0-RELEASE

>Description:

	fmt strips any characters for which isprint(3) returns false.
	A Bad Thing (TM) for those of us using iso-latin character sets.

>How-To-Repeat:

	% echo -e '\377' | fmt   # in bash
	%

>Fix:

Here is a patch which adds an option to fmt to make it 8 bit clean.
Currently fmt will strip all characters for which isprint returns
false - a Bad Thing (TM) for those of us using iso-latin character
sets.

The patch adds the -8 option, which tells fmt not to perform the
isprint test. The default behaviour remains unchanged, so no
existing script will be surprised. It also adds a few words to
the man page.

Please have a look at it and if you think it's ok, commit it.

Bye, Jens


--- fmt.c	Sun Dec 24 17:41:12 1995
+++ fmt.c-new	Mon Dec 25 16:04:38 1995
@@ -59,10 +59,12 @@
 #define	NOSTR	((char *) 0)	/* Null string pointer for lint */
 
 /* LIZ@UOM 6/18/85 --New variables goal_length and max_length */
+/* Jens Schweikhardt 12/25/95 --New variable pass_8_bits */
 #define GOAL_LENGTH 65
 #define MAX_LENGTH 75
 int	goal_length;		/* Target or goal line length in output */
 int	max_length;		/* Max line length in output */
+int	pass_8_bits;		/* Wheter nonprintable chars are removed */
 int	pfx;			/* Current leading blank count */
 int	lineno;			/* Current input line */
 int	mark;			/* Last place we saw a head line */
@@ -90,6 +92,18 @@
 	lineno = 1;
 	mark = -10;
 	/*
+	 * Jens Schweikhardt 12/25/95 -- Test for -8 option. The whole stuff
+	 * should actually be rewritten using getopt().
+	 * But I hesitate because it would break current scripts if we required
+	 * them to use option characters to specify goal and max lengths.
+	 */
+	pass_8_bits = 0;
+	if (argc > 1 && (strcmp (argv[1], "-8") == 0)) {
+		argv++;
+		argc--;
+		pass_8_bits = 1;
+	}
+	/*
 	 * LIZ@UOM 6/18/85 -- Check for goal and max length arguments
 	 */
 	if (argc > 1 && (1 == (sscanf(argv[1], "%d", &number)))) {
@@ -103,8 +117,8 @@
 		}
 	}
 	if (max_length <= goal_length) {
-		fprintf(stderr, "Max length must be greater than %s\n",
-			"goal length");
+		fprintf(stderr, "Max length must be greater than"
+		" goal length\n");
 		exit(1);
 	}
 	if (argc < 2) {
@@ -115,7 +129,7 @@
 	while (--argc) {
 		if ((fi = fopen(*++argv, "r")) == NULL) {
 			perror(*argv);
-			errs++;
+			errs = 1;
 			continue;
 		}
 		fmt(fi);
@@ -151,9 +165,11 @@
 				c = getc(fi);
 				continue;
 			}
-			if (!isprint(c) && c != '\t') {
-				c = getc(fi);
-				continue;
+			if (!pass_8_bits) {
+				if (!isprint(c) && c != '\t') {
+					c = getc(fi);
+					continue;
+				}
 			}
 			*cp++ = c;
 			c = getc(fi);
--- fmt.1	Mon Dec 25 13:52:04 1995
+++ fmt.1-new	Mon Dec 25 14:21:46 1995
@@ -39,6 +39,7 @@
 .Nd simple text formatter
 .Sh SYNOPSIS
 .Nm fmt
+.Op Fl 8
 .Oo
 .Ar goal
 .Op Ar maximum
@@ -72,6 +73,12 @@
 .Pp
 will reformat a paragraph,
 evening the lines.
+.Pp
+The options are as follows:
+.Bl -tag -width Ds
+.It Fl 8
+Do not strip the input from non-printable characters.
+If this option is not specified, isprint(3) determines what gets stripped.
 .Sh SEE ALSO
 .Xr nroff 1 ,
 .Xr mail 1
>Release-Note:
>Audit-Trail:

From: J Wunsch <j@uriah.heep.sax.de>
To: schweikh@ito.uni-stuttgart.de
Cc: FreeBSD-gnats-submit@freebsd.org
Subject: Re: bin/931: fmt strips 8bit characters (bad)
Date: Thu, 4 Jan 1996 20:42:54 +0100 (MET)

 As schweikh@ito.uni-stuttgart.de wrote:
 > 
 > 	fmt strips any characters for which isprint(3) returns false.
 > 	A Bad Thing (TM) for those of us using iso-latin character sets.
 
 Your fix is wrong.  The correct fix was to use setlocale(), and it's
 already present in -current.
 
 For 2.0.5 (and presumably 2.1, too), ENABLE_STARTUP_LOCALE should have
 the same effect.  (Of course, you need to set your LANG variable to
 something that's supporting ISO-8859-1.)
 
 -- 
 cheers, J"org
 
 joerg_wunsch@uriah.heep.sax.de -- http://www.sax.de/~joerg/ -- NIC: JW11-RIPE
 Never trust an operating system you don't have sources for. ;-)
State-Changed-From-To: open->closed 
State-Changed-By: joerg 
State-Changed-When: Sat Jan 6 16:04:32 MET 1996 
State-Changed-Why:  
This was an internationalization problem, and it has already been fixed 
by Andrey's big setlocale() action. 

>Unformatted:
