From nobody@FreeBSD.org  Tue Jan 19 18:42:55 2010
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 1BC8D1065670
	for <freebsd-gnats-submit@FreeBSD.org>; Tue, 19 Jan 2010 18:42:55 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (www.freebsd.org [IPv6:2001:4f8:fff6::21])
	by mx1.freebsd.org (Postfix) with ESMTP id 0AE758FC0A
	for <freebsd-gnats-submit@FreeBSD.org>; Tue, 19 Jan 2010 18:42:55 +0000 (UTC)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by www.freebsd.org (8.14.3/8.14.3) with ESMTP id o0JIgsBP094013
	for <freebsd-gnats-submit@FreeBSD.org>; Tue, 19 Jan 2010 18:42:54 GMT
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.14.3/8.14.3/Submit) id o0JIgseY094012;
	Tue, 19 Jan 2010 18:42:54 GMT
	(envelope-from nobody)
Message-Id: <201001191842.o0JIgseY094012@www.freebsd.org>
Date: Tue, 19 Jan 2010 18:42:54 GMT
From: vermaden <vermaden@interia.pl>
To: freebsd-gnats-submit@FreeBSD.org
Subject: /dev/random can not be source for tr(1) | tr: Illegal byte sequence
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         142986
>Category:       bin
>Synopsis:       /dev/random can not be source for tr(1) | tr: Illegal byte sequence
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    freebsd-bugs
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Tue Jan 19 18:50:05 UTC 2010
>Closed-Date:    Wed Jan 20 22:47:12 UTC 2010
>Last-Modified:  Wed Jan 20 22:47:12 UTC 2010
>Originator:     vermaden
>Release:        8.0-RELEASE i386
>Organization:
>Environment:
Stock i386 8.0-RELEASE output.
>Description:
While I can use /dev/random for tr(1) source (password generation for example) on Linux (CentO@ 5.4 to be precise) and also on gtr(1) from coreutils port, the BSD tr(1) does not want to cooperate with /dev/random output, as below:

FreeBSD% head -c 200 /dev/random | tr -d -c 'A-Za-z0-9'
tr: Illegal byte sequence

FreeBSD% head -c 200 /dev/random| gtr -d -c 'A-Za-z0-9'
x9SMkHI4INJOwwAHhHocHElFdrVJOEMYxgtcVndR

CentOS$ head -c 200 /dev/random | tr -d -c 'A-Za-z0-9'
KMLqkAymFbFlpXPNrs5PaQ6McrHW3xzeBo9zm3aaL


>How-To-Repeat:
FreeBSD% head -c 200 /dev/random | tr -d -c 'A-Za-z0-9'
tr: Illegal byte sequence

FreeBSD% head -c 200 /dev/random| gtr -d -c 'A-Za-z0-9'
x9SMkHI4INJOwwAHhHocHElFdrVJOEMYxgtcVndR

CentOS$ head -c 200 /dev/random | tr -d -c 'A-Za-z0-9'
KMLqkAymFbFlpXPNrs5PaQ6McrHW3xzeBo9zm3aaL
>Fix:


>Release-Note:
>Audit-Trail:

From: Yuri Pankov <yuri.pankov@gmail.com>
To: vermaden <vermaden@interia.pl>
Cc: bug-followup@freebsd.org
Subject: Re: bin/142986: /dev/random can not be source for tr(1) | tr:
 Illegal byte sequence
Date: Tue, 19 Jan 2010 23:51:00 +0300

 /dev/random just isn't guaranteed to produce correct multibyte characters
 (and I guess you are using multibyte locale). Try setting LC_CTYPE to
 single byte locale:
 > locale
 LANG=en_US.UTF-8
 LC_CTYPE="en_US.UTF-8"
 LC_COLLATE="en_US.UTF-8"
 LC_TIME="en_US.UTF-8"
 LC_NUMERIC="en_US.UTF-8"
 LC_MONETARY="en_US.UTF-8"
 LC_MESSAGES="en_US.UTF-8"
 LC_ALL=
 > head -c 200 /dev/urandom | tr -c -d 'A-Za-z0-9'
 tr: Illegal byte sequence
 > head -c 200 /dev/urandom | env LC_CTYPE=C tr -c -d 'A-Za-z0-9'
 myRUBav1EHfJrD0HCSuurHRPYmqMPzWErJhHRgp6ze0YisDoA4x
State-Changed-From-To: open->closed 
State-Changed-By: linimon 
State-Changed-When: Wed Jan 20 22:46:52 UTC 2010 
State-Changed-Why:  
From misfiled PR bin/142986: 

Date: 20 Jan 2010 20:46:31 +0100 
From: vermaden <vermaden@interia.pl> 
Sender: vermaden@interia.pl 
To: Yuri Pankov <yuri.pankov@gmail.com> 
Subject: =?UTF-8?q?Re:_bin/142986:_/dev/random_can_not_be_source_for_tr(1)_|_tr:_Illegal_byte_sequence?= 

Thanks you, locale was the problem, with 
LC_ALL/LC_CTYPE works as desired, You 
should close this PR as solved/unneded. 

Regards, 
vermaden 

http://www.freebsd.org/cgi/query-pr.cgi?pr=142986 
>Unformatted:
