From nobody@FreeBSD.org  Mon Apr 16 18:45:01 2007
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52])
	by hub.freebsd.org (Postfix) with ESMTP id 27CA816A401
	for <freebsd-gnats-submit@FreeBSD.org>; Mon, 16 Apr 2007 18:45:01 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (www.freebsd.org [69.147.83.33])
	by mx1.freebsd.org (Postfix) with ESMTP id 1652E13C455
	for <freebsd-gnats-submit@FreeBSD.org>; Mon, 16 Apr 2007 18:45:01 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by www.freebsd.org (8.13.1/8.13.1) with ESMTP id l3GIj0Ww094195
	for <freebsd-gnats-submit@FreeBSD.org>; Mon, 16 Apr 2007 18:45:00 GMT
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.13.1/8.13.1/Submit) id l3GIdwTn092958;
	Mon, 16 Apr 2007 18:39:59 GMT
	(envelope-from nobody)
Message-Id: <200704161839.l3GIdwTn092958@www.freebsd.org>
Date: Mon, 16 Apr 2007 18:39:59 GMT
From: Zhang Weiwu<zhangweiwu@realss.com>
To: freebsd-gnats-submit@FreeBSD.org
Subject: request ftpd conform to RFC 2640
X-Send-Pr-Version: www-3.0

>Number:         111714
>Category:       bin
>Synopsis:       [patch] request ftpd(8) conform to RFC 2640
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    yar
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          update
>Submitter-Id:   current-users
>Arrival-Date:   Mon Apr 16 18:50:01 GMT 2007
>Closed-Date:    Tue May 01 12:20:51 GMT 2007
>Last-Modified:  Tue May 01 12:20:51 GMT 2007
>Originator:     Zhang Weiwu
>Release:        FreeBSD 6.1 Stable RELEASE
>Organization:
Real Softservice
>Environment:
FreeBSD exupery.realss.com 6.1-RELEASE FreeBSD 6.1-RELEASE #1: Sun Dec  3 00:54:09 CST 2006     root@hatch.realss.com:/usr/obj/usr/src/sys/HATCH  i386

>Description:
Compare to those who only use English for a life time, probably we Asian
users feel much more painful for character-set issue. To Asian users the
UTF8 compliance as defined in RFC2640 is very important. As FreeBSD is
an international project, UTF8 and related RFC compliance is AFAIK basic
requirement. The attached path added an option -8 to ftpd, when used, it
indicate the file system on server is using UTF-8 and the FTP client
(e.g. smart ftp, filezilla, lftp, gftp ...) should convert the filenames
and path names to local character set (which they do fine). FTP server
tell client to do character conversion if necessary by including "UTF8"
in FEAT.

RFC2640 is the only way to make international ftp server: it is the only
standard way to deliver non-latin filenames to different OSs. a.k.a.
Suppose you set up an FTP server and you want users use the Chinese file
names for both Simplified Chinese version of Windows user and
Traditional Chinese version of Windows.

P.S. RFC 2640 is based on RFC2389 which also gets implemented here.
Although I tried my best compare the command-set of FreeBSD ftpd and
RFC959 to implementation  RFC2389, the implementation of RFC2389 may
still be incorrect.
>How-To-Repeat:

>Fix:
See my patch. I don't have FreeBSD 6.2 to try (and upgrade is not easy
on my fully loaded server) and hope this patch can be applied to 6.2's
ftpd directly.

Patch attached with submission follows:

diff -u ftpd.FreeBSD-6.1/ftpcmd.y ftpd/ftpcmd.y
--- ftpd.FreeBSD-6.1/ftpcmd.y	Thu Nov 18 21:46:29 2004
+++ ftpd/ftpcmd.y	Mon Apr 16 19:27:31 2007
@@ -95,6 +95,7 @@
 extern	int usedefault;
 extern  char tmpline[];
 extern	int readonly;
+extern	int utf8_mode;
 extern	int noepsv;
 extern	int noretr;
 extern	int noguestretr;
@@ -135,7 +136,7 @@
 	ABOR	DELE	CWD	LIST	NLST	SITE
 	STAT	HELP	NOOP	MKD	RMD	PWD
 	CDUP	STOU	SMNT	SYST	SIZE	MDTM
-	LPRT	LPSV	EPRT	EPSV
+	LPRT	LPSV	EPRT	EPSV	FEAT
 
 	UMASK	IDLE	CHMOD	MDFIVE
 
@@ -687,6 +688,16 @@
 			if ($4 != NULL)
 				free($4);
 		}
+	| FEAT CRLF
+		{
+			lreply(211, "Features:");
+			printf(" EPSV\r\n");
+			printf(" EPRT\r\n");
+			printf(" MDTM\r\n");
+			printf(" SIZE\r\n");
+			if (utf8_mode) printf(" UTF8\r\n");
+			reply(211, "End");
+		}
 	| SYST check_login CRLF
 		{
 			if ($2) {
@@ -1112,6 +1123,7 @@
 	{ "NLST", NLST, OSTR, 1,	"[ <sp> path-name ]" },
 	{ "SITE", SITE, SITECMD, 1,	"site-cmd [ <sp> arguments ]" },
 	{ "SYST", SYST, ARGS, 1,	"(get type of operating system)" },
+	{ "FEAT", FEAT, ARGS, 1,	"(get features supported)" },
 	{ "STAT", STAT, OSTR, 1,	"[ <sp> path-name ]" },
 	{ "HELP", HELP, OSTR, 1,	"[ <sp> <string> ]" },
 	{ "NOOP", NOOP, ARGS, 1,	"" },
diff -u ftpd.FreeBSD-6.1/ftpd.8 ftpd/ftpd.8
--- ftpd.FreeBSD-6.1/ftpd.8	Sun Jan 29 21:21:05 2006
+++ ftpd/ftpd.8	Mon Apr 16 20:03:35 2007
@@ -40,7 +40,7 @@
 .Nd Internet File Transfer Protocol server
 .Sh SYNOPSIS
 .Nm
-.Op Fl 46ADdEhMmOoRrSUvW
+.Op Fl 468ADdEhMmOoRrSUvW
 .Op Fl l Op Fl l
 .Op Fl a Ar address
 .Op Fl P Ar port
@@ -78,6 +78,11 @@
 is specified, accept connections via
 .Dv AF_INET6
 socket.
+.It Fl 8
+Server is running in UTF-8 mode, an RFC2640 compliant  client should convert
+all paths and messages from server from UTF-8 to client's local character-set.
+Note enalbing this option does not make server do any filesystem encoding
+conversion, it only tell client to do so.
 .It Fl A
 Allow only anonymous ftp access.
 .It Fl a
@@ -250,6 +255,7 @@
 .It DELE Ta "delete a file [RW]"
 .It EPRT Ta "specify data connection port, multiprotocol"
 .It EPSV Ta "prepare for server-to-server transfer, multiprotocol"
+.It FEAT Ta "give server feature information"
 .It HELP Ta "give help information"
 .It LIST Ta "give list files in a directory" Pq Dq Li "ls -lgA"
 .It LPRT Ta "specify data connection port, multiprotocol"
diff -u ftpd.FreeBSD-6.1/ftpd.c ftpd/ftpd.c
--- ftpd.FreeBSD-6.1/ftpd.c	Thu Mar  9 17:12:44 2006
+++ ftpd/ftpd.c	Tue Apr 17 01:26:06 2007
@@ -128,6 +128,7 @@
 int	restricted_data_ports = 1;
 int	paranoid = 1;	  /* be extra careful about security */
 int	anon_only = 0;    /* Only anonymous ftp allowed */
+int	utf8_mode = 0;    /* server file system is in UTF-8 */
 int	guest;
 int	dochroot;
 char	*chrootdir;
@@ -308,7 +309,7 @@
 	openlog("ftpd", LOG_PID | LOG_NDELAY, LOG_FTP);
 
 	while ((ch = getopt(argc, argv,
-	                    "46a:AdDEhlmMoOp:P:rRSt:T:u:UvW")) != -1) {
+	                    "468a:AdDEhlmMoOp:P:rRSt:T:u:UvW")) != -1) {
 		switch (ch) {
 		case '4':
 			family = (family == AF_INET6) ? AF_UNSPEC : AF_INET;
@@ -316,6 +317,10 @@
 
 		case '6':
 			family = (family == AF_INET) ? AF_UNSPEC : AF_INET6;
+			break;
+
+		case '8':
+			utf8_mode = 1;
 			break;
 
 		case 'a':

>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->yar 
Responsible-Changed-By: yar 
Responsible-Changed-When: Wed Apr 18 07:33:07 UTC 2007 
Responsible-Changed-Why:  
My area, ftpd. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=111714 
State-Changed-From-To: open->patched 
State-Changed-By: yar 
State-Changed-When: Wed Apr 18 22:44:10 UTC 2007 
State-Changed-Why:  
Committed to HEAD with some changes.  Thanks! 

http://www.freebsd.org/cgi/query-pr.cgi?pr=111714 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: bin/111714: commit references a PR
Date: Wed, 18 Apr 2007 22:43:49 +0000 (UTC)

 yar         2007-04-18 22:43:40 UTC
 
   FreeBSD src repository
 
   Modified files:
     libexec/ftpd         ftpcmd.y ftpd.8 ftpd.c 
   Log:
   Add support for RFC 2389 (FEAT) and RFC 2640 (UTF8) to ftpd(8).
   
   The support for RFC 2640 (UTF8) is optional and rudimentary.
   The server just advertises its capability to handle UTF-8 file
   names and relies on its own 8-bit cleanness, as well as on
   the backward compatibility of UTF-8 with ASCII.  So uploaded
   files will have UTF-8 names, but the initial server contents
   should be prepared in UTF-8 by hand, no on-the-fly conversion
   of file names will be done.
   
   PR:             bin/111714
   Submitted by:   Zhang Weiwu <see email in the PR>
   MFC after:      1 week
   
   Revision  Changes    Path
   1.66      +22 -1     src/libexec/ftpd/ftpcmd.y
   1.72      +17 -1     src/libexec/ftpd/ftpd.8
   1.212     +6 -1      src/libexec/ftpd/ftpd.c
 _______________________________________________
 cvs-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/cvs-all
 To unsubscribe, send any mail to "cvs-all-unsubscribe@freebsd.org"
 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: bin/111714: commit references a PR
Date: Tue,  1 May 2007 11:40:50 +0000 (UTC)

 yar         2007-05-01 11:40:44 UTC
 
   FreeBSD src repository
 
   Modified files:        (Branch: RELENG_6)
     libexec/ftpd         ftpcmd.y ftpd.8 ftpd.c 
   Log:
   MFC:
           ftpd.c          1.212
           ftpcmd.y        1.66
           ftpd.8          1.72-1.73
   
   Add support for RFC 2389 (FEAT) and RFC 2640 (UTF8) to ftpd(8).
   
   PR:     bin/111714
   
   Revision   Changes    Path
   1.64.2.2   +22 -1     src/libexec/ftpd/ftpcmd.y
   1.69.2.2   +17 -2     src/libexec/ftpd/ftpd.8
   1.206.2.3  +6 -1      src/libexec/ftpd/ftpd.c
 _______________________________________________
 cvs-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/cvs-all
 To unsubscribe, send any mail to "cvs-all-unsubscribe@freebsd.org"
 
State-Changed-From-To: patched->closed 
State-Changed-By: yar 
State-Changed-When: Tue May 1 12:19:51 UTC 2007 
State-Changed-Why:  
The requested feature has been implemented in CURRENT and 
6-STABLE.  Thanks! 

http://www.freebsd.org/cgi/query-pr.cgi?pr=111714 
>Unformatted:
