From nobody@FreeBSD.org  Sat Sep 14 23:20:21 2013
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1])
	(using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
	(No client certificate requested)
	by hub.freebsd.org (Postfix) with ESMTP id B85E6921
	for <freebsd-gnats-submit@FreeBSD.org>; Sat, 14 Sep 2013 23:20:21 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from oldred.freebsd.org (oldred.freebsd.org [8.8.178.121])
	(using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
	(No client certificate requested)
	by mx1.freebsd.org (Postfix) with ESMTPS id A252A2111
	for <freebsd-gnats-submit@FreeBSD.org>; Sat, 14 Sep 2013 23:20:21 +0000 (UTC)
Received: from oldred.freebsd.org ([127.0.1.6])
	by oldred.freebsd.org (8.14.5/8.14.7) with ESMTP id r8ENKLLg097827
	for <freebsd-gnats-submit@FreeBSD.org>; Sat, 14 Sep 2013 23:20:21 GMT
	(envelope-from nobody@oldred.freebsd.org)
Received: (from nobody@localhost)
	by oldred.freebsd.org (8.14.5/8.14.5/Submit) id r8ENKKjV097551;
	Sat, 14 Sep 2013 23:20:20 GMT
	(envelope-from nobody)
Message-Id: <201309142320.r8ENKKjV097551@oldred.freebsd.org>
Date: Sat, 14 Sep 2013 23:20:20 GMT
From: Henry Hu <henry.hu.sh@gmail.com>
To: freebsd-gnats-submit@FreeBSD.org
Subject: bug in libiconv_none prevents MPD from starting
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         182102
>Category:       kern
>Synopsis:       [libiconv] bug in libiconv_none prevents MPD from starting
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    freebsd-bugs
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Sat Sep 14 23:30:00 UTC 2013
>Closed-Date:    Sun Dec 15 04:31:57 UTC 2013
>Last-Modified:  Sun Dec 15 04:31:57 UTC 2013
>Originator:     Henry Hu
>Release:        FreeBSD 10-CURRENT
>Organization:
Columbia University
>Environment:
FreeBSD pepsi 10.0-CURRENT FreeBSD 10.0-CURRENT #1 r255532M: Sat Sep 14 17:30:47 PDT 2013     root@pepsi:/usr/obj/usr/src/sys/MYKERNEL  amd64

>Description:
Recently I updated to latest -CURRENT which enables WITH_ICONV.
Then I found MPD(audio/musicpd) failed to start.
It says:

path: invalid filesystem charset: UTF-8

MPD now uses the system iconv. I found the call in MPD to g_convert() failed.
After some tracing, I found that the iconv() call is not working properly.
Here is the test program:

#include <stdio.h>
#include <iconv.h>

int main() {
	iconv_t cd = iconv_open("UTF-8", "UTF-8");
	char *inbuf = " ";
	char *inp = inbuf;
	size_t inl = 1;
	char outbuf[100];
	char *outp = outbuf;
	size_t oul = 1;
	printf("%p %d %p %d\n", inp, inl, outp, oul);
	int ret = iconv(cd, &inp, &inl, &outp, &oul);
	printf("%d %p %d %p %d\n", ret, inp, inl, outp, oul);
}

This essentially loads the iconv_none module, because the input and output encodings are the same.
If you run it, you get something like this:

0x40094c 1 0x7fffffffd4d0 1
0 0x40094c 0 0x7fffffffd4d0 0

However, according to iconv()'s manpage, after iconv():

     *src      Pointer to the byte just after the last character fetched.

     *srcleft  Number of remaining bytes in the source buffer.

     *dst      Pointer to the byte just after the last character stored.

     *dstleft  Number of remainder bytes in the destination buffer.

So both src (inp) and dst (outp) should move forward, but they stay the same. This makes g_convert() think that the conversion is not completed and there are partial inputs.

In /usr/src/lib/libiconv_modules/iconv_none/citrus_iconv_none.c, we have:

static int
/*ARGSUSED*/
_citrus_iconv_none_iconv_convert(struct _citrus_iconv * __restrict ci __unused,
    char * __restrict * __restrict in, size_t * __restrict inbytes,
    char * __restrict * __restrict out, size_t * __restrict outbytes,
    uint32_t flags __unused, size_t * __restrict invalids)
{
        ....
	memcpy(*out, *in, len);
	in += len;
	*inbytes -= len;
	out += len;
	*outbytes -= len;
	*invalids = 0;
        ....

Here, the function attempted to move the in & out pointers forward. However, now it just moves the local copy. It should do

*in += len;
..
*out += len;

instead.
After applying the patch, it works correctly, and MPD starts.

Please fix this.
>How-To-Repeat:
Compile and run this:

#include <stdio.h>
#include <iconv.h>

int main() {
	iconv_t cd = iconv_open("UTF-8", "UTF-8");
	char *inbuf = " ";
	char *inp = inbuf;
	size_t inl = 1;
	char outbuf[100];
	char *outp = outbuf;
	size_t oul = 1;
	printf("%p %d %p %d\n", inp, inl, outp, oul);
	int ret = iconv(cd, &inp, &inl, &outp, &oul);
	printf("%d %p %d %p %d\n", ret, inp, inl, outp, oul);
}

And observe that the input & output pointers of iconv() are not increased.
>Fix:
Patch attached.

Patch attached with submission follows:

Index: lib/libiconv_modules/iconv_none/citrus_iconv_none.c
===================================================================
--- lib/libiconv_modules/iconv_none/citrus_iconv_none.c	(版本 255575)
+++ lib/libiconv_modules/iconv_none/citrus_iconv_none.c	(工作副本)
@@ -115,9 +115,9 @@
 		len = *outbytes;
 	}
 	memcpy(*out, *in, len);
-	in += len;
+	*in += len;
 	*inbytes -= len;
-	out += len;
+	*out += len;
 	*outbytes -= len;
 	*invalids = 0;
 	if (e2big)


>Release-Note:
>Audit-Trail:
State-Changed-From-To: open->feedback 
State-Changed-By: linimon 
State-Changed-When: Mon Sep 16 05:17:46 UTC 2013 
State-Changed-Why:  
Does this apply to the system iconv, or to the port? 

http://www.freebsd.org/cgi/query-pr.cgi?pr=182102 
State-Changed-From-To: feedback->open 
State-Changed-By: linimon 
State-Changed-When: Mon Sep 16 05:25:53 UTC 2013 
State-Changed-Why:  
reclassify. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=182102 

From: Jilles Tjoelker <jilles@stack.nl>
To: bug-followup@FreeBSD.org, henry.hu.sh@gmail.com
Cc:  
Subject: Re: kern/182102: [libiconv] bug in libiconv_none prevents MPD from
 starting
Date: Fri, 20 Sep 2013 18:44:10 +0200

 In PR kern/182102, you wrote:
 > Then I found MPD(audio/musicpd) failed to start. It says:
 
 > path: invalid filesystem charset: UTF-8
 
 > MPD now uses the system iconv. I found the call in MPD to g_convert()
 > failed. After some tracing, I found that the iconv() call is not
 > working properly.
 
 > This essentially loads the iconv_none module, because the input and
 > output encodings are the same.
 
 As of Subversion r254080 (August 8, 2013), iconv_none is no longer used
 because it does not check the encoding as specified by POSIX and
 required by various applications. So there seems to be something wrong
 with either your version specification or your analysis.
 
 As for a fix, I think it would be a good idea to drop iconv_none
 entirely. Also, iconv_std might be folded into lib/libc/iconv/; there is
 little benefit in dlopening such a small module.
 
 -- 
 Jilles Tjoelker

From: Henry Hu <henry.hu.sh@gmail.com>
To: bug-followup@freebsd.org, henry.hu.sh@gmail.com
Cc:  
Subject: Re: kern/182102: [libiconv] bug in libiconv_none prevents MPD from starting
Date: Fri, 20 Sep 2013 11:18:54 -0700

 --90e6ba614f0e38901a04e6d4b5de
 Content-Type: text/plain; charset=ISO-8859-1
 
 Ah, I'm sorry. I'm on the project/uefi branch, and I did not notice that
 iconv_none is no longer used. I just checked the latest code and found that
 iconv_none still had this bug.
 Removing it should make things cleaner.
 
 -- 
 Cheers,
 Henry
 
 --90e6ba614f0e38901a04e6d4b5de
 Content-Type: text/html; charset=ISO-8859-1
 Content-Transfer-Encoding: quoted-printable
 
 <div dir=3D"ltr"><div>Ah, I&#39;m sorry. I&#39;m on the project/uefi branch=
 , and I did not notice that iconv_none is no longer used. I just checked th=
 e latest code and found that iconv_none still had this bug.<br></div>Removi=
 ng it should make things cleaner.<br clear=3D"all">
 
 <div><div><div><div><div><br>-- <br>Cheers,<br>Henry
 </div></div></div></div></div></div>
 
 --90e6ba614f0e38901a04e6d4b5de--

From: Henry Hu <henry.hu.sh@gmail.com>
To: bug-followup@freebsd.org, henry.hu.sh@gmail.com
Cc:  
Subject: Re: kern/182102: [libiconv] bug in libiconv_none prevents MPD from starting
Date: Sat, 14 Dec 2013 21:09:31 -0500

 --089e0149cd5eced06804ed89308b
 Content-Type: text/plain; charset=ISO-8859-1
 
 Because the code is no longer used, please close this PR.
 
 -- 
 Cheers,
 Henry
 
 --089e0149cd5eced06804ed89308b
 Content-Type: text/html; charset=ISO-8859-1
 
 <div dir="ltr">Because the code is no longer used, please close this PR.<br clear="all"><div><br>-- <br>Cheers,<br>Henry
 </div></div>
 
 --089e0149cd5eced06804ed89308b--
State-Changed-From-To: open->closed 
State-Changed-By: eadler 
State-Changed-When: Sun Dec 15 04:31:57 UTC 2013 
State-Changed-Why:  
Per submitter request 

http://www.freebsd.org/cgi/query-pr.cgi?pr=182102 
>Unformatted:
