From nobody@FreeBSD.org  Tue Oct 15 12:38:07 2013
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1])
	(using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
	(No client certificate requested)
	by hub.freebsd.org (Postfix) with ESMTP id 8017D51A
	for <freebsd-gnats-submit@FreeBSD.org>; Tue, 15 Oct 2013 12:38:07 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from oldred.freebsd.org (oldred.freebsd.org [8.8.178.121])
	(using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
	(No client certificate requested)
	by mx1.freebsd.org (Postfix) with ESMTPS id 6C0152E9D
	for <freebsd-gnats-submit@FreeBSD.org>; Tue, 15 Oct 2013 12:38:07 +0000 (UTC)
Received: from oldred.freebsd.org ([127.0.1.6])
	by oldred.freebsd.org (8.14.5/8.14.7) with ESMTP id r9FCc6wh027615
	for <freebsd-gnats-submit@FreeBSD.org>; Tue, 15 Oct 2013 12:38:06 GMT
	(envelope-from nobody@oldred.freebsd.org)
Received: (from nobody@localhost)
	by oldred.freebsd.org (8.14.5/8.14.5/Submit) id r9FCc6GI027605;
	Tue, 15 Oct 2013 12:38:06 GMT
	(envelope-from nobody)
Message-Id: <201310151238.r9FCc6GI027605@oldred.freebsd.org>
Date: Tue, 15 Oct 2013 12:38:06 GMT
From: Fabian Keil <fk@fabiankeil.de>
To: freebsd-gnats-submit@FreeBSD.org
Subject: [patch][regression] claws-mail deadlocking in base iconv
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         182994
>Category:       bin
>Synopsis:       [patch][regression] claws-mail deadlocking in base iconv
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    delphij
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Tue Oct 15 12:40:00 UTC 2013
>Closed-Date:    Tue Mar 18 20:07:34 UTC 2014
>Last-Modified:  Tue Mar 18 20:07:34 UTC 2014
>Originator:     Fabian Keil
>Release:        HEAD
>Organization:
>Environment:
FreeBSD r500.local 11.0-CURRENT FreeBSD 11.0-CURRENT #607 r256450+1287ba1(fk): Mon Oct 14 23:08:55 CEST 2013     fk@r500.local:/usr/obj/usr/src/sys/ZOEY  amd64

>Description:
Since the base iconv is available for ports, the rebuilt claws-mail
started to deadlock in iconv every now and then on my system.

This prevented claws-mail from rendering windows or reacting
to input. I didn't bisect, but I suspect the offending commit
was r254273 or one of the following iconv-related changes.
>How-To-Repeat:
So far I haven't been able to reproduce this intentionally and various
rebuilds of ports, kernel and userland (mainly for other reasons) had
no effect.

When the problem occurs, trying to attach to the process causes
gdb and gdb76 to crash which also crashes claws-mail, but sending
SIGABRT causes a proper core dump that can be analysed with gdb.

The backtraces always show that there is only one thread running and
it's trying to lock cm_lock in _citrus_mapper_close(), which apparently
is already locked due to a _citrus_mapper_close() recursion. Examples:

#0  _umtx_op_err () at /usr/src/lib/libthr/arch/amd64/amd64/_umtx_op_err.S:37
37	RSYSCALL_ERR(_umtx_op)
[New Thread 80a806400 (LWP 100487/claws-mail)]
(gdb) where
#0  _umtx_op_err () at /usr/src/lib/libthr/arch/amd64/amd64/_umtx_op_err.S:37
#1  0x00000008084861a6 in __thr_rwlock_wrlock (rwlock=0x80a8a47c0, tsp=<value optimized out>) at /usr/src/lib/libthr/thread/thr_umtx.c:296
#2  0x0000000808489b1d in rwlock_wrlock_common (rwlock=<value optimized out>, abstime=0x0) at /usr/src/lib/libthr/thread/thr_rwlock.c:267
#3  0x0000000808489a8b in _pthread_rwlock_wrlock (rwlock=0x80a8a47c0) at /usr/src/lib/libthr/thread/thr_rwlock.c:289
#4  0x000000080911e848 in _citrus_mapper_close (cm=0x80b5a2d80) at /usr/src/lib/libc/iconv/citrus_mapper.c:375
#5  0x000000080d205d18 in _citrus_mapper_serial_mapper_uninit (cm=0x80b5a2d40) at /usr/src/lib/libiconv_modules/mapper_parallel/../mapper_serial/citrus_mapper_serial.c:110
#6  0x000000080911e8d7 in mapper_close (cm=0x80b5a2d40) at /usr/src/lib/libc/iconv/citrus_mapper.c:188
#7  0x000000080911e88c in _citrus_mapper_close (cm=<value optimized out>) at /usr/src/lib/libc/iconv/citrus_mapper.c:384
#8  0x000000080c4e83f3 in close_srcs (sl=0x80b591140) at /usr/src/lib/libiconv_modules/iconv_std/citrus_iconv_std.c:206
#9  0x000000080c4e7dc9 in _citrus_iconv_std_iconv_uninit_shared (ci=<value optimized out>) at /usr/src/lib/libiconv_modules/iconv_std/citrus_iconv_std.c:415
#10 0x00000008090f3f95 in release_shared (ci=0x80a8ee630) at /usr/src/lib/libc/iconv/citrus_iconv.c:99
#11 0x00000008090f4002 in _citrus_iconv_close (cv=0x80d88d5d0) at /usr/src/lib/libc/iconv/citrus_iconv.c:335
#12 0x00000008090f1ca6 in iconv_close (handle=0x80a8a47c0) at /usr/src/lib/libc/iconv/iconv.c:131
#13 0x000000000046376d in conv_iconv_strdup (inbuf=0x7fffffff58b0 "\n", src_code=0x80b5b4db0 "Windows-1252", dest_code=0x6f03d0 "UTF-8") at codeconv.c:895
#14 0x0000000000463d13 in conv_convert (conv=0x80b5a4e80, outbuf=0x7fffffff3720 "", outlen=8192, inbuf=0x7fffffff58b0 "\n") at codeconv.c:734
#15 0x00000000005e22ac in textview_write_line (textview=0x80a959cc0, str=0x7fffffff58b0 "\n", conv=0x80b5a4e80, do_quote_folding=1) at textview.c:1573
#16 0x00000000005df8e4 in textview_write_body (textview=0x80a959cc0, mimeinfo=0x80aad2d00) at textview.c:1177
#17 0x00000000005e5363 in textview_add_part (textview=0x80a959cc0, mimeinfo=0x80aad2d00) at textview.c:826
#18 0x00000000005e4053 in recursive_add_parts (textview=0x80a959cc0, node=0x80a826190) at textview.c:839
#19 0x00000000005e4302 in recursive_add_parts (textview=0x80a959cc0, node=0x80aa81d20) at textview.c:888
#20 0x00000000005e4302 in recursive_add_parts (textview=0x80a959cc0, node=0x80a828890) at textview.c:888
#21 0x00000000005defa1 in textview_add_parts (textview=0x80a959cc0, mimeinfo=0x80b610700) at textview.c:898
#22 0x00000000005deb85 in textview_show_part (textview=0x80a959cc0, mimeinfo=0x80b610700, fp=0x8094319a0) at textview.c:645
[...]

#0  0x0000000808491b9c in __error () from /lib/libthr.so.3
#1  0x000000080848bb1d in rwlock_wrlock_common (rwlock=<value optimized out>, abstime=0x0) at /usr/src/lib/libthr/thread/thr_rwlock.c:267
#2  0x000000080848ba8b in _pthread_rwlock_wrlock (rwlock=0x80a8ede20) at /usr/src/lib/libthr/thread/thr_rwlock.c:289
#3  0x000000080911f848 in _citrus_mapper_close (cm=0x80a8bfc40) at /usr/src/lib/libc/iconv/citrus_mapper.c:375
#4  0x000000080ce02d18 in _citrus_mapper_serial_mapper_uninit (cm=0x80a8bfc00) at /usr/src/lib/libiconv_modules/mapper_parallel/../mapper_serial/citrus_mapper_serial.c:110
#5  0x000000080911f8d7 in mapper_close (cm=0x80a8bfc00) at /usr/src/lib/libc/iconv/citrus_mapper.c:188
#6  0x000000080911f88c in _citrus_mapper_close (cm=<value optimized out>) at /usr/src/lib/libc/iconv/citrus_mapper.c:384
#7  0x000000080c5893f3 in close_srcs (sl=0x80a8edda0) at /usr/src/lib/libiconv_modules/iconv_std/citrus_iconv_std.c:206
#8  0x000000080c588dc9 in _citrus_iconv_std_iconv_uninit_shared (ci=<value optimized out>) at /usr/src/lib/libiconv_modules/iconv_std/citrus_iconv_std.c:415
#9  0x00000008090f4f95 in release_shared (ci=0x80b408890) at /usr/src/lib/libc/iconv/citrus_iconv.c:99
#10 0x00000008090f5002 in _citrus_iconv_close (cv=0x80b782630) at /usr/src/lib/libc/iconv/citrus_iconv.c:335
#11 0x00000008090f2ca6 in iconv_close (handle=0x80a8ede20) at /usr/src/lib/libc/iconv/iconv.c:131
#12 0x000000000046376d in conv_iconv_strdup (
    inbuf=0x80b600e00 "[...]"..., src_code=0x80b782640 "ISO-8859-15", dest_code=0x80c07ab24 "UTF-8") at codeconv.c:895
#13 0x0000000000463f2f in conv_codeset_strdup (
    inbuf=0x80b600e00 "[...]"..., src_code=0x80b782640 "ISO-8859-15", dest_code=0x80c07ab24 "UTF-8") at codeconv.c:774
#14 0x000000080c07a1a7 in get_part_as_string (mimeinfo=0x80aad2d80) at pgpinline.c:153
#15 0x000000080c078333 in pgpinline_is_encrypted (mimeinfo=0x80aad2d80) at pgpinline.c:375
[...]

#0  _umtx_op_err () at /usr/src/lib/libthr/arch/amd64/amd64/_umtx_op_err.S:37
#1  0x00000008084861a6 in __thr_rwlock_wrlock (rwlock=0x80bf93f40, tsp=<value optimized out>) at /usr/src/lib/libthr/thread/thr_umtx.c:296
#2  0x0000000808489b1d in rwlock_wrlock_common (rwlock=<value optimized out>, abstime=0x0) at /usr/src/lib/libthr/thread/thr_rwlock.c:267
#3  0x0000000808489a8b in _pthread_rwlock_wrlock (rwlock=0x80bf93f40) at /usr/src/lib/libthr/thread/thr_rwlock.c:289
#4  0x000000080911e848 in _citrus_mapper_close (cm=0x80bfa0b40) at /usr/src/lib/libc/iconv/citrus_mapper.c:375
#5  0x000000080d08ad18 in _citrus_mapper_serial_mapper_uninit (cm=0x80bfa0b00) at /usr/src/lib/libiconv_modules/mapper_parallel/../mapper_serial/citrus_mapper_serial.c:110
#6  0x000000080911e8d7 in mapper_close (cm=0x80bfa0b00) at /usr/src/lib/libc/iconv/citrus_mapper.c:188
#7  0x000000080911e88c in _citrus_mapper_close (cm=<value optimized out>) at /usr/src/lib/libc/iconv/citrus_mapper.c:384
#8  0x000000080ae2c3f3 in close_srcs (sl=0x80bf9fbc0) at /usr/src/lib/libiconv_modules/iconv_std/citrus_iconv_std.c:206
#9  0x000000080ae2bdc9 in _citrus_iconv_std_iconv_uninit_shared (ci=<value optimized out>) at /usr/src/lib/libiconv_modules/iconv_std/citrus_iconv_std.c:415
#10 0x00000008090f3f95 in release_shared (ci=0x80bd59ac0) at /usr/src/lib/libc/iconv/citrus_iconv.c:99
#11 0x00000008090f4002 in _citrus_iconv_close (cv=0x80dc1a3d0) at /usr/src/lib/libc/iconv/citrus_iconv.c:335
#12 0x00000008090f1ca6 in iconv_close (handle=0x80bf93f40) at /usr/src/lib/libc/iconv/iconv.c:131
#13 0x000000000046376d in conv_iconv_strdup (inbuf=0x80bd570c0 "[...]", src_code=0x7fffffff5d60 "ISO-8859-2", dest_code=0x6f03d0 "UTF-8") at codeconv.c:895
#14 0x0000000000463f2f in conv_codeset_strdup (inbuf=0x80bd570c0 "[...]", src_code=0x7fffffff5d60 "ISO-8859-2", dest_code=0x0) at codeconv.c:774
#15 0x00000000005f63f0 in unmime_header (encoded_str=0x7fffffff7e94 "[...]", addr_field=1) at unmime.c:144
#16 0x0000000000464a56 in conv_unmime_header (str=0x7fffffff7e94 "[...]", default_encoding=0x0, addr_field=1)
    at codeconv.c:1516
#17 0x000000000059588a in parse_stream (data=0x8094319a0, isstring=0, flags={perm_flags = 3, tmp_flags = 0}, full=0, decrypted=0) at procheader.c:558
#18 0x0000000000595234 in procheader_parse_stream (fp=0x8094319a0, flags={perm_flags = 3, tmp_flags = 0}, full=0, decrypted=0) at procheader.c:461
#19 0x0000000000595195 in procheader_parse_file (file=0x80d4202b0 "/home/fk/.claws-mail/tempfolder/processing/18", flags={perm_flags = 3, tmp_flags = 0}, full=0, decrypted=0) at procheader.c:354
[...]

Maciej Suszko reported that the problem is reproducible using large IMAP folders:
http://lists.freebsd.org/pipermail/freebsd-current/2013-October/045391.html

Using IMAP is not a requirement for the deadlock, though, I build claws-mail without IMAP support.
>Fix:
The attached patch prevents the deadlocks.

It has been successfully tested by other claws-mail users after being posted to freebsd-current@:
http://lists.freebsd.org/pipermail/freebsd-current/2013-October/045323.html

For claws-mail the patch seems to be sufficient, but Tijl Coosemans noted that there may be a similar problem:
http://lists.freebsd.org/pipermail/freebsd-current/2013-October/045393.html


Patch attached with submission follows:

From f37c3c4403298a3a9ab8b0c191893e09e06c180e Mon Sep 17 00:00:00 2001
From: Fabian Keil <fk@fabiankeil.de>
Date: Sat, 5 Oct 2013 12:34:13 +0200
Subject: [PATCH] Let _citrus_mapper_close() unlock cm_lock before calling
 mapper_close()

mapper_close() may call _citrus_mapper_close() again in which
case an already locked cm_lock would cause a deadlock.

mapper_close() doesn't seems to require the lock an is already
called without locking in other places.
---
 lib/libc/iconv/citrus_mapper.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/lib/libc/iconv/citrus_mapper.c b/lib/libc/iconv/citrus_mapper.c
index b1ee8fb..43fa4b0 100644
--- a/lib/libc/iconv/citrus_mapper.c
+++ b/lib/libc/iconv/citrus_mapper.c
@@ -381,7 +381,9 @@ _citrus_mapper_close(struct _citrus_mapper *cm)
 			_CITRUS_HASH_REMOVE(cm, cm_entry);
 			free(cm->cm_key);
 		}
+		UNLOCK(&cm_lock);
 		mapper_close(cm);
+		return;
 quit:
 		UNLOCK(&cm_lock);
 	}
-- 
1.8.4



>Release-Note:
>Audit-Trail:
State-Changed-From-To: open->patched 
State-Changed-By: delphij 
State-Changed-When: Mon Oct 21 07:58:47 UTC 2013 
State-Changed-Why:  
Patch committed as r256833 against -HEAD, MFC reminder. 


Responsible-Changed-From-To: freebsd-bugs->delphij 
Responsible-Changed-By: delphij 
Responsible-Changed-When: Mon Oct 21 07:58:47 UTC 2013 
Responsible-Changed-Why:  
Take. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=182994 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: bin/182994: commit references a PR
Date: Mon, 21 Oct 2013 07:58:49 +0000 (UTC)

 Author: delphij
 Date: Mon Oct 21 07:58:37 2013
 New Revision: 256833
 URL: http://svnweb.freebsd.org/changeset/base/256833
 
 Log:
   Drop cm_lock before calling mapper_close, which in turn could call
   _citrus_mapper_close again and result in a deadlock otherwise.
   
   This is similar to NetBSD PR/24023 (fixed in their r1.5 of this file).
   
   PR:		bin/182994
   Submitted by:	Fabian Keil <fk fabiankeil de>
   MFC after:	3 days
 
 Modified:
   head/lib/libc/iconv/citrus_mapper.c
 
 Modified: head/lib/libc/iconv/citrus_mapper.c
 ==============================================================================
 --- head/lib/libc/iconv/citrus_mapper.c	Mon Oct 21 07:49:36 2013	(r256832)
 +++ head/lib/libc/iconv/citrus_mapper.c	Mon Oct 21 07:58:37 2013	(r256833)
 @@ -381,7 +381,9 @@ _citrus_mapper_close(struct _citrus_mapp
  			_CITRUS_HASH_REMOVE(cm, cm_entry);
  			free(cm->cm_key);
  		}
 +		UNLOCK(&cm_lock);
  		mapper_close(cm);
 +		return;
  quit:
  		UNLOCK(&cm_lock);
  	}
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: bin/182994: commit references a PR
Date: Thu, 24 Oct 2013 05:01:57 +0000 (UTC)

 Author: delphij
 Date: Thu Oct 24 05:01:49 2013
 New Revision: 257039
 URL: http://svnweb.freebsd.org/changeset/base/257039
 
 Log:
   MFC r256833:
   
   Drop cm_lock before calling mapper_close, which in turn could call
   _citrus_mapper_close again and result in a deadlock otherwise.
   
   This is similar to NetBSD PR/24023 (fixed in their r1.5 of this file).
   
   PR:		bin/182994
   Submitted by:	Fabian Keil <fk fabiankeil de>
   Approved by:	re (hrs)
 
 Modified:
   stable/10/lib/libc/iconv/citrus_mapper.c
 Directory Properties:
   stable/10/lib/libc/   (props changed)
 
 Modified: stable/10/lib/libc/iconv/citrus_mapper.c
 ==============================================================================
 --- stable/10/lib/libc/iconv/citrus_mapper.c	Thu Oct 24 03:36:39 2013	(r257038)
 +++ stable/10/lib/libc/iconv/citrus_mapper.c	Thu Oct 24 05:01:49 2013	(r257039)
 @@ -381,7 +381,9 @@ _citrus_mapper_close(struct _citrus_mapp
  			_CITRUS_HASH_REMOVE(cm, cm_entry);
  			free(cm->cm_key);
  		}
 +		UNLOCK(&cm_lock);
  		mapper_close(cm);
 +		return;
  quit:
  		UNLOCK(&cm_lock);
  	}
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 
State-Changed-From-To: patched->closed 
State-Changed-By: delphij 
State-Changed-When: Tue Mar 18 20:07:04 UTC 2014 
State-Changed-Why:  
MFC done as r257039 on Oct 24, 2013.  Thanks for reporting 
this issue! 

http://www.freebsd.org/cgi/query-pr.cgi?pr=182994 
>Unformatted:
