From dds@istlab.dmst.aueb.gr  Thu Dec 30 14:11:06 2004
Return-Path: <dds@istlab.dmst.aueb.gr>
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id 3082816A4CE
	for <FreeBSD-gnats-submit@freebsd.org>; Thu, 30 Dec 2004 14:11:06 +0000 (GMT)
Received: from istlab.dmst.aueb.gr (istlab.dmst.aueb.gr [195.251.249.147])
	by mx1.FreeBSD.org (Postfix) with ESMTP id 6BDF043D45
	for <FreeBSD-gnats-submit@freebsd.org>; Thu, 30 Dec 2004 14:11:05 +0000 (GMT)
	(envelope-from dds@istlab.dmst.aueb.gr)
Received: from istlab.dmst.aueb.gr (localhost [127.0.0.1])
	by istlab.dmst.aueb.gr (8.13.1/8.13.1) with ESMTP id iBUEB1HY019500
	for <FreeBSD-gnats-submit@freebsd.org>; Thu, 30 Dec 2004 16:11:01 +0200 (EET)
	(envelope-from dds@istlab.dmst.aueb.gr)
Received: (from dds@localhost)
	by istlab.dmst.aueb.gr (8.13.1/8.13.1/Submit) id iBUEB12S019499;
	Thu, 30 Dec 2004 16:11:01 +0200 (EET)
	(envelope-from dds)
Message-Id: <200412301411.iBUEB12S019499@istlab.dmst.aueb.gr>
Date: Thu, 30 Dec 2004 16:11:01 +0200 (EET)
From: Diomidis Spinellis <dds@istlab.dmst.aueb.gr>
Reply-To: Diomidis Spinellis <dds@istlab.dmst.aueb.gr>
To: FreeBSD-gnats-submit@freebsd.org
Cc:
Subject: libc/regexp leaks memory
X-Send-Pr-Version: 3.113
X-GNATS-Notify:

>Number:         75656
>Category:       bin
>Synopsis:       libc/regexp leaks memory
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    dds
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Thu Dec 30 14:20:22 GMT 2004
>Closed-Date:    Thu Jan 13 08:43:59 GMT 2005
>Last-Modified:  Thu Jan 13 08:43:59 GMT 2005
>Originator:     Diomidis Spinellis
>Release:        FreeBSD 4.10-STABLE i386
>Organization:
AUEB
>Environment:
System: FreeBSD istlab.dmst.aueb.gr 4.10-STABLE FreeBSD 4.10-STABLE #23: Fri Oct 8 15:53:45 EEST 2004 dds@istlab.dmst.aueb.gr:/usr/obj/usr/src/sys/ISTLAB i386
>Description:
The C library regular expression code will leak memory
by not freeing in one case the m->pmatch and m->lastpos
pointers before freing the m block in engine.c:matcher
>How-To-Repeat:
While writing a chapter on memory management for the "Code Properties"
book, run the sed(1) Towers of Hanoi test case under Valgrind's memcheck
tool (http://valgrind.kde.org/tools.html):

$ echo ':abc: : :' | valgrind  --tool=memcheck --leak-check=yes ./sed -f TEST/hanoi.sed
==19363== Memcheck, a memory error detector for x86-linux.
==19363== Copyright (C) 2002-2004, and GNU GPL'd, by Julian Seward.
==19363== Using valgrind-2.1.2.CVS, a program supervision framework for x86-linux.
==19363== Copyright (C) 2000-2004, and GNU GPL'd, by Julian Seward.
==19363== For more details, rerun with: -v
==19363==
==19363== Warning: ignoring --pointercheck=yes, because i386_set_ldt failed (errno=45)
:abc:   :   :
:ab :c  :   :
:a  :c  :b  :
:a  :   :bc :
:   :a  :bc :
:c  :a  :b  :
:c  :ab :   :
:   :abc:   :
Done!  Try another, or end with ^D.
==19363==
==19363== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
==19363== malloc/free: in use at exit: 54476 bytes in 435 blocks.
==19363== malloc/free: 746 allocs, 311 frees, 205173 bytes allocated.
==19363== For counts of detected errors, rerun with: -v
==19363== searching for pointers to 435 not-freed blocks.
==19363== checked 903192 bytes.
==19363==
==19363== 136 bytes in 17 blocks are definitely lost in loss record 8 of 22
==19363==    at 0x3C026659: malloc (in /usr/local/lib/valgrind/vgpreload_memcheck.so)
==19363==    by 0x8054D9E: lmatcher (engine.c:285)
==19363==    by 0x8057918: regexec (regexec.c:180)
==19363==    by 0x804D1D6: regexec_e (process.c:535)
==19363==
==19363==
==19363== 296 bytes in 37 blocks are definitely lost in loss record 11 of 22
==19363==    at 0x3C026659: malloc (in /usr/local/lib/valgrind/vgpreload_memcheck.so)
==19363==    by 0x8051E1A: smatcher (engine.c:285)
==19363==    by 0x80578F2: regexec (regexec.c:178)
==19363==    by 0x804D1D6: regexec_e (process.c:535)
==19363==
==19363==
==19363== 1184 bytes in 37 blocks are definitely lost in loss record 14 of 22
==19363==    at 0x3C026659: malloc (in /usr/local/lib/valgrind/vgpreload_memcheck.so)
==19363==    by 0x8051D05: smatcher (engine.c:272)
==19363==    by 0x80578F2: regexec (regexec.c:178)
==19363==    by 0x804D1D6: regexec_e (process.c:535)
==19363==
==19363==
==19363== 1360 bytes in 17 blocks are definitely lost in loss record 16 of 22
==19363==    at 0x3C026659: malloc (in /usr/local/lib/valgrind/vgpreload_memcheck.so)
==19363==    by 0x8054C79: lmatcher (engine.c:272)
==19363==    by 0x8057918: regexec (regexec.c:180)
==19363==    by 0x804D1D6: regexec_e (process.c:535)
==19363==
==19363== LEAK SUMMARY:
==19363==    definitely lost: 2976 bytes in 108 blocks.
==19363==    possibly lost:   0 bytes in 0 blocks.
==19363==    still reachable: 51500 bytes in 327 blocks.
==19363==         suppressed: 0 bytes in 0 blocks.
==19363== Reachable blocks (those to which a pointer was found) are not shown.
==19363== To see them, rerun with: --show-reachable=yes

Repeat the process with more elements and you will get even more leaked
blocks:
$ echo ':abcd: : :' | valgrind  --tool=memcheck --leak-check=yes ./sed -f TEST/hanoi.sed
[...]
==19457== LEAK SUMMARY:
==19457==    definitely lost: 6016 bytes in 212 blocks.
==19457==    possibly lost:   0 bytes in 0 blocks.
==19457==    still reachable: 51503 bytes in 327 blocks.
==19457==         suppressed: 0 bytes in 0 blocks.

>Fix:

	Apply the following patch:

Index: engine.c
===================================================================
RCS file: /home/ncvs/src/lib/libc/regex/engine.c,v
retrieving revision 1.5.8.1
diff -u -r1.5.8.1 engine.c
--- engine.c	31 Jul 2000 06:30:37 -0000	1.5.8.1
+++ engine.c	30 Dec 2004 14:08:27 -0000
@@ -242,6 +242,10 @@
 	for (;;) {
 		endp = fast(m, start, stop, gf, gl);
 		if (endp == NULL) {		/* a miss */
+			if (m->pmatch != NULL)
+				free((char *)m->pmatch);
+			if (m->lastpos != NULL)
+				free((char *)m->lastpos);
 			STATETEARDOWN(m);
 			return(REG_NOMATCH);
 		}

>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->dds 
Responsible-Changed-By: dds 
Responsible-Changed-When: Thu Dec 30 14:24:10 GMT 2004 
Responsible-Changed-Why:  
Finder keeper. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=75656 
State-Changed-From-To: open->analyzed 
State-Changed-By: dds 
State-Changed-When: Thu Dec 30 14:25:25 GMT 2004 
State-Changed-Why:  
Already analyzed, before submitting. 


http://www.freebsd.org/cgi/query-pr.cgi?pr=75656 
State-Changed-From-To: analyzed->patched 
State-Changed-By: dds 
State-Changed-When: Thu Dec 30 14:28:12 GMT 2004 
State-Changed-Why:  
Checking in engine.c; 
/home/ncvs/src/lib/libc/regex/engine.c,v  <--  engine.c 
new revision: 1.16; previous revision: 1.15 


http://www.freebsd.org/cgi/query-pr.cgi?pr=75656 
State-Changed-From-To: patched->closed 
State-Changed-By: dds 
State-Changed-When: Thu Jan 13 08:43:34 GMT 2005 
State-Changed-Why:  
MFC'd to RELENG_4 and RELENG_5 


http://www.freebsd.org/cgi/query-pr.cgi?pr=75656 
>Unformatted:
