From dan@dan.emsphone.com  Tue Jun  3 08:59:37 2003
Return-Path: <dan@dan.emsphone.com>
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP
	id 7491637B401; Tue,  3 Jun 2003 08:59:37 -0700 (PDT)
Received: from dan.emsphone.com (dan.emsphone.com [199.67.51.101])
	by mx1.FreeBSD.org (Postfix) with ESMTP
	id A7F7B43FA3; Tue,  3 Jun 2003 08:59:36 -0700 (PDT)
	(envelope-from dan@dan.emsphone.com)
Received: (from dan@localhost)
	by dan.emsphone.com (8.12.9/8.12.9) id h53FxZgS089446;
	Tue, 3 Jun 2003 10:59:35 -0500 (CDT)
	(envelope-from dan)
Message-Id: <200306031559.h53FxZgS089446@dan.emsphone.com>
Date: Tue, 3 Jun 2003 10:59:35 -0500 (CDT)
From: Dan Nelson <dnelson@allantgroup.com>
To: FreeBSD-gnats-submit@freebsd.org
Cc: phk@freebsd.org
Subject: [PATCH] more malloc options for debugging programs
X-Send-Pr-Version: 3.113
X-GNATS-Notify:

>Number:         52907
>Category:       kern
>Synopsis:       [libc] [patch] more malloc options for debugging programs
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    phk
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          change-request
>Submitter-Id:   current-users
>Arrival-Date:   Tue Jun 03 09:00:28 PDT 2003
>Closed-Date:    Thu May 17 19:03:35 GMT 2007
>Last-Modified:  Thu May 17 19:03:35 GMT 2007
>Originator:     Dan Nelson
>Release:        FreeBSD 5.1-BETA i386
>Organization:
The Allant Group
>Environment:
System: FreeBSD dan.emsphone.com 5.1-BETA FreeBSD 5.1-BETA #271: Thu May 29 16:33:28 CDT 2003 dan@dan.emsphone.com:/usr/src/sys/i386/compile/DANSMP i386


	
>Description:
	
>How-To-Repeat:
	
>Fix:

This patch adds two things to malloc:  The ability to mark each
allocated and freed area with a unique byte pattern instead of a
constant 0xd0, and the ability to die on command at a given malloc/free
operation.

The two options are collapsed onto the C malloc flag, but could easily
enough be separated.  Both make use of an internal running counter of
how many calls to malloc/free/realloc have been made.  To help
differentiate the two types of junked data, the patch also alters the J
flag to fill mallocs with 0xd0 and frees with 0xd1.

"C" all by itself just sticks the counter in the low-order bytes of
each 4-byte word when filling with junk.  Very useful in conjunction
with the next option:

"C" with a numeric argument will cause malloc to abort() the Nth time
malloc/free/realloc is called.  For example, use "C1" to make sure you
never malloc or free any data :)

These two options are great for tracking down uninitialized pointers,
and reuse of free'd memory.  For example, say your program seg faulted
on a bad pointer lookup (pointer value is 0xd0d0a492).  Where's the
bug?  Run your program again with MALLOC_OPTIONS="CC0xa492", and you
will get a coredump right at the malloc() call.  Chances are the bug is
nearby.

I find it useful enough that on my system I have "J" do what "C" does,
and C is only used to set the abort counter.

Index: malloc.3
===================================================================
RCS file: /home/ncvs/src/lib/libc/stdlib/malloc.3,v
retrieving revision 1.60
diff -u -p -r1.60 malloc.3
--- malloc.3	24 Dec 2002 13:41:45 -0000	1.60
+++ malloc.3	3 Jun 2003 15:10:13 -0000
@@ -177,19 +177,45 @@ flags being set) become fatal.
 The process will call
 .Xr abort 3
 in these cases.
+.It C
+This option sets the 
+.Dq J
+option, keeps a count of the total number of memory allocation calls,
+and when filling memory with junk, initializes the low-order bytes in each 
+4-byte word with the running count.
+For example, the 255th malloc operation will be initialized with 0xd0d0d0ff, 
+and the next free after that will have its memory set to 0xd1d10100.
+Note that there is some ambiguity in interpreting these values;
+0xd0d0d0ff could also have been the 53503rd or 13684991th operation.
+.It Cn
+Keep a count of the total number of memory allocation calls,
+and abort at the nth operation.
+n can be specified in decimal, or it can be hexidecimal (prefixed with 0x).
+If a hex value is used, ensure that no options that may be confused with
+hex digits follow this option.
+A lowercase 
+.Dq c
+will clear both the 
+.Dq C
+and 
+.Dq Cn
+options.
+.Dq C0
+can be used to clear only this option, if needed.
 .It J
 Each byte of new memory allocated by
 .Fn malloc ,
 .Fn realloc
 or
-.Fn reallocf
-as well as all memory returned by
+.Fn reallocf 
+will be initialized to 0xd0.
+All memory returned by
 .Fn free ,
 .Fn realloc
 or
 .Fn reallocf
-will be initialized to 0xd0.
-This options also sets the
+will be initialized to 0xd1.
+This option also sets the
 .Dq R
 option.
 This is intended for debugging and will impact performance negatively.
@@ -281,6 +307,12 @@ If the environment variable
 .Ev MALLOC_OPTIONS
 is set, the characters it contains will be interpreted as flags to the
 allocation functions.
+.It Ev MALLOC_COUNT
+If the environment variable
+.Ev MALLOC_COUNT
+is set, it will be interpreted as a count value (see the 
+.Dq Cn
+option).  This value will override any counts specified in MALLOC_OPTIONS.
 .El
 .Sh RETURN VALUES
 The
Index: malloc.c
===================================================================
RCS file: /home/ncvs/src/lib/libc/stdlib/malloc.c,v
retrieving revision 1.76
diff -u -p -r1.76 malloc.c
--- malloc.c	1 Jun 2003 09:16:50 -0000	1.76
+++ malloc.c	3 Jun 2003 15:21:12 -0000
@@ -26,7 +26,8 @@ __FBSDID("$FreeBSD: src/lib/libc/stdlib/
  * What to use for Junk.  This is the byte value we use to fill with
  * when the 'J' option is enabled.
  */
-#define SOME_JUNK	0xd0		/* as in "Duh" :-) */
+#define ALLOC_JUNK	0xd0		/* as in "Duh" :-) */
+#define FREE_JUNK	0xd1		/* as in "Die" :-) */
 
 /*
  * The basic parameters you can tweak.
@@ -248,6 +249,15 @@ static int malloc_zero;
 /* junk fill ?  */
 static int malloc_junk = 1;
 
+/* junk fill with a counter ? */
+static int malloc_counter_junk = 0;
+
+/* keep track of the total number of malloc/realloc/frees done */
+static u_int32_t malloc_counter = 0;
+
+/* If this is nonzero, die when malloc_counter == malloc_counter_abort */
+static u_int32_t malloc_counter_abort = 0;
+
 #ifdef HAS_UTRACE
 
 /* utrace ?  */
@@ -288,6 +298,7 @@ static int extend_pgdir(u_long index);
 static void *imalloc(size_t size);
 static void ifree(void *ptr);
 static void *irealloc(void *ptr, size_t size);
+static void fillrange(void *ptr, size_t size, u_int32_t value, u_char pad);
 
 static void
 wrtmessage(const char *p1, const char *p2, const char *p3, const char *p4)
@@ -445,6 +456,16 @@ malloc_init ()
 		case '<': malloc_cache   >>= 1; break;
 		case 'a': malloc_abort   = 0; break;
 		case 'A': malloc_abort   = 1; break;
+		case 'c': malloc_counter_abort = 0; malloc_counter_junk = 0; break;
+		case 'C':
+		    if (p[1] >= '0' && p[1] <= '9') {
+		        errnosave = errno;
+		        malloc_counter_abort = strtoul(p+1, &p, 0);
+		        errno = errnosave;
+		        p--; /* decrement to compensate for the for loop */
+		    } else
+		        malloc_counter_junk = 1;
+		    break;
 		case 'h': malloc_hint    = 0; break;
 		case 'H': malloc_hint    = 1; break;
 		case 'r': malloc_realloc = 0; break;
@@ -471,6 +492,13 @@ malloc_init ()
 	}
     }
 
+    p = getenv("MALLOC_COUNT");
+    if (p) {
+	errnosave = errno;
+	malloc_counter_abort = strtoul(p, NULL, 0);
+	errno = errnosave;
+    }
+
     /*
      * Sensitive processes, somewhat arbitrarily defined here as setuid,
      * setgid, root and wheel cannot afford to have malloc mistakes.
@@ -481,6 +509,13 @@ malloc_init ()
     UTRACE(0, 0, 0);
 
     /*
+     * Filling junk with a running count implies filing with junk in the
+     * first place.
+     */
+    if (malloc_counter_junk)
+	malloc_junk=1;
+
+    /*
      * We want junk in the entire allocation, and zero only in the part
      * the user asked for.
      */
@@ -598,7 +633,7 @@ malloc_pages(size_t size)
 	    page_dir[index+i] = MALLOC_FOLLOW;
 
 	if (malloc_junk)
-	    memset(p, SOME_JUNK, size << malloc_pageshift);
+	    fillrange(p, size << malloc_pageshift, malloc_counter, ALLOC_JUNK);
     }
 
     if (delay_free) {
@@ -733,7 +768,7 @@ malloc_bytes(size_t size)
     k <<= bp->shift;
 
     if (malloc_junk)
-	memset((u_char*)bp->page + k, SOME_JUNK, bp->size);
+	fillrange((u_char*)bp->page + k, bp->size, malloc_counter, ALLOC_JUNK);
 
     return (u_char *)bp->page + k;
 }
@@ -891,7 +926,7 @@ free_pages(void *ptr, u_long index, stru
     l = i << malloc_pageshift;
 
     if (malloc_junk)
-	memset(ptr, SOME_JUNK, l);
+	fillrange(ptr, l, malloc_counter, FREE_JUNK);
 
     if (malloc_hint)
 	madvise(ptr, l, MADV_FREE);
@@ -1012,7 +1047,7 @@ free_bytes(void *ptr, u_long index, stru
     }
 
     if (malloc_junk)
-	memset(ptr, SOME_JUNK, info->size);
+	fillrange(ptr, info->size, malloc_counter, FREE_JUNK);
 
     info->bits[i/MALLOC_BITS] |= 1<<(i%MALLOC_BITS);
     info->free++;
@@ -1093,6 +1128,28 @@ ifree(void *ptr)
     return;
 }
 
+static void fillrange(void *ptr, size_t size, u_int32_t value, u_char pad)
+{
+    int i;
+
+    if (malloc_counter_junk)
+    {
+	u_int32_t fillvalue;
+	if (value < 256) // fits in one byte: D0D0D012
+	    fillvalue = pad<<24 | pad<<16 | pad<<8 | value;
+	else if (value < 65536) // fits in two bytes D0D01234
+	    fillvalue = pad<<24 | pad<<16 | value;
+	else // D0123456
+	    fillvalue = pad<<24 | (value & 0x00FFFFFF);
+
+	for(i = 0; i < (size/4)*4 ; i += 4) 
+	    ((u_int32_t *)ptr)[i/4] = fillvalue;
+    }
+
+    for( ; i < size ; i++) 
+	((u_char *)ptr)[i] = pad;
+}
+
 /*
  * These are the public exported interface routines.
  */
@@ -1114,6 +1171,9 @@ malloc(size_t size)
     }
     if (!malloc_started)
 	malloc_init();
+    malloc_counter++;
+    if (malloc_counter_abort && malloc_counter == malloc_counter_abort)
+    	wrterror("Aborting as requested\n");
     if (malloc_sysv && !size)
 	r = 0;
     else if (!size)
@@ -1142,6 +1202,9 @@ free(void *ptr)
 	errno = EDOOFUS;
 	return;
     }
+    malloc_counter++;
+    if (malloc_counter_abort && malloc_counter == malloc_counter_abort)
+    	wrterror("Aborting as requested\n");
     if (ptr != ZEROSIZEPTR)
 	    ifree(ptr);
     UTRACE(ptr, 0, 0);
@@ -1171,6 +1234,9 @@ realloc(void *ptr, size_t size)
     }		
     if (!malloc_started)
 	malloc_init();
+    malloc_counter++;
+    if (malloc_counter_abort && malloc_counter == malloc_counter_abort)
+    	wrterror("Aborting as requested\n");
     if (ptr == ZEROSIZEPTR)
 	ptr = NULL;
     if (malloc_sysv && !size) {

	


>Release-Note:
>Audit-Trail:

From: Dan Nelson <dnelson@allantgroup.com>
To: freebsd-gnats-submit@FreeBSD.org
Cc:  
Subject: Re: bin/52907: [PATCH] more malloc options for debugging programs
Date: Thu, 16 Oct 2003 09:47:08 -0500

 An updated patch, to keep up with changes in -current:
 
 Index: lib/libc/stdlib/malloc.3
 ===================================================================
 RCS file: /home/ncvs/src/lib/libc/stdlib/malloc.3,v
 retrieving revision 1.60
 diff -p -u -r1.60 malloc.3
 --- lib/libc/stdlib/malloc.3	24 Dec 2002 13:41:45 -0000	1.60
 +++ lib/libc/stdlib/malloc.3	3 Jun 2003 15:10:13 -0000
 @@ -177,19 +177,45 @@ flags being set) become fatal.
  The process will call
  .Xr abort 3
  in these cases.
 +.It C
 +This option sets the 
 +.Dq J
 +option, keeps a count of the total number of memory allocation calls,
 +and when filling memory with junk, initializes the low-order bytes in each 
 +4-byte word with the running count.
 +For example, the 255th malloc operation will be initialized with 0xd0d0d0ff, 
 +and the next free after that will have its memory set to 0xd1d10100.
 +Note that there is some ambiguity in interpreting these values;
 +0xd0d0d0ff could also have been the 53503rd or 13684991th operation.
 +.It Cn
 +Keep a count of the total number of memory allocation calls,
 +and abort at the nth operation.
 +n can be specified in decimal, or it can be hexidecimal (prefixed with 0x).
 +If a hex value is used, ensure that no options that may be confused with
 +hex digits follow this option.
 +A lowercase 
 +.Dq c
 +will clear both the 
 +.Dq C
 +and 
 +.Dq Cn
 +options.
 +.Dq C0
 +can be used to clear only this option, if needed.
  .It J
  Each byte of new memory allocated by
  .Fn malloc ,
  .Fn realloc
  or
 -.Fn reallocf
 -as well as all memory returned by
 +.Fn reallocf 
 +will be initialized to 0xd0.
 +All memory returned by
  .Fn free ,
  .Fn realloc
  or
  .Fn reallocf
 -will be initialized to 0xd0.
 -This options also sets the
 +will be initialized to 0xd1.
 +This option also sets the
  .Dq R
  option.
  This is intended for debugging and will impact performance negatively.
 @@ -281,6 +307,12 @@ If the environment variable
  .Ev MALLOC_OPTIONS
  is set, the characters it contains will be interpreted as flags to the
  allocation functions.
 +.It Ev MALLOC_COUNT
 +If the environment variable
 +.Ev MALLOC_COUNT
 +is set, it will be interpreted as a count value (see the 
 +.Dq Cn
 +option).  This value will override any counts specified in MALLOC_OPTIONS.
  .El
  .Sh RETURN VALUES
  The
 Index: lib/libc/stdlib/malloc.c
 ===================================================================
 RCS file: /home/ncvs/src/lib/libc/stdlib/malloc.c,v
 retrieving revision 1.79
 diff -p -u -r1.79 malloc.c
 --- lib/libc/stdlib/malloc.c	27 Sep 2003 18:58:26 -0000	1.79
 +++ lib/libc/stdlib/malloc.c	30 Sep 2003 18:45:11 -0000
 @@ -26,7 +26,8 @@ __FBSDID("$FreeBSD: src/lib/libc/stdlib/
   * What to use for Junk.  This is the byte value we use to fill with
   * when the 'J' option is enabled.
   */
 -#define SOME_JUNK	0xd0		/* as in "Duh" :-) */
 +#define ALLOC_JUNK	0xd0		/* as in "Duh" :-) */
 +#define FREE_JUNK	0xd1		/* as in "Die" :-) */
  
  /*
   * The basic parameters you can tweak.
 @@ -248,6 +249,15 @@ static int malloc_zero;
  /* junk fill ?  */
  static int malloc_junk = 1;
  
 +/* junk fill with a counter ? */
 +static int malloc_counter_junk = 0;
 +
 +/* keep track of the total number of malloc/realloc/frees done */
 +static u_int32_t malloc_counter = 0;
 +
 +/* If this is nonzero, die when malloc_counter == malloc_counter_abort */
 +static u_int32_t malloc_counter_abort = 0;
 +
  #ifdef HAS_UTRACE
  
  /* utrace ?  */
 @@ -288,6 +298,7 @@ static int extend_pgdir(u_long index);
  static void *imalloc(size_t size);
  static void ifree(void *ptr);
  static void *irealloc(void *ptr, size_t size);
 +static void fillrange(void *ptr, size_t size, u_int32_t value, u_char pad);
  
  static void
  wrtmessage(const char *p1, const char *p2, const char *p3, const char *p4)
 @@ -445,6 +456,16 @@ malloc_init ()
  		case '<': malloc_cache   >>= 1; break;
  		case 'a': malloc_abort   = 0; break;
  		case 'A': malloc_abort   = 1; break;
 +		case 'c': malloc_counter_abort = 0; malloc_counter_junk = 0; break;
 +		case 'C':
 +		    if (p[1] >= '0' && p[1] <= '9') {
 +		        errnosave = errno;
 +		        malloc_counter_abort = strtoul(p+1, &p, 0);
 +		        errno = errnosave;
 +		        p--; /* decrement to compensate for the for loop */
 +		    } else
 +		        malloc_counter_junk = 1;
 +		    break;
  		case 'h': malloc_hint    = 0; break;
  		case 'H': malloc_hint    = 1; break;
  		case 'r': malloc_realloc = 0; break;
 @@ -471,6 +492,13 @@ malloc_init ()
  	}
      }
  
 +    p = getenv("MALLOC_COUNT");
 +    if (p) {
 +	errnosave = errno;
 +	malloc_counter_abort = strtoul(p, NULL, 0);
 +	errno = errnosave;
 +    }
 +
      /*
       * Sensitive processes, somewhat arbitrarily defined here as setuid,
       * setgid, root and wheel cannot afford to have malloc mistakes.
 @@ -481,6 +509,13 @@ malloc_init ()
      UTRACE(0, 0, 0);
  
      /*
 +     * Filling junk with a running count implies filing with junk in the
 +     * first place.
 +     */
 +    if (malloc_counter_junk)
 +	malloc_junk=1;
 +
 +    /*
       * We want junk in the entire allocation, and zero only in the part
       * the user asked for.
       */
 @@ -598,7 +633,7 @@ malloc_pages(size_t size)
  	    page_dir[index+i] = MALLOC_FOLLOW;
  
  	if (malloc_junk)
 -	    memset(p, SOME_JUNK, size << malloc_pageshift);
 +	    fillrange(p, size << malloc_pageshift, malloc_counter, ALLOC_JUNK);
      }
  
      if (delay_free) {
 @@ -733,7 +768,7 @@ malloc_bytes(size_t size)
      k <<= bp->shift;
  
      if (malloc_junk)
 -	memset((u_char*)bp->page + k, SOME_JUNK, bp->size);
 +	fillrange((u_char*)bp->page + k, bp->size, malloc_counter, ALLOC_JUNK);
  
      return ((u_char *)bp->page + k);
  }
 @@ -906,7 +941,7 @@ free_pages(void *ptr, u_long index, stru
      l = i << malloc_pageshift;
  
      if (malloc_junk)
 -	memset(ptr, SOME_JUNK, l);
 +	fillrange(ptr, l, malloc_counter, FREE_JUNK);
  
      if (malloc_hint)
  	madvise(ptr, l, MADV_FREE);
 @@ -1028,7 +1063,7 @@ free_bytes(void *ptr, u_long index, stru
      }
  
      if (malloc_junk)
 -	memset(ptr, SOME_JUNK, info->size);
 +	fillrange(ptr, info->size, malloc_counter, FREE_JUNK);
  
      info->bits[i/MALLOC_BITS] |= 1<<(i%MALLOC_BITS);
      info->free++;
 @@ -1109,6 +1144,28 @@ ifree(void *ptr)
      return;
  }
  
 +static void fillrange(void *ptr, size_t size, u_int32_t value, u_char pad)
 +{
 +    int i = 0;
 +
 +    if (malloc_counter_junk)
 +    {
 +	u_int32_t fillvalue;
 +	if (value < 256) // fits in one byte: D0D0D012
 +	    fillvalue = pad<<24 | pad<<16 | pad<<8 | value;
 +	else if (value < 65536) // fits in two bytes D0D01234
 +	    fillvalue = pad<<24 | pad<<16 | value;
 +	else // D0123456
 +	    fillvalue = pad<<24 | (value & 0x00FFFFFF);
 +
 +	for( ; i < (size/4)*4 ; i += 4) 
 +	    ((u_int32_t *)ptr)[i/4] = fillvalue;
 +    }
 +
 +    for( ; i < size ; i++) 
 +	((u_char *)ptr)[i] = pad;
 +}
 +
  /*
   * These are the public exported interface routines.
   */
 @@ -1128,6 +1185,11 @@ malloc(size_t size)
  	errno = EDOOFUS;
  	return (NULL);
      }
 +    if (!malloc_started)
 +	malloc_init();
 +    malloc_counter++;
 +    if (malloc_counter_abort && malloc_counter == malloc_counter_abort)
 +    	wrterror("Aborting as requested\n");
      if (malloc_sysv && !size)
  	r = NULL;
      else if (!size)
 @@ -1156,6 +1218,9 @@ free(void *ptr)
  	errno = EDOOFUS;
  	return;
      }
 +    malloc_counter++;
 +    if (malloc_counter_abort && malloc_counter == malloc_counter_abort)
 +    	wrterror("Aborting as requested\n");
      if (ptr != ZEROSIZEPTR)
  	    ifree(ptr);
      UTRACE(ptr, 0, 0);
 @@ -1179,6 +1244,15 @@ realloc(void *ptr, size_t size)
  	errno = EDOOFUS;
  	return (NULL);
      }
 +    if (ptr && !malloc_started) {
 +	wrtwarning("malloc() has never been called\n");
 +	ptr = 0;
 +    }		
 +    if (!malloc_started)
 +	malloc_init();
 +    malloc_counter++;
 +    if (malloc_counter_abort && malloc_counter == malloc_counter_abort)
 +    	wrterror("Aborting as requested\n");
      if (ptr == ZEROSIZEPTR)
  	ptr = NULL;
      if (malloc_sysv && !size) {
 
Responsible-Changed-From-To: freebsd-bugs->phk 
Responsible-Changed-By: kris 
Responsible-Changed-When: Thu Nov 6 18:52:41 PST 2003 
Responsible-Changed-Why:  
Assign to malloc author 

http://www.freebsd.org/cgi/query-pr.cgi?pr=52907 
State-Changed-From-To: open->closed 
State-Changed-By: phk 
State-Changed-When: Thu May 17 19:02:56 UTC 2007 
State-Changed-Why:  
This PR got overtaken by other events and is no longer relevant. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=52907 
>Unformatted:
