From hsn@netmag.cz  Wed Jan 21 14:18:59 2004
Return-Path: <hsn@netmag.cz>
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id D058716A4CF
	for <FreeBSD-gnats-submit@freebsd.org>; Wed, 21 Jan 2004 14:18:48 -0800 (PST)
Received: from mail.tiscali.cz (stateless3.tiscali.cz [213.235.135.72])
	by mx1.FreeBSD.org (Postfix) with ESMTP id B736643D2D
	for <FreeBSD-gnats-submit@freebsd.org>; Wed, 21 Jan 2004 14:18:47 -0800 (PST)
	(envelope-from hsn@netmag.cz)
Received: from asura.bsd (212.90.236.232) by mail.tiscali.cz (6.7.018)
        id 3FB9693501075949 for FreeBSD-gnats-submit@freebsd.org; Wed, 21 Jan 2004 23:18:46 +0100
Received: from hsn by asura.bsd with local (Exim 4.24 #4 (Debian))
	id 1AjPRc-0000PN-1v
	for <FreeBSD-gnats-submit@freebsd.org>; Wed, 21 Jan 2004 21:59:24 +0100
Message-Id: <E1AjPRc-0000PN-1v@asura.bsd>
Date: Wed, 21 Jan 2004 21:59:24 +0100
From: Radim Kolar <hsn@netmag.cz>
Reply-To: Radim Kolar <hsn@netmag.cz>
To: FreeBSD-gnats-submit@freebsd.org
Cc:
Subject: very bad performance of realloc()
X-Send-Pr-Version: 3.113
X-GNATS-Notify:

>Number:         61691
>Category:       kern
>Synopsis:       very bad performance of realloc()
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    freebsd-bugs
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Wed Jan 21 14:20:10 PST 2004
>Closed-Date:    Thu Jan 22 06:57:51 PST 2004
>Last-Modified:  Wed Feb 27 10:00:00 UTC 2013
>Originator:     Radim Kolar
>Release:        FreeBSD 5.2-RELEASE i386
>Organization:
Sanatana Dharma
>Environment:
System: FreeBSD asura.bsd 5.2-RELEASE FreeBSD 5.2-RELEASE #0: Thu Jan 15 18:35:03 CET 2004 root@asura.bsd:/usr/obj/usr/src/sys/GENERIC i386

>Description:
Prob 1: 
Current implementation of realloc(3) is slow, it looks like it copies data
on reallocation even when it is not needed. This causes that programs
which are using more realloc calls runs very slowly. I think that
default malloc flag do not includes 'R', so it should run very fast.

Prob 2: Second problem is if you reallocate to more than 18 MB of memory,
malloc eats  about  3  times  more memory from system, than needed for
allocation. Maybe this have something to do with possible exhausting malloc
cache size (default 16 pages). 

I have used default malloc flags, using madvice has zero effect.
>How-To-Repeat:
I have wrote a small testing program.
You can get it from http://home.worldonline.cz/~cz210552/forkbomb.html

prob1:
run it ./forkbomb -m -l 64 under FreeBSD and Linux/gnulibc 2.3.2.

prob2:
run it ./forkbomb -m -l 20, see top(1) how many memory  is allocated (61MB)
run it ./forkbomb -m -l 16, looks okay (except slowness) (17.7MB)
>Fix:
Port glibc2.3 memory allocation alg. to freebsd libc.
	
>Release-Note:
>Audit-Trail:
State-Changed-From-To: open->closed 
State-Changed-By: phk 
State-Changed-When: Thu Jan 22 06:56:15 PST 2004 
State-Changed-Why:  
You seem to be confused as to what responsibilities the programmer 
has when it comes to practicing sensible memory management. 

Your also seem to be unaware of the difference between swap and VM  
based memory management. 


http://www.freebsd.org/cgi/query-pr.cgi?pr=61691 

From: Radim Kolar <hsn@netmag.cz>
To: Poul-Henning Kamp <phk@FreeBSD.org>
Cc: freebsd-bugs@FreeBSD.org, bug-followup@freebsd.org
Subject: Re: kern/61691: very bad performance of realloc()/brk()
Date: Fri, 23 Jan 2004 16:20:35 +0100

 > You seem to be confused as to what responsibilities the programmer
 > has when it comes to practicing sensible memory management.
 I have run some benchmarks for you. There are doing 32 times realloc() in
 1 MB chunks. I don't think that this is example of bad programming
 practice. The major problem is that realloc() copies data while Linux
 doesn't.
 
 FreeBSD 5.2
 ===========
 It looks that brk() syscall is quite slow in FreeBSD when comparing
 against Linux. FreeBSD calls 32 times brk() and 10 times mmap+munmap (for page directory).
 
 (hsn@ttyv0):~/forkbomb% time ./forkbomb -l 32 -i 256 -M --quit            12:45
 Safety alarm at 300 sec. enabled.
 Actions: alloc 32 MB (step 1024 kB) and touch it.
 Forkbomb 1.2 started.
 ./forkbomb -l 32 -i 256 -M --quit  4.58s user 5.28s system 74% cpu 13.159 total
 
 linux2.4+glibc2.3.2
 ===================
 Linux does 1 times mmap + 31 times mremap syscall
 
 Safety alarm at 300 sec. enabled.
 Actions: alloc 32 MB (step 1024 kB) and touch it.
 Forkbomb 1.2 started.
 ./forkbomb -l 32 -i 256 -M --quit  0.00s user 0.30s system 107% cpu 0.280 total
 
 Well my pr-report/wish is: optimize realloc() function (which is about 3 pages
 long) to avoid copying data while brk() is sufficient , because the reallocated block is
 last block. 

From: "Poul-Henning Kamp" <phk@phk.freebsd.dk>
To: Radim Kolar <hsn@netmag.cz>
Cc: freebsd-bugs@FreeBSD.org, bug-followup@FreeBSD.org
Subject: Re: kern/61691: very bad performance of realloc()/brk() 
Date: Fri, 23 Jan 2004 21:51:36 +0100

 In message <20040123152035.GA2311@asura.bsd>, Radim Kolar writes:
 >> You seem to be confused as to what responsibilities the programmer
 >> has when it comes to practicing sensible memory management.
 >I have run some benchmarks for you. There are doing 32 times realloc() in
 >1 MB chunks. I don't think that this is example of bad programming
 >practice. The major problem is that realloc() copies data while Linux
 >doesn't.
 
 No, as I said, the problem is you use realloc the wrong way.
 
 You should not
 
 	malloc(1M)
 	realloc(2M)
 	realloc(3M)
 
 You should
 
 	realloc(32M)
 	do stuff
 	realloc(now_i_know_the_size)
 
 For whatever value of 32M 99% of your data fit in.
 	
 
 -- 
 Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
 phk@FreeBSD.ORG         | TCP/IP since RFC 956
 FreeBSD committer       | BSD since 4.3-tahoe    
 Never attribute to malice what can adequately be explained by incompetence.

From: Niclas Zeising <zeising@daemonic.se>
To: bug-followup@FreeBSD.org, hsn@netmag.cz
Cc:  
Subject: Re: kern/61691: very bad performance of realloc()
Date: Wed, 27 Feb 2013 10:36:55 +0100

 Ping!
 Is this still an issue with the new malloc subsystem (jmemalloc)?
 -- 
 Niclas Zeising

From: Niclas Zeising <zeising@daemonic.se>
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/61691: very bad performance of realloc()
Date: Wed, 27 Feb 2013 10:50:41 +0100

 Repeat after me:
 No PRs before coffee.  No PRs before coffee.
 Regards!
 -- 
 Niclas Zeising
>Unformatted:
