From nobody@FreeBSD.ORG  Mon Nov 13 13:48:20 2000
Return-Path: <nobody@FreeBSD.ORG>
Received: by hub.freebsd.org (Postfix, from userid 32767)
	id 109C437B479; Mon, 13 Nov 2000 13:48:20 -0800 (PST)
Message-Id: <20001113214820.109C437B479@hub.freebsd.org>
Date: Mon, 13 Nov 2000 13:48:20 -0800 (PST)
From: hanche@math.ntnu.no
Sender: nobody@FreeBSD.ORG
To: freebsd-gnats-submit@FreeBSD.org
Subject: Memory limits have no effect in linux compatibility
X-Send-Pr-Version: www-1.0

>Number:         22826
>Category:       kern
>Synopsis:       [linux] Memory limits have no effect in linux compatibility
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    freebsd-emulation
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Mon Nov 13 13:50:01 PST 2000
>Closed-Date:    Sat Oct 29 16:33:55 GMT 2005
>Last-Modified:  Sat Oct 29 16:33:55 GMT 2005
>Originator:     Harald Hanche-Olsen
>Release:        4.1-RELEASE
>Organization:
Norwegian University of Science and Technology
>Environment:
FreeBSD fiinbeck.math.ntnu.no 4.1-RELEASE FreeBSD 4.1-RELEASE #2: Fri Sep i386 unknown

>Description:
Memory limits, soft or hard, appear to have no effect on programs
running under Linux emulation, allowing them to allocate memory until
the machine runs out of virtual memory.
>How-To-Repeat:
Set a low memory limit.  Soft or hard, it doesn't matter:
$ ulimit -H -m 8192
Run a linux program which likes to allocate lots of memory,
and watch it grow, and grow, and grow.
I discovered this using acroread 4.05, which has
a marvelous memory leak - when the display covers most of the screen,
it grows by several MB each time you move to a different page.
I had it grow to over 100 MB with a memory limit of just 8 MB.
>Fix:


>Release-Note:
>Audit-Trail:

From: Bruce Evans <bde@zeta.org.au>
To: hanche@math.ntnu.no
Cc: freebsd-gnats-submit@FreeBSD.ORG
Subject: Re: kern/22826: Memory limits have no effect in linux compatibility
Date: Tue, 14 Nov 2000 20:50:20 +1100 (EST)

 On Mon, 13 Nov 2000 hanche@math.ntnu.no wrote:
 
 > >Description:
 > Memory limits, soft or hard, appear to have no effect on programs
 > running under Linux emulation, allowing them to allocate memory until
 > the machine runs out of virtual memory.
 > >How-To-Repeat:
 > Set a low memory limit.  Soft or hard, it doesn't matter:
 > $ ulimit -H -m 8192
 
 I think you want to use -d (data segment size) and maybe -s (stack
 size) instead of -m (max memory size) (better described as max
 resident set in setrlimit(2)).  -m limits use of real memory but
 doesn't limit use of virtual memory.
 
 > Run a linux program which likes to allocate lots of memory,
 > and watch it grow, and grow, and grow.
 > I discovered this using acroread 4.05, which has
 > a marvelous memory leak - when the display covers most of the screen,
 > it grows by several MB each time you move to a different page.
 > I had it grow to over 100 MB with a memory limit of just 8 MB.
 
 Marvelous indeed.  It has (effectively) hung my system a couple of
 times with earlier versions of acroread4.
 
 Bruce
 
 

From: Harald Hanche-Olsen <hanche@math.ntnu.no>
To: bde@zeta.org.au
Cc: freebsd-gnats-submit@FreeBSD.ORG
Subject: Re: kern/22826: Memory limits have no effect in linux compatibility
Date: Tue, 14 Nov 2000 12:24:32 +0100

 + Bruce Evans <bde@zeta.org.au>:
 
 | On Mon, 13 Nov 2000 hanche@math.ntnu.no wrote:
 | 
 | > Set a low memory limit.  Soft or hard, it doesn't matter:
 | > $ ulimit -H -m 8192
 | 
 | I think you want to use -d (data segment size) and maybe -s (stack
 | size) instead of -m (max memory size) (better described as max
 | resident set in setrlimit(2)).  -m limits use of real memory but
 | doesn't limit use of virtual memory.
 
 Oh, I didn't know that.  (In fact, it's a bit tricky to correlate the
 limits described in setrlimit(1) with those handled by the (u)limit
 command in various shells.
 
 More to the point, however, in my experiment I used both -d and -s as
 well as -m, and still the process grew without bound.  I'm sorry, I
 should have mentioned that.
 
 - Harald
 

From: Bruce Evans <bde@zeta.org.au>
To: Harald Hanche-Olsen <hanche@math.ntnu.no>
Cc: freebsd-gnats-submit@FreeBSD.ORG
Subject: Re: kern/22826: Memory limits have no effect in linux compatibility
Date: Wed, 15 Nov 2000 00:47:13 +1100 (EST)

 On Tue, 14 Nov 2000, Harald Hanche-Olsen wrote:
 
 > More to the point, however, in my experiment I used both -d and -s as
 > well as -m, and still the process grew without bound.  I'm sorry, I
 > should have mentioned that.
 
 mmap() ignores all of the limits even for native processes, so the
 limits are no use for limiting buggy or malicious programs that use
 mmap().
 
 vm has surprisingly few references to the limits.  Most are just to
 set the defaults.  The others are:
 - limit on normal stack growth in vm_map.c should work.  I think
   programs can still build abnormally large stacks using mmap().
 - limit on locked memory in vm_mmap.c should work.
 - limit on rss in vm_pageout.c should work.  It is only used when
   the system becomes short of memory.
 - limit on data size in vm_unix.c works, but only affects the
   break(2).  This limits malloc(3).
 
 The data limit also affects loading of programs, but there are
 problems there too.  It has no effect for elf programs.
 
 Bruce
 
 
Responsible-Changed-From-To: freebsd-bugs->marcel 
Responsible-Changed-By: marcel 
Responsible-Changed-When: Sun Dec 3 13:56:47 PST 2000 
Responsible-Changed-Why:  
My plate. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=22826 

From: Marcel Moolenaar <marcel@cup.hp.com>
To: freebsd-gnats-submit@FreeBSD.org, hanche@math.ntnu.no
Cc: bde@FreeBSD.org
Subject: Re: kern/22826: Memory limits have no effect in linux compatibility
Date: Sun, 03 Dec 2000 16:30:05 -0800

 The reason Linux binaries don't respect the limits is as follows:
 
 In glibc2 allocations larger than a threshold (128K) use mmap(). Below
 the threshold brk() is used. Since our native mmap doesn't respect the
 limits, Linux binaries won't always fail when allocating more than the
 limit.
 
 The following program has been used to verify this.
 
 \begin{mem.c}
 #include <ctype.h>
 #include <limits.h>
 #include <stdlib.h>
 #include <unistd.h>
 
 int main(int argc, char *argv[])
 {
         size_t blksize, blkcount, memsize;
         char *ptr, *suffix;
 
         if (argc != 2)
                 errx(1, "add size of allocated memory");
 
         memsize = strtoul(argv[1], &suffix, 0);
         if (suffix == argv[1])
                 errx(1, "invalid memory size");
 
         switch (toupper(*suffix)) {
         case '\0':
                 break;
         case 'K':
                 memsize *= 1024;
                 break;
         case 'M':
                 memsize *= 1024 * 1024;
                 break;
         default:
                 errx(1, "invalid suffix");
         }
 
         blksize = 64*1024;
         blkcount = 0;
 
 #ifdef SINGLE_ALLOC
 
         ptr = malloc(memsize);
         if (ptr == NULL)
                 errx(1, "malloc failed");
 
 #else
 
         while (memsize) {
                 blkcount++;
                 ptr = malloc(blksize);
                 if (ptr == NULL)
                         errx(1, "malloc failed: block %d; %d remaining",
                             blkcount, memsize);
                 memsize -= blksize;
         }
 
 #endif
 
         return 0;
 }
 \end{mem.c}
 
 If compiled with -DSINGLE_ALLOC, the linux version (linuxmem) will
 succeed where the FreeBSD version (bsdmem) will fail:
 
 % limit datasize 16M
 % ./bsdmem 17M
 bsdmem: malloc failed
 % ./linuxmem 17M
 % 
 
 If the allocation size is reduced to below the mmap() threshold, both
 versions will fail as expected:
 
 % limit datasize 16M
 % ./bsdmem 17M
 bsdmem: malloc failed: block 256; 1114112 remaining
 % ./linuxmem 17M
 linuxmem: malloc failed: block 256; 1114112 remaining
 %
 
 The bug therefore is in our native mmap() syscall and not in the
 Linuxulator.
 
 I'll see if I can fix our mmap, although I'm no VM guy. If I don't
 manage (likely, but you'll never know until you try), I'll either file a
 new PR or bug someone who should be able to fix it.
 
 FYI,
 
 -- 
 Marcel Moolenaar
   mail: marcel@cup.hp.com / marcel@FreeBSD.org
   tel:  (408) 447-4222
 
Responsible-Changed-From-To: marcel->emulation 
Responsible-Changed-By: marcel 
Responsible-Changed-When: Sat Nov 17 22:16:29 PST 2001 
Responsible-Changed-Why:  
Assign to emulation@FreeBSD.org. It is not going to be addressed if 
it's assigned to me and I don't do it. Maintainership of the Linuxulator 
has been passed on to emulation@FreeBSD.org as well. 

http://www.FreeBSD.org/cgi/query-pr.cgi?pr=22826 

From: Nate Eldredge <nge@cs.hmc.edu>
To: bug-followup@FreeBSD.org, hanche@math.ntnu.no
Cc:  
Subject: Re: kern/22826: Memory limits have no effect in linux compatibility
Date: Mon, 17 Oct 2005 23:23:12 -0700 (PDT)

 The "bug" here is that mmap does not respect the datasize limit (ulimit 
 -d).  But this may actually be correct.  mmap'ed space is in some sense 
 not part of the "data segment"; at least, it is not necessarily contiguous 
 with the main data segment.  I tested Linux and Solaris, and neither one 
 subjects mmap to the datasize limit.  The Linux malloc is a little 
 counter-intuitive in that it may use mmap to acquire its memory, thus 
 placing it outside the data segment and beyond the purview of ulimit -d, 
 but I'm not sure I would call this a bug.
 
 mmap is subject to the virtual memory limit (ulimit -v) on FreeBSD, as it 
 is on Linux and Solaris as well.  So this is probably the right limit to 
 set if you want to restrict how much memory the process can use.
 
 Maintainers, consider closing this?
 
 -- 
 Nate Eldredge
 nge@cs.hmc.edu

From: Harald Hanche-Olsen <hanche@math.ntnu.no>
To: bug-followup@FreeBSD.org
Cc: nge@cs.hmc.edu
Subject: Re: kern/22826: Memory limits have no effect in linux compatibility
Date: Sat, 29 Oct 2005 18:10:12 +0200 (CEST)

 As the originator of this PR, I guess I should chime in, though this
 is a bit outside my expertise.  Nate's explanation seems reasonable to
 me, so I think it is indeed time to close this one.  (And the problem
 doesn't bite me anymore, anyway.)
 
 - Harald
State-Changed-From-To: open->closed 
State-Changed-By: linimon 
State-Changed-When: Sat Oct 29 16:33:25 GMT 2005 
State-Changed-Why:  
Closed at submitter's request. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=22826 
>Unformatted:
