Subj : Re: Memory growth problem
To   : Mike McIntosh <mike@jeezo.com>
From : Brendan Eich <brendan@meer.net>
Date : Sat May 17 2003 01:14 pm

Mike McIntosh wrote:

>Eric/Brendan,
>
>Thanks for those links. I've spent best part of a week trying to be sure
>that I'm not leaking in my app. I found a few minor things on the way (which
>is good) but it seems ok. [We can run same test transaction in non-JS mode
>which basically goes through same app code without any increase in  memory.]
>

The leaks or bloat wouldn't be in your app alone, but only in 
combination with the JS engine, especially via roots and other strong 
references in the live-thing graph the GC scans.

>The results from this _appear_ to be showing memory being allocated per
>transaction in the engine (see below). I've also added rogue leaks to both
>the engine and my app and they _do_ get reported by the CRTDBG so I have
>_reasonable_ confidence in it. Blocks allocated by my app are marked as
>client blocks, while spidermonkey will be normal blocks.
>

The problem with such tracing is that it doesn't show stack backtraces, 
so common subroutines that allocate get blame, and the ancestral callers 
who trigger the wrongful bloat or leak are hard to see, or guess at.  
That's why I was recommended trace-malloc: it gives a full 
(cycle-compressed, to reduce depth due to recursion) stack backtrace per 
allocation, using a callsite tree.

>I've also run this under Boundschecker and it does not leak at exit so the
>problem seems one of accumulation of memory which is eventually released,
>not a leak as such.
>
>I was really interested in whether the scoping technique was the correct way
>to clean up after each execution?. But the finalizers for custom objects
>_do_ seem to be triggered at end of scope. Am I misinterpreting how the GC
>works?.  Where/when is memory allocated by JS_GetStringChars (which takes no
>context as parameter) released?
>

JS_GetStringChars doesn't allocate memory in the usual case.  If it does 
allocate, to flatten a dependent string and give it a zero terminator, 
it's as if you called JS_NewUCStringCopy* to create the string to create 
an independent string with the same chars content up front.  In all 
cases, strings are garbage collected, so unless you have a stuck GC 
root, or some equivalent strong reference, the string will be collected 
when it is no longer in use.

If you'd said JS_GetStringBytes, I would have pointed to the 
deflated_string_cache (jsstr.c), which does accumulate 8-bit-char copies 
of JS strings.  But again, when the JS strings from which those deflated 
strings originated (via calls to JS_GetStringBytes) become unreachable 
and are finalized, the deflated string versions are freed too.  See 
js_FinalizeString in jsstr.c.

>FWIW, the first entry here for 6336 bytes seems to be exactly twice the size
>of our transacton response which we call JS_GetStringChars on to get char*
>

Do you mean JS_GetStringBytes?  JS_GetStringChars returns jschar *, not 
char *.

>to return to our client-side. Maybe just a poisson-rouge?.
>

Can you show me some of your code that recovers the transaction response 
from the JS engine, and then calls JS_GetString{Bytes,Chars}?  Thanks,

/be

>
>Mike
>
>A typical (per transaction) dump looks like :-
>
>Dumping objects ->
>c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584980} normal block at
>0x04776888, 6336 bytes long.
>Data: << P r o v i d e > 3C 00 50 00 72 00 6F 00 76 00 69 00 64 00 65 00
>c:\xxs\sdk\mozilla\js\src\jsapi.c(1443) : {584863} normal block at
>0x04727E80, 112 bytes long.
>Data: < P $ @S ! M > 1B 00 00 00 50 7F 24 04 40 53 85 03 21 FA 4D 03
>c:\xxs\sdk\mozilla\js\src\jsscope.c(339) : {584862} normal block at
>0x04727AC8, 896 bytes long.
>Data: < pm > 00 00 00 00 00 00 00 00 10 70 6D 04 00 00 00 00
>c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584861} normal block at
>0x04727A80, 24 bytes long.
>Data: < PS "p ?a > 05 00 00 00 50 53 85 03 D8 22 70 04 D9 3F 61 03
>c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584860} normal block at
>0x04727A28, 32 bytes long.
>Data: < @#p @ E > 01 00 00 00 40 23 70 04 40 16 45 03 00 00 00 00
>c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584859} normal block at
>0x047279E0, 24 bytes long.
>Data: < PS "p ?a > 05 00 00 00 50 53 85 03 D8 22 70 04 D9 3F 61 03
>c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584858} normal block at
>0x04727988, 32 bytes long.
>Data: < 8#p H > 01 00 00 00 38 23 70 04 D0 97 48 03 00 00 00 00
>c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584857} normal block at
>0x04727940, 24 bytes long.
>Data: < PS "p ?a > 05 00 00 00 50 53 85 03 D8 22 70 04 D9 3F 61 03
>c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584856} normal block at
>0x047278E8, 32 bytes long.
>Data: < 0#p H > 01 00 00 00 30 23 70 04 10 95 48 03 00 00 00 00
>c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584855} normal block at
>0x047278A0, 24 bytes long.
>Data: < PS "p ?a > 05 00 00 00 50 53 85 03 D8 22 70 04 D9 3F 61 03
>c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584854} normal block at
>0x0472DCC0, 32 bytes long.
>Data: < (#p H > 01 00 00 00 28 23 70 04 B0 8D 48 03 00 00 00 00
>c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584852} normal block at
>0x047277E0, 24 bytes long.
>Data: < PS "p ?a > 05 00 00 00 50 53 85 03 D8 22 70 04 D9 3F 61 03
>c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584851} normal block at
>0x04727788, 32 bytes long.
>Data: < #p H > 01 00 00 00 20 23 70 04 90 91 48 03 00 00 00 00
>c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584850} normal block at
>0x04727740, 24 bytes long.
>Data: < PS "p ?a > 05 00 00 00 50 53 85 03 D8 22 70 04 D9 3F 61 03
>c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584849} normal block at
>0x0472DE88, 32 bytes long.
>Data: < #p |H > 01 00 00 00 18 23 70 04 D0 7C 48 03 00 00 00 00
>c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584848} normal block at
>0x0472DE40, 24 bytes long.
>Data: < PS "p ?a > 05 00 00 00 50 53 85 03 D8 22 70 04 D9 3F 61 03
>c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584847} normal block at
>0x0472DDE8, 32 bytes long.
>Data: < #p yH > 01 00 00 00 10 23 70 04 D0 79 48 03 00 00 00 00
>c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584845} normal block at
>0x0472DD28, 24 bytes long.
>Data: < PS "p ?a > 05 00 00 00 50 53 85 03 D8 22 70 04 D9 3F 61 03
>c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584844} normal block at
>0x0472DAD0, 32 bytes long.
>Data: < #p mH > 01 00 00 00 08 23 70 04 E0 6D 48 03 00 00 00 00
>c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584842} normal block at
>0x0472DC78, 24 bytes long.
>Data: < PS "p ?a > 05 00 00 00 50 53 85 03 D8 22 70 04 D9 3F 61 03
>c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584841} normal block at
>0x0472DC20, 32 bytes long.
>Data: < #p vH > 01 00 00 00 00 23 70 04 00 76 48 03 00 00 00 00
>c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584840} normal block at
>0x0472DBD8, 24 bytes long.
>Data: < PS "p ?a > 05 00 00 00 50 53 85 03 D8 22 70 04 D9 3F 61 03
>c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584839} normal block at
>0x0472DB80, 32 bytes long.
>Data: < "p sH > 01 00 00 00 F8 22 70 04 00 73 48 03 00 00 00 00
>c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584838} normal block at
>0x0472D888, 24 bytes long.
>Data: < PS "p ?a > 05 00 00 00 50 53 85 03 D8 22 70 04 D9 3F 61 03
>c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584837} normal block at
>0x0472DB28, 32 bytes long.
>Data: < "p pH > 01 00 00 00 F0 22 70 04 E0 70 48 03 00 00 00 00
>c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584835} normal block at
>0x0472DA88, 24 bytes long.
>Data: < PS "p ?a > 05 00 00 00 50 53 85 03 D8 22 70 04 D9 3F 61 03
>c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584834} normal block at
>0x0472DA30, 32 bytes long.
>Data: < "p H > 01 00 00 00 E8 22 70 04 E0 82 48 03 00 00 00 00
>c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584833} normal block at
>0x0472D9A8, 88 bytes long.
>Data: < 8Fa > 01 00 00 00 38 46 61 03 1B 00 00 00 1B 00 00 00
>c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584832} normal block at
>0x0472D960, 24 bytes long.
>Data: < PS "p ?a > 05 00 00 00 50 53 85 03 D8 22 70 04 D9 3F 61 03
>c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584831} normal block at
>0x0472D908, 32 bytes long.
>Data: < "p H > 01 00 00 00 E0 22 70 04 D0 7F 48 03 00 00 00 00
>c:\program files\microsoft visual studio\vc98\include\crtdbg.h(552) :
>{584830} normal block at 0x0472D8D0, 12 bytes long.
>Data: <h`r g > 68 60 72 04 F8 67 BB 05 00 00 00 00
>c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584111} normal block at
>0x0470A530, 108 bytes long.
>Data: << S d s C r y p > 3C 00 53 00 64 00 73 00 43 00 72 00 79 00 70 00
>c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {583665} normal block at
>0x046F3380, 6 bytes long.
>Data: < 1 > 20 00 31 00 00 00
>c:\xxs\sdk\mozilla\js\src\jsapi.c(1443) : {581143} normal block at
>0x04724358, 376 bytes long.
>Data: <] @S @S M > 5D 00 00 00 40 53 85 03 40 53 85 03 B1 E9 4D 03
>c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {581132} normal block at
>0x046D3858, 88 bytes long.
>Data: < 8Fa ] > 01 00 00 00 38 46 61 03 5D 00 00 00 04 00 00 00
>c:\program files\microsoft visual studio\vc98\include\crtdbg.h(552) :
>{581130} normal block at 0x046EEF60, 36 bytes long.
>Data: < 3m m 3m > C0 33 6D 04 10 B5 6D 04 C0 33 6D 04 02 00 00 00
>{581129} normal block at 0x046D3970, 97 bytes long.
>Data: < <tc><provider s> 00 3C 74 63 3E 3C 70 72 6F 76 69 64 65 72 20 73
>c:\program files\microsoft visual studio\vc98\include\crtdbg.h(552) :
>{581128} normal block at 0x046DB510, 36 bytes long.
>Data: < 3m o ` n > C0 33 6D 04 90 F2 6F 04 60 EF 6E 04 01 00 00 00
>{581127} normal block at 0x046D2CC8, 897 bytes long.
>Data: < <FlightPriceRQ>> 00 3C 46 6C 69 67 68 74 50 72 69 63 65 52 51 3E
>c:\program files\microsoft visual studio\vc98\include\crtdbg.h(552) :
>{581126} normal block at 0x046FF290, 36 bytes long.
>Data: < m m ` n > 10 B5 6D 04 10 B5 6D 04 60 EF 6E 04 CD CD CD CD
>c:\program files\microsoft visual studio\vc98\include\crtdbg.h(552) :
>{581124} normal block at 0x046DFF40, 16 bytes long.
>Data: < o > CC CC CD CD 90 F2 6F 04 00 CD CD CD 02 00 00 00
>c:\xxs\src\xxjse\context.cpp(310) : {581117} client block at 0x04710F90,
>subtype 0, 25 bytes long.
>Data: <JOE JORDAN > 4A 4F 45 20 4A 4F 52 44 41 4E 00 CD CD CD CD CD
>Object dump complete.
>
>
>"Eric Brueggemann" <eric_brueggemann@yahoo.com> wrote in message
>news:3EC44B68.1050509@yahoo.com...
>  
>
>>Valgrind is another outstanding memory debugging tool:
>>http://developer.kde.org/~sewardj/
>>
>>Brendan Eich wrote:
>>    
>>
>>>Maybe you're leaking memory somewhere.  Best to track the leaks.  Check
>>>out http://lxr.mozilla.org/mozilla/source/tools/trace-malloc/lib/ and
>>>the tools above it one directory.
>>>
>>>/be
>>>
>>>      
>>>
>>>>How do I track these allocations down?. I'm on Win32 with latest beta
>>>>trunk.
>>>>I have JS_THREADSAFE, GC_MARK_DEBUG defined.
>>>>
>>>>Mike
>>>>
>>>>
>>>>
>>>>
>>>>        
>>>>
>
>
>  
>

.