Subj : Re: Memory growth problem
To   : netscape.public.mozilla.jseng
From : "Mike McIntosh" <mike@jeezo.com>
Date : Mon May 19 2003 03:01 pm

Hi Brendan,

Thanks for your response(s).

I've noticed a fundamental flaw in how I was trying to detect this.

First, if I run a really simple script which just reads my ClientRequest
property and sets a ClientResponse I see the following calls to JS_malloc.

Alloc of 24 bytes -- JS_NewObject   --> These are freed via JS_free
Alloc of 88 bytes -- js_NewScope     --> on subsequent execution
Alloc of 482 bytes -- ClientRequest    --> This is released via
jstr/js_FinalizeString

These first 2 objects are not released after first transaction but I _do_
see them being released on the subsequent transaction. So my object dump
trace showing lots of allocated objects is incorrect. This is really showing
memory which has been allocated by the engine but which _will_ be released
during the next GC. Would that be correct?.

Anyway, for the simplest script I can see all memory being released if I
watch JS_Alloc/JS_Free in the debugger. I also see the finalize being called
on the ClientRequest string.

I _was_ talking about GetStringBytes and here is my code to set the client
response :-

static JSBool gbl_SetClientResponse(JSContext *cx, JSObject *obj, uintN
argc, jsval *argv, jsval *rval)
{
 TRY()

 *rval = BOOLEAN_TO_JSVAL(FALSE);

 if ((argc == 1) && JSVAL_IS_STRING(argv[0]))
 {
  ////////////////////////////////
  // Access the context info...
  //
  privData* pd = (privData*)JS_GetContextPrivate(cx);
  if (pd && (pd->m_hctx != 0))
  {
   //////////////////////////////
   // Get the response argument
   //
   JSString* jszResponse = JS_ValueToString(cx, argv[0]);
   if (jszResponse)
   {
    argv[0] = STRING_TO_JSVAL(jszResponse);
    char * pszBytes = JS_GetStringBytes(jszResponse);
    if (pszBytes)
    {
    // This sends the response string back to _our_ client
     BOOL bVal = XxeSetClientResponse(pd->m_hctx, pszBytes,
strlen(pszBytes));
     if (bVal)
     {
      *rval = BOOLEAN_TO_JSVAL(bVal);
     }
    }
   }
  }
 }
 CATCH_JS()

 return JS_TRUE;
}

I can see calls to js_PurgeDeflatedStringCache which seems to release the
strings allocated by JS_GetStringBytes.

In order to get somewhere with the more complicated case I'll try to use the
trace-malloc tool (is it only available for Linux as the README states?.)

Mike

"Brendan Eich" <brendan@meer.net> wrote in message
news:3EC68A2E.2060902@meer.net...
> Mike McIntosh wrote:
>
> >Eric/Brendan,
> >
> >Thanks for those links. I've spent best part of a week trying to be sure
> >that I'm not leaking in my app. I found a few minor things on the way
(which
> >is good) but it seems ok. [We can run same test transaction in non-JS
mode
> >which basically goes through same app code without any increase in
memory.]
> >
>
> The leaks or bloat wouldn't be in your app alone, but only in
> combination with the JS engine, especially via roots and other strong
> references in the live-thing graph the GC scans.
>
> >The results from this _appear_ to be showing memory being allocated per
> >transaction in the engine (see below). I've also added rogue leaks to
both
> >the engine and my app and they _do_ get reported by the CRTDBG so I have
> >_reasonable_ confidence in it. Blocks allocated by my app are marked as
> >client blocks, while spidermonkey will be normal blocks.
> >
>
> The problem with such tracing is that it doesn't show stack backtraces,
> so common subroutines that allocate get blame, and the ancestral callers
> who trigger the wrongful bloat or leak are hard to see, or guess at.
> That's why I was recommended trace-malloc: it gives a full
> (cycle-compressed, to reduce depth due to recursion) stack backtrace per
> allocation, using a callsite tree.
>
> >I've also run this under Boundschecker and it does not leak at exit so
the
> >problem seems one of accumulation of memory which is eventually released,
> >not a leak as such.
> >
> >I was really interested in whether the scoping technique was the correct
way
> >to clean up after each execution?. But the finalizers for custom objects
> >_do_ seem to be triggered at end of scope. Am I misinterpreting how the
GC
> >works?.  Where/when is memory allocated by JS_GetStringChars (which takes
no
> >context as parameter) released?
> >
>
> JS_GetStringChars doesn't allocate memory in the usual case.  If it does
> allocate, to flatten a dependent string and give it a zero terminator,
> it's as if you called JS_NewUCStringCopy* to create the string to create
> an independent string with the same chars content up front.  In all
> cases, strings are garbage collected, so unless you have a stuck GC
> root, or some equivalent strong reference, the string will be collected
> when it is no longer in use.
>
> If you'd said JS_GetStringBytes, I would have pointed to the
> deflated_string_cache (jsstr.c), which does accumulate 8-bit-char copies
> of JS strings.  But again, when the JS strings from which those deflated
> strings originated (via calls to JS_GetStringBytes) become unreachable
> and are finalized, the deflated string versions are freed too.  See
> js_FinalizeString in jsstr.c.
>
> >FWIW, the first entry here for 6336 bytes seems to be exactly twice the
size
> >of our transacton response which we call JS_GetStringChars on to get
char*
> >
>
> Do you mean JS_GetStringBytes?  JS_GetStringChars returns jschar *, not
> char *.
>
> >to return to our client-side. Maybe just a poisson-rouge?.
> >
>
> Can you show me some of your code that recovers the transaction response
> from the JS engine, and then calls JS_GetString{Bytes,Chars}?  Thanks,
>
> /be
>
> >
> >Mike
> >
> >A typical (per transaction) dump looks like :-
> >
> >Dumping objects ->
> >c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584980} normal block at
> >0x04776888, 6336 bytes long.
> >Data: << P r o v i d e > 3C 00 50 00 72 00 6F 00 76 00 69 00 64 00 65 00
> >c:\xxs\sdk\mozilla\js\src\jsapi.c(1443) : {584863} normal block at
> >0x04727E80, 112 bytes long.
> >Data: < P $ @S ! M > 1B 00 00 00 50 7F 24 04 40 53 85 03 21 FA 4D 03
> >c:\xxs\sdk\mozilla\js\src\jsscope.c(339) : {584862} normal block at
> >0x04727AC8, 896 bytes long.
> >Data: < pm > 00 00 00 00 00 00 00 00 10 70 6D 04 00 00 00 00
> >c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584861} normal block at
> >0x04727A80, 24 bytes long.
> >Data: < PS "p ?a > 05 00 00 00 50 53 85 03 D8 22 70 04 D9 3F 61 03
> >c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584860} normal block at
> >0x04727A28, 32 bytes long.
> >Data: < @#p @ E > 01 00 00 00 40 23 70 04 40 16 45 03 00 00 00 00
> >c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584859} normal block at
> >0x047279E0, 24 bytes long.
> >Data: < PS "p ?a > 05 00 00 00 50 53 85 03 D8 22 70 04 D9 3F 61 03
> >c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584858} normal block at
> >0x04727988, 32 bytes long.
> >Data: < 8#p H > 01 00 00 00 38 23 70 04 D0 97 48 03 00 00 00 00
> >c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584857} normal block at
> >0x04727940, 24 bytes long.
> >Data: < PS "p ?a > 05 00 00 00 50 53 85 03 D8 22 70 04 D9 3F 61 03
> >c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584856} normal block at
> >0x047278E8, 32 bytes long.
> >Data: < 0#p H > 01 00 00 00 30 23 70 04 10 95 48 03 00 00 00 00
> >c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584855} normal block at
> >0x047278A0, 24 bytes long.
> >Data: < PS "p ?a > 05 00 00 00 50 53 85 03 D8 22 70 04 D9 3F 61 03
> >c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584854} normal block at
> >0x0472DCC0, 32 bytes long.
> >Data: < (#p H > 01 00 00 00 28 23 70 04 B0 8D 48 03 00 00 00 00
> >c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584852} normal block at
> >0x047277E0, 24 bytes long.
> >Data: < PS "p ?a > 05 00 00 00 50 53 85 03 D8 22 70 04 D9 3F 61 03
> >c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584851} normal block at
> >0x04727788, 32 bytes long.
> >Data: < #p H > 01 00 00 00 20 23 70 04 90 91 48 03 00 00 00 00
> >c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584850} normal block at
> >0x04727740, 24 bytes long.
> >Data: < PS "p ?a > 05 00 00 00 50 53 85 03 D8 22 70 04 D9 3F 61 03
> >c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584849} normal block at
> >0x0472DE88, 32 bytes long.
> >Data: < #p |H > 01 00 00 00 18 23 70 04 D0 7C 48 03 00 00 00 00
> >c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584848} normal block at
> >0x0472DE40, 24 bytes long.
> >Data: < PS "p ?a > 05 00 00 00 50 53 85 03 D8 22 70 04 D9 3F 61 03
> >c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584847} normal block at
> >0x0472DDE8, 32 bytes long.
> >Data: < #p yH > 01 00 00 00 10 23 70 04 D0 79 48 03 00 00 00 00
> >c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584845} normal block at
> >0x0472DD28, 24 bytes long.
> >Data: < PS "p ?a > 05 00 00 00 50 53 85 03 D8 22 70 04 D9 3F 61 03
> >c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584844} normal block at
> >0x0472DAD0, 32 bytes long.
> >Data: < #p mH > 01 00 00 00 08 23 70 04 E0 6D 48 03 00 00 00 00
> >c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584842} normal block at
> >0x0472DC78, 24 bytes long.
> >Data: < PS "p ?a > 05 00 00 00 50 53 85 03 D8 22 70 04 D9 3F 61 03
> >c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584841} normal block at
> >0x0472DC20, 32 bytes long.
> >Data: < #p vH > 01 00 00 00 00 23 70 04 00 76 48 03 00 00 00 00
> >c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584840} normal block at
> >0x0472DBD8, 24 bytes long.
> >Data: < PS "p ?a > 05 00 00 00 50 53 85 03 D8 22 70 04 D9 3F 61 03
> >c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584839} normal block at
> >0x0472DB80, 32 bytes long.
> >Data: < "p sH > 01 00 00 00 F8 22 70 04 00 73 48 03 00 00 00 00
> >c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584838} normal block at
> >0x0472D888, 24 bytes long.
> >Data: < PS "p ?a > 05 00 00 00 50 53 85 03 D8 22 70 04 D9 3F 61 03
> >c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584837} normal block at
> >0x0472DB28, 32 bytes long.
> >Data: < "p pH > 01 00 00 00 F0 22 70 04 E0 70 48 03 00 00 00 00
> >c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584835} normal block at
> >0x0472DA88, 24 bytes long.
> >Data: < PS "p ?a > 05 00 00 00 50 53 85 03 D8 22 70 04 D9 3F 61 03
> >c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584834} normal block at
> >0x0472DA30, 32 bytes long.
> >Data: < "p H > 01 00 00 00 E8 22 70 04 E0 82 48 03 00 00 00 00
> >c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584833} normal block at
> >0x0472D9A8, 88 bytes long.
> >Data: < 8Fa > 01 00 00 00 38 46 61 03 1B 00 00 00 1B 00 00 00
> >c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584832} normal block at
> >0x0472D960, 24 bytes long.
> >Data: < PS "p ?a > 05 00 00 00 50 53 85 03 D8 22 70 04 D9 3F 61 03
> >c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584831} normal block at
> >0x0472D908, 32 bytes long.
> >Data: < "p H > 01 00 00 00 E0 22 70 04 D0 7F 48 03 00 00 00 00
> >c:\program files\microsoft visual studio\vc98\include\crtdbg.h(552) :
> >{584830} normal block at 0x0472D8D0, 12 bytes long.
> >Data: <h`r g > 68 60 72 04 F8 67 BB 05 00 00 00 00
> >c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {584111} normal block at
> >0x0470A530, 108 bytes long.
> >Data: << S d s C r y p > 3C 00 53 00 64 00 73 00 43 00 72 00 79 00 70 00
> >c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {583665} normal block at
> >0x046F3380, 6 bytes long.
> >Data: < 1 > 20 00 31 00 00 00
> >c:\xxs\sdk\mozilla\js\src\jsapi.c(1443) : {581143} normal block at
> >0x04724358, 376 bytes long.
> >Data: <] @S @S M > 5D 00 00 00 40 53 85 03 40 53 85 03 B1 E9 4D 03
> >c:\xxs\sdk\mozilla\js\src\jsapi.c(1434) : {581132} normal block at
> >0x046D3858, 88 bytes long.
> >Data: < 8Fa ] > 01 00 00 00 38 46 61 03 5D 00 00 00 04 00 00 00
> >c:\program files\microsoft visual studio\vc98\include\crtdbg.h(552) :
> >{581130} normal block at 0x046EEF60, 36 bytes long.
> >Data: < 3m m 3m > C0 33 6D 04 10 B5 6D 04 C0 33 6D 04 02 00 00 00
> >{581129} normal block at 0x046D3970, 97 bytes long.
> >Data: < <tc><provider s> 00 3C 74 63 3E 3C 70 72 6F 76 69 64 65 72 20 73
> >c:\program files\microsoft visual studio\vc98\include\crtdbg.h(552) :
> >{581128} normal block at 0x046DB510, 36 bytes long.
> >Data: < 3m o ` n > C0 33 6D 04 90 F2 6F 04 60 EF 6E 04 01 00 00 00
> >{581127} normal block at 0x046D2CC8, 897 bytes long.
> >Data: < <FlightPriceRQ>> 00 3C 46 6C 69 67 68 74 50 72 69 63 65 52 51 3E
> >c:\program files\microsoft visual studio\vc98\include\crtdbg.h(552) :
> >{581126} normal block at 0x046FF290, 36 bytes long.
> >Data: < m m ` n > 10 B5 6D 04 10 B5 6D 04 60 EF 6E 04 CD CD CD CD
> >c:\program files\microsoft visual studio\vc98\include\crtdbg.h(552) :
> >{581124} normal block at 0x046DFF40, 16 bytes long.
> >Data: < o > CC CC CD CD 90 F2 6F 04 00 CD CD CD 02 00 00 00
> >c:\xxs\src\xxjse\context.cpp(310) : {581117} client block at 0x04710F90,
> >subtype 0, 25 bytes long.
> >Data: <JOE JORDAN > 4A 4F 45 20 4A 4F 52 44 41 4E 00 CD CD CD CD CD
> >Object dump complete.
> >
> >
> >"Eric Brueggemann" <eric_brueggemann@yahoo.com> wrote in message
> >news:3EC44B68.1050509@yahoo.com...
> >
> >
> >>Valgrind is another outstanding memory debugging tool:
> >>http://developer.kde.org/~sewardj/
> >>
> >>Brendan Eich wrote:
> >>
> >>
> >>>Maybe you're leaking memory somewhere.  Best to track the leaks.  Check
> >>>out http://lxr.mozilla.org/mozilla/source/tools/trace-malloc/lib/ and
> >>>the tools above it one directory.
> >>>
> >>>/be
> >>>
> >>>
> >>>
> >>>>How do I track these allocations down?. I'm on Win32 with latest beta
> >>>>trunk.
> >>>>I have JS_THREADSAFE, GC_MARK_DEBUG defined.
> >>>>
> >>>>Mike
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >
> >
> >
> >
>

.