This should crash ? lazarus / 21-General ---------------------------------------------------------------------- dbannon · 2021-02-05 11:45 I recently, inadvertently used something like the following code, and was quite surprised to find it worked, raised no warnings, does not leak and did not crash. Personally, I think it should at least have the decency to crash !  Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i program testptr; uses sysutils;var        ptr : pchar;        St : ansistring;begin        St := 'some content, might be quite big';      ptr := pchar(St);                  // that looks reasonable        ptr := pchar(uppercase(St));       // that is not so reasonable      writeln(ptr);end. -------output : SOME CONTENT, MIGHT BE QUITE BIG  My question is, just where is the uppercase content stored ?  I have not allocated space for it, the 'ptr' variable is just a pointer, initially, it was pointing to the start of the ST content, thats fine, but the uppercase() function creates new content, and sets 'ptr' to point to that new content. Where ? I know, usually people write in saying my code crashes but I think its OK, well, I am saying my code is not OK but does not crash ! Davo ---------------------------------------------------------------------- marcov · 2021-02-05 11:57 uppercase(st) is an expression that generates a temp ansistring. This code is not safe, a pointer to a temp type is only valid for the statement it was last used in. (and the temp is last used in the line with uppercase). Also it is usually may crash, guarantees on crashes when you do something wrong don't exist :-) ---------------------------------------------------------------------- dbannon · 2021-02-05 12:08 Thanks for that marcov, it sure looked unsafe to me but seems to work fine. In real app, I then proceed to search up and down that temp string and have not seen a problem (yet). But I sure will recode before any end users get their hands on it. Its the old story, if it seems too good to be true, then its probably not true ! Davo ---------------------------------------------------------------------- 440bx · 2021-02-05 12:37 Quote from: dbannon on February 05, 2021, 12:45:18 pmMy question is, just where is the uppercase content stored ? It would be reasonable to believe the uppercased string resides on the stack but, that is _not_ the case. Quote from: dbannon on February 05, 2021, 12:45:18 pm  I have not allocated space for it, the 'ptr' variable is just a pointer, initially, it was pointing to the start of the ST content, thats fine, but the uppercase() function creates new content, and sets 'ptr' to point to that new content. Where ? Somewhere along the line uppercase() allocated a block of memory in the heap.  I didn't see anywhere code that would free that block, it looks like that would cause a memory leak. Quote from: dbannon on February 05, 2021, 12:45:18 pmI know, usually people write in saying my code crashes but I think its OK, well, I am saying my code is not OK but does not crash ! That's the crux of the problem, when the compiler starts doing things the programmer didn't ask for, all bets are off. Look at the attachments, the first attachment shows that ptr is at address $155AB14 and shows the string stored there.  The second attachment is simply a memory dump showing the string at that address (trust but verify philosophy.)  The third attachment is a memory map of the program and, it shows that the address $155AB14 is in the application's heap (not the stack).  Since it's on the heap, it's persistent (which would very likely result in a memory leak since it doesn't seem to be freed anywhere before the program ends.) HTH. ---------------------------------------------------------------------- MarkMLl · 2021-02-05 13:06 Quote from: dbannon on February 05, 2021, 01:08:26 pmThanks for that marcov, it sure looked unsafe to me but seems to work fine. In real app, I then proceed to search up and down that temp string and have not seen a problem (yet). But I sure will recode before any end users get their hands on it. Its the old story, if it seems too good to be true, then its probably not true ! That's a "use after dispose" error. In the past there's been discussion of whether strings should be zeroed when their reference count indicates that they can be freed because of the security implications, but the core developers decided against it. MarkMLl ---------------------------------------------------------------------- PascalDragon · 2021-02-05 13:38 Quote from: 440bx on February 05, 2021, 01:37:55 pmQuote from: dbannon on February 05, 2021, 12:45:18 pmMy question is, just where is the uppercase content stored ? It would be reasonable to believe the uppercased string resides on the stack but, that is _not_ the case. The string data itself is on the heap, but the string reference itself is indeed on the stack (see the highlighted line): Code: ASM  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i .section .text.n__main,"x"        .balign 16,0x90.globl  PASCALMAINPASCALMAIN:.globl  _main_main:# Temps allocated between ebp-4 and ebp+0        pushl   %ebp        movl    %esp,%ebp        leal    -4(%esp),%esp        pushl   %ebx        call    fpc_initializeunits        movl    $0,-4(%ebp)        xorl    %eax,%eax        pushl   $P$TESTPTR_$$_fin$00000003        pushl   %ebp        pushl   $__FPC_finally_handler        pushl   %fs:(%eax)      movl    %esp,%fs:(%eax)# [7] St := 'some content, might be quite big';        movl    $U_$P$TESTPTR_$$_ST,%eax        movl    $.Ld1,%edx        call    fpc_ansistr_assign# [8] ptr := pchar(St);                // that looks reasonable        movl    U_$P$TESTPTR_$$_ST,%eax        testl   %eax,%eax        jne     .Lj11        movl    $FPC_EMPTYCHAR,%eax.Lj11:        movl    %eax,U_$P$TESTPTR_$$_PTR# [9] ptr := pchar(uppercase(St));       // that is not so reasonable        movl    U_$P$TESTPTR_$$_ST,%eax      leal    -4(%ebp),%edx        call    SYSUTILS_$$_UPPERCASE$ANSISTRING$$ANSISTRING        movl    -4(%ebp),%eax        testl   %eax,%eax        jne     .Lj12        movl    $FPC_EMPTYCHAR,%eax.Lj12:        movl    %eax,U_$P$TESTPTR_$$_PTR# [10] writeln(ptr);        call    fpc_get_output        movl    %eax,%ebx        movl    U_$P$TESTPTR_$$_PTR,%ecx        movl    %ebx,%edx        movl    $0,%eax        call    fpc_write_text_pchar_as_pointer        call    fpc_iocheck        movl    %ebx,%eax        call    fpc_writeln_end      call    fpc_iocheck        xorl    %eax,%eax        popl    %edx      addl    $12,%esp        movl    %edx,%fs:(%eax)        movl    %ebp,%eax        call    P$TESTPTR_$$_fin$00000003        call    fpc_do_exit        popl    %ebx        movl    %ebp,%esp        popl  %ebp        ret Don't forget that we're dealing with an AnsiString here, thus only pointers are moved around. The release of the temporary variable happens inside the implicit finalizer which is only executed at the end of the main program and (in case of Win64) is as follows: Code: ASM  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i .section .text.n_p$testptr_$$_fin$00000003,"x"        .balign 16,0x90P$TESTPTR_$$_fin$00000003:# [tstrcrash.pp]# [11] end.        pushl   %ebp        movl    %eax,%ebp# [6] begin        leal    -4(%ebp),%eax        call    fpc_ansistr_decr_ref        popl    %ebp        ret  And just to avoid this question: it is in no way guaranteed when FPC frees a temporary variable. It could be right after the call or - as it is now - at the end of the routine. Or anywhere in between. ---------------------------------------------------------------------- 440bx · 2021-02-05 13:51 Quote from: PascalDragon on February 05, 2021, 02:38:36 pmThe string data itself is on the heap, but the string reference itself is indeed on the stack (see the highlighted line): Ok, I saw that. Quote from: PascalDragon on February 05, 2021, 02:38:36 pmThe release of the temporary variable happens inside the implicit finalizer which is only executed at the end of the main program and (in case of Win64) is as follows: Code: ASM  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i .section .text.n_p$testptr_$$_fin$00000003,"x"        .balign 16,0x90P$TESTPTR_$$_fin$00000003:# [tstrcrash.pp]# [11] end.        pushl   %ebp        movl    %eax,%ebp# [6] begin        leal    -4(%ebp),%eax        call    fpc_ansistr_decr_ref        popl    %ebp        ret Just in case, would he finalizer be called at the end of a function if the call to uppercase() had occurred in a function instead of the main program ?  I ask because since the pointer is kept on the stack, once the function has returned the pointer is lost, this suggests that the finalizer should be called in the function just prior to its returning to the caller, is this correct ? Quote from: PascalDragon on February 05, 2021, 02:38:36 pmAnd just to avoid this question: it is in no way guaranteed when FPC frees a temporary variable. It could be right after the call or - as it is now - at the end of the routine. Or anywhere in between. Could the rules used by the compiler for when such temporaries are freed be made public ? (even if they come with a disclaimer that they may change in the future.) ---------------------------------------------------------------------- PascalDragon · 2021-02-05 13:56 Quote from: 440bx on February 05, 2021, 02:51:34 pmJust in case, would he finalizer be called at the end of a function if the call to uppercase() had occurred in a function instead of the main program ?  I ask because since the pointer is kept on the stack, once the function has returned the pointer is lost, this suggests that the finalizer should be called in the function just prior to its returning to the caller, is this correct ? That finalization is part of any routine that uses managed, temporary variables. Essentially such a routine looks like: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i procedure MyProc;var  tmp1, tmp2, ...;begin  initTemps;  try    // do something  finally    freeTemps;  end;end; Quote from: 440bx on February 05, 2021, 02:51:34 pmQuote from: PascalDragon on February 05, 2021, 02:38:36 pmAnd just to avoid this question: it is in no way guaranteed when FPC frees a temporary variable. It could be right after the call or - as it is now - at the end of the routine. Or anywhere in between. Could the rules used by the compiler for when such temporaries are freed be made public ? (even if they come with a disclaimer that they may change in the future.) No, because we don't want users to rely on them for any reason. Even if you say that you're only going to use say version 3.2.0 and nothing else we can't guarantee that some optimization might not change something in that behaviour. ---------------------------------------------------------------------- 440bx · 2021-02-05 14:06 @PascalDragon, Thank you.  That clarifies things.