[Solved] Varible not correctly initialized if declared third lazarus / 21-General ---------------------------------------------------------------------- Frate · 2022-06-11 20:23 Hello, today I found some funny stuff going on with Pascal. This drove me crazy for around 30 minutes after which I realized that the boolean value I was using was not initialized to False (as I think it's supposed to). I went trough the code with a colleague and we found out that this is happening only if you have a variable declared third. The same happened with integers and strings (probably more but haven't tried). I made sure to make a clean build, changed optimization, variable names, function, and more. It seems that some value gets off the memory boundaries it's supposed to stay in, and writes to on the third variable declaration. Never seen something like this before. Can it be a compiler bug? Thanks Elia Win 11, Lazarus 2.2.0 (rev lazarus_2_2_0) FPC 3.2.2 i386-win32-qt5 Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i function readChangelog(version, filename: string): string;var  Lines: TStringList;  line: string;  sectionBegin: Boolean;begin  sectionBegin := False;  Lines := TStringList.Create;  try    LogDebug({$I %file%}, {$I %CurrentRoutine%}, 'Reading changelog ' + filename + ' ' + version);    version := LowerCase(version).Replace('v', '').Replace('a', '').Replace('b', '');    Lines.LoadFromFile(filename);    for line in Lines do begin      if line.StartsWith('## ') and line.Contains(version) then begin // search for version string        sectionBegin := True;        Result := '# Changelog' + #13#10 + line;        Continue;      end;      if sectionBegin then begin        if line.StartsWith('## ') then        break;        Result := Result + #13#10 + line;      end;    end;  finally    Lines.Free;  end;end; ---------------------------------------------------------------------- MarkMLl · 2022-06-11 20:31 Variables other than strings etc. are not initialised. /Particularly/ if they're local variables declared in a function etc., they contain whatever was left on the stack... which might not even have been in the valid range (0 or 1) for a Boolean. MarkMLl ---------------------------------------------------------------------- VTwin · 2022-06-11 20:50 https://www.freepascal.org/docs-html/ref/refse24.html ---------------------------------------------------------------------- Thaddy · 2022-06-11 20:58 Quote from: MarkMLl on June 11, 2022, 10:31:32 pmVariables other than strings etc. are not initialised. /Particularly/ if they're local variables declared in a function etc., they contain whatever was left on the stack... which might not even have been in the valid range (0 or 1) for a Boolean. MarkMLl Wrong, Mark. Globals are always initialized. Only stack based locals are never initialized. This behavior is documented and guaranteed. In this case OP refers to local, so he/she needs to initialize. For globals this is not the case. ---------------------------------------------------------------------- MarkMLl · 2022-06-11 21:22 Quote from: Thaddy on June 11, 2022, 10:58:51 pmWrong, Mark. Globals are always initialized. Only stack based locals are never initialized. This behavior is documented and guaranteed. In this case OP refers to local, so he/she needs to initialize. For globals this is not the case. You're right, in an unhelpful way. Link given by Vtwin explicitly states that globals are zeroed, then goes on to demonstrate that zeroing a variable might not be appropriate initialisation: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i  Type    TWeekDays =      (monday,tuesday,wednesday,thursday,friday,saturday,sunday);    TWeekend = saturday..sunday;   var    W : TWeekend;   begin    Writeln(W);  end. The above will result, when run, in an error:Runtime error 107 at $000000000040024A  $000000000040024A  $000000000042BF70  $00000000004001D2  What's more, even in the case of globals one has to ask /why/ they're zeroed: I'm inclined to argue that that's more a feature of how the linker/loader conventionally treats a particular section in an executable than any behaviour of the language. MarkMLl ---------------------------------------------------------------------- 440bx · 2022-06-11 21:44 Quote from: MarkMLl on June 11, 2022, 11:22:45 pmWhat's more, even in the case of globals one has to ask /why/ they're zeroed: I'm inclined to argue that that's more a feature of how the linker/loader conventionally treats a particular section in an executable than any behaviour of the language. MarkMLl Just for the record, when the compiler finds a global in the code, it has to allocate space in initialized data segment, because of that, it has to set that space to some value and the simple thing is to put binary zeroes there.  Otherwise, the compiler would have to set the initial value based on the type and basically use various "preferences" depending on the data type (e.g, what initial value should an enumeration have ? or what initial value should a date type have and so on..., it's just not practical and could easily cause confusion.) ---------------------------------------------------------------------- Frate · 2022-06-11 21:59 Thank you all for you replies. So now I get that it's better to initialize everything. The weird thing is that behavior is inconsistent. For example the following code initializes all variables correctly; Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i function readChangelog(version, filename: string): string;var  Lines: TStringList;  sectionBegin: Boolean;begin  sectionBegin := False;  try    LogDebug({$I %file%}, {$I %CurrentRoutine%}, 'Reading changelog ' + filename + ' ' + version);    version := LowerCase(version).Replace('v', '').Replace('a', '').Replace('b', '');    Lines.LoadFromFile(filename);   [...] After running I found that sectionBegin is initialized to False. Weird stuff "They can contain rubbish" That's a first for me, especially with a boolean variable. Just learnt something new! ---------------------------------------------------------------------- 440bx · 2022-06-11 22:20 Quote from: Frate on June 11, 2022, 11:59:40 pmThe weird thing is that behavior is inconsistent. Actually, it's very consistent.  Upon entering the function, the compiler adjusts the stack pointer to create space for its local variables, this effectively _overlays_ the variables in that area of the stack and their initial value is going to be whatever happened to be on the stack at that location. IOW, it's consistently inconsistent. I suggest always setting the initial value of local variables.  It prevent bugs and, if there is a bug, it makes debugging easier because, since the variables are initialized, they have predictable values.  Predictability is a nice feature in a program. ---------------------------------------------------------------------- Warfley · 2022-06-12 02:06 Except for managed types (strings, dynamic arrays, managed records), local variabels are never initialized. So any value they have is just the value that happend to be at that memory location at that time. For example: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i procedure WriteVar(A: Integer);var  i: Integer;begin  i := A;end; procedure ReadVar;var  i: Integer;begin  WriteLn(i);end; begin  WriteVar(1024);  ReadVar;Gives without any optimizations the value 1024 because the variable i maps to the same memory region on the stack in both calls. But if we change optimization behavior, e.g by changing the definition of WriteVar to the following: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i procedure WriteVar(A: Integer); inline; Now the result is on my machine 4200080. So it is pretty simple, the value of a local variable is always the value that was written at the same memory location beforehand, so in this sense it is very consistent. But knowing what was previously on the exact location is in a high level language like pascal practically impossible, meaning that you should always assume that the value in there is basically just random. With global variables, as already discussed are initialized to 0. The reason for this is simply that this is how the respective sections in the executable are defined. Both ELF (Unix) and PE (windows) define global variables to be initialized by 0 (BSS segment) or with a compiletime defined value (DATA segment). This is defined by the OS and I don't know if this is guaranteed by the FPC for all architectures (i.e. that it is part of the language). My suggestion is, always treat any uninitialized variables like mines, that if you touch them they will blow up in your face. That said, personally I find there is one exception to this, and this are threadlocal variables. Here assuming that they are 0 initialized is actually quite helpful (due to the different handling they require between mainthread and worker threads, and it's sometimes hard to separate those two in a general fashion) ---------------------------------------------------------------------- Frate · 2022-06-12 05:56 Quote from: 440bx on June 12, 2022, 12:20:56 amPredictability is a nice feature in a program. Absolutely right , thanks for the explanation. Quote from: Warfley on June 12, 2022, 04:06:36 amSo it is pretty simple, the value of a local variable is always the value that was written at the same memory location beforehand Oh this is the eyeopener for me. Btw thanks all, awesome community as always! ---------------------------------------------------------------------- MarkMLl · 2022-06-12 07:47 Quote from: 440bx on June 12, 2022, 12:20:56 amActually, it's very consistent.  Upon entering the function, the compiler adjusts the stack pointer to create space for its local variables, this effectively _overlays_ the variables in that area of the stack and their initial value is going to be whatever happened to be on the stack at that location. Oh no it's not! /Most/ of the time that will be true, until an interrupt has occurred during execution of the calling function which has changed the content of the not-yet-allocated portion of the stack. Or for that matter, if function A calls B and then C sequentially, and B fills a variable by calling Random(), C might overlay that variable with a Boolean. MarkMLl ---------------------------------------------------------------------- MarkMLl · 2022-06-12 08:01 Quote from: 440bx on June 11, 2022, 11:44:35 pmJust for the record, when the compiler finds a global in the code, it has to allocate space in initialized data segment, Actually, no it /doesn't/ has to. Compilers do, by default, put globals into an initialised segment. But in the general case (I'm not sure how you'd actually do this with FPC) you can override the allocation of a variable to put it into any segment you want. So as a specific example, on something like GCC for an AVR processor you can set up a global Code: C  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i unsigned char _reset_reason __attribute__ ((section (".noinit")));  which is not zeroed at startup hence can be used to retain the cause of a reset. Hence by convention and by default, globals are zeroed. Locals are not zeroed, although strings and dynamic arrays etc. start off in a safe state. And I'd also note that locals are not cleared when freed (or, in the case of strings etc., resized), which has implications for things like password handlers. And of course the variable associated with a for statement usually has no defined value on completion. It is safest to NOT assume that globals are initialised, since if you move a global to be a local in a function you can easily overlook the fact that it now has no defined value. MarkMLl ---------------------------------------------------------------------- PascalDragon · 2022-06-13 12:55 Quote from: MarkMLl on June 12, 2022, 09:47:52 am/Most/ of the time that will be true, until an interrupt has occurred during execution of the calling function which has changed the content of the not-yet- allocated portion of the stack. That's only the case if you're doing embedded programming and then the interrupt handler simply needs to make sure that it doesn't trash any registers by (re)storing them (if the hardware supports a separate stack for interrupt routines, even better). If you develop some userspace application for e.g. Windows you don't need to worry about that. ---------------------------------------------------------------------- MarkMLl · 2022-06-13 13:13 Quote from: PascalDragon on June 13, 2022, 02:55:14 pmThat's only the case if you're doing embedded programming and then the interrupt handler simply needs to make sure that it doesn't trash any registers by (re)storing them (if the hardware supports a separate stack for interrupt routines, even better). If you develop some userspace application for e.g. Windows you don't need to worry about that. Sorry, I disagree there. I'm not talking about registers being trashed, I'm talking about an ISR (noting your point: using the same stack) asynchronously changing the content of memory which hasn't yet been allocated to a stackframe. So if we assume that all memory was initially zeroed, main program M calls function A which runs for a while and calls function B, then B will still see zero in its local variables. But if we assume that M calls A, which gets interrupted by a handler H before it calls B, then B will see whatever was left on the stack by H's local variables as the starting state of its locals. I don't know whether async I/O-completion callbacks would have the same effect. MarkMLl ---------------------------------------------------------------------- PascalDragon · 2022-06-13 13:37 Quote from: MarkMLl on June 13, 2022, 03:13:40 pmQuote from: PascalDragon on June 13, 2022, 02:55:14 pmThat's only the case if you're doing embedded programming and then the interrupt handler simply needs to make sure that it doesn't trash any registers by (re)storing them (if the hardware supports a separate stack for interrupt routines, even better). If you develop some userspace application for e.g. Windows you don't need to worry about that. Sorry, I disagree there. I'm not talking about registers being trashed, I'm talking about an ISR (noting your point: using the same stack) asynchronously changing the content of memory which hasn't yet been allocated to a stackframe. If some ISR does that then you're either developing for an embedded target and you need to fix your ISR or you need to ditch your OS, because it's crap in preserving user state.