[SOLVED] Bug in DynArrays
lazarus / 64-FPC development

----------------------------------------------------------------------
Key-Real · 2025-09-21 19:18

Hi,

I'm working on the PlayStation 1 Port.
I run into a issue with setlength()

It is a bigger setup, so I can't provide an easy example.

I'll try to simplify the situation:
got:

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i Type
Vector2 = record                       x, y : longint;               
end;v3dObject = record                       ...                     
 positions : array of Vector3;                        uvs : array of
Vector2;                       ...                     end; function
importObject: v3dObject;begin....   setLength(result.positions,
12345);....   setLength(result.uvs, 12345);                <--   Here
it crash....end; 

observations:
if i do in the main program
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i ...var   
 a1 : array of dword;     a2 : array of dword;begin     setLength(a1,
12345);     setLength(a2, 12345);     ....end. 
everything works, i do not do those 2 setLength, it crashes as
described above.

further observations:

he goes into:

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i procedure
fpc_dynarray_setlength(var p : pointer;pti : pointer; dimcount :
sizeint;dims : pdynarrayindex);[Public,Alias:'FPC_DYNARR_SETLENGTH'];
compilerproc;....     if assigned(p) then        ....     else     
 begin          oldlen:=0;          {$IFDEF PS1}writeln('!!!!!!!
AllocMem');{$ENDIF}              newp:=AllocMem(size);               
                                     <---  INTO THIS          {$IFDEF
PS1}writeln('!!!!!!! AllocMem done');{$ENDIF}       end; 
there now we are in heap.inc:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i function
SysAllocMem(size: ptruint): pointer;begin  {$IFDEF
PS1}writeln('!!!!!!! SysAllocMem()');{$ENDIF}  result :=
SysGetMem(size);                                                    
<--- INTO THIS  if Assigned(result) then    FillChar(result^,
SysMemSize(result), 0);end; than:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i function
SysGetMem(size : ptruint):pointer;...  if (size < GrowHeapSize2 div 2)
and (size <= HeapInc.MaxVarHeaderAndPayload - HeapInc.VarHeaderSize)
then begin    {$IFDEF PS1}writeln('!!!!!!! AllocVar()');{$ENDIF}   
result := ts^.AllocVar(size, false);                                 
       <--- INTO THIS  end else begin  {$IFDEF PS1}writeln('!!!!!!!
AllocHuge()');{$ENDIF}    result := ts^.AllocHuge(size); 
end;... than:

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i function
HeapInc.ThreadState.AllocVar(size: SizeUint; isArena: boolean):
pointer;...  {$IFDEF PS1}writeln('!!!!!!! in AllocVar');{$ENDIF}   
binIndex := VarSizeToBinIndexUp(size + VarHeaderSize);    size :=
BinIndexToVarSize(binIndex);    fv := varFree.Find(binIndex);     <---
   Here is an imporatant place A, see below    ----> OK NOW HE FINDS
THE fv, so later he goes into   ...    end else begin     
varFree.Remove(fv);                          <--- he goes into tihs  
end;   ... 
so:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i procedure
HeapInc.VarFreeMap.Remove(c: pFreeVarChunk);...begin    prev :=
c^.prev;    next := c^.next;       if Assigned(next) then     
next^.prev := prev;     if Assigned(prev) then begin      prev^.next
:= next;                          <-- HE CRUSH HERE !!!!!!!!!    end
else    ... 
conclusion:
he find a not existing item in the linked list because he finds fv :=
varFree.Find(binIndex);

observation:
if i do in the Important Place A:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i {$IFDEF
PS1}fv:= nil;{$ENDIF} everything works.

so now:
IDEAS??????

----------------------------------------------------------------------
runewalsh · 2025-09-21 19:39

I’d bet on memory corruption (double / incorrect FreeMem) on your
part...

Or maybe varFree somehow ended up uninitialized? It it assumed to be
zero-initialized, being a global (or thread-local) variable. Try
FillChar(HeapInc.thisTs, sizeof(HeapInc.thisTs), 0) somewhere in the
initialization routines (InitHeap / InitHeapThread / RelocateHeap, IDK
how PS1 works).

----------------------------------------------------------------------
Key-Real · 2025-09-21 19:51

the fillchar() unfortunly changes nothing.

i implemented, in sysheap.inc on my target:

function SysOSAlloc (size: ptruint): pointer;
procedure SysOSFree(p: pointer; size: ptruint);


what else i have to implement?

----------------------------------------------------------------------
runewalsh · 2025-09-21 20:36

Most likely the memory corruption, the freelist is easily corrupted
with incorrect overwrites.

Example:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i var       
a, b: array of int32;begin        SetLength(a, 200);       
FillDWord(pointer(a)^, length(a) * 2 { writes 200 items past the end
}, $ABABABAB);        SetLength(b, 200);end. 
It crashes the EXACTLY SAME way as your code, and due to the nature of
the corruption, heaptrc won't help you (without scanning the entire
heap for corruptions every time). :\

----------------------------------------------------------------------
Key-Real · 2025-09-21 20:58

so the positions array overwrites the uvs array?

because first setlength(positions,123);
and then uvs follow.
?

can't be, i comment out all the writes to positions array

----------------------------------------------------------------------
runewalsh · 2025-09-21 21:20

Something is overwriting something, perhaps in an unrelated place.
Another example that crashes (almost) the same way (but only without
heaptrc, for... reasons):

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i var       
a, b: pByte;begin        a := GetMem(1000);        b := GetMem(1000); 
      FreeMem(b);        b[0] := 1; // corruption        b :=
GetMem(1000); // crashend.

----------------------------------------------------------------------
Thaddy · 2025-09-22 17:52

Simple observation: you do not know jack dung about alignment. You
just assume memory is contiguous.
Sequential <> contiguous.
Back to school.
The use of getmem() in the hands of people like you makes me shiver. 
Horror!! 
You don't have a license to use getmem(). 
Period.

----------------------------------------------------------------------
PascalDragon · 2025-09-22 20:07

Quote from: Thaddy on September 22, 2025, 07:52:53 pmThe use of
getmem() in the hands of people like you makes me shiver. 
Horror!! 
You don't have a license to use getmem(). 
Period.

Key-Real is trying to debug an issue with dynamic arrays where they
stepped down into where SysGetMem is used. So they aren't using GetMem
directly, so please keep your opinion to yourself. And runewalsh is
only demonstrating a possible cause. And he definitely is allowed to
demonstrate using GetMem considering that he implemented the current
heap implementation.

----------------------------------------------------------------------
Key-Real · 2025-09-23 07:42

Quote from: runewalsh on September 21, 2025, 11:20:59 pmSomething is
overwriting something, perhaps in an unrelated place.
Another example that crashes (almost) the same way (but only without
heaptrc, for... reasons):

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i var       
a, b: pByte;begin        a := GetMem(1000);        b := GetMem(1000); 
      FreeMem(b);        b[0] := 1; // corruption        b :=
GetMem(1000); // crashend.


It was indeed memory corruption as You say. Thx a lot.