Possible Bug (Mac M1/AArch)?
lazarus / 64-FPC development

----------------------------------------------------------------------
Grahame Grieve · 2022-11-03 18:02

Consider this code:

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i procedure
foo(s : TStream);var  b : TBytes;begin  SetLength(b, 4000);  s.read(b,
4000);end; My code has plenty of examples of this, and I believe it
has always worked for me before (Delphi, and FPC windows, unix, and
Mac). But on my current Mac m1 with Lazarus 2.2.5 (rev
lazarus_2_2_4-3-g564e1e8244) FPC 3.2.2 aarch64-darwin-cocoa, that
results in memory corruption and complete failure of the executable
thereafter. If I change it to

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i procedure
foo(s : TStream);var  b : TBytes;begin  SetLength(b, 4000); 
s.read(b[0], 4000);end;
that works fine. This feels like a compiler bug to me, should I report
it? You can see real world fixes here:
https://github.com/HealthIntersections/fhirserver/pull/190/files

----------------------------------------------------------------------
Leledumbo · 2022-11-03 19:16

I believe it should not instead. TStream.Read expects untyped
parameter, which is treated differently based on actual TStream
descendant used. But the semantic should be similar to pure array of
bytes, no metadata whatsoever to skip. Therefore, b and b[0] are
totally different. The former will point to the start of the TBytes
structure, which contains metadata (length, refcount, etc.) while the
latter is the start of the data part only, which is what you want.

----------------------------------------------------------------------
domasz · 2022-11-03 19:29

Quote from: Grahame Grieve on November 03, 2022, 07:02:48 pm
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i procedure
foo(s : TStream);var  b : TBytes;begin  SetLength(b, 4000); 
s.read(b[0], 4000);end;

I've always (over 10 years) used this way in Delphi. This is the only
one that makes sense to me.
Also you can read to strings:

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i b:
AnsiString;[...]S.Read(b[1], 4000);

----------------------------------------------------------------------
ASerge · 2022-11-04 04:29

Quote from: domasz on November 03, 2022, 08:29:09 pmI've always (over
10 years) used this way in Delphi. This is the only one that makes
sense to me.
Delphi has the methodCode: Pascal  [Select][+][-]window.onload =
function(){var x1 = document.getElementById("main_content_section");
if (x1) { var x = document.getElementsByClassName("geshi");for (var i
= 0; i TStream.function Read(var Buffer: TBytes; Count: Longint):
Longint;But FPC does not have it.

----------------------------------------------------------------------
PascalDragon · 2022-11-04 06:35

Quote from: ASerge on November 04, 2022, 05:29:26 amQuote from: domasz
on November 03, 2022, 08:29:09 pmI've always (over 10 years) used this
way in Delphi. This is the only one that makes sense to me.
Delphi has the methodCode: Pascal  [Select][+][-]window.onload =
function(){var x1 = document.getElementById("main_content_section");
if (x1) { var x = document.getElementsByClassName("geshi");for (var i
= 0; i TStream.function Read(var Buffer: TBytes; Count: Longint):
Longint;But FPC does not have it.

FPC 3.3.1 has it as well. For 3.2.2 you could introduce a class helper
that adds that method after all it only does a Read(Buffer[0], Count)
in the end (though yes, it goes through the method that provides the
Offset parameter, but that doesn't change the principle).

----------------------------------------------------------------------
Grahame Grieve · 2022-11-04 06:48

I don't see how this from TBytesStream is different to my first code:


Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i function
TBytesStream.Realloc(var NewCapacity: PtrInt): Pointer;begin  // adapt
TMemoryStream code to use with dynamic array  if NewCapacity<0 Then  
NewCapacity:=0  else    begin      if (NewCapacity>Capacity) and
(NewCapacity < (5*Capacity) div 4) then        NewCapacity :=
(5*Capacity) div 4;      NewCapacity := (NewCapacity + (TMSGrow-1))
and not (TMSGROW-1);    end;  if NewCapacity=Capacity then   
Result:=Pointer(FBytes)  else    begin     
SetLength(FBytes,Newcapacity);      Result:=Pointer(FBytes);      if
(Result=nil) and (Newcapacity>0) then        raise
EStreamError.Create(SMemoryStreamError);    end;end;

----------------------------------------------------------------------
ASerge · 2022-11-06 06:14

Quote from: Grahame Grieve on November 04, 2022, 07:48:34 amI don't
see how this from TBytesStream is different to my first code:
This code does not use the Read method, in which an error occurs when
parameters are passed incorrectly.

----------------------------------------------------------------------
Grahame Grieve · 2022-11-08 00:35

no but it uses the pointer to the TBytes directly, rather than a
pointer to TBytes[0]. I don't see how that's different?

----------------------------------------------------------------------
Jonas Maebe · 2022-11-08 20:09

A dynamic array is internally a pointer. TBytesStream.Realloc changes
that pointer to point to a different block of memory of larger/smaller
size. That is the purpose of a pointer, so this code is correct.

Your code was reading array data and storing it in the pointer itself,
spilling out in memory coming after that pointer. That is wrong, since
the read data should be stored in the memory block to which the
pointer points (and the pointer just contains the address of that
block). That memory can be addressed using tbytesvar[0].

----------------------------------------------------------------------
PascalDragon · 2022-11-08 21:58

You can use the following class helper with the FPC 3.2.x series to
achieve the same that's possible in FPC 3.3.1 now:

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i //
interface section{$ifdef ver3_2}type  TStreamHelper = class helper for
TStream  public    function Read(var Buffer: TBytes; Count: Longint):
Longint; overload;    function Read(Buffer: TBytes; aOffset, Count:
Longint): Longint; overload;    function Write(const Buffer: TBytes;
Offset, Count: Longint): Longint; overload;    function Write(const
Buffer: TBytes; Count: Longint): Longint; overload;  end;{$endif} //
implementation section{$ifdef ver3_2}function TStreamHelper.Read(var
Buffer: TBytes; Count: Longint): Longint;begin  Result := Read(Buffer,
0, Count);end; function TStreamHelper.Read(Buffer: TBytes; aOffset,
Count: Longint): Longint;begin  Result := Read(Buffer[aOffset],
Count);end; function TStreamHelper.Write(const Buffer: TBytes; Offset,
Count: Longint): Longint;begin  Result := Write(Buffer[Offset],
Count);end; function TStreamHelper.Write(const Buffer: TBytes; Count:
Longint): Longint;begin  Result := Write(Buffer, 0,
Count);end;{$endif} 
Just make sure that the unit is in scope when you want to use the
added methods.

----------------------------------------------------------------------
Arioch · 2022-11-18 13:21

Quote from: domasz on November 03, 2022, 08:29:09 pmCode: Pascal
 [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i procedure
foo(s : TStream);  s.read(b[0], 4000);end;I've always (over 10 years)
used this way in Delphi

Call me paranoid, but i wrote Code: Pascal
 [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i  
s.read(b[Low(b)], 4000);
Experience with moving legacy projects to Unicode Delphi and new
datatypes (like gen-dyn-arrays instead of raw pointers) and how all
hell was breaking loose with the undocumented and subconscious
programming against implementation details suddenly were blessing me
as unreproduccible shroedin-bugs in the twisted code whose design
logic no one remembered any more.