static array [0..0] now fail at run time lazarus / 62-General ---------------------------------------------------------------------- rossh_lz · 2020-11-21 12:56 The old method of defining a static type of array with an open ended size, now fails a runtime.  sample:   PCells = array [0..3] of Byte;   TPArray = array [0..0] of PCells;   PTPArray = ^TPArray; The TPArray array can be sized to suit with GetMem, or assigning existing mem to PTPArray. Now, as of FPC 3.2.0, I get range check fatal error on every attempt to access an array index > 0.  Code that still runs OK in Delphi, and for the last 12 years in Lazarus, now fails. Range checking is not active (and never affected this anyway). Why the implied range checking now here?  How to get rid of it? Thanks. ---------------------------------------------------------------------- Jonas Maebe · 2020-11-21 13:15 The range check at compile time was added because it has always generated code that gave unexpected results in some cases (even when range checking was inactive). You can find a very long thread about this at https://forum.lazarus.freepascal.org/index.php/topic,44655.0.html . It seemed more prudent to give a compile time error than to silently generate unexpected code. Since both FPC and Delphi support indexing pointers as arrays, the portable and safe way to implement what you want is to declare PTPArray as ^PCells instead. If you're using {$mode delphi}, and for Delphi proper, you also need to enable the pointermath switch. You should not need to make any other changes to your code. ---------------------------------------------------------------------- marcov · 2020-11-21 13:21 Quote from: rossh_lz on November 21, 2020, 01:56:56 pmThe old method of defining a static type of array with an open ended size, now fails a runtime.  You are comparing with Delphi, I assume you have enabled delphi mode? ---------------------------------------------------------------------- nanobit · 2020-11-21 13:22 Quote from: rossh_lz on November 21, 2020, 01:56:56 pmsample:   PCells = array [0..3] of Byte;   TPArray = array [0..0] of PCells;   PTPArray = ^TPArray; You can define a larger range: type TPArray = array[0..(high(sizeint) div sizeof(PCells))-1] of PCells; if you need also negative indices, then you need pointermath instead. type {$PointerMath on} PTPArray = ^PCells; {$PointerMath off} But FPC has a known bug here (if item is static array, issue 35745) under {$modeswitch autoderef} and delphi mode: Then "var pArrays: PTPArray;" has this pointermath bug: pArrays[1] is next byte, but should be next array (PCells). ( pArrays^[1] (non-pointermath expression) is correct for returning a byte) Workaround: (pArrays + 1)^ is next array. @pArrays[1] refers to next byte, but should refer to next array. Workaround: (pArrays + 1) refers to next array. ---------------------------------------------------------------------- PascalDragon · 2020-11-21 17:18 Quote from: rossh_lz on November 21, 2020, 01:56:56 pmNow, as of FPC 3.2.0, I get range check fatal error on every attempt to access an array index > 0.  Code that still runs OK in Delphi, and for the last 12 years in Lazarus, now fails. This code fails to compile in current Delphi versions as well (also with range checks disabled): Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i type  PCells = array [0..3] of Byte;  TPArray = array [0..0] of PCells;  PTPArray = ^TPArray; var  p: PTPArray;begin  GetMem(p, 10 * SizeOf(PCells));  Writeln(p^[2]^);end. The error generated is E1012: constant expression violates subrange bounds. @Jonas: maybe this should be mentioned in User Changes 3.2.0? ---------------------------------------------------------------------- 440bx · 2020-11-21 17:46 Quote from: PascalDragon on November 21, 2020, 06:18:09 pmThe error generated is E1012: constant expression violates subrange bounds. Does it generate correct code in 64bit mode when the index is a 32bit integer _variable_ less than zero ? ---------------------------------------------------------------------- rossh_lz · 2020-11-21 18:56 Quote from: PascalDragon on November 21, 2020, 06:18:09 pmQuote from: rossh_lz on November 21, 2020, 01:56:56 pmNow, as of FPC 3.2.0, I get range check fatal error on every attempt to access an array index > 0.  Code that still runs OK in Delphi, and for the last 12 years in Lazarus, now fails. This code fails to compile in current Delphi versions as well (also with range checks disabled): Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i type  PCells = array [0..3] of Byte;  TPArray = array [0..0] of PCells;  PTPArray = ^TPArray; var  p: PTPArray;begin  GetMem(p, 10 * SizeOf(PCells));  Writeln(p^[2]^);end. The error generated is E1012: constant expression violates subrange bounds. @Jonas: maybe this should be mentioned in User Changes 3.2.0? Yes, your sample above fails, because the hard coded index of 2, is beyond range at compile time. But this code below works and it's the beauty of using static 0..0 arrays.  This was a way to get dynamic arrays, before real dynamic array were available. Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i     var    i, j: Integer;      p: PTPArray;    begin      GetMem(p, 10 * SizeOf(PCells));      i := 7;      j := 1;      p^[i][j] := 4;      for i := 1 to 9 do        p^[i] := p^[i-1];    end; The issue is, the above works just fine in Delphi, and all previous FPC, except the latest 3.2.0, where it seems that runtime range checking of static arrays has been added.  This is not mentioned in the notes (and not welcome either). ---------------------------------------------------------------------- nanobit · 2020-11-21 19:11 Quote from: rossh_lz on November 21, 2020, 07:56:37 pmBut this code below works and it's the beauty of using static 0..0 arrays.  array [0..0] means single element, this is less than beautiful for larger arrays. ---------------------------------------------------------------------- jamie · 2020-11-21 19:29 the easy fix which I am sure will work in all versions is this. Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i Type  PCells = array [0..3] of Byte;  PTPArray = ^PCells;   ///---- Code land ---//Var  p:PTpArray;begin  getmem(p, sizeof(Pcells)*10);  P[5][0] := 10;end;                                 ect. ---------------------------------------------------------------------- rossh_lz · 2020-11-21 19:32 Quote from: nanobit on November 21, 2020, 08:11:38 pmQuote from: rossh_lz on November 21, 2020, 07:56:37 pmBut this code below works and it's the beauty of using static 0..0 arrays.  array [0..0] means single element, this is less than beautiful for larger arrays. No.   The beauty is that it's open ended for size, and the use of indexes at runtime, mean you are not constrained to declaring array sizes at design time.   This was how you had to do it before dynamic arrays were available. ---------------------------------------------------------------------- nanobit · 2020-11-21 19:36 Quote from: jamie on November 21, 2020, 08:29:21 pmthe easy fix which I am sure will work in all versions is this. Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i Type  PCells = array [0..3] of Byte;  PTPArray = ^PCells; Not all versions. As stated earlier, works only with {$PointerMath on}, and if none of the following is true: {$modeswitch autoderef}, {$mode delphi} ---------------------------------------------------------------------- PascalDragon · 2020-11-21 19:43 Quote from: 440bx on November 21, 2020, 06:46:54 pmQuote from: PascalDragon on November 21, 2020, 06:18:09 pmThe error generated is E1012: constant expression violates subrange bounds. Does it generate correct code in 64bit mode when the index is a 32bit integer _variable_ less than zero ? I don't have a 64-bit Delphi. Quote from: rossh_lz on November 21, 2020, 07:56:37 pmThe issue is, the above works just fine in Delphi, and all previous FPC, except the latest 3.2.0, where it seems that runtime range checking of static arrays has been added.  This is not mentioned in the notes (and not welcome either). As long as I don't enable range checking the following code compiles and runs without problems on FPC 3.2.0 and newer (and if I enable $R there'll be a runtime error in both FPC and Delphi): Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i program tarrtest; {$mode objfpc} type  PCells = array [0..3] of Byte;  TPArray = array [0..0] of PCells;  PTPArray = ^TPArray; var i, j: Integer; p: PTPArray;begin GetMem(p, 10 * SizeOf(PCells)); i := 7; j := 1; p^[i][j] := 4; for i := 1 to 9 do   p^[i] := p^[i-1];end. So please provide a full example including compiler parameters that fails for you that you think shouldn't fail. ---------------------------------------------------------------------- 440bx · 2020-11-21 19:48 Quote from: PascalDragon on November 21, 2020, 08:43:32 pmQuote from: 440bx on November 21, 2020, 06:46:54 pmQuote from: PascalDragon on November 21, 2020, 06:18:09 pmThe error generated is E1012: constant expression violates subrange bounds. Does it generate correct code in 64bit mode when the index is a 32bit integer _variable_ less than zero ? I don't have a 64-bit Delphi. I wasn't asking about Delphi.  It works in Delphi.  I was asking if FPC 3.2.0 generates correct code when in 64bit mode with an index that is a 32bit integer variable with a value less than zero.  (FPC 3.0.4 does not because it fails to sign extend the 32bit integer variable.) ---------------------------------------------------------------------- PascalDragon · 2020-11-21 20:07 Quote from: 440bx on November 21, 2020, 08:48:05 pmQuote from: PascalDragon on November 21, 2020, 08:43:32 pmQuote from: 440bx on November 21, 2020, 06:46:54 pmQuote from: PascalDragon on November 21, 2020, 06:18:09 pmThe error generated is E1012: constant expression violates subrange bounds. Does it generate correct code in 64bit mode when the index is a 32bit integer _variable_ less than zero ? I don't have a 64-bit Delphi. I wasn't asking about Delphi.  It works in Delphi.  I was asking if FPC 3.2.0 generates correct code when in 64bit mode with an index that is a 32bit integer variable with a value less than zero.  (FPC 3.0.4 does not because it fails to sign extend the 32bit integer variable.) Nothing was changed in this regard compared to the previous thread that was already linked here. ---------------------------------------------------------------------- nanobit · 2020-11-22 07:57 Quote from: rossh_lz on November 21, 2020, 08:32:00 pmQuote from: nanobit on November 21, 2020, 08:11:38 pmarray [0..0] means single element, this is less than beautiful for larger arrays. No. The beauty is that it's open ended for size, and the use of indexes at runtime, mean you are not constrained to declaring array sizes at design time.   This was how you had to do it before dynamic arrays were available. Truly open-ended (unchecked) is only pointermath (see previous comments). array[0..max] is open-ended with maximum, which is range-checked {$R}. Open-ended because you use PTPArray only with dynamic array-size, but not: var a: TPArray; sizeof(TPArray); high(TPArray). array[0..0] is just one example of non-empty ranges. (max+1)*sizeOf(element) must lie within the maximum dynamic allocation size of your array (constrained by memory manager or api function if you don't know a lower maximum). Index (0 or larger) can still point to invalid dynamic memory, regardless of range declaration. If your variables use only type PTPArray, you can change your example to: type TPArray = array[0..(high(sizeint) div sizeof(PCells))-1] of PCells; type PTPArray = ^TPArray; var p: PTPArray; ---------------------------------------------------------------------- rossh_lz · 2020-11-22 14:46 The underlying problem I have is this error below.  It shows at most places that the compiler has to do indexing of underlying array pointers.   As I said before this code is many years old, and runs fine in all versions prior to the current 3.2.0. I started modify the code to replace code like   p^.data^[j].something , where data is a pointer to a static array.  I changed this to a wrapper function that does the pointer incrementing:  inc(data_ptr, j).  Of course this is the compilers job.   The fix works, to force the code to be more specific.  But this project is too big to be fiddled with, so I'm going back a version of the FPC and see how that works. I can't give you a simple example to test - simple examples don't crash.   But a complex app with large mem usage is another story. The 3.2.0 notes talk about changes to array type parameter passing.  My instinct says it seems to be some error here that is conflicting. ***** "Project xyz raised exception class 'ERangeError' with message with message range check error.  address 1000013FA0. '/usr/local/bin/fpc'  -MDelphi -Scghi -O1 -gw3 -gl -k-framework -kCocoa -l -vewnhibq -Filib/x86_64-darwin -Fu../codeadd -Fu../codeadd/registry -Fu../codeadd/Abbrevia_5.2/source -Fu../codeadd/synapse_tcp-ip_209 -Fu../../.lazarus/lib/Printer4Lazarus/lib/x86_64-darwin/cocoa -Fu../../.lazarus/lib/ImagesForLazarus/lib/x86_64-darwin/cocoa -Fu../../.lazarus/lib/cairocanvas_pkg/lib/x86_64-darwin/cocoa -Fu../../.lazarus/lib/units/x86_64-darwin/cocoa -Fu../../.lazarus/lib/LCLBase/units/x86_64-darwin -Fu/Library/Lazarus/components/lazutils/lib/x86_64-darwin -Fu/Library/Lazarus/packager/units/x86_64-darwin -Fu. -FElib/x86_64-darwin -olib/x86_64-darwin/xyz -dLCL -dLCLcocoa -dCURSOR_ERROR -dUseCThreads -dDEBUG ---------------------------------------------------------------------- nanobit · 2020-11-22 16:53 Take your old code and just change your static-array type (used only by array pointer) from array[0..0] to array[0..max]. Max (some constant) will suppress range-check errors if max is large enough to hold all actual index values. ---------------------------------------------------------------------- Thaddy · 2020-11-22 17:03 range checks are local, so this will also hide the error Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i program untitled; {$mode delphi}{$rangechecks on}{$pointermath on}var{$push}{$rangechecks off}  arr:array[0..0] of byte;{$pop} // restore on.beginend.Now the arr variable will never be range checked. Basically it restores the old, less restrictive behavior. ---------------------------------------------------------------------- Jonas Maebe · 2020-11-22 17:07 Quote from: Thaddy on November 22, 2020, 06:03:15 pmrange checks are local, so this will also hide the error Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i program untitled; {$mode delphi}{$rangechecks on}{$pointermath on}var{$push}{$rangechecks off}  arr:array[0..0] of byte;{$pop} // restore on.beginend.Now the arr variable will never be range checked. That is completely wrong. ---------------------------------------------------------------------- PascalDragon · 2020-11-22 17:08 Quote from: rossh_lz on November 22, 2020, 03:46:53 pmI can't give you a simple example to test - simple examples don't crash.   But a complex app with large mem usage is another story. Without an example we can't help much. ---------------------------------------------------------------------- Thaddy · 2020-11-22 17:14 Quote from: Jonas Maebe on November 22, 2020, 06:07:16 pmThat is completely wrong. I saw it was wrong after testing. IMO it should work. ---------------------------------------------------------------------- Jonas Maebe · 2020-11-22 17:16 Quote from: Thaddy on November 22, 2020, 06:14:04 pmQuote from: Jonas Maebe on November 22, 2020, 06:07:16 pmThat is completely wrong. I saw it was wrong after testing. IMO it should work. Range checking is a setting that applies to code, not to data. ---------------------------------------------------------------------- rossh_lz · 2020-11-23 11:42 Quote from: PascalDragon on November 22, 2020, 06:08:25 pmWithout an example we can't help much. Yes, understood.   Here is the issue in assembler.  Code is made with O1., range check off.   The code is doing unwanted range checking on the array size.  Hence the fatal failure. Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i    PTData = ^TData;  TData = packed record    Info: TDataInfo;    Segment: array [0..0] of TDataSeg;  end;  var  Data: PTData;  /Users/rossh/Lazarus/Datalog/Utils.pas:1017  i := 2;00000001003082CA                          : c7 85 6c ff ff ff 02 00 00 00     movl   $0x2, -0x94(%rbp)/Users/rossh/Lazarus/Datalog/Utils.pas:1018  if Data^.Segment[i].First = $CC then00000001003082D4                        : 48 8b 5d a8                       movq   -0x58(%rbp), %rbx00000001003082D8                          : 8b 85 6c ff ff ff               movl   -0x94(%rbp), %eax00000001003082DE                        : 85 c0                             testl  %eax, %eax00000001003082E0                          : 76 05                           jbe    0x1003082e7               ; <+6719> at Utils.pas:1018:1700000001003082E2                          : e8 89 e6 d0 ff                    callq  0x100016970               ; FPC_RANGEERROR00000001003082E7                          : 8b 85 6c ff ff ff                 movl   -0x94(%rbp), %eax00000001003082ED                        : 48 6b c0 5c                       imulq  $0x5c, %rax, %rax00000001003082F1                          : 81 bc 03 b9 01 00 00 cc 00 00 00  cmpl   $0xcc, 0x1b9(%rbx,%rax)00000001003082FC                        : 0f 84 bf 1c 00 00                 je     0x100309fc1               ; <+14105> at Utils.pas       Segment: array [0..$FFFF] of TDataSeg;  /Users/rossh/Lazarus/Datalog/Utils.pas:1017  i := 2;00000001003081DA                          : c7 85 6c ff ff ff 02 00 00 00     movl   $0x2, -0x94(%rbp)/Users/rossh/Lazarus/Datalog/Utils.pas:1018  if Data^.Segment[i].First = $CC then00000001003081E4                        : 48 8b 5d a8                       movq   -0x58(%rbp), %rbx00000001003081E8                          : 8b 85 6c ff ff ff               movl   -0x94(%rbp), %eax00000001003081EE                        : 3d ff ff 00 00                    cmpl   $0xffff, %eax           ; imm = 0xFFFF 00000001003081F3                          : 76 05                           jbe    0x1003081fa               ; <+6722> at Utils.pas:1018:1700000001003081F5                          : e8 86 e6 d0 ff                    callq  0x100016880               ; FPC_RANGEERROR00000001003081FA                          : 8b 85 6c ff ff ff                 movl   -0x94(%rbp), %eax0000000100308200                        : 48 6b c0 5c                       imulq  $0x5c, %rax, %rax0000000100308204                          : 81 bc 03 b9 01 00 00 cc 00 00 00  cmpl   $0xcc, 0x1b9(%rbx,%rax)000000010030820F                        : 0f 84 c0 1c 00 00                 je     0x100309ed5               ; <+14109> at Utils.pas ---------------------------------------------------------------------- nanobit · 2020-11-23 13:18 Such code works for me with {$R-}. And your code is different from your previous descriptions. ---------------------------------------------------------------------- rossh_lz · 2020-11-23 13:53 Quote from: nanobit on November 23, 2020, 02:18:14 pmSuch code works for me with {$R-}. And your code is different from your previous descriptions. Yes I have various different constructs in use, but all end up with the same error. I thought $R- was the default?  Seemed to be so before.  You mean I have to force $R-, in every unit? ---------------------------------------------------------------------- nanobit · 2020-11-23 15:01 {$R-} before index-use would be needed technically, but {$R-} is not best practice. And working under {$R+} would need changes in your types. Here is one potential idea, but I cannot say which compilers support this efficiently: This still uses the classic array type TDataSegs, because TDataSeg itself might be array type and then pointermath has issues as said earlier. type TDataSeg = record first, last: byte; end; type TDataSegs = array[0..high(sizeInt) div sizeof(TDataSeg)-1] of TDataSeg; type PDataSegs = ^TDataSegs;    PTData = ^TData;   TData = packed record     Info: TDataInfo;     function Segment: PDataSegs; inline;   end;    function TData.Segment: PDataSegs; inline;    begin result := PDataSegs(PByte(@self) + sizeof(Info)); end; ---------------------------------------------------------------------- PascalDragon · 2020-11-23 15:29 Quote from: rossh_lz on November 23, 2020, 02:53:42 pmI thought $R- was the default?  Seemed to be so before.  You mean I have to force $R-, in every unit? $R- indeed is the default. Maybe something resulted in it being enabled? You can use {$ifopt R+}{$warning Range checks enabled}{$endif} to warn at compile time whether the option is enabled at a specific code point. Additionally you could check the compiler command line as well as the used fpc.cfg whether something passes -Cr while it shouldn't. ---------------------------------------------------------------------- Almir.Bispo · 2020-11-23 18:30 Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i procedure TForm1.Button1Click(Sender: TObject);var  S :array  of byte;   //dynamic array declared as byte    i :integer;beginsetlength(S,12); //dynamic dimension declared for i:=0 to 10 do begin   s[i]:=i;   showmessage('Test: '+inttostr(S[i]));  //test   showmessage('Test: '+inttostr( length(S)) );  //test end; end;     Visit my Blog: http://adltecnologia.blogspot.com