Is this a type helper bug?
lazarus / 62-General

----------------------------------------------------------------------
avra · 2021-03-26 14:51

Type helpers do not seam to work for subrange types. For example,
following type helper works only if I change type of TNewJ1939PGN from
subrange to dword.
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i program
forumtest; {$mode delphi} {$H+}{$modeswitch typehelpers} uses 
cthreads; type  TNewJ1939PGN = 0..(1 shl 18)-1; // does not work with
subrange, but works if type is changed to dword   TNewJ1939CanID =
bitpacked record    SA:       byte;         // if PGN was a record,
bit packing would not work and    PGN:      TNewJ1939PGN; // Priority
bits would not be immediatelly after PGN bits    Priority: 0..7;     
  // since PGN record would then be aligned to byte by pascal  end;  
                  // and there is no solution for that, so let's try
type helpers instead   TNewJ1939PGNHelper = type helper for
TNewJ1939PGN  private    function  GetPS: byte;    procedure
SetPS(const ANewPS: byte);  public    property  PS: byte read GetPS
write SetPS;  end; function TNewJ1939PGNHelper.GetPS: byte;begin 
Result := Self and $FF; //  and %00 00000000 11111111 (PS is in lowest
8 bits)end; procedure TNewJ1939PGNHelper.SetPS(const ANewPS:
byte);begin // debug says that Self was 100 and becomes 255, but PGN
value is not changed  Self := (Self and $3FF00) or ANewPS; //  and %11
11111111 00000000end;  // If I change type TNewJ1939PGN to dword, Self
becomes 255 as it should var  NewJ1939CanID: TNewJ1939CanID;begin 
NewJ1939CanID.PGN    := 100;  NewJ1939CanID.PGN.PS := 255; 
WriteLn('BitSizeOf(NewJ1939CanID)           = ',
BitSizeOf(NewJ1939CanID));  WriteLn('BitSizeOf(NewJ1939CanID.SA)     
 = ', BitSizeOf(NewJ1939CanID.SA)); 
WriteLn('BitSizeOf(NewJ1939CanID.PGN)       = ',
BitSizeOf(NewJ1939CanID.PGN)); 
WriteLn('BitSizeOf(NewJ1939CanID.Priority)  = ',
BitSizeOf(NewJ1939CanID.Priority));  WriteLn('NewJ1939CanID.PGN      
          = ', NewJ1939CanID.PGN);  WriteLn('NewJ1939CanID.PGN.PS    
         = ', NewJ1939CanID.PGN.PS);end.
Checked on FPC 3.3.1 SVN64623 x64 Linux.

----------------------------------------------------------------------
avk · 2021-03-26 16:14

It looks like a helper bug, and only when writing.(Linux,
FPC-3.3.1-r49028)
Maybe replace it with this one?
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i  
//TNewJ1939PGNHelper = type helper for TNewJ1939PGN 
TNewJ1939PGNHelper = type helper for TNewJ1939CanID  private   
function  GetPS: byte;    procedure SetPS(const ANewPS: byte); 
public    property  PS: byte read GetPS write SetPS;  end; function
TNewJ1939PGNHelper.GetPS: byte;begin  //Result := Self and $FF; //
 and %00 00000000 11111111 (PS is in lowest 8 bits)  Result :=
Self.PGN and $FF;end; procedure TNewJ1939PGNHelper.SetPS(const ANewPS:
byte);begin // debug says that Self was 100 and becomes 255, but PGN
value is not changed  //Self := (Self and $3FF00) or ANewPS; //  and
%11 11111111 00000000  Self.PGN := (Self.PGN and $3FF00) or
ANewPS;end;

----------------------------------------------------------------------
avra · 2021-03-29 06:32

Quote from: avk on March 26, 2021, 05:14:29 pmIt looks like a helper
bug, and only when writing.(Linux, FPC-3.3.1-r49028)Thanks for
confirmation.

Quote from: avk on March 26, 2021, 05:14:29 pmMaybe replace it with
this one?Moving PS from TNewJ1939PGN to TNewJ1939CanID is not
desirable. I wanted to keep PS in TNewJ1939PGN which is inside of
TNewJ1939CanID. Otherwise, I could simply avoid record in a record
which would then allow full bitpacking without non-desirable byte
alignment effect triggering, and type helpers would not be needed at
all. I like record in a record ease of use, so I hoped that type
helpers could come to the rescue. Unfortunately they do not seam to
work properly for subranges (which I do need in this case).

Quote from: jamie on March 26, 2021, 11:00:18 pmI would wager a little
to say its the old auto type level conversion taking place...

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i  Self :=
WORD((Self and $3FF00) or ANewPS); Besides word() type casting loosing
higher bits, and dword() dealing with more bits then needing, both do
not work and result is still 100 instead of 255.

I will file this as a bug and see what devs think about it...

----------------------------------------------------------------------
avra · 2021-03-29 06:46

Reported: https://bugs.freepascal.org/view.php?id=38678

----------------------------------------------------------------------
avra · 2021-04-07 09:18

Reported helper bug for subrange types is still there, but I have
found a work around for my use case. Initial goal was to have
bitpacked PGN record inside of another record to achieve dotted field
access like this:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i  
J1939CanID.PGN.Value    := 100;  J1939CanID.PGN.PS.Value := 255; 
J1939CanID.PGN.PF := 122;  J1939CanID.PGN.DP := true;Unfortunately PGN
is 18 bits in length so when put inside of another record it expanded
it self to 24 bits, which was bad since I needed 3 Priority bits right
after PGN record structure (with side effect that whole record was now
1 byte bigger then original C structure). At first I was able to do it
only after I gave up on PGN record (to avoid byte alignment of
bitpacked sub-record structure) which looked really ugly:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i  
J1939CanID.PGN    := 100;  J1939CanID.PGN_PS.Value := 255; 
J1939CanID.PGN_PF := 122;  J1939CanID.PGN_DP := true;
Finally, I have now managed to solve it by multiple overlays and dummy
bit inserts, Maybe it will be of use to someone else, so here it is:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i program
testforum; {$mode delphi} {$H+} uses  baseunix; type  T1939SimplePGN =
0..(1 shl 18)-1;   TJ1939PS = record case byte of    1: (Value:
cuint8);    2: (DA:    cuint8);    3: (GE:    cuint8);  end;  
TNewJ1939PGN = bitpacked record case byte of // J1939 PGN (Parameter
Group Number), as used in J1939 29 bit CAN ID    1: (Value:
T1939SimplePGN; );              // full access helper    2: (PS:
TJ1939PS; // 8 bit: PS in PDU2 , else 0 (PDU specific)        PF:
cuint8;   // 8 bit: PF (PDU format). PDU1 (0..239) indicates a
destination address in PS, PDU2 (250-255) indicates extension to PDU
Format (PF).        DP: boolean;  // 1 bit: DP (Data Page), page
selector for PDU (Protocol Data Unit) Format (PF) field. Currently at
0, pointing to Page 0. Page 1 is for future purposes)        R:
 boolean); // 1 bit: R  (Reserved), should always be set to 0 when
transmitting messages  end;   TNewJ1939CanID = bitpacked record case
byte of    1: ( SA:   byte;         case byte of           1: ( PGN: 
     TNewJ1939PGN; ); // PGN's 18 bits expand to 24 bits when used in
record           2: ( __DummyPGN: T1939SimplePGN;  // so we need
__DummyPGN just to skip 18 bits                Priority:   0..7; ); );
     // and then put these Priority 3 bits immediately after PGN's 18
bits    2: (Bit:   bitpacked array[0..31] of boolean); // bit access
helper    3: (Full:  cuint32);                           // full
access helper  end; var  NewJ1939CanID: TNewJ1939CanID;begin 
NewJ1939CanID.PGN.Value    := 100;  NewJ1939CanID.PGN.PS.Value :=
255;  //NewJ1939CanID.PGN.Value := (1 shl 18)-1; 
//NewJ1939CanID.PGN.DP := true;  //NewJ1939CanID.Priority := 7; 
WriteLn('BitSizeOf(NewJ1939CanID)           = ',
BitSizeOf(NewJ1939CanID));  WriteLn('BitSizeOf(NewJ1939CanID.SA)     
 = ', BitSizeOf(NewJ1939CanID.SA)); 
WriteLn('BitSizeOf(NewJ1939CanID.PGN)       = ',
BitSizeOf(NewJ1939CanID.PGN)); 
WriteLn('BitSizeOf(NewJ1939CanID.Priority)  = ',
BitSizeOf(NewJ1939CanID.Priority));  WriteLn('NewJ1939CanID.PGN.Value
          = ', NewJ1939CanID.PGN.Value); 
WriteLn('NewJ1939CanID.PGN.PS.Value         = ',
NewJ1939CanID.PGN.PS.Value);  WriteLn('BinStr(NewJ1939CanID.Full, 32)
   = ', BinStr(NewJ1939CanID.Full, 32));end.
You can explore comments for more explanation.

----------------------------------------------------------------------
avra · 2021-04-28 08:10

UPDATE: Here is a little better example. It shows that plain subrange
types do not have this type helper bug, plain records holding the same
subrange type do not have this bug, but bitpacked records holding this
subrange type do have this type helper bug!. There is still no
activity in bugtracker regarding this bug report.

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i program
testtypehelperbug;{$mode delphi} {$H+}{$modeswitch typehelpers} type 
TNewJ1939PGN = 0..(1 shl 18)-1;   // Both PGN and ID.PGN work well if
we replace 18 bits subrange type with dword type   TNewJ1939CanID =
bitpacked record // if TNewJ1939PGN is 18bits then ID.PGN works well
only when record is not bitpacked    SA: byte;    PGN: TNewJ1939PGN; 
 Priority: 0..7;  end;   TNewJ1939PGNHelper = type helper for
TNewJ1939PGN  private    function GetPS: byte;    procedure
SetPS(const ANewPS: byte);  public    property PS: byte read GetPS
write SetPS;  end; function TNewJ1939PGNHelper.GetPS: byte;begin 
Result := Self and $FF; // and %00 00000000 11111111 (PS is in lowest
8 bits)end; procedure TNewJ1939PGNHelper.SetPS(const ANewPS: byte); //
works well only when type is not used in a recordbegin // debug says
that Self was 100 and becomes 255, but PGN value is not changed  Self
:= (Self and $3FF00) or ANewPS; // and %11 11111111 00000000end; // If
I change type TNewJ1939PGN to dword, although debug says that Self
becomes 255 it does not propagate further var  PGN: TNewJ1939PGN;  ID:
TNewJ1939CanID;begin  PGN    := 100;  PGN.PS := 255;  WriteLn('PGN  
 = ', PGN);          // PGN is 255 as it should be  WriteLn('PGN.PS =
', PGN.PS);       // PGN.PS is 255 as it should be   ID.PGN    :=
100;  ID.PGN.PS := 255;  WriteLn('ID.PGN    = ', ID.PGN);    // PGN is
100 (WRONG!!!)  WriteLn('ID.PGN.PS = ', ID.PGN.PS); // PGN.PS is 100
(WRONG!!!)end.

----------------------------------------------------------------------
avra · 2021-04-29 06:31

Quote from: jamie on April 29, 2021, 03:25:35 amInteresting
problem.Unfortunately 

Quote from: jamie on April 29, 2021, 03:25:35 amThis is one of those
things that is kind of hard for the compiler to detect.
...
I would say the compiler will simply load and write that field as it
was a stand alone because I can't see how the helper knows it is
living in a bitpacked field.Well, when I directly assign value to 18
bit type in a bitpacked record (ID.PGN := 100;), compiler knows how to
handle it correctly and does all needed bit shifting and masking (even
with such a hard example as leading 2 bits before that 18 bit type and
3 trailing bits after that 18 bit type, which puts types totally out
of record byte boundaries). I would expect the same behavior when
helper on such 18 bit type in a bitpacked helper is used (ID.PGN.PS :=
255;). That is a special case which needs additional handling, but it
should be done for completeness. Unfortunately, compiler does not
treat it like that and we do have a bug. How it should be handled may
be debated, but it is a bug.

----------------------------------------------------------------------
avra · 2021-05-01 20:06

Quote from: jamie on April 29, 2021, 11:03:39 pmwhen you directly
read/write to the field as a stand alone item the compiler is treating
as a 32 bit because that is the next logical upswing for the byte
orders and since there is no bitpacking indicating there the helper
simply acceses as a 32 bit.

 When you place that same type in a bitpacked record now that field is
sharing the priority field of the bits that exist.

 being a helper at work I can't how the compiler will know the
difference here..

Take a look at the little modified example below. ID.PGN and ID.PGN.PS
both point to the same memory location in a bitpacked record not
aligned to a byte. ID.PGN is a field in the record and it can modify
it self properly. ID.PGN.PS is a helper and it can not modify it's
location properly. I still think that if a field in a bitpacked record
can work well (probably by masking and bit shifting), then helper
should be able to do the same.

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i program
testtypehelperbug;{$mode delphi} {$H+}{$modeswitch typehelpers} type 
T18bits = 0..(1 shl 18)-1;   // Both PGN and ID.PGN work well if we
replace 18 bits subrange type with dword type   TBitPackedRec =
bitpacked record // if T18bits is 18bits then ID.PGN works well only
when record is not bitpacked    SA:  0..(1 shl 7)-1;    PGN: T18bits; 
  PR:  0..7;  end;   T18bitsHelper = type helper for T18bits  private 
  function GetPS: byte;    procedure SetPS(const ANewPS: byte); 
public    property PS: byte read GetPS write SetPS;  end; function
T18bitsHelper.GetPS: byte;begin  Result := Self and $FF; // and %00
00000000 11111111 (PS is in lowest 8 bits)end; procedure
T18bitsHelper.SetPS(const ANewPS: byte);begin  Self := (Self and
$3FF00) or ANewPS; // and %11 11111111 00000000end; var  PGN:
T18bits;  ID:  TBitPackedRec;begin  PGN    := 100;  WriteLn('1. PGN  
 = ', PGN);          // PGN is 100 as it should be  WriteLn('1. PGN.PS
= ', PGN.PS);       // PGN.PS is 100 as it should be  PGN.PS := 255; 
WriteLn('2. PGN    = ', PGN);          // PGN is 255 as it should be 
WriteLn('2. PGN.PS = ', PGN.PS);       // PGN.PS is 255 as it should
be   ID.SA  := $55;  ID.PR  := 3;  ID.PGN := 100;  WriteLn('1. ID.PGN
  = ', ID.PGN);    // PGN is 100 as it should be  WriteLn('1.
ID.PGN.PS = ', ID.PGN.PS); // PGN.PS is 100 as it should be 
WriteLn('1. ID.SA     = ', ID.SA);     // PGN.SA is 85 as it should
be  WriteLn('1. ID.PR     = ', ID.PR);     // PGN.PR is 3 as it should
be  ID.PGN.PS   := 255;  WriteLn('2. ID.PGN    = ', ID.PGN);    // PGN
is 100 (WRONG!!! IT SHOULD BE 255)  WriteLn('2. ID.PGN.PS = ',
ID.PGN.PS); // PGN.PS is 100 (WRONG!!! IT SHOULD BE 255)  WriteLn('2.
ID.SA     = ', ID.SA);     // PGN.SA is 85 as it should be 
WriteLn('2. ID.PR     = ', ID.PR);     // PGN.PR is 3 as it should
beend.