How to declare a variable absolute with respect to a dynamic array?
lazarus / 61-Beginners

----------------------------------------------------------------------
Lupp · 2017-02-17 18:21

What I actually get passed to a procedure for a var parameter of a
dynamic array type when replacing it with an actual dynamic array is
obviously a pointer into the heap.

To do some work efficiently on a level of byte manipulation, I would
like to declare a differently typed variable absoulte with respect to
the data space of the (actual) parameter.
What exactly is kept in RAM for a dynamic array and in what way is
this organized?

Is it possible to declare a pocedure in a way that it will accept
actual dynamic arrays of different element types without writing tons
of declarations for overloaded procedures with a common name?

You see, I like strict typing, but the overload concept allows anyway
to weaken it.  In a few well considered cases I would like to do so
without lots of formalities.

----------------------------------------------------------------------
ASerge · 2017-02-17 19:11

Quote from: Lupp on February 17, 2017, 07:21:14 pmIs it possible to
declare a procedure in a way that it will accept actual dynamic arrays
of different element types without writing tons of declarations for
overloaded procedures with a common name?
You can use generics, like this:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i program
Project1; {$APPTYPE CONSOLE} type  generic TGArray<_T> =
class(TObject)  public    class procedure SumToFirst(var A: array of
_T); static;  end; class procedure TGArray.SumToFirst(var A: array of
_T);var  X: _T;  Sum: _T;begin  Sum := Default(_T);  for X in A do   
Sum := Sum + X;  A[Low(A)] := Sum;end; procedure Test;type  TIntArray
= specialize TGArray<Integer>;  TStrArray = specialize
TGArray<string>;var  IA: array of Integer;  SA: array of string;begin 
SetLength(IA, 2);  IA[0] := 1;  IA[1] := 2;  SetLength(SA, 2);  SA[0]
:= 'one';  SA[1] := 'two';  TIntArray.SumToFirst(IA); 
TStrArray.SumToFirst(SA);  WriteLn(IA[0], ' ', SA[0]);end; begin 
Test;  Readln;end.

----------------------------------------------------------------------
molly · 2017-02-17 19:18

Quote from: Lupp on February 17, 2017, 07:21:14 pmIs it possible to
declare a pocedure in a way that it will accept actual dynamic arrays
of different element types without writing tons of declarations for
overloaded procedures with a common name?
something like array of const you meant ? or perhaps even variants ?

----------------------------------------------------------------------
howardpc · 2017-02-17 20:22

You can use an untyped var parameter to pass 'generic' dynamic arrays
to a procedure, as in the following example:

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i program
project1; {$Mode objfpc}{$H+} type  TStringArray = array of string; 
TIntegerArray = array of integer;  TDoubleArray = array of
double; procedure ShowArray(var anArray; aType: TVarRec);var  sa:
TStringArray;  ia: TIntegerArray;  da: TDoubleArray;  i, maxi:
integer;begin  case aType.VType of    vtAnsiString: begin     
sa:=TStringArray(anArray);      maxi:=High(sa);      for i:=0 to maxi
do        if (i <> maxi) then          Write(sa[i],', ')        else
WriteLn(sa[i]);    end;    vtInteger: begin     
ia:=TIntegerArray(anArray);      maxi:=High(ia);      for i:=0 to maxi
do        if (i <> maxi) then          Write(ia[i],', ')        else
WriteLn(ia[i]);    end;    vtExtended: begin     
da:=TDoubleArray(anArray);      maxi:=High(da);      for i:=0 to maxi
do        if (i <> maxi) then          Write(da[i]:4:2,', ')       
else WriteLn(da[i]:4:2);    end;  end;end; var  sa: TStringArray;  ia:
TIntegerArray;  da: TDoubleArray;  vr: TVarRec; begin  SetLength(sa,
4);  sa:=TStringArray.Create('zero','one','two','three'); 
vr.VType:=vtAnsiString;  ShowArray(sa, vr);   SetLength(ia, 4); 
ia:=TIntegerArray.Create(0, 1, 2, 3);  vr.VType:=vtInteger; 
ShowArray(ia, vr);   SetLength(da, 4);  da:=TDoubleArray.Create(0.5,
1.5, 2.5, 3.5);  vr.VType:=vtExtended;  ShowArray(da, vr);   WriteLn; 
WriteLn('Press Enter to exit');  ReadLn;end.

----------------------------------------------------------------------
Lupp · 2017-02-18 00:29

(My experiences with Pascal were based on Turbo Pascal long ago. I did
a bit of programming with Delph 1/2 and very few with 7, subsequently
I had a look now and then into very old versions of FreePascal and
Lazarus.)

Thanks to all of you. As I already stated, I lack experience with some
of the modern concepts. I will have to study your suggestions. 
(I would surely like something like "types as values of variables" if
available handily.)

What I wanted to achieve here primarily was rather simple, however:

The var parameter of a procedure, say v, it may be typed TaSmallint
(type TaSmallint = array of Smallint) e.g, gives access to the data.
It also comes with information about the length of the array set by
SetLength.
Suppose I now want to analyze/manipulate the elements bytewise. I
declare Code: Pascal  [Select][+][-]window.onload = function(){var x1
= document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i b: array of
Byte absolute v; But now the High(v) value is also applied to b. If I
use {$R+} access to the single bytes of the upper half of my Smallints
will fail due to violation of the index range. An attempt to resize b
would probably move the b-array to elewhere on the heap.

Is there a simple way  to get the indices for b rangechecked as if
High(b)=High(v)*2+1 ? The compiler an my program as well have the
information needed to calculate the maximum length of b usable without
violating the range v brought in.

The behaviour I described does not cange if v is untyped. The crude
example
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i procedure
Test(var v);{$R-}var  a: TaSmall absolute v;  b: TaByte  absolute v; 
j: Longint;beginWriteLn(High(a), High(b):5);for j:=0 to 9 do 
WriteLn(a[j], b[j*2]:6, b[j*2+1]:6, b[j*2]+b[j*2+1]*256:8);end; again
assumes the same High() value for a and b.

----------------------------------------------------------------------
Thaddy · 2017-02-18 07:58

In the case of dynamic arrays there is no real simple way like for
static arrays AND maintain range checks..
Due to the nature of dynamic arrays, even if b is declared absolute to
a, on setlength() they become two distinct arrays. (see docs)
There are many options, though:
1.- My favorite: write a type helper for one of the array types that
has methods to represent it as another type. (AsByte,AsWord,etc)
2.- What I used to do and now strongly discourage: manipulate the
array descriptor directly. (Just because it can be done does not mean
it should be done)
3.- Use a pointer type to array and calculate Low/High yourself from
the first array low/high. What is also what is in 1..

----------------------------------------------------------------------
Lupp · 2017-02-18 10:46

Thanks again.
@Thaddy: You surely know the technical details of the implementation
of dynamic arrays with FP and of interoperability issues concerning
data exchange in a world also using different programming tools to
some degree.

Where and in what way is the High information stored?
Does the "structure" of dynamic arrays also care about the element
size? If so, in what way? If not, why?
Does the "structure" of dynamic arrays also provide means to
store/make accessible additional metadata?
Is there a text describing this and related topics in a way an amateur
like me might understand without intensely studying terminology first?

(Due to hubris I dared to assume the High information for a dynamic
array a should be stored some few bytes below the dataspace starting
with a[0] then. If so, is there a way to extend the range for metadata
downwards?)

Regards and thanks in advance.

(Editing.)
Testing with
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i procedure
Test(var v);var  p1: Pointer absolute v;  p2: ^Longint;  a: TaSmall
absolute v;  b: TaByte  absolute v;  j:
Longint;beginp2:=p1-8;WriteLn(p2^);for j:=0 to High(v) do 
{Something};end;  
resulted in favor of my assumption with 8 Byte being the size of High
information. (Everybody else surely knew that in advance.)

----------------------------------------------------------------------
Thaddy · 2017-02-18 12:51

Oh, well. I said I discourage that!  But you are on the right track.
Some more information:
- You need to keep in mind the native pointer size, so it's not always
8.... (64 pit systems have 64 bit pointers) of valid information
- You need to test for the alignment used, so it can be a lot more
than 8...and can be more than 8 in total information .
- Dynamic arrays also have a refcount in the descriptor
But indeed the array descriptor is at a negative offset - and not
portable as you may have guessed from the above ....

This is implementation detail and can change between ANY point release
of FPC. It isn't even the same as in Delphi, which also differs
between versions.
Stricktly for hacking purposes and not for production code.

----------------------------------------------------------------------
howardpc · 2017-02-18 12:54

Quote from: Lupp on February 18, 2017, 01:29:25 amThe crude example
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i procedure
Test(var v);{$R-}var  a: TaSmall absolute v;  b: TaByte  absolute v; 
j: Longint;beginWriteLn(High(a), High(b):5);for j:=0 to 9 do 
WriteLn(a[j], b[j*2]:6, b[j*2+1]:6, b[j*2]+b[j*2+1]*256:;end; 
I'm guessing, but are you after the following?
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i program
project1; {$Mode objfpc}{$H+} type  TSmallIntArray = array of
SmallInt;   procedure Test(anArray: TSmallIntArray);  var    s:
SmallInt;  begin    for s in anArray do      WriteLn('s:',s:2,'
hi(s):',hi(s):3,' lo(s):',lo(s));  end; var  sma:
TSmallIntArray; begin  SetLength(sma, 7); 
sma:=TSmallIntArray.Create(-3,-2,-1,0,1,2,3);  Test(sma);   WriteLn; 
WriteLn('Press Enter to exit');  ReadLn;end.

----------------------------------------------------------------------
Thaddy · 2017-02-18 13:01

Actually what he wants is two types of dynamic array using the same
memory space, just like  two static arrays, one of which is declared
absolute to the other's address.
That can only somewhat be achieved by any of the methods I provided,
since after a setlength the arrays become distinctive.
But you can read (Low() (That is actually always zero for dynamic
arrays anyway) and ) High() from the first array, use a pointer, and
then use the memory space as if it was an array of a different type.
Get it? Sort of absolute for dynamic arrays. (Yes you can declare them
absolute, but that won't work after a setlength()......)
Do not use it... can be done... I discourage it... Use at your own
risk.

The best solution for his problem is to write an array helper. That
can be type safe and memory safe.

----------------------------------------------------------------------
Lupp · 2017-02-18 13:36

He (that's me) is aiming
1) at a better understanding.
2) at a few things I consider to write libraries for (to compile to
dll for Win, e.g.).
3) at doing so minimizing overhead.
4) at getting high efficiency.
5) not at resizable arrays. This is about dynamic arrays created once
in another place and passed to procedures as transients to work on
them.

I am aware (hope so) of the issues concerning dependencies on FP
version and OS. (In addition calling conventions.) As there obviously
not is access to an implementation independent view of array
descriptors, not even concerning ElementCount / High and ElementSize,
I might either design a class providing this in it's own way, or write
procedures directly getting passed the needed information via
dedicated parameters. I now will try to do it the second way
concerning a first example.

----------------------------------------------------------------------
Thaddy · 2017-02-18 13:55

5) is a dynamic array and therefor by definition a resizable array in
pascal.
Alas we have no syntax for it like in C, but we have alternative
methods.
Note this is also not in Kernigan and Richie C, the mother of all
C(++)..

I think I understand your issue, but this is as it stands.
Just like in C we can manipulate a memory area in any way we feel, but
not with a really simple construct.
And C doesn't have High() anyway...
C == un-C-afe by default. If you make many things look easy, you can
expect many accidents.
A type helper is just about the best I can think of and that could be
made a part of the RTL if sufficiently mature and still respects
Pascal efforts to make things safe.

----------------------------------------------------------------------
Lupp · 2017-02-18 17:16

Quote5) is a dynamic array and therefore by definition a resizable
array in pascal.
Yes I expressed myself badly again. I had understood and accepted
this. My point 5) was about not aiming at making use of SetLength on
arrays passed as parameters because different parts of the program or
a different program calling an external routine are in charge of
sizing.

Omitting range checks in very specific cases for very short parts of a
routine to avoid overhead and to enhance efficiency should be safe
enough. I will not go the C-do. (I always deemed C ugly when I had to
try to understand a piece of code. My original training more than 50
years ago was with ALGOL60 - very strictly typed.)

----------------------------------------------------------------------
Thaddy · 2017-02-18 20:48

Well, you over-stressed my humble experience since 1978....(Bugger..
) 
But if I can be of any service to olderbies I am glad to help out when
possible.
Maybe I should add my mother is 86 and is perfectly capable to write
Delphi database applications (true!)
She didn't switch to Lazarus yet,... Oh, give her time... I am very
proud about that fact...and my mother of course,

----------------------------------------------------------------------
Blestan · 2017-02-19 00:03

hahahahaha . speaking of parents .. my father was (born in 1943 and he
passed away in 2013) the first programers in bulgaria ever ..
programed in algol in the early 60 .. then fortran in the '70 ....
cics/cobol  for ibm 360 in the 80'then i theached him to delphi in the
'90 .... ))
thaddy is one of the best agents in this forum ... his comments aways
acted for me like a kick in the face but very provacative/productive
))\
cheers ))
p.s if you do not take in consideration thaddy's comment you are a
complete idiot / morron / nerd ... believe me )

----------------------------------------------------------------------
molly · 2017-02-19 06:30

Quote from: Lupp on February 18, 2017, 02:36:48 pmAs there obviously
not is access to an implementation independent view of array
descriptors, not even concerning ElementCount / High and ElementSize,
I might either design a class providing this in it's own way, or write
procedures directly getting passed the needed information via
dedicated parameters.
The first isn't true, see code below. (ok, ok.. i cheated by providing
that info   )

For the later you'd better as otherwise things becomes a mess. What
will happen if someone would to pass a simple integer to your
function, or a byte ? It is called an untyped parameter for a reason 

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i program
something; {$MODE OBJFPC}{$H+}{$RANGECHECKS ON} uses  typinfo; Type 
PByteArray = ^TByteArray;  TByteArray = array of byte;  TWordArray =
array of Word;  TLongWordArray = array of LongWord; procedure
DoSomethingWithSomething(var v; ti: PTypeInfo);var  a   : PByte;  i  
: integer;begin  a := Addr(v);   if (ti^.Kind = tkDynArray) then 
begin    if (DynArrayDim(ti) = 1) then    begin      for i := 0 to
Pred(GetTypeData(ti)^.elSize * DynArraySize(pointer(v))) do     
begin        Write('$', HexStr(a[i],2),' ');      end;      WriteLn; 
 end    else      WriteLn('ERROR: Multidimensional arrays are not
supported');  end  else    WriteLn('ERROR: type ', ti^.kind, ' is
unsupported');end; var  x: TbyteArray;  y: TWordArray;  z:
TLongWordArray;begin  x := TByteArray.Create(9,8,7,6,5,4,3,2,1);    y
:= TWordArray.Create($1122,$3344,$5566,$7788,$9900);  z :=
TLongWordArray.Create($12345678,$23456789,$34567890);  
DoSomethingWithSomething(x,TypeInfo(x));  WriteLn; 
DoSomethingWithSomething(y,TypeInfo(y));  WriteLn; 
DoSomethingWithSomething(z,TypeInfo(z));  WriteLn;    SetLength(x,
0);  SetLength(y, 0);  SetLength(z, 0);end.

----------------------------------------------------------------------
Lupp · 2017-02-19 13:14

1) Thank you, @molly.
2) I did not yet know the unit 'typinfo'. (Do you know the reason for
not naming it 'typeinfo'?)

3) It's not quite the style I would prefer as it seems to be against
the grain somehow. (Implementing features the language doesn't want to
support basically.)
3.a) Functionally I would see a language/compiler modification.
However, I do not understand how registering packages works.
3.b) The 'TypeInfo' function returns a pointer that must be
dereferenced explicitly.
3.c) I feel slightly reminded of the implemenattion of mouse-awareness
by Turbo Pascal in old times.
3.d) What is returned by TypeInfo(ti^).Kind? (Or TypeInfo(ti^)^.Kind?)

4) Ok. There's a dilemma.  If I expect meta information to be
availabel about variables, I will also expect to be able to save it to
files.  (And to pass it to different applications?)
5) It's a mess. The world is too complicated. At least this fact is
simple.
6) I would better stay silent.

7) I didn't understand the remark Quote(ok, ok.. i cheated by
providing that info   ) Suppose it's fun. A lesson in English -or
whatever- available?

----------------------------------------------------------------------
howardpc · 2017-02-22 14:16

Quote from: Lupp on February 19, 2017, 02:14:38 pm2) I did not yet
know the unit 'typinfo'. (Do you know the reason for not naming it
'typeinfo'?)

Simply to avoid a name clash with the compiler-intrinsic TypeInfo()
function that can be applied to any type at all, returning a
PTypeInfo.

----------------------------------------------------------------------
molly · 2017-02-25 13:12

Quote from: Lupp on February 19, 2017, 02:14:38 pm1) Thank you,
@molly.
You're welcome 

Quote3) It's not quite the style I would prefer as it seems to be
against the grain somehow. (Implementing features the language doesn't
want to support basically.)
Pascal is a strongly typed language, you seem to want to 'overrule'
the strong type. That causes issues that have to be dealt with.

Quote3.a) Functionally I would see a language/compiler modification.
However, I do not understand how registering packages works.
That will not happen. Because you made the parameter 'typeless' no
rtti information is available ergo you are unable to determine its
type using the usual constructs.

Quote3.b) The 'TypeInfo' function returns a pointer that must be
dereferenced explicitly.
yes.

Quote3.c) I feel slightly reminded of the implemenattion of mouse-
awareness by Turbo Pascal in old times.
You asked for 'trouble' so... 

Quote3.d) What is returned by TypeInfo(ti^).Kind? (Or
TypeInfo(ti^)^.Kind?)
That depends on the type 

More information can be read inside documentation of unit typinfo
(press the types section at the top).

Quote4) Ok. There's a dilemma.  If I expect meta information to be
availabel about variables, I will also expect to be able to save it to
files.  (And to pass it to different applications?)
That should be no problem ?

Quote5) It's a mess. The world is too complicated. At least this fact
is simple.
That fact wasn't simple for me because this post slipped through my
todo list....   sorry for the delay.

QuoteSuppose it's fun. A lesson in English -or whatever- available?
you originally wrote:

QuoteAs there obviously not is access to an implementation independent
view of array descriptors,

To which i replied:
QuoteThe first isn't true, see code below.

But i later added, that i cheated. I cheated because _I_ am supplying
the information that is needed for the routine in order to have a
"implementation independent view of array descriptors""

You could just as well invent your own 'design' to solve the issue in
a similar manner. Point being: you would have to provide the
information about the array structure. A well designed record and
passing that would also work but, is basically the same as what
TTypeInfo does (and the RTTI does a better job than i ever could ;-)

Again, sorry for the delay.