Some incompatibilities with Turbo Pascal regarding string handling lazarus / 62-General ---------------------------------------------------------------------- Avinash · 2022-08-25 00:33 Well, there is such a type as a pascal string. And there are procedures that have pascal strings as parameters. But there are also procedures that require a pointer to a pascal string (PString). And here difficulties arise when it is necessary to pass the result of a string expression or function to such procedures. You have to define an extra variable: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i var S2: String;...S2 := FExpand(S);XProc(@S2); However, there is a trick to make things easier: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i function StrPtr(const S: String): PString;begin  StrPtr := @S;end;Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i XProc(StrPtr(FExpand(S)));In this case, the compiler reserves a hidden string variable in the stack, and the pointer produced by StrPtr is valid until the procedure in which we do this is completed. This simplifies the source code and makes the binaries a bit smaller. But let's ask a question, what will happen if there are several StrPtr calls in one procedure? Will multiple hidden variables be reserved, or will the same one be used? Let's check: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i type  PString = ^String;   function StrPtr(const S: String): PString;  begin    StrPtr := @S;  end;   procedure Test(PS1, PS2, PS3, PS4: PString);  begin    WriteLn(PS1^);    WriteLn(PS2^);    WriteLn(PS3^);    WriteLn(PS4^);  end; var  S1, S2, S3, S4, S5, S6, S7, S8: String; begin  S1 := '1111';  S2 := '2222';  S3 := '3333';  S4 := '4444';  S5 := '5555';  S6 := '6666';  S7 := '7777';  S8 := '8888';   Test(StrPtr(S1+S2), StrPtr(S3+S4), StrPtr(S5+S6), StrPtr(S7+S8));end. Turbo Pacal by Borland is unbreakable: Code: Text  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i 11112222333344445555666677778888 Free Pascal: Code: Text  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i 11112222111122221111222211112222(or 77778888 depending on calling convention) ---------------------------------------------------------------------- PascalDragon · 2022-08-25 06:57 You are relying on implementation details here. It is nowhere documented that temporary variables (in this case the result of the string concatenations) can not be reused right away which is what FPC is doing and it is perfectly in its right to do so. ---------------------------------------------------------------------- 440bx · 2022-08-25 12:23 Quote from: PascalDragon on August 25, 2022, 08:57:13 amYou are relying on implementation details here. It is nowhere documented that temporary variables (in this case the result of the string concatenations) can not be reused right away which is what FPC is doing and it is perfectly in its right to do so. That doesn't sound right.  As a matter of course, the compiler should treat every parameter as being totally independent of any other parameter and, for this reason, it should be creating a temporary for each one. ---------------------------------------------------------------------- BobDog · 2022-08-25 16:25 In the function StrPtr, I cannot assign a new value (inside the function) for the string parameter S, so I assume that S is actually a proper constant. (I get:Error: Can't assign values to const variable) So in light of that, supply that function with an actual constant. concatenation of two constants = another constant. Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i type  PString = ^String;   function StrPtr(const S: String): PString;  begin    StrPtr := @s;  end;   procedure Test(PS1, PS2, PS3, PS4: PString);  begin    WriteLn(PS1^);    WriteLn(PS2^);    WriteLn(PS3^);    WriteLn(PS4^);  end; const  S1 = '1111';  S2 = '2222';  S3 = '3333';  S4 = '4444';  S5 = '5555';  S6 = '6666';  S7 = '7777';  S8 = '8888'; begin  Test(StrPtr(S1+S2), StrPtr(S3+S4), StrPtr(S5+S6), StrPtr(S7+S8));readln;end. ---------------------------------------------------------------------- 440bx · 2022-08-25 18:11 Quote from: BobDog on August 25, 2022, 06:25:43 pmIn the function StrPtr, I cannot assign a new value (inside the function) for the string parameter S, so I assume that S is actually a proper constant. (I get:Error: Can't assign values to const variable) So in light of that, supply that function with an actual constant. concatenation of two constants = another constant. The "const" in StrPtr tells the compiler that the parameter cannot be changed by the function, it doesn't mean the parameter must be a constant. As long as the parameter is a "String" the parameter's type requirements are met. Also, the concatenation of two string constants cannot be a constant. The resulting concatenation has to be placed in read/write memory otherwise it is not possible to create the concatenation. ---------------------------------------------------------------------- BobDog · 2022-08-25 19:00 Maybe I worded it wrongly. (I did word it wrongly) The pointers generated at compile time are all unique (from any two constants added), At runtime the pointer to S is always the same location,  it takes the address of the first one processed. These are my observations only. But it should run to give the correct result IMO 11112222 33334444 55556666 77778888 I tested in C and freebasic, both give the above result. OK, before anybody says it, this is not a C or freebasic forum, but I give my observations. ---------------------------------------------------------------------- Avinash · 2022-08-25 19:47 In this case the already concatenated strings in the data segment are just generated. (some characters in the dump part, like the first byte of length S[0]=#8 of these strings didn't showing by the forum font) 000079F4: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  │................ 00007A04: 08 37 37 37 37 38 38 38 38 00 00 00 08 35 35 35  │77778888...555 00007A14: 35 36 36 36 36 00 00 00 08 33 33 33 33 34 34 34  │56666...3333444 00007A24: 34 00 00 00 08 31 31 31 31 32 32 32 32 00 00 00  │4...11112222... 00007A34: 00 00 00 00 39 54 68 69 73 20 62 69 6E 61 72 79  │....9This binary 00007A44: 20 68 61 73 20 6E 6F 20 73 74 72 69 6E 67 20 63  │ has no string c 00007A54: 6F 6E 76 65 72 73 69 6F 6E 20 73 75 70 70 6F 72  │onversion suppor 00007A64: 74 20 63 6F 6D 70 69 6C 65 64 20 69 6E 2E 00 00  │t compiled in... 00007A74: 67 52 65 63 6F 6D 70 69 6C 65 20 74 68 65 20 61  │gRecompile the a 00007A84: 70 70 6C 69 63 61 74 69 6F 6E 20 77 69 74 68 20  │pplication with 00007A94: 61 20 75 6E 69 74 20 74 68 61 74 20 69 6E 73 74  │a unit that inst 00007AA4: 61 6C 6C 73 20 61 20 75 6E 69 63 6F 64 65 73 74  │alls a unicodest 00007AB4: 72 69 6E 67 20 6D 61 6E 61 67 65 72 20 69 6E 20  │ring manager in 00007AC4: 74 68 65 20 70 72 6F 67 72 61 6D 20 75 73 65 73  │the program uses 00007AD4: 20 63 6C 61 75 73 65 2E 00 00 00 00 0E 52 75 6E  │ clause.....Run 00007AE4: 74 69 6D 65 20 65 72 72 6F 72 20 00 05 20 61 74  │time error . at 00007AF4: 20 24 00 00 03 20 20 24 00 00 00 00 6B 65 72 6E  │ $..  $....kern 00007B04: 65 6C 33 32 00 00 00 00 53 65 74 54 68 72 65 61  │el32....SetThrea 00007B14: 64 44 65 73 63 72 69 70 74 69 6F 6E 00 00 00 00  │dDescription.... ---------------------------------------------------------------------- 440bx · 2022-08-25 20:03 Quote from: BobDog on August 25, 2022, 09:00:37 pm11112222 33334444 55556666 77778888 I tested in C and freebasic, both give the above result. OK, before anybody says it, this is not a C or freebasic forum, but I give my observations. There is nothing wrong with that and, IMO, pointing out that those are the results obtained in other languages is completely fair. ---------------------------------------------------------------------- MarkMLl · 2022-08-25 20:08 Quote from: Avinash on August 25, 2022, 09:47:49 pmIn this case the already concatenated strings in the data segment are just generated. If PascalDragon says that you're relying on internal compiler/runtime behaviour, that's pretty much the end of it as far as this forum is concerned. I agree that things relating to calling convention etc. can be pretty fuzzy at times, but the number of people hacking TP in its prime meant that code was being written that worked /despite/ being significantly removed from the supported APIs. But as a specific example, would you insist that since (working from fuzzy memory here) TP returned a 32-bit result in AXDX that that behaviour should be replicated by FPC? I've got the sources of a Prolog interpreter which handles garbage collection by making lots of assumptions about Turbo Pascal heap management based on Borland's description of implementation details and what people had worked out. As such it is, regrettably impossible to port it to FPC: "darned shame". MarkMLl ---------------------------------------------------------------------- PascalDragon · 2022-08-26 07:17 Quote from: 440bx on August 25, 2022, 02:23:58 pmQuote from: PascalDragon on August 25, 2022, 08:57:13 amYou are relying on implementation details here. It is nowhere documented that temporary variables (in this case the result of the string concatenations) can not be reused right away which is what FPC is doing and it is perfectly in its right to do so. That doesn't sound right.  As a matter of course, the compiler should treat every parameter as being totally independent of any other parameter and, for this reason, it should be creating a temporary for each one. The point here is that in total five calls are involved: four ones to the StrPtr and then the one to Test. The temporary variable is involved when concatenating the strings to pass them to StrPtr. The other temporary variables involved are those that receive the results of these StrPtr calls. As the StrPtr calls are independent of each other the compiler will reuse the temporary variable that is passes to the StrPtr functions, but of course not the temporary variables that capture the results of these calls. Again: this relies on implementation details of both TP and FPC. It is nowhere documented how this should behave and thus the compiler is free to do as it wants and here the potential for optimizations wins out. Quote from: MarkMLl on August 25, 2022, 10:08:24 pmQuote from: Avinash on August 25, 2022, 09:47:49 pmIn this case the already concatenated strings in the data segment are just generated. If PascalDragon says that you're relying on internal compiler/runtime behaviour, that's pretty much the end of it as far as this forum is concerned. Correct. ---------------------------------------------------------------------- dje · 2022-08-26 07:50 Simple solution is to implement your own temp string pool. Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i var  StringPoolPos: integer;  StringPool: array[0..15] of string; (* Length must be power of 2 *)  function StrPtr(const S: string): PString;  begin    StringPoolPos := (StringPoolPos + 1) and High(StringPool);  StringPool[StringPoolPos] := S;    StrPtr := @StringPool[StringPoolPos];  end; ---------------------------------------------------------------------- BobDog · 2022-08-26 08:57 You could do it this way: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i type  PString = ^String;   function StrPtr(const S: String): PString;  const    memory:array of string=nil;  var temp:Pstring;  beginsetlength (memory,length(memory)+1);    temp := @memory[high(memory)];    temp^:=S;exit(temp);  end;   procedure Test(PS1, PS2, PS3, PS4: PString);  begin    WriteLn(PS1^);    WriteLn(PS2^);    WriteLn(PS3^);    WriteLn(PS4^);  end; var  S1, S2, S3, S4, S5, S6, S7, S8: String; begin  S1 := '1111';  S2 := '2222';  S3 := '3333';  S4 := '4444';  S5 := '5555';  S6 := '6666';  S7 := '7777';  S8 := '8888';   Test(StrPtr(S1+S2), StrPtr(S3+S4), StrPtr(S5+S6), StrPtr(S7+S8));readln;end. ---------------------------------------------------------------------- MarkMLl · 2022-08-26 09:14 Quote from: PascalDragon on August 26, 2022, 09:17:32 amAgain: this relies on implementation details of both TP and FPC. It is nowhere documented how this should behave and thus the compiler is free to do as it wants and here the potential for optimizations wins out. *BOTH* compilers. Also, is OP basing his expectations on shortstring behaviour? As soon as other string types are introduced reference counting etc. comes into play, which is most definitely an area in which I'd not even dream of challenging a member of the core team. MarkMLl ---------------------------------------------------------------------- BobDog · 2022-08-26 10:08 Is this topic not just a form of bug investigating, using comparison between two pascal compilers, and the developer says yea or nay, nay in this instance. Is this code a viable work around for the situation? Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i type  PString = ^String;   function StrPtr(const S: String): PString;  const temp:Pstring=nil;  begin    GetMem(temp,sizeof(string));    temp^:=S;    StrPtr:=temp;  end;   procedure Test(PS1, PS2, PS3, PS4: PString);  begin    WriteLn(PS1^);    WriteLn(PS2^);    WriteLn(PS3^);    WriteLn(PS4^);  end; var  S1, S2, S3, S4, S5, S6, S7, S8: String; begin  S1 := '1111';  S2 := '2222';  S3 := '3333';  S4 := '4444';  S5 := '5555';  S6 := '6666';  S7 := '7777';  S8 := '8888';   Test(StrPtr(S1+S2), StrPtr(S3+S4), StrPtr(S5+S6), StrPtr(S7+S8));readln;end. Or must I use a global pointer and freemem at the end. I have not used memory allocation in freepascal before, but Google is very good for freepascal help, I use it all the time. ---------------------------------------------------------------------- dje · 2022-08-26 10:26 Quote from: BobDog on August 26, 2022, 12:08:02 pmIs this code a viable work around for the situation? I would say no. Memory is not freed. Same with your previous code. "array of string" is not in TP mode, and is not freed. I posted the solution I used on the Atari ST using Highspeed Pascal & PurePascal. I used a circular string pool. As long as I didn't call 'StrPtr' more than 16 times in one call, it works. No memory leaks. ---------------------------------------------------------------------- SymbolicFrank · 2022-08-26 11:06 So, there is a trick to make things easier, which uses manual pointer assignment, and this should work by default? I don't get it. If you're using pointers, all bets are off and you're expected to know what you are doing. It's unreasonable to expect the compiler to read your mind and implement what you want that pointer to represent. ---------------------------------------------------------------------- Thaddy · 2022-08-26 11:37 Why not simply:Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i {$IFDEF FPC}{$MODE TP}{$ENDIF}var  S1, S2, S3, S4, S5, S6, S7, S8: String;procedure test(a,b,c,d:string);begin  writeln(a);  writeln(b);  writeln(c);  writeln(d);end; begin  S1 := '1111';  S2 := '2222';  S3 := '3333';  S4 := '4444';  S5 := '5555';  S6 := '6666';  S7 := '7777';  S8 := '8888';   Test(S1+S2, S3+S4, S5+S6, S7+S8);  readln;end.This should be is also TP compatible, since TP allows the + operator for string concatenation and defaults to shortstring. And the above code is not dependent on any internals... ---------------------------------------------------------------------- Martin_fr · 2022-08-26 11:59 Quote from: BobDog on August 26, 2022, 12:08:02 pmIs this topic not just a form of bug investigating, using comparison between two pascal compilers, Not really... Lets say, I have 2 cars, and they are specified to have a top speed of 100 m/h. Now I drive down a steep road, and have strong backwind. One of the cars speeds up to 140m/h. But the other car doesn't. Is the other car "buggy"? Well, only if the spec of the car says: top speed on flat 100. top speed on downhill (min 10%) 140. The @ of string-concate example is actually very similar to the often discussed "scope of an interface" "scope of an interface"... With some compilers people used an interface (ref counted / auto freed) to destroy their objects. They made use of the observation that those compilers will free an interface at the very end of the procedure in which it is used. (even if it isn't used, after a certain point in the procedure). From what I read in discussions, this is an observation  only. It's not documented. The documentation apparently just states that it will be "dec-refed", but not when exactly. So FPC takes a more optimized approach (yes fpc has optimizations that some people would rather not to be...). FPC dec-refs the interface right after it's last use. And that is way too early for the trick to auto-free the object behind. (It is of course way better in terms of resource management) So if you observed some behaviour, but it is not documented, then you can only relay on it being present in that very exact compiler. But with any other compiler for the same language (even with a new version, or changed settings for the same compiler) you need to verify the observation again. And if it behaves different, then that is the norm. Getting the same observation of undocumented behaviour is like winning a raffle: It happens, but only sometimes. In the concrete example. Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i  Test(StrPtr(S1+S2), StrPtr(S3+S4), StrPtr(S5+S6), StrPtr(S7+S8)); "S1+S2" is a temporary value. It is not stored in any variable. You can not take the address of it Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i @(S1 + S2)Quoteproject1.lpr(21,10) Error: Variable identifier expectedThe position is the "+" operator. The "@" is expecting a single variable, not an expression. So you "worked around" by passing it to a function. Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i StrPtr(S1+S2)Inside the function, the value is stored in a variable. That is the parameter "const S: String". This parameter does have the same address as the temporary storage of the concatenation (because of the "const" modifier). But the scope of this variable ends at the end of the function "StrPtr" So the following happens Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i  Test(  StrPtr(   StrPtr(S1+S2),      S3+S4    // 1: The values are concatenated. The result is stored in a temporary place.                 //      The concatenated value is only passed to StrPtr. So the temp place is only "used" until the function returns.                 //      After StrPtr returns the temp place can be used again (and will be used again)                   //      The temp place holds the "AnsiString" which is the pointer to the text.                  //      (Remember PString is a pointer to a (internal) pointer to some text)    ),  // 2: StrPtr returns the Pointer (to pointer to text).     //     The temp place can be re-used.   StrPtr(S5+S6),    StrPtr(S7+S8)); Like Delphi with interfaces, TP may have been less optimized about realizing when it can re-use the temp-storage. So by lack of optimization, the code may work in (the specific version) of TP. Remember a pointer does not extend the scope or lifetime of the value it points to. (There is no documentation that says it would, and docs about scope/lifetime do not declare any such exception, so such exception do not exist in docs.) If the value's lifetime ends, the pointer becomes dangling. In the example code, by coincidence the dangling pointers all end up on the same value. But even that is not guaranteed. The dangling pointers could point to just anything at all. ---------------------------------------------------------------------- MarkMLl · 2022-08-26 12:23 Quote from: Martin_fr on August 26, 2022, 01:59:38 pmRemember a pointer does not extend the scope or lifetime of the value it points to. (There is no documentation that says it would, and docs about scope/lifetime do not declare any such exception, so such exception do not exist in docs.) If the value's lifetime ends, the pointer becomes dangling. OP also appears to be overlooking the fact that ^String is an opaque descriptor with the actual text being at something like ^^String. There is no guarantee that the lifetime of the text is the same as the lifetime of the descriptor, i.e. there are potentially two refcounts here. Relying on undocumented behaviour is unwise. It's also potentially illegal if the end product and its documentation is supposed to comply with ISO 9000. MarkMLl ---------------------------------------------------------------------- PascalDragon · 2022-08-26 12:48 Quote from: MarkMLl on August 26, 2022, 11:14:30 amQuote from: PascalDragon on August 26, 2022, 09:17:32 amAgain: this relies on implementation details of both TP and FPC. It is nowhere documented how this should behave and thus the compiler is free to do as it wants and here the potential for optimizations wins out. *BOTH* compilers. Also, is OP basing his expectations on shortstring behaviour? As soon as other string types are introduced reference counting etc. comes into play, which is most definitely an area in which I'd not even dream of challenging a member of the core team. With non ShortString strings you wouldn't need to use tricks like PString in the first place, cause that only exists to avoid the need to copy 256 Byte per String parameter on the stack. With the managed string types you're dealing with references anyway. Quote from: BobDog on August 26, 2022, 12:08:02 pmIs this code a viable work around for the situation? While your code would solve the specific problem you would have a memory leak as dje mentioned. Quote from: Thaddy on August 26, 2022, 01:37:36 pmWhy not simply:Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i {$IFDEF FPC}{$MODE TP}{$ENDIF}var  S1, S2, S3, S4, S5, S6, S7, S8: String;procedure test(a,b,c,d:string);begin  writeln(a);  writeln(b);  writeln(c);  writeln(d);end; begin  S1 := '1111';  S2 := '2222';  S3 := '3333';  S4 := '4444';  S5 := '5555';  S6 := '6666';  S7 := '7777';  S8 := '8888';   Test(S1+S2, S3+S4, S5+S6, S7+S8);  readln;end.This should be is also TP compatible, since TP allows the + operator for string concatenation and defaults to shortstring. And the above code is not dependent on any internals... The preface was that there are (pre-existing) functions that take PString as parameter. Quote from: Martin_fr on August 26, 2022, 01:59:38 pmThe @ of string- concate example is actually very similar to the often discussed "scope of an interface" "scope of an interface"... With some compilers people used an interface (ref counted / auto freed) to destroy their objects. They made use of the observation that those compilers will free an interface at the very end of the procedure in which it is used. (even if it isn't used, after a certain point in the procedure). From what I read in discussions, this is an observation  only. It's not documented. The documentation apparently just states that it will be "dec-refed", but not when exactly. So FPC takes a more optimized approach (yes fpc has optimizations that some people would rather not to be...). FPC dec-refs the interface right after it's last use. And that is way too early for the trick to auto-free the object behind. (It is of course way better in terms of resource management) So if you observed some behaviour, but it is not documented, then you can only relay on it being present in that very exact compiler. But with any other compiler for the same language (even with a new version, or changed settings for the same compiler) you need to verify the observation again. And if it behaves different, then that is the norm. Getting the same observation of undocumented behaviour is like winning a raffle: It happens, but only sometimes. And guess what: the compiler in question *cough* Delphi *cough* did in fact change this non documented behaviour in recent versions. Quote from: MarkMLl on August 26, 2022, 02:23:21 pmQuote from: Martin_fr on August 26, 2022, 01:59:38 pmRemember a pointer does not extend the scope or lifetime of the value it points to. (There is no documentation that says it would, and docs about scope/lifetime do not declare any such exception, so such exception do not exist in docs.) If the value's lifetime ends, the pointer becomes dangling. OP also appears to be overlooking the fact that ^String is an opaque descriptor with the actual text being at something like ^^String. There is no guarantee that the lifetime of the text is the same as the lifetime of the descriptor, i.e. there are potentially two refcounts here. Pointers and ShortString (which is what we're talking about here in comparison to the managed string types) do not have any reference counts. ---------------------------------------------------------------------- MarkMLl · 2022-08-26 12:57 Quote from: PascalDragon on August 26, 2022, 02:48:33 pmPointers and ShortString (which is what we're talking about here in comparison to the managed string types) do not have any reference counts. OP's original code, which defines his expectations, has no indication of compiler mode and declares strings with no associated length. I interpret that as meaning that he is either not using ShortStrings, or is mixing short strings with managed types. MarkMLl ---------------------------------------------------------------------- PascalDragon · 2022-08-26 13:06 Quote from: MarkMLl on August 26, 2022, 02:57:08 pmQuote from: PascalDragon on August 26, 2022, 02:48:33 pmPointers and ShortString (which is what we're talking about here in comparison to the managed string types) do not have any reference counts. OP's original code, which defines his expectations, has no indication of compiler mode and declares strings with no associated length. I interpret that as meaning that he is either not using ShortStrings, or is mixing short strings with managed types. Considering that they compared with TP I simply assumed ShortString (also I just tested the code with {$H+} and there it fails in a completely different way, meaning String = ShortString is a rather safe bet ). ---------------------------------------------------------------------- MarkMLl · 2022-08-26 13:15 I wasn't assuming anything, I'm an engineer (dammit), not a programmer :-) MarkMLl ---------------------------------------------------------------------- 440bx · 2022-08-26 16:47 Quote from: PascalDragon on August 26, 2022, 09:17:32 amThe point here is that in total five calls are involved: four ones to the StrPtr and then the one to Test. The temporary variable is involved when concatenating the strings to pass them to StrPtr. The other temporary variables involved are those that receive the results of these StrPtr calls. As the StrPtr calls are independent of each other the compiler will reuse the temporary variable that is passes to the StrPtr functions, but of course not the temporary variables that capture the results of these calls. Again: this relies on implementation details of both TP and FPC. It is nowhere documented how this should behave and thus the compiler is free to do as it wants and here the potential for optimizations wins out. I understand all that but, the compiler is _not_ free to assume that the result of a function is always the same regardless of the parameters passed to it.  In the case of StrPtr, the function always returns the same address (address of a string) but, the compiler cannot assume that the _contents_ of the string are constant which is what it is implicitly doing. By using a single temporary and re-using it for each parameter the compiler is implicitly making the incorrect assumption that the contents of the string pointed to by the result is a constant. Since the compiler has accepted the burden of creating temporary strings to hold results, in the case of "Test" is has to create a temporary for each one of the parameter to preserve parameter independence. In addition to that, there is another problem somewhere.  I made some minor modifications to the OP's original program.  The modified program is:Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i program _ParameterIndependence; uses  sysutils  ;  type  PString = ^String;   function StrPtr(const S: String): PString;  begin    StrPtr := @S;    writeln(IntToHex(ptruint(result), 0));  end;   procedure Test(PS1, PS2, PS3, PS4: PString);  begin    WriteLn(PS1^);    WriteLn(PS2^);  WriteLn(PS3^);    WriteLn(PS4^);  end; var  S1, S2, S3, S4, S5, S6, S7, S8: String; begin  S1 := '1111';  S2 := '2222';  S3 := '3333';  S4 := '4444';  S5 := '5555';  S6 := '6666';  S7 := '7777';  S8 := '8888';   case sizeof(pointer) of    4 : writeln('32 bit results');  8 : writeln('64 bit results');  end;   writeln;  writeln('case 1');  writeln;   writeln(S1+S2, ' ',          S3+S4, ' ',          S5+S6, ' ',          S7+S8);   writeln;  writeln('case 2');  writeln;   writeln(StrPtr(S1+S2)^, ' ',          StrPtr(S3+S4)^, ' ',          StrPtr(S5+S6)^, ' ',          StrPtr(S7+S8)^);    writeln;  writeln('case 3');  writeln;   writeln(StrPtr(S1+S2)^);  writeln(StrPtr(S3+S4)^);  writeln(StrPtr(S5+S6)^);  writeln(StrPtr(S7+S8)^);   writeln;  writeln('case 4');  writeln;   { FPC (v3.0.4 and 3.2.2) isn't handling this correctly                    }   Test(StrPtr(S1+S2), StrPtr(S3+S4), StrPtr(S5+S6), StrPtr(S7+S8));   writeln;  writeln('press ENTER/RETURN to end this program');  readln;end. Separate from the issue the OP mentioned, even disregarding the implicit violation of parameter independence FPC is making, the program above does _not_ work when compiled for 32bit. In 32bit, it doesn't even show the results of "case 4".  Attached is a screenshot of the output in 64bit and 32bit for comparison. ---------------------------------------------------------------------- Avinash · 2022-08-27 03:39 The string is not stored somewhere «inside» or «by» the StrPtr. In fact, StrPtr itself does not receive a string at all, but only a pointer to the ready string. When it is written in the function parameters (const S: String) or (var S: String) it is really (S: Pointer). The fact that the programmer refers to the string as S and not as S^ is just a service provided by Pascal syntax. So StrPtr does nothing with the any string, it simply returns the parameter passed to it. When passing parameters through registers, and such a function receives a parameter in eax and must return the result in eax, then it can be written completely empty: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i function StrPtr(const S: String): PString; assembler;asmend; This is a trick to get a pointer there when the compiler doesn't want to give it to us by normal means. We can't write somehow Move('Hello world', S, 12) or Move(@'Hello world'^, S, 12) despite the fact that string definitely has an address in memory. But we can write Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i   function StrPtr(const S: String): Pointer; assembler; register;  asm  end; var S: String;begin  S := 'Heavens';  WriteLn(S);  Move(StrPtr('HELL')^, S, 5);  WriteLn(S);end. And the point here is not that FPC uses one parameter instead of four when passing them to Test, but that it uses the same temporary string when passing it to StrPtr. In pseudocode it's like this: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i var   Temp: String;  P1, P2, P3, P4: Pointer; Temp := S1 + S2;P1 := StrPtr(Temp); // that results in P1 := @Temp;Temp := S3 + S4;P2 := StrPtr(Temp);   // P2 := @Temp;Temp := S5 + S6;P3 := StrPtr(Temp);   // P3 := @Temp;Temp := S7 + S8;P4 := StrPtr(Temp);   // P4 := @Temp; Test(P1, P2, P3, P4);  // Test(@Temp, @Temp, @Temp, @Temp); Therefore, we get the output of just the last content of the Temp string (11112222 or 77778888 depending on calling convention). ---------------------------------------------------------------------- BobDog · 2022-08-27 15:31 Hi Avenash. Your code is good on 32 bits, but seems to fail on 64 bits. The original function StrPtr and your asm StrPtr behave identically on 32 bits. The original StrPtr works on both. Testing the logic of the original StrPtr, (changing the const string in situ) Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i type  PString = ^String;   function StrPtr(const S: String): Pstring;   begin  StrPtr:=@s;  end; varvariable:string;begin  variable:='ABC';  writeln(variable);Strptr(variable)^ := 'Press return to end';  writeln(variable);  readln; end. Win 10 3.2.2 with Geany ide. ---------------------------------------------------------------------- Avinash · 2022-08-27 17:24 For 64 bits the calling convention is different, and the function must return the result in a different register than it is passed to it. This should work i think: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i {$ASMMODE INTEL}function StrPtr(const S: String): PString; assembler;asm  mov rax, rcxend; ---------------------------------------------------------------------- BobDog · 2022-08-27 19:15 That works in 64 bits. Thanks. You don't need the keyword move. string(Strptr(S)^) :='HELL'; works. ---------------------------------------------------------------------- Avinash · 2022-08-27 20:07 This does not work with FPC due to data protection, but can be done with Turbo Pascal: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i StrPtr('Heavens')^ :='HELL';WriteLn('Heavens'); ---------------------------------------------------------------------- BobDog · 2022-08-28 10:03 I get literally nothing printed to the console, a complete black empty void, which is what these two things probably are anyway. ---------------------------------------------------------------------- Avinash · 2022-08-28 15:10 Strange, everything works for me both when compiling and when running from the IDE (Borland Pascal 7.0 and Turbo Pascal 7.1), it printing «HELL». Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i type  PString = ^String;   function StrPtr(const S: String): PString;  begin    StrPtr := @S;  end; begin  StrPtr('Heavens')^ :='HELL';  WriteLn('Heavens');end. ---------------------------------------------------------------------- PascalDragon · 2022-08-28 16:48 Quote from: 440bx on August 26, 2022, 06:47:56 pmQuote from: PascalDragon on August 26, 2022, 09:17:32 amThe point here is that in total five calls are involved: four ones to the StrPtr and then the one to Test. The temporary variable is involved when concatenating the strings to pass them to StrPtr. The other temporary variables involved are those that receive the results of these StrPtr calls. As the StrPtr calls are independent of each other the compiler will reuse the temporary variable that is passes to the StrPtr functions, but of course not the temporary variables that capture the results of these calls. Again: this relies on implementation details of both TP and FPC. It is nowhere documented how this should behave and thus the compiler is free to do as it wants and here the potential for optimizations wins out. I understand all that but, the compiler is _not_ free to assume that the result of a function is always the same regardless of the parameters passed to it.  In the case of StrPtr, the function always returns the same address (address of a string) but, the compiler cannot assume that the _contents_ of the string are constant which is what it is implicitly doing. By using a single temporary and re-using it for each parameter the compiler is implicitly making the incorrect assumption that the contents of the string pointed to by the result is a constant. Since the compiler has accepted the burden of creating temporary strings to hold results, in the case of "Test" is has to create a temporary for each one of the parameter to preserve parameter independence. You still understand this incorrectly: it's not the temporaries passed to Test, but those passed to StrPtr that are reused. The compiler essentially does this: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i var  tmp1: String;  tmp2, tmp3, tmp4, tmp5: PString;begin  S1 := '1111';  S2 := '2222';  S3 := '3333';  S4 := '4444';  S5 := '5555';  S6 := '6666';  S7 := '7777';  S8 := '8888';   tmp1 := S1+S2;  tmp2 := StrPtr(tmp1);   tmp1 := S3+S4;  tmp3 := StrPtr(tmp1);   tmp1 := S5+S6;  tmp4 := StrPtr(tmp1);   tmp1 := S7+S8;  tmp5 := StrPtr(tmp1);   Test(tmp2, tmp3, tmp4, tmp5);end. And that is something that the compiler is perfectly free to do (sidenote: Delphi 10.2 behaves the same here (when using (P)ShortString instead of (P)String)). Quote from: 440bx on August 26, 2022, 06:47:56 pmSeparate from the issue the OP mentioned, even disregarding the implicit violation of parameter independence FPC is making, the program above does _not_ work when compiled for 32bit. In 32bit, it doesn't even show the results of "case 4".  Attached is a screenshot of the output in 64bit and 32bit for comparison. I get no difference for both 32- and 64-bit and both 3.0.4 and 3.2.2 on Windows. In all cases the results of “case 4” are as written in the original post. ---------------------------------------------------------------------- 440bx · 2022-08-28 18:02 Quote from: PascalDragon on August 28, 2022, 06:48:24 pmit's not the temporaries passed to Test, but those passed to StrPtr that are reused. The problem is that it is using the same temporary for each parameter.  All this said, I think that your argument is very reasonable but, I also think that it is reasonable to expect the compiler to use a different temporary for each parameter where the result of the function is stored. I accept the way FPC has chosen to do it but, I also think it would have been "nice" if it had done it the other way but, as you stated, optimizations won the day. Quote from: PascalDragon on August 28, 2022, 06:48:24 pmI get no difference for both 32- and 64-bit and both 3.0.4 and 3.2.2 on Windows. In all cases the results of “case 4” are as written in the original post. For some reason, case 4 doesn't work for me with either compiler.  One of these days I'll run it under the VS debugger (assembler view) to figure out why. ETA: Thank you for the detailed reply. ---------------------------------------------------------------------- kupferstecher · 2022-08-28 20:45 Quote from: 440bx on August 28, 2022, 08:02:04 pmI accept the way FPC has chosen to do it but, I also think it would have been "nice" if it had done it the other way but, as you stated, optimizations won the day. How I see it: The hack by the TE was done because Pascal doesn't allow to pass the address of an implicit temporary variable (e.g. after string concatenation). And this restriction is useful, as the implicit temporary immediately looses scope after it was processed. With his hack the TE tries to prolong the scope of the implicit temporary. Why should it work? QuoteI also think it would have been "nice" if it had done it the other wayHow long should the scope prevail? Over the complete statement, over the complete function? Should the compiler recognize the situation, or should it be in all situations? I don't know compiler internals, but I guess such temporary variables are used in assignments as well. An expression like Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i   S1:= S2 + S3 + S4 + S5; internally may be something like that: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i temp:= S2 + S3;temp:= temp + S4;S1:= temp + S5; Using different temporarys in this case would be strange (But as said, I don't know the internals). ---------------------------------------------------------------------- 440bx · 2022-08-28 22:06 Quote from: kupferstecher on August 28, 2022, 10:45:56 pmUsing different temporarys in this case would be strange (But as said, I don't know the internals). The internals are subject to the choices made by the compiler writer(s) and also the language definition when applicable.  The problem, in the case of strings, is that the compiler assumed the responsibility of creating whatever temporary variables are necessary.  How expressions that use those temporary variables are dealt with, is where the problem resides. ---------------------------------------------------------------------- KodeZwerg · 2022-08-28 23:05 To find out the "why" you simply should need to compare the binary debug assembly at runtime from those other compilers plus the platform you are running on with your tp version. Than all those "undocumented" or "observed" things become to coding reality ---------------------------------------------------------------------- Avinash · 2022-08-29 04:51 Let us now consider the following example: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i type  PString = ^String;   function StrPtr(const S: String): PString;  begin    StrPtr := @S;  end;   function Test(const S: String): String;  begin    Test := '@@@@@@@@';    Test := S;  end; var   S1, S2, Sx: String;  P: PString;begin  S1 := '1111';  S2 := '2222';   Sx := Test(StrPtr(S1 + S2)^);  WriteLn(Sx);  { outputs '11112222' }   P := StrPtr(S1 + S2);  Sx := Test(P^);  WriteLn(Sx);  { outputs '@@@@@@@@' }end. I looked at the assembler listing, in both cases it is absolutely identical with the exception of the fact that: in the first case the S1+S2 is placed in [EBP-512] and the Test puts the result to [EBP-256]in the second case [EBP-256] is used both timesAs you can see, the second case is just a manual repetition of what the compiler does in the first case (the generated assembler code is the same). But in the first case two variables are used, and in the second case just one. In the pseudo-code: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i Temp1 := S1 + S2;P := StrPtr(Temp1);Temp2 := Test(P^);Sx := Temp2;Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i Temp1 := S1 + S2;P := StrPtr(Temp1);   // P := @Temp1;Temp1 := Test(P^);   // Temp1 := Test(@Temp1);Sx := Temp1;           //  '@@@@@@@@' It seems that in the first case there is also no reason to use more than one variable.  Is there any intention here so that in the first case two are used? ---------------------------------------------------------------------- SymbolicFrank · 2022-08-29 07:33 Quote from: 440bx on August 29, 2022, 12:06:24 amQuote from: kupferstecher on August 28, 2022, 10:45:56 pmUsing different temporarys in this case would be strange (But as said, I don't know the internals). The internals are subject to the choices made by the compiler writer(s) and also the language definition when applicable.  The problem, in the case of strings, is that the compiler assumed the responsibility of creating whatever temporary variables are necessary.  How expressions that use those temporary variables are dealt with, is where the problem resides. Why would that be a problem? It's an external library, you shouldn't make assumptions on the internal workings in the first place. If you want pointers to the temp vars, create and assign them yourself. ---------------------------------------------------------------------- 440bx · 2022-08-29 08:25 Quote from: SymbolicFrank on August 29, 2022, 09:33:44 amWhy would that be a problem? It's an external library, you shouldn't make assumptions on the internal workings in the first place. If you want pointers to the temp vars, create and assign them yourself. What external library ? ... StrPtr is not external library.  As far as assumptions, the only "assumption" I make is that if the compiler takes on the responsibility to create temporary variables, as FPC does, then it should create as many as necessary.  IOW, it should generate code that yields a correct result. I guess that's the problem, expecting the compiler to meet the requirements it has assumed responsibility for. ---------------------------------------------------------------------- PascalDragon · 2022-08-29 12:00 Quote from: 440bx on August 28, 2022, 08:02:04 pmQuote from: PascalDragon on August 28, 2022, 06:48:24 pmI get no difference for both 32- and 64-bit and both 3.0.4 and 3.2.2 on Windows. In all cases the results of “case 4” are as written in the original post. For some reason, case 4 doesn't work for me with either compiler.  One of these days I'll run it under the VS debugger (assembler view) to figure out why. Would be interesting should you find out the reason. Quote from: Avinash on August 29, 2022, 06:51:09 amIt seems that in the first case there is also no reason to use more than one variable.  Is there any intention here so that in the first case two are used? That's simply how temporary variables are handled currently. It's nowhere said that it will stay exactly like this or whether the compiler might reuse the temporary variable in the first case as well in the future. That's why this is considered an implementation detail. Quote from: 440bx on August 29, 2022, 10:25:24 amQuote from: SymbolicFrank on August 29, 2022, 09:33:44 amWhy would that be a problem? It's an external library, you shouldn't make assumptions on the internal workings in the first place. If you want pointers to the temp vars, create and assign them yourself. What external library ? ... StrPtr is not external library.  As far as assumptions, the only "assumption" I make is that if the compiler takes on the responsibility to create temporary variables, as FPC does, then it should create as many as necessary.  IOW, it should generate code that yields a correct result. I guess that's the problem, expecting the compiler to meet the requirements it has assumed responsibility for. The problem is that a compiler can either reuse all temporaries (with some restrictions) or none, because it can't make further assumptions about the called functions. E.g. StrPtr could store the passed in parameter in a global variable and a different function could return that value again. This would only work correctly if the compiler would never reuse temporary variables. FPC's stance on this is once temporaries have been passed in to a function they are fair game again for reuse. ---------------------------------------------------------------------- marcov · 2022-08-31 19:27 Quote from: 440bx on August 28, 2022, 08:02:04 pmQuote from: PascalDragon on August 28, 2022, 06:48:24 pmit's not the temporaries passed to Test, but those passed to StrPtr that are reused. The problem is that it is using the same temporary for each parameter.  All this said, I think that your argument is very reasonable but, I also think that it is reasonable to expect the compiler to use a different temporary for each parameter where the result of the function is stored. Such assumptions are not reasonable for a HLL. ---------------------------------------------------------------------- 440bx · 2022-08-31 19:36 Quote from: marcov on August 31, 2022, 09:27:28 pmSuch assumptions are not reasonable for a HLL. It is _not_ reasonable for the compiler to assume StrPtr does not return a pointer to its parameter.  That is perfectly legal and the compiler is making that unjustified assumption when using a single temporary. I understand why the compiler has chosen to make that implicit assumption.  At this point, I think that such assumptions should be documented.  If it were documented I wouldn't be arguing that the compiler should create a temporary for every parameter. ---------------------------------------------------------------------- marcov · 2022-08-31 19:38 Quote from: 440bx on August 31, 2022, 09:36:08 pmQuote from: marcov on August 31, 2022, 09:27:28 pmSuch assumptions are not reasonable for a HLL. It is _not_ reasonable for the compiler to assume StrPtr does not return a pointer to its parameter.  That is perfectly legal and the compiler is making that unjustified assumption when using a single temporary. It is not reasonable to assume the compiler will extend the lifetime of a variable beyond that of the logical lifetime (the called procedure), just because erroneous usage is possible. PEBKAC Keep in mind that in standard pascal taking the address is not even possible, the only pointers are the ones allocated with NEW ---------------------------------------------------------------------- MarkMLl · 2022-08-31 19:56 Quote from: 440bx on August 31, 2022, 09:36:08 pmI understand why the compiler has chosen to make that implicit assumption.  At this point, I think that such assumptions should be documented.  If it were documented I wouldn't be arguing that the compiler should create a temporary for every parameter. Point of order: the compiler does not need to document that it is- or is not- making a particular assumption where this does not impact on the HLL definition. It should, however, document that details such as the allocation of storage for temporaries is an implementation detail, and that assumptions should not be made by the users. There is obviously a grey area if manually-written assembler can be expanded inline, but I don't think that anybody in this thread has provided adequate test code demonstrating that. I'd suggest that the bottom line for everybody is: provide test cases if you think the behaviour of the compiler is inconsistent with the HLL definition on one or more platforms. MarkMLl ---------------------------------------------------------------------- 440bx · 2022-08-31 20:06 Quote from: marcov on August 31, 2022, 09:38:49 pmIt is not reasonable to assume the compiler will extend the lifetime of a variable beyond that of the logical lifetime (the called procedure), just because erroneous usage is possible. PEBKAC Keep in mind that in standard pascal taking the address is not even possible, the only pointers are the ones allocated with NEW We're not talking about Standard Pascal here.  What is not reasonable at all is for the compiler to assume that in the following statement:Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i   Test(StrPtr(S1+S2), StrPtr(S3+S4), StrPtr(S5+S6), StrPtr(S7+S8));the result of (S1+S2) is the same as the result of (S3+S4) is the same as the result of (S5+S6), etc.  Which is what the compiler is implicitly _assuming_, contrary to what is logically expected, when it uses the same temporary for each function call result. Effectively making the result the same for all expressions no matter how different they may be. If the compiler is going to give itself the liberty of making that assumption then, it should be documented. ---------------------------------------------------------------------- marcov · 2022-08-31 20:08 Quote from: MarkMLl on August 31, 2022, 09:56:18 pmQuote from: 440bx on August 31, 2022, 09:36:08 pmI understand why the compiler has chosen to make that implicit assumption.  At this point, I think that such assumptions should be documented.  If it were documented I wouldn't be arguing that the compiler should create a temporary for every parameter. Point of order: the compiler does not need to document that it is- or is not- making a particular assumption where this does not impact on the HLL definition. It should, however, document that details such as the allocation of storage for temporaries is an implementation detail, and that assumptions should not be made by the users. Such choices might vary with target and are not set in stone forever as the code generator and optimizer mutates and improves. Documenting means effectively expanding the language, which is not intended in such cases. ---------------------------------------------------------------------- marcov · 2022-08-31 20:10 Quote from: 440bx on August 31, 2022, 10:06:14 pmQuote from: marcov on August 31, 2022, 09:38:49 pmIt is not reasonable to assume the compiler will extend the lifetime of a variable beyond that of the logical lifetime (the called procedure), just because erroneous usage is possible. PEBKAC Keep in mind that in standard pascal taking the address is not even possible, the only pointers are the ones allocated with NEW We're not talking about Standard Pascal here.  What is not reasonable at all is for the compiler to assume that in the following statement You are putting assumptions on lifetime details of variables that are compiler managed, and do so with low level tricks like taking the address of an allocation that you don't manually manage. That is what is not reasonable. ---------------------------------------------------------------------- 440bx · 2022-08-31 20:59 Quote from: marcov on August 31, 2022, 10:10:22 pmlow level tricks like taking the address of an allocation that you don't manually manage. That is what is not reasonable. There is nothing unreasonable about taking the address of a string.  Since the compiler assumed the responsibility of creating temporary variables, it is also responsible for managing them without making unsupported assumptions about their usage, _unless_ it documents any limitations arising from the way it manages them. ---------------------------------------------------------------------- marcov · 2022-09-01 10:32 Quote from: 440bx on August 31, 2022, 10:59:20 pmQuote from: marcov on August 31, 2022, 10:10:22 pmlow level tricks like taking the address of an allocation that you don't manually manage. That is what is not reasonable. There is nothing unreasonable about taking the address of a string.  It is low level usage. And then you need to play by the rules, not using the pointer outside the string's formal scope. QuoteSince the compiler assumed the responsibility of creating temporary variables, it is also responsible for managing them without making unsupported assumptions about their usage, _unless_ it documents any limitations arising from the way it manages them. If you assign that much care/responsibility to the compiler, then the procedural boundary for the temp var is also arbitrary.  You could pass the pointer to some global datastructure, and so the compiler could never ever deallocate the string. ---------------------------------------------------------------------- dseligo · 2022-09-01 10:51 Quote from: marcov on September 01, 2022, 12:32:17 pmQuote from: 440bx on August 31, 2022, 10:59:20 pmThere is nothing unreasonable about taking the address of a string.  It is low level usage. And then you need to play by the rules, not using the pointer outside the string's formal scope. Exactly. If one want pointer for temporary string, he can put string in a variable and use address of that variable. ---------------------------------------------------------------------- Avinash · 2022-09-01 12:10 It can be called a question of «philosophy». When the user looks at some complex expressions, he does not understand well how the compiler does it all. But a common understanding is formed that everything he sees inside such an expression somehow exists until the whole expression is calculated. Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i WriteLn(Upc ase(S1+S2));WriteLn(Upcase(S3+S4));WriteLn(Upcase(S5+S6)); WriteLn(Upc ase(S1+S2) + Upcase(S3+S4) + Upcase(S5+S6)); And so it becomes completely unexpected when such things just as well exist separately, but suddenly cease to exist when they get into expression: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i WriteLn(Str Ptr(S1+S2)^);WriteLn(StrPtr(S3+S4)^);WriteLn(StrPtr(S5+S6)^); WriteLn( StrPtr(S1+S2)^ + StrPtr(S3+S4)^ + StrPtr(S5+S6)^); ---------------------------------------------------------------------- marcov · 2022-09-01 12:18 Quote from: Avinash on September 01, 2022, 02:10:13 pmIt can be called a question of «philosophy». It is hard to form a philosophy if your code is a mix  from one (automated memory allocation) domain into another (manually manipulating pointers). As I said earlier, I don't think you can come up with a reasonable solution for this within the current language. QuoteWhen the user looks at some complex expressions, he does not understand well how the compiler does it all. It is not reasonable to believe that the compiler should prevent you doing something dangerous when using very low level features with which you basically declare that you want to work outside the compiler's control. Also note that the proposed enhancement of the temp scope is a hackish solution to keep some faulty code running, not a grand new philosophy that blends HLL and low level features seamlessly. For a new language a better route would be not to allow to pass temp strings to pchar routines, but here and now that hursts backwards compatibility too much (and also would trigger a discussion about language philosophy, hold your hands language vs a mixed paradigm production language) ---------------------------------------------------------------------- Avinash · 2022-09-01 13:12 Okay, let's forget about StrPtr. I'm starting to understand what 440bx is talking about: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i type  PString = ^String; var  I: Integer;  S: String;   function Test: PString;  begin    Test := @S;    Inc(I);    case I of      1: S := '1111';      2: S := '2222';      3: S := '3333';      4: S := '4444';    end;  end; begin  WriteLn(Test^ + Test^ + Test^ + Test^);end. Is the 4444444444444444 here the correct result? Turbo Pascal outputs 1111222233334444 ---------------------------------------------------------------------- marcov · 2022-09-01 13:37 I'm a bit in dubio. I think the code is bad, and has a hidden assumption that the partial string is written out before the next function call (test) is done. FPC however first collects results and then concatenates to a string and writes them out. I don't see anything wrong with that, which illustrates perfectly that trying to fixate such behaviour you make optimizations harder. ---------------------------------------------------------------------- Avinash · 2022-09-01 14:02 But a dereferenced pointer is a value, isn't it? I specify the parentheses, i command to get the string first, why shouldn't this work? Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i WriteLn(String(Test^) + String(Test^) + String(Test^) + String(Test^)); These two lines give different results, it looks illogical: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i WriteLn(Concat(Test^) + Concat(Test^) + Concat(Test^) + Concat(Test^));   { '4444444444444444' }Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i WriteLn(Upcase(Test^) + Upcase(Test^) + Upcase(Test^) + Upcase(Test^));   { '1111222233334444' } ---------------------------------------------------------------------- marcov · 2022-09-01 14:05 Quote from: Avinash on September 01, 2022, 04:02:44 pmBut a dereferenced pointer is a value, isn't it? No, the pointer itself is the value. The compiler doesn't track what it points too, as soon as you start messing with pointers that is your job. Code: [Select]I specify the parentheses, i [i]command[/i] to get the string first, why shouldn't this work? [code=pascal]WriteLn(String(Test^) + String(Test^) + String(Test^) + String(Test^)); test^ is already string, so that cast is a no-op? ---------------------------------------------------------------------- 440bx · 2022-09-01 15:34 Quote from: marcov on September 01, 2022, 12:32:17 pmIf you assign that much care/responsibility to the compiler, then the procedural boundary for the temp var is also arbitrary.  You could pass the pointer to some global datastructure, and so the compiler could never ever deallocate the string. I don't assign responsibility to the compiler, the compiler (its writers) _chose_ to assume that responsibility, particularly when they claim(ed) that FPC is TP compatible. Any area where FPC is _not_ compatible with TP should be documented.  The reason is very simple, because a programmer that buys TP's claim of compatibility should be made aware of the exceptions that break the compatibility and, the instances where that compatibility isn't met be documented. I don't assign responsibility, I simply ask that the responsibilities that have been willingly and explicitly assumed be met. ---------------------------------------------------------------------- marcov · 2022-09-01 16:25 Quote from: 440bx on September 01, 2022, 05:34:39 pmQuote from: marcov on September 01, 2022, 12:32:17 pmIf you assign that much care/responsibility to the compiler, then the procedural boundary for the temp var is also arbitrary.  You could pass the pointer to some global datastructure, and so the compiler could never ever deallocate the string. I don't assign responsibility to the compiler, the compiler (its writers) _chose_ to assume that responsibility, particularly when they claim(ed) that FPC is TP compatible. Yes. Language compatible. Not undocumented implementation detail, since that is hopeless due to 16-bit vs 32-bit difference anyway. QuoteAny area where FPC is _not_ compatible with TP should be documented.  If TP doesn't document it, I don't see why we should.  We state to be a compatible compiler, not an emulator. ---------------------------------------------------------------------- 440bx · 2022-09-01 16:52 Quote from: marcov on September 01, 2022, 06:25:46 pmYes. Language compatible. Not undocumented implementation detail, since that is hopeless due to 16-bit vs 32-bit difference anyway. If TP doesn't document it, I don't see why we should.  We state to be a compatible compiler, not an emulator. "language compatible" ?? nice nugget of creativity there!.  I might be wrong but, I believe that when someone reads that one thing is compatible with something else, it is because the _results_ are going to be the same with either one of the "items" involved.  Compatible doesn't mean, item A resembles item B, it means it works the same way. Whether some behavior is documented or not is irrelevant.  When item B claims compatibility with Item A then Item B has assumed the responsibility to behave as closely as possible as Item A.  If Item B does not behave the same as Item A then Item B either corrects the behavior to make  it the same or documents the difference/discrepancy. ETA: @Avinash Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i type  PString = ^String; var  I: Integer;  S: String;   function Test: PString;  begin    Test := @S;    Inc(I);    case I of      1: S := '1111';      2: S := '2222';      3: S := '3333';      4: S := '4444';    end;  end; begin  WriteLn(Test^ + Test^ + Test^ + Test^);end.Excellent example!. ---------------------------------------------------------------------- marcov · 2022-09-01 18:38 Quote from: 440bx on September 01, 2022, 06:52:57 pm"language compatible" ?? nice nugget of creativity there!. Not really. The documented part of the language is documented in the "Language guide" tome of the BP manuals. Similarly for Delphi. QuoteI might be wrong but, I believe that when someone reads that one thing is compatible with something else, it is because the _results_ are going to be the same with either one of the "items" involved.  Compatible doesn't mean, item A resembles item B, it means it works the same way. The results with proper use. Duplication of improper use is a different thing. If TP doesn't document it, it isn't guaranteed not to change between versions either. QuoteWhether some behavior is documented or not is irrelevant.  When item B claims compatibility with Item A then Item B has assumed the responsibility to behave as closely as possible as Item A.  If Item B does not behave the same as Item A then Item B either corrects the behavior to make  it the same or documents the difference/discrepancy. As said that ship had already sailed before we even started due to TP's liberal use of 16-bit assembler everywhere. ---------------------------------------------------------------------- 440bx · 2022-09-01 18:46 Quote from: marcov on September 01, 2022, 08:38:40 pmThe results with proper use. Duplication of improper use is a different thing. If TP doesn't document it, it isn't guaranteed not to change between versions either.There is nothing "improper" in taking and returning the address of a string. Quote from: marcov on September 01, 2022, 08:38:40 pmAs said that ship had already sailed before we even started due to TP's liberal use of 16-bit assembler everywhere. I don't think that ship has sailed, I think that ship sunk.  FPC claims compatibility with TP.  Because of that, it should document the cases where it does not meet its claim. A section in the FPC manuals where all incompatibilities are documented would be very useful to programmers porting code from TP and/or Delphi to FPC. ---------------------------------------------------------------------- marcov · 2022-09-01 19:31 Quote from: 440bx on September 01, 2022, 08:46:05 pmThere is nothing "improper" in taking and returning the address of a string. There is in accessing it outside of the scope of the variables that you point it too. I don't know by heart about TP, but Delphi clearly documents that the lifetime of temps is for till after the statement where they are needed. Quote from: 440bx on September 01, 2022, 08:46:05 pmQuote from: marcov on September 01, 2022, 08:38:40 pmAs said that ship had already sailed before we even started due to TP's liberal use of 16-bit assembler everywhere. I don't think that ship has sailed, I think that ship sunk.  Empty phrases all derived from two words in the compiler description. But the FPC position has been clear for 25 years now. QuoteFPC claims compatibility with TP.  Because of that, it should document the cases where it does not meet its claim. A section in the FPC manuals where all incompatibilities are documented would be very useful to programmers porting code from TP and/or Delphi to FPC. Beside that it will never be a blind recompile due to 16/32-bit and OS differences,  the not having to characterize undocumented compiler behaviour is a big reason for how it is actually. It is particularly hard to distinguish between behaviour that is usually the case, and behaviour that is always the case. Constant WIP. And that includes version differences on TPs side. Did you test all TP compiler modes?  Windows? Protected mode? TP6 vs TP7? Updates and upgrades? But anyway, if you feel really strong about this, the wiki is the place for you. ---------------------------------------------------------------------- 440bx · 2022-09-01 21:08 Quote from: marcov on September 01, 2022, 09:31:42 pmThere is in accessing it outside of the scope of the variables that you point it too. In the last example given, the function is returning pointers to constants, therefore they are not out of scope. Quote from: marcov on September 01, 2022, 09:31:42 pm I don't know by heart about TP, but Delphi clearly documents that the lifetime of temps is for till after the statement where they are needed. because the OP mentioned TP, not Delphi. Quote from: marcov on September 01, 2022, 09:31:42 pmEmpty phrases all derived from two words in the compiler description. You're the one who brought ships into the discussion, I just observed that they didn't sail, they sunk instead. Quote from: marcov on September 01, 2022, 09:31:42 pmBut the FPC position has been clear for 25 years now. What position is that ?... the one that says, differences/incompatibilities won't be documented ? ... is that the one ? Quote from: marcov on September 01, 2022, 09:31:42 pmConstant WIP. Constant WIP is right.... most compilers (those that are not dead) are.  That comes with the territory, it cannot be used as a reason to keep incompatibilities undocumented. Quote from: marcov on September 01, 2022, 09:31:42 pmAnd that includes version differences on TPs side. Did you test all TP compiler modes?  Windows? Protected mode? TP6 vs TP7? Updates and upgrades? since you don't seem to be willing to even document the incompatibility, what could possibly be the point of doing all that testing ?... what's important is that, it is known that under some circumstances (which the more accurately defined and, ideally documented, the better) the behavior of the "TP compatible compiler" (i.e, FPC) does not match the behavior of what it claims to be compatible with. Quote from: marcov on September 01, 2022, 09:31:42 pmBut anyway, if you feel really strong about this, the wiki is the place for you. I don't use Pascal strings and, I don't use TP.  Philosophically, incompatibilities/discrepancies should be documented for the benefit of programmers who rely on FPC.  Some of them are even documented in the FPC manuals... imagine that!. ---------------------------------------------------------------------- marcov · 2022-09-01 21:40 Quote from: 440bx on September 01, 2022, 11:08:58 pmIn the last example given, the function is returning pointers to constants, therefore they are not out of scope. That one is different and was confusing for me too initially. But it depends on writeln() evaluation details. Show me where it is documented that each parameter of writeln is wholly evaluated before the next in TP. QuoteQuote from: marcov on September 01, 2022, 09:31:42 pm I don't know by heart about TP, but Delphi clearly documents that the lifetime of temps is for till after the statement where they are needed. because the OP mentioned TP, not Delphi. Show me the TP docs that say otherwise then. I have a BP7 manual set right here, just name the volume and the page. QuoteQuote from: marcov on September 01, 2022, 09:31:42 pmEmpty phrases all derived from two words in the compiler description. You're the one who brought ships into the discussion, I just observed that they didn't sail, they sunk instead. I used a common expression. Nothing weird about that. QuoteQuote from: marcov on September 01, 2022, 09:31:42 pmBut the FPC position has been clear for 25 years now. What position is that ?... the one that says, differences/incompatibilities won't be documented ? ... is that the one ? That the compatibility means compatibility to the documented behaviour. QuoteQuote from: marcov on September 01, 2022, 09:31:42 pmConstant WIP. Constant WIP is right.... most compilers (those that are not dead) are.  That comes with the territory, it cannot be used as a reason to keep incompatibilities undocumented. This is also not known for dead compilers like TP, specially its version matrix. Discovering that would also be a WIP as new details would be found and would require changes. I'm not using anything as a reason, only guessing at the reasons. It was decided long ago not to go there. Period. You can file a bug to get the documented policy more formally stated in the docs if you like. It will also get you more developer's opinion, and not just my recollection on policy. QuoteQuote from: marcov on September 01, 2022, 09:31:42 pmAnd that includes version differences on TPs side. Did you test all TP compiler modes?  Windows? Protected mode? TP6 vs TP7? Updates and upgrades? since you don't seem to be willing to even document the incompatibility, what could possibly be the point of doing all that testing ? By that logic, it not being there in the almost 20 year old wiki, suggests nobody cares. Quote... what's important is that, it is known that under some circumstances (which the more accurately defined and, ideally documented, the better) the behavior of the "TP compatible compiler" (i.e, FPC) does not match the behavior of what it claims to be compatible with. ... according to your (extreme) definition. In case of doubt people embarking on a big project could of course simply ask, since they might not find and/or correctly interpret a politically phrased footnote in the docs anyway. QuoteQuote from: marcov on September 01, 2022, 09:31:42 pmBut anyway, if you feel really strong about this, the wiki is the place for you. I don't use Pascal strings and, I don't use TP.  Philosophically, incompatibilities/discrepancies should be documented for the benefit of programmers who rely on FPC.  Some of them are even documented in the FPC manuals... imagine that!. [/quote] I'm not just talking about TP, but Delphi too. ---------------------------------------------------------------------- SymbolicFrank · 2022-09-13 09:48 The problem with documenting the current implementation details is, that as soon as you do so, you can never change it due to backwards compatibility. At that point, the only ways forward are either to stop offering that backwards compatibility and restricting all discussions to that specific version of the compiler, or implementing a whole new feature set for improvements. I don't think many people would want that. ---------------------------------------------------------------------- MarkMLl · 2022-09-13 10:03 Quote from: SymbolicFrank on September 13, 2022, 11:48:08 amThe problem with documenting the current implementation details is, that as soon as you do so, you can never change it due to backwards compatibility. At that point, the only ways forward are either to stop offering that backwards compatibility and restricting all discussions to that specific version of the compiler, or implementing a whole new feature set for improvements. I don't think many people would want that. I'd like to offer a qualified disagreement. There is nothing wrong with documenting current implementation details, provided that (a) there is a robust understanding of what "current" actually means and (b) the author uses sufficiently robust language to convince even the densest reader that relying on the information is profoundly ill- advised. Quote"All of the true things that I am about to tell you are shameless lies." MarkMLl ---------------------------------------------------------------------- 440bx · 2022-09-13 10:08 Quote from: SymbolicFrank on September 13, 2022, 11:48:08 amThe problem with documenting the current implementation details is, that as soon as you do so, you can never change it due to backwards compatibility. No. What should be documented aren't "implementation details", what should be documented are the _differences_ and, if those differences are removed then, it should be documented that the difference has been removed.  It _might_ be reasonable to gloss over the reasons for the differences or leave them out altogether but, the facts (differences) those should be clearly documented. This is nothing unreasonable.  Through FPC's history there have been a number of times when a newer version of the compiler would behave differently than the older version and those differences were documented which is the right thing to do. When a product claims compatibility then it should document the incompatibilities.  For the record, compatibility is more than "syntactic compatibility", it's also functional compatibility, otherwise a programmer could write just a TP parser that doesn't even generate code and claim "compatibility" with it (that would certainly be very "useful".) @MarkMLI posted while I was composing this reply, I agree with his position. ---------------------------------------------------------------------- SymbolicFrank · 2022-09-13 10:54 If you are going to document it, people are going to use it. And as soon as it changes (like it did with newer versions of TP), people will complain that their application stopped working, proclaim the compiler to be broken and demand the old behavior back. No matter how often you tell them: "Here be Dragons! Use at your own risk!". Just like it was done in the first post, to "speed things up". It worked then, so it should keep on working the same way. QED. We even see that here with things like recompilations of 16-bit applications that contain assembly language. So... ---------------------------------------------------------------------- 440bx · 2022-09-13 11:00 Quote from: SymbolicFrank on September 13, 2022, 12:54:22 pmIf you are going to document it, people are going to use it. People are going to use the documented incompatibilities ?... that doesn't make sense... most programmers don't purposely shoot themselves in the foot by creating code that is documented to behave differently between the compatible products. ---------------------------------------------------------------------- PascalDragon · 2022-09-13 11:35 Quote from: 440bx on September 13, 2022, 12:08:58 pmWhen a product claims compatibility then it should document the incompatibilities.  For the record, compatibility is more than "syntactic compatibility", it's also functional compatibility, otherwise a programmer could write just a TP parser that doesn't even generate code and claim "compatibility" with it (that would certainly be very "useful".) First of one would need to know such differences. Up until this thread it wasn't known that TP behaved in any way different there. Second we'll only document differences to documented behaviour. Since it isn't documented how TP behaves in that case: though luck, we won't document that difference. (Though we will probably document that FPC handles parameters independently, because it provides better optimization opportunities) ---------------------------------------------------------------------- MarkMLl · 2022-09-13 11:47 Quote from: 440bx on September 13, 2022, 01:00:00 pmPeople are going to use the documented incompatibilities ?... that doesn't make sense... most programmers don't purposely shoot themselves in the foot by creating code that is documented to behave differently between the compatible products. Some take pride in it. In part it's one-upmanship, in part it's because they have the hubris to believe that they have the perception to know what can reasonably be expected to not change, and in part it's because there's a certain type of PHB that rewards any performance improvement without an understanding of the risks being taken. As an engineer of sorts, I'm a firm believer in keeping an adequate minute-by-minute log of design decisions etc. The real problem is cross-referencing and ideally reconciling a mixture of memos, personal notes, sourcecode comments and possibly a wiki to deduce a plausible position. MarkMLl ---------------------------------------------------------------------- 440bx · 2022-09-13 12:59 Quote from: PascalDragon on September 13, 2022, 01:35:16 pmFirst of one would need to know such differences. Up until this thread it wasn't known that TP behaved in any way different there. I have no problem with that. When the difference isn't known, it obviously cannot be documented. Quote from: PascalDragon on September 13, 2022, 01:35:16 pmSecond we'll only document differences to documented behaviour. Since it isn't documented how TP behaves in that case: though luck, we won't document that difference. Now that is a very unfortunate choice because it means that a now known incompatibility isn't documented, leaving the door open for programmers who are porting code from TP to FPC to be bitten by a known but, purposely left undocumented, incompatibilty.  The "documented behavior" "standard" is a rather poor one.  A program, compiler or other, is rather unlikely to be 100% documented and,  when a product claims compatibility with something else, it has taken upon itself the burden to be compatible (per its claim.)  When the product fails to meet its claim, it should document that failure regardless of its cause so that programmers that run into that incompatibility don't waste their time each having to figure out that FPC isn't compatible with TP in some particular case. Simply stated: documenting incompatibilities is a good faith effort not to cause individual programmers to waste their time figuring out what is already known to some. It is just plain courtesy to document the incompatibilities.  It's not only common sense, it's common courtesy too. ---------------------------------------------------------------------- MarkMLl · 2022-09-13 13:12 Quote from: 440bx on September 13, 2022, 02:59:04 pmSimply stated: documenting incompatibilities is a good faith effort not to cause individual programmers to waste their time figuring out what is already known to some. It is just plain courtesy to document the incompatibilities.  It's not only common sense, it's common courtesy too. Furthermore, we already know from this forum how often programmers- even "old hands" who should know better- will consult dodgy blogs and arbitrary Facebook groups and then claim that those are (or are as good as) official documentation. If there is neither documentation nor a "this is intentionally undocumented" notice for a topic then it is difficult to counter that argument. In addition there is the matter of ensuring that development is moderately open, so that if somebody genuinely wishes to help there is adequate material to explain how the major components of the FPC compiler and associated subprojects hang together. MarkMLl