Unneeded copy of string
lazarus / 21-General

----------------------------------------------------------------------
LemonParty · 2025-04-15 16:23

Hello.

I have this simple code:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i {$mode
objfpc}{$H+} procedure P1(constref Ch: AnsiChar);begin 
Writeln(Ch);end; var  S: AnsiString = '12345'; begin 
P1(S[1]);end. When I disassebled this code I found this:
Code: ASM  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i # [32]
P1(S[1]);  leaq  TC_$P$PROGRAM_$$_S(%rip),%rcx  call 
fpc_ansistr_unique  movq  %rax,%rcx  call  P$PROGRAM_$$_P1$CHAR The
question is why compiler make copy of a string?

----------------------------------------------------------------------
PascalDragon · 2025-04-15 21:06

Quote from: LemonParty on April 15, 2025, 06:23:14 pmThe question is
why compiler make copy of a string?

constref enforces the passing of Ch as a reference aka a Pointer. To
ensure that another potential reference to the string doesn't
interfere with that passed in reference the compiler ensures a unique
copy of the string. For a primitive argument type const is better. In
general you should prefer to use const because the compiler will pick
the most efficient way to pass the parameter - unless you really need
a reference to the passed in value.

----------------------------------------------------------------------
LemonParty · 2025-04-16 08:56

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i {$mode
objfpc}{$H+} procedure P1(constref Ch: AnsiChar);begin  Writeln('P1
Char');end; procedure P1(constref Ch: WideChar);begin  Writeln('P1
Wide');end; procedure P2(P: PAnsiChar);begin  Writeln('P2
Char');end; procedure P2(P: PWideChar);begin  Writeln('P2
Wide');end; var  S: AnsiString = '12345';  W: WideString =
'12345'; begin  P1(S[1]);  P1(W[1]);  P2(@S[1]);  P2(@W[1]); 
Writeln('1');end. This code demonstrate a problem when use something
else than constref. It would be nice if compiler have an option to
turn off such hidden copying.

----------------------------------------------------------------------
Thaddy · 2025-04-16 09:14

well, then make it const as PascalDragon suggested.

----------------------------------------------------------------------
LemonParty · 2025-04-16 09:22

I need a reference.

----------------------------------------------------------------------
Thaddy · 2025-04-16 09:45

PascalDragon explained that in that case you can't avoid a copy.
That is related to the type being a managed type.

----------------------------------------------------------------------
H₂SO₄ · 2025-04-16 09:46

Quote from: LemonParty on April 16, 2025, 10:56:16 amCode: Pascal
 [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i {$mode
objfpc}{$H+} procedure P1(constref Ch: AnsiChar);begin  Writeln('P1
Char');end; procedure P1(constref Ch: WideChar);begin  Writeln('P1
Wide');end; procedure P2(P: PAnsiChar);begin  Writeln('P2
Char');end; procedure P2(P: PWideChar);begin  Writeln('P2
Wide');end; var  S: AnsiString = '12345';  W: WideString =
'12345'; begin  P1(S[1]);  P1(W[1]);  P2(@S[1]);  P2(@W[1]); 
Writeln('1');end. This code demonstrate a problem when use something
else than constref. It would be nice if compiler have an option to
turn off such hidden copying.

This can be solved by using  {$TYPEDADDRESS ON}  aka  {$T+}

----------------------------------------------------------------------
Thaddy · 2025-04-16 09:53

Did you check the assembler output?

----------------------------------------------------------------------
H₂SO₄ · 2025-04-16 11:15

Quote from: Thaddy on April 16, 2025, 11:53:21 amDid you check the
assembler output?

I assume the problem @LemonParty was referring to is the compiler
selecting the "wrong" overload in line 31 (PWideChar  is supplied, but
under  {$T-}  the compiler chooses the  PAnsiChar  version). It's not
about string copying.

The output should be  P1 Char, P1 Wide, P2 Char, P2 Wide  but without 
{$T+}  it's actually  P1 Char, P1 Wide, P2 Char, P2 Char.
With constref the @ operator could be avoided, preventing loss of type
information in that case and allowing the compiler to select the
correct overload.

@LemonParty is this the actual problem you were trying to solve? I've
answered a similiar question about overloads before (that you posted).

----------------------------------------------------------------------
Thaddy · 2025-04-16 11:48

In this case we need the assembler output.
Will check when I am home.

----------------------------------------------------------------------
LemonParty · 2025-04-16 13:19

QuoteThis can be solved by using  {$TYPEDADDRESS ON}  aka  {$T+}Is
this going to work when I place procedures in separate unit or whole
project should be compiled with this switch?

----------------------------------------------------------------------
beria2 · 2025-04-16 13:33

Quote from: LemonParty on April 16, 2025, 11:22:45 amI need a
reference.
it is the link that is needed - use only the Pointer type. I always do
that. And you always know exactly what you have.

----------------------------------------------------------------------
ASerge · 2025-04-16 14:03

Quote from: LemonParty on April 16, 2025, 03:19:50 pmQuoteThis can be
solved by using  {$TYPEDADDRESS ON}  aka  {$T+}Is this going to work
when I place procedures in separate unit or whole project should be
compiled with this switch?
A separate unit is enough.
Interestingly, the discussion turned to another issue, not the one
stated in the topic.

----------------------------------------------------------------------
H₂SO₄ · 2025-04-16 14:24

Quote from: LemonParty on April 16, 2025, 03:19:50 pmQuoteThis can be
solved by using  {$TYPEDADDRESS ON}  aka  {$T+}Is this going to work
when I place procedures in separate unit or whole project should be
compiled with this switch?

It's a local directive, meaning that it can be toggled at the token
level.
You could even enable it just for a single part of an expression if
you really wanted to:

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i
P2({$push}{$T+} @W[1] {$pop});
Quote from: ASerge on April 16, 2025, 04:03:38 pmInterestingly, the
discussion turned to another issue, not the one stated in the topic.

Turns out it was an XY problem after all!

----------------------------------------------------------------------
LemonParty · 2025-04-16 15:25

I have built a test program
u1.pas:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i {$mode
objfpc}{$H+} {$T+} unit u1; interface procedure P2(P:
PAnsiChar);procedure P2(P: PWideChar); implementation procedure P2(P:
PAnsiChar);begin  Writeln('P2 Char');end; procedure P2(P:
PWideChar);begin  Writeln('P2 Wide');end; end.
prog.pas:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i {$mode
objfpc}{$H+} uses u1; var  S: AnsiString = '12345';  W: WideString =
'12345'; begin  {$Push}{$T+}  P2(@S[1]);  P2(@W[1]);  {$Pop} 
{$Push}{$T-}  P2(@S[1]);  P2(@W[1]);  {$Pop}end.Output is:
QuoteP2 Char
P2 Wide
P2 Char
P2 Char
This mean all project should be built with {T+} option. This also mean
we can't use overloaded functions in units that rely on strings.
I think behavior of compiler around constref should be changed. As I
suppouse idea of constref is that we reference something, but not
change it. So compiler should not do copy of a string on constref.

----------------------------------------------------------------------
ASerge · 2025-04-17 01:55

Quote from: LemonParty on April 16, 2025, 05:25:49 pmThis mean all
project should be built with {T+} option. This also mean we can't use
overloaded functions in units that rely on strings.
I think behavior of compiler around constref should be changed. As I
suppouse idea of constref is that we reference something, but not
change it. So compiler should not do copy of a string on constref.
You've mixed two different things.
1. The issue from your example, when the compiler does not warn that
there is a conflict when choosing from two overloaded functions, but
chooses the first one that comes to hand. According to the
documentation, the @ operator for a variable has the Pointer type in
the {$T-} state, which means both functions are suitable.
2. Using the UniqueString procedure before calling a function that
uses part of a string. This is not a issue. If you want to avoid
unnecessary calls, cast it to the PChar type:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i
P1(PChar(Pointer(S))[0]);Code: ASM  [Select][+][-]window.onload =
function(){var x1 = document.getElementById("main_content_section");
if (x1) { var x = document.getElementsByClassName("geshi");for (var i
= 0; i # [31] P1(PChar(Pointer(S))[0]);        movq   
TC_$P$PROGRAM_$$_S(%rip),%rcx        call    P$PROGRAM_$$_P1$CHAR In
both cases, constref has nothing to do with it.

----------------------------------------------------------------------
LemonParty · 2025-04-17 06:13

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i
P1(PChar(Pointer(S))[0]);This code is ugly.
Can anyone show an example when copying a string on constref is a
necessery move?

----------------------------------------------------------------------
H₂SO₄ · 2025-04-17 07:17

Putting aside the discussion whether  constref  should copy the
underlying string, what exactly are you trying to do?

Are you writing functions that utilize character pointers as C
strings? What are these functions supposed to do?

From what I've gathered, you want overloads for both ansi and wide
strings, but without either  {$T+}  or  constref  the compiler can't
choose the correct overload, so you insisted on abusing  constref 
just to help the compiler discern types, while the actual point of 
constref  is to guarantee immutable pass-by-reference.

Coming back to the  constref  discussion, I think it's reasonable for
the compiler to ensure that references are valid. For this purpose FPC
has  var,  out  and  constref  while C++ has its own kind of non-
nullable references (e.g.  const char&). If you don't need such
guarantees, just use a plain pointer instead.

----------------------------------------------------------------------
Martin_fr · 2025-04-17 07:58

Quote from: LemonParty on April 17, 2025, 08:13:11 amCan anyone show
an example when copying a string on constref is a necessery move?

It all depends on the level of optimization, and other settings. Maybe
even compiler version.
To demonstrate the effect of the missing "uniquestring", I used the
PAnsiChar version.

The below passes a pointer to 'a', but prints '5' (which is something
random that went in place of the released string).

If you comment the first line, and uncomment the 2nd, then it prints
's', the first char of the 2nd string.

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i Program
foo; {$mode objfpc}{$H+}uses SysUtils; var  S, S2:
AnsiString; //procedure P1(constref Ch: AnsiChar);procedure P1( Ch:
PAnsiChar);begin  s := '';  s2 := 'something else' +
IntToStr(random(9)); // avoid compiletime const eval  //s :=
'something else' + IntToStr(random(9)); // avoid compiletime const
eval  Writeln(Ch^);end; begin  s := 'abcdef' + inttostr(random(9)); //
avoid compiletime const eval  P1(@S[1]);   readln;end. 

In real life, I have seen stuff like this when using "const s:
ansistring"
- because that is a much more common construct
- because that does not do the uniquesting (in fact it is used because
it does not)

And last time I saw it, the string was a field in an object. It wasn't
directly modified by the called procedure, but that procedure had
callbacks (events that it called, and about 10 calls deep into the
stack on the callback was some code that modified the field.


@PascalDragon: I only partly agree with your statement.

1) UniqueString is overkill. (for constref)

It is not only a reference. It is also const.

It does only need an Increment-refcount. If anyone else writes to it,
and the refcount is greater 1 then the other will make a copy of their
own.
Sure you can bypass that with pchar magic. But then you intentionally
break protection.

The general idea is, that having any code that holds a ref to a
string, can trust it. If the caller in this case holds that ref, then
the callee is fine by those means.

( That differs for a "var c: ansichar" param (or "out"), because then
"c" itself can be changed, and at that point it must act on a uniquely
referenced copy of the string)

2) It is inconsistent....
While it is nice that there is safety first...

constref is a form of const.

const does not add that form of protection.

In fact it has been stated countless times, that const is a contract
where the user (programmer) tells the compiler, that the variable will
not change.
And "will not change" includes, that it will not be changed by any
code while the called code is running (has not returned to its
caller).

With const too, as I stated other code may change (breaking the
contract) a string that was passed without protection. And that is by
design.

Why is that design not applied for constref?
I may have missed something, but I though constref is the same
optimization as const? Except for adding a ref?

EDIT /APPEND

And then, if the string is protected because "constref" takes a
pointer (reference) to a char in the string, then why is it not
protected if
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i @s[1]takes
the same pointer? (only without read-only protection...)

----------------------------------------------------------------------
LemonParty · 2025-04-17 08:24

QuotePutting aside the discussion whether  constref  should copy the
underlying string, what exactly are you trying to do?
I am writing a library. And I want to have overload of a function that
can handle both AnsiChar and WideChar buffers. They look like this:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i function
Pos(constref Buf: AnsiChar; C: AnsiChar; Range: SizeUInt):
SizeUInt;function Pos(constref Buf: WideChar; C: WideChar; Range:
SizeUInt): SizeUInt; Buf in this case is not a single character, but a
pointer to the buffer. This works fine, until we get to strings.

----------------------------------------------------------------------
LemonParty · 2025-04-17 08:35

QuoteAnd then, if the string is protected because "constref" takes a
pointer (reference) to a char in the string, then why is it not
protected ifThis express my point very well. It is illogical that
constref protect when taking a pointer, but taking a pointer not
protect.

----------------------------------------------------------------------
Thaddy · 2025-04-17 10:12

Quote from: Martin_fr on April 17, 2025, 09:58:57 amIt all depends on
the level of optimization, and other settings. Maybe even compiler
version.
Optimization should never affect result.

----------------------------------------------------------------------
Martin_fr · 2025-04-17 10:45

Quote from: Thaddy on April 17, 2025, 12:12:08 pmQuote from: Martin_fr
on April 17, 2025, 09:58:57 amIt all depends on the level of
optimization, and other settings. Maybe even compiler version.
Optimization should never affect result.

Yes, and actually in the example that is obeyed. Optimization does not
affect the result. The result of the example I gave is undefined. That
is according to the documentation.

Optimization does not affect the undefined-ness. It only affect how it
can be observed / how it manifests.

But that is ok. Undefined is undefined. In fact, it would not be
undefined, if it would always manifest in one and the same way.

----------------------------------------------------------------------
LemonParty · 2025-04-17 16:11

Do trunk have this problem?
Should I report this to bugtracker?

----------------------------------------------------------------------
Martin_fr · 2025-04-17 16:50

Quote from: LemonParty on April 17, 2025, 06:11:21 pmDo trunk have
this problem?
Should I report this to bugtracker?

For your original snippet of code, todays fpc 3.3.1 at
eba0624535cc504fcaf367055cd3adeab56097a4 generates the uniquestring
call.

As for "problem", it may or may not be. PascalDragon is part of the
FPC team. They have to decide what the desired behaviour is.  (I am
part of the Lazarus team, I just communicated my personal thoughts on
the topic).

----------------------------------------------------------------------
LemonParty · 2025-04-17 19:23

I found temporary decision. If call procedure this way
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i P(PChar(
@S[1] )^)compiler not put copying.

----------------------------------------------------------------------
Martin_fr · 2025-04-17 19:28

You can have that a bit shorter
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i
P1(pchar(S)[0]);
Mind the index changes to 0-based.

And you can shorten it with "type p=pchar;" / though I wouldn't.

----------------------------------------------------------------------
nanobit · 2025-04-17 20:20

Quote from: LemonParty on April 16, 2025, 05:25:49 pmThis mean all
project should be built with {T+} option.

{$T+} is an old invention, and has some caveat in modern Pascal (since
pointermath):
Most programmers are accustomed to think that "@" means untyped ({$T-}
default).
{$T+} allows type-checking (matching) in pointer-assignments,
but also changes "@" behavior (to typed pointer) in pointermath
(generated code).
Therefore I use {$T-} only.

----------------------------------------------------------------------
ASerge · 2025-04-17 20:47

Quote from: Martin_fr on April 17, 2025, 09:28:06 pmCode: Pascal
 [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i
P1(pchar(S)[0]);The same as I indicated above, only my version is more
optimal, but @LemonParty rejected it.