Feedback request: Syntax for "intrinsic functions" in FpDebug
lazarus / 12-Debugger

----------------------------------------------------------------------
Martin_fr · 2022-09-07 02:57

FpDebug is getting "intrinsic functions" for watches (like
https://sourceware.org/gdb/current/onlinedocs/gdb/Convenience-
Funs.html#Convenience-Funs )

For now it is the function "length" that returns the len of an array
or (under dwarf 3) a string. This does not call "length" in the target
app. This is computed inside FpDebug. It just uses function syntax.

If the syntax is just "length(myvar)" then this can conflict with a
user defined function "length" in the target app (if function eval is
enabled, and functions in the target app can be called).
Additionally it is possible that in future there may be intrinsics
that don't take any argument, in which case the brackets are optional.
And then such an intrinsic could also hide any variable of the same
name.

Therefore the idea is, that the intrinsics must be clearly
distinguishable from any other Pascal syntax. (GDB prefixes them with
$_ )

Looking at options using a prefix that can not naturally occur in
Pascal

!length():length()

----------------------------------------------------------------------
440bx · 2022-09-07 04:42

Quote from: Martin_fr on September 07, 2022, 04:57:34 amTherefore the
idea is, that the intrinsics must be clearly distinguishable from any
other Pascal syntax. (GDB prefixes them with $_ )

Looking at options using a prefix that can not naturally occur in
Pascal
I voted for the exclamation mark specifically because the exclamation
mark does not occur in the Pascal syntax.  My second choice would be
the colon ":" because it is very easy to type but, isn't my first
choice because it does occur in the Pascal syntax, though it is rather
unlikely it could ever be a source of ambiguity given that it would
always be the first character (and wouldn't be acting as a
separator/delimiter as it does in Pascal.)

ETA:

The colon ":" does have one advantage over the exclamation mark, it is
easier to visually distinguish from whatever letter follows it unlike
the exclamation mark followed by a lowercase "L".

----------------------------------------------------------------------
af0815 · 2022-09-07 07:47

Quote from: 440bx on September 07, 2022, 06:42:31 amThe colon ":" does
have one advantage over the exclamation mark, it is easier to visually
distinguish from whatever letter follows it unlike the exclamation
mark followed by a lowercase "L".
+1

a !l or !L is no so good to read, like a :l or :L

----------------------------------------------------------------------
Martin_fr · 2022-09-08 14:53

Just adding on here.

Naming suggestions for the below are welcome.
IMHO it should be a short name. "CC" may be just fine.

Currently
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i
:CC(Sender)CC is short for ChildClass.

This is similar to "auto type cast to instance class". Except you can
do it anywhere inside the expression, not just the final result.

So where previously you needed to write
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i
TButton(Sender).Tagyou can now write
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i
:CC(Sender).Tagand it will work for TButton, TCheckBox, ....

Leaves the question, why not extend the existing "auto type cast"
option?
Well, it could be ambiguous.
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i type  TBar
= class // (TSomeBase)  strict private    Foo: Integer;  end;  
TBarChild = class(TBar)  private    Foo: Integer;  end;
If the user is not aware (could be cross units) he may expect the
TBarChild's Foo for a watch like "SomeBar.Foo".

":CC(SomeBar).Foo" is more clear.

---
EDIT: add parent class for TBarChild

----------------------------------------------------------------------
Arioch · 2022-09-08 15:01

There is one more option - question mark

?length(...)

The "gut feeling" is like naming function with prefixes,
TMyButton.IsClicked vs TMyButton.DoClick

Exclamation mark to me hints at Do It! attitude and belongs to
procedures not functions.


P.S. why should brackets be opitional? why not instead use C approach
with mandatory ones? I am not sure what their use in debug evaluator
would even be. In some templates-fueled logger or exception dumper
there could be intrinsics like ?OS-Version() or ?Source-Line-Number()
but in variable watches, parameterless function (basically, a
constant?)? why?

P.P.S. yet a more crazy idea would be to reverse the name/baracket
syntax and use non-Pascalish brackets.

So, in this case, it would become something like {length s} or {copy
s, 10, 20}
Potential advantages: easier parsing (Lazarus and human) agaisnt
potential typos with nested/mismatched parenthesis, the evaluating
expression can be just copy-pasted into Pascal sources, where it would
be bona fide comment, easy to pick up and copy back to eval dialog.
Potential disadvantage: well, it IS weird-looking.

----------------------------------------------------------------------
Arioch · 2022-09-08 15:33

Quote from: Martin_fr on September 08, 2022, 04:53:07 pmLeaves the
question, why not extend the existing "auto type cast" option?
Well, it could be ambiguous.

I don't get it. Would you bring multiple record helpers in, then
perhaps.
But in your example TBarChild  and TBar are unrelated types, so
SomeBar variable can be either this type or that but not both.
What is the ambiguity and how is it resolved by :CC ?

If you mean that both units had global SomeBar variables of different
types, then only an explicit non-automatic typecast would be the least
ambiguous option.

----------------------------------------------------------------------
Martin_fr · 2022-09-08 15:52

Quote from: Arioch on September 08, 2022, 05:01:40 pmThere is one more
option - question mark
There are many more:
- copy gdb $_
- Just $ (works if every intrinsic name has at least one char outside
a..f => does not work for "CC()")
  Both $ prefixes can conflict with mangled names in some debug
symbols (currently not accessible)
- % as long as it does not conflict with cpu registers (which in turn
can always happen with new cpu)
- \ backslash


Quote?length(...)

Well, yes. But I want to keep another avenue open. (not yet decided if
it will be taken)

While we wont see it in FreePascal, we can still have it in the
debugger: Introduce the ? mark as trinary operator. Though that
already conflicts with the ":"
    a ? b : c

Well, in most cases (if not always) the parser should expect an
operator after "b". So it should be able to tell.

QuoteP.S. why should brackets be opitional? why not instead use C
approach with mandatory ones? I am not sure what their use in debug
evaluator would even be. In some templates-fueled logger or exception
dumper there could be intrinsics like ?OS-Version() or ?Source-Line-
Number() but in variable watches, parameterless function (basically, a
constant?)? why?

Indeed an intrinsic without parameter is very unlikely.

Not necessarily a constant. A sequential number generator ":SeqNum"
could exist. Maybe if used as argument to  a function evaluated in the
debug target "UserFunc(:SeqNum)". But to be honest, I don't see it.

More likely (once "out param" are supported), if in a watch-function-
eval calling a procedure with out param(s).
An intrinsic could supply a dummy param, so the function can be called
without modifying data in the target (which would be the case if you
need to pass an actual variable).

But, yes even then brackets could be mandatory. Well at least if no
prefix is used. If the intrinsic is prefixed, then the brackets would
be optional.
So just keeping as many roads open as possible.


There will anyway be an option for users to choose: prefix or not.
This poll just helps with finding a default.


Other than (currently) the votes indicate that some prefix is more
desirable, there also is the consideration of new user experience.

If you have a function in your code CC() that you want to call in a
watch, and if there is no prefix by default => then that will be a big
suprise.
And because the user likely does not even know what is causing the
unexpected result, they will have a hard time googling it (what terms
should they google).

Yet if you want to use an intrinsic, and "CC()" doesn't work. You have
a name for the feature that you can google (and if it is the wrong
name, you still can describe it)

----------------------------------------------------------------------
Martin_fr · 2022-09-08 16:08

Quote from: Arioch on September 08, 2022, 05:33:53 pmQuote from:
Martin_fr on September 08, 2022, 04:53:07 pmLeaves the question, why
not extend the existing "auto type cast" option?
Well, it could be ambiguous.
I don't get it. Would you bring multiple record helpers in, then
perhaps.

Not sure what you mean, but maybe this is based on my typo and no
longer relevant?
Otherwise, more explanation needed.

Quote from: Arioch on September 08, 2022, 05:33:53 pmBut in your
example TBarChild  and TBar are unrelated types, so SomeBar variable
can be either this type or that but not both.
Typo in my previous post. Fixed. TBarChild should inherit from TBar

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i var  
SomeBar: TBar;begin  SomeBar := TBarChild.create;
If you consider the watch "Sender.Tag" the debugger could say, well
there is no "Tag", so lets try to typecast to instance class.

If TBar did not have a field "Foo" then the watch "SomeBar.Foo" would
be typecast too.
If TBar is in a 3rd party package, and did not use to have "Foo", but
now unbeknown to you "Foo" was added, then the typecast no longer
happen. You still get a result, and you trust it, not knowing that it
is not what you though it is.


QuoteWhat is the ambiguity and how is it resolved by :CC ?

If you mean that both units had global SomeBar variables of different
types, then only an explicit non-automatic typecast would be the least
ambiguous option.
Well currently auto typecast is a checkbox in the watch properties.
Its default (on/off) can be set in the global IDE options.

But the way it currently only acts on the overall result, if it does
do a typecast then the result contains the typename to which it did
cast.

But if you then extract a single field "Sender.Tag", that does not
tell you the typename to which sender was cast. (Or if it was cast at
all)

Of course even with CC it could cast to a class that you did not
expect. But it takes out some of the risk. And it only happens when
you explicitly put the CC() in the expression.

----------------------------------------------------------------------
Arioch · 2022-09-08 19:26

> Introduce the ? mark as trinary operator. Though that already
conflicts with the ":"

exactly :-) i thought about it too.  Colon and question are equally
bad and equally good.

However, if you want tersely-named built-in functions - then that can
be used too.
 x ? a : b :3( x, a, b ) or !3( x, a, b){? x, a, b)Is there much
difference for typing in watch window?

Also, why only ternary not quaternary, or MS SQL-like DECODE( index,
value1, value2, ....)
Ternary operator seems to be just the most narrow case of it (assuming
ord(true) = +1). Well, with reversed order :-)

And then { [ordinal expression] #  Value1, Value2, Value 3, ....}

----------------

:CC merits seem a bit offtopic here, needing a separat "ticket" :-)
But i am not convinced yet.

> And it only happens when you explicitly put the CC() in the
expression

Does current autotype has benefits over your supposed CC() ? If a user
is bothered by that, rather edge case, then he can just supress auto-
typing and key implicit typecast into watch expression.  TBar(x) or
TBarChild(x).

If type is not fixed, then perhaps it is not critically important for
user now.

If you want to guard relaxed users from potential mistakes - then make
member vars duplication a mistake for evaluator, requiring explicit
typecast. For example SomeBar.Foo#TBarChild or SomeBar.Foo{TBarChild}

Afterall, what if i want to watch or read BOTH those variables ? Your
:CC approach limites me to only the outermost variable, does it not?

Frankly, as a user i do not feel there are advantages for explicitly
keying pseudo-function vs ticking auto-cast checkbox.

Typically i want to see the same things, as my program code would, so
SomeVar.Foo should access exacty what my CODE would've accessed.
Or, in rare cases i would care about inheritance chain, then i would
probably want to access both .Foo variables anyway.

Same goes about expressions. Modify evaluator: if auto-cast checkbox
was set - then do casts for all the terms, which were not explicitly
typecased by the user himself.

Scenarios when i would want auto-casted half of terms and anopther
half to be taken verbatim looks really exotic to me, does any actually
exist?

>  char outside a..f => does not work for "CC()

If you make brackets mandatory - then it doees work :-)
$CC is number, $CC() is function call.
But i agree this would be bad idea, too fragile.

BTW, what about detching IDE/OS vaiables? like $(PATH) from bash, or
%PATH% from DOS
$( prefix might be useful, and even implemented as a function with
NULL name :-D

But no, not in Pascal.

$_ or other two-char is less bad. But then they consume more space,
and watches names is not always wide a column.

> - % .... can always happen with new cpu

You do not need "new CPU" for it.  Enter GNU Assembler!

https://en.wikipedia.org/wiki/GNU_Assembler  ->  "movl  $4, %eax   # 4
(code for "write" syscall) -> EAX register"


> A sequential number generator ":SeqNum" could exist

LOL no, it can not be!

The syntax from Firebird/Interbase SQL would be GEN_ID( name,  diff  )
with idiomatic things like "calculating"  GEN_ID(  NNN, -GEN_ID( NNN,
0) + 10)

For sequences you would soon want
several sequences, not only onecontrol when to restart from zero,
would closing Lazarus IDE be enough for it or removing project.lps
would be needed?way to check current value without changing it, like
for generating file paths like 'path/Row0123/Row0123Column654.txt
When you type ":SeqNum" - you just hide those parameters in some
hidden context and make them non-manageable.

Then, there can be "volatile constants" like current source file an
line in it - but those make good sense for error reporting and
logging, but hardly for variable watches/evaluating.

> An intrinsic could supply a dummy param

Again, niche case. This would work for out-parameters, but not var-
parameters (both in and out)

I mean, okay, "extract me Nth parameter as return value) makes perfect
sense to me.  Guess it was calles xargs in Linux bash.

As of dummies - there is a convention in functional languages that is
no permeating into imperative language as well.
A special variable name of single underscore, like FuncName(10, x, _,
_, y, _)
In those languages though it is a first class citizen of the language,
used not only for function call. Like, vector comparison can be if [x,
b, q] = [y, a, _] then ..., also records unpacking, etc.

Okay, in Pascal single underscore is valid variable name, though i
never saw it in the wild. Any other single special char could do.

So, data extraction into expression, if it is needed much. Like { bx
|  FuncName( a, _, b, c, bx, _) }

Yes, i deviate here from my infix notation {funcName param, param,
param} but special symbols between first term and 2nd term could be
used for distinction. Ot maybe those ideas are mutually exclusive.

Or, perhaps just have two placeholders, like  FuncName( a, _, b, c,
=!=, _)

> There will anyway be an option for users to choose: prefix or not.

Why? Isn't Pascal, unlike C, is about preventing users from shooting
their own foot?
Think user support.
Someone posted in the forum or tracker a screenshot with evaluating
field. Or is making a tutorials for novices.
If prefix is mandatory - everyone instantly sees when there is
intrinsic, and where thereis some entity from the program.
If not - it would always be a guesswork. Who would enjoy guessing over
program correctness?

----------------------------------------------------------------------
Martin_fr · 2022-09-08 20:21

Quote from: Arioch on September 08, 2022, 09:26:07 pm> And it only
happens when you explicitly put the CC() in the expression

Does current autotype has benefits over your supposed CC() ? If a user
is bothered by that, rather edge case, then he can just supress auto-
typing and key implicit typecast into watch expression.  TBar(x) or
TBarChild(x).

They are different things.

The question should be: How does  CC()  compare to an (not yet
existing) "auto type cast individual parts of the entire expression".
CC() gives the power to select which parts.

QuoteAfterall, what if i want to watch or read BOTH those variables ?
Your :CC approach limites me to only the outermost variable, does it
not?
If you want 2 values, you need to watches.
* SomeBar.Foo
* :CC(SomeBar).Foo  

The latter is the same as TBarChild(SomeBar.)Foo. Except the later
also works if you have more classes that inherit from TBar, and you
don't know which one it will be. Because the power of declaring "var
SomeBar: TBar" after all is that it can be any subclass. (and
subclasses can reintroduce staff)

Btw, in neither case will the existing autotypecast affect which Foo
you get to see.

I suggest, to wait till I committed, and then to test it.


QuoteTypically i want to see the same things, as my program code
would, so SomeVar.Foo should access exacty what my CODE would've
accessed.
And you do.

QuoteSame goes about expressions. Modify evaluator: if auto-cast
checkbox was set - then do casts for all the terms, which were not
explicitly typecased by the user himself.
Sorry, not sure what you mean?

QuoteWhy? Isn't Pascal, unlike C, is about preventing users from
shooting their own foot?
No, Pascal just requires you to sign a document that you know what you
do, and the consequences are on you.

----------------------------------------------------------------------
Martin_fr · 2022-09-09 11:23

Documentation has been added https://wiki.freepascal.org/FpDebug-
Watches-Intrinsic-Functions

The default is now ":", but can be configured (":", "!" or none).

----------------------------------------------------------------------
Arioch · 2022-09-09 17:01

Quote from: Martin_fr on September 08, 2022, 10:21:54 pmQuoteSame goes
about expressions. Modify evaluator: if auto-cast checkbox was set -
then do casts for all the terms, which were not explicitly typecased
by the user himself.
Sorry, not sure what you mean?

I was arguing that if CC is a better auto-type-cast, then it would
better just replace the current implementation with it, than to keep
two different implementations.

As for selective aapplicatio nto some terms but not all, in those i
think rare cases it would be needed - it could've be doen liek this.

1.  expression:  AAAA.foo + BBBB.bar * CCCC.baz     [  ] auto-type
checkbox: clear.  
Result - none of the variables "uplifted" to their most specific known
type

2.  expression:  AAAA.foo + BBBB.bar * CCCC.baz      auto-type
checkbox: set.  
Result - CC applied to all three variables

3.  expression:  AAAA.foo + TMyBBBType(BBBB).bar * CCCC.baz      auto-
type checkbox: set.  
Result - CC applied to AAAA and CCCC but not to BBBB


QuoteIf you want 2 values, you need to watches.
* SomeBar.Foo
* :CC(SomeBar).Foo  

But it obviously is not scaling up beyon f two!

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i type 
 TSomeType = class      strict private  Foo: integer;   end;  
 TSubType = class(TSomeType)      strict private Foo: integer; 
 end;    TSubSubType = class(TSubType)      protected Foo: integer; 
 end; var Dummy: TSometype; begin    Dummy := TSubSubType.Create; 
This is the gist of the problem with member variables being hidden?
If that is a real problem, then it would be good to solve it geerally,
to make evaluator able to specify all three of them. (or 4, or 5,
etc);

----------------------------------------------------------------------
Arioch · 2022-09-09 17:18

An idea for intrinsic - watch (non-destructively) references counter
for objects, interfaces, strings and dynarrays.
All with the same name, not type-specific accessors.

None-destructive... well...

The Delphi debugger seems to use RTL functions, properly incrementing
and decrementing counters.
Well, when those counters were correct from get go. But we usually
debug not-correct programs, right?

So, occasionally Delphi eval/watch tends to DESTROY the very object
being looked into.
Like when you are debugging the very process of creating a ref-counted
object.

Also, the classic Delphi problem of being too smartie-flexible.
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i // correct
use with manual memory managementvar d1: TXmlDocument; // classbegin 
 d1 := TXmlDocument.Create(Application); // increments refcounter
artificially   try      ....d1.Node(..)..., etc  //refcounter goes up
and down, but never zeroes   finally       d1.Free;   end;end; //
correct use with ARC (auto ref-counting)var d2: iXmlDocument; //
interfacebegin   d2 := TXmlDocument.Create(nil); // refcounter is
naturally +1, the rest is the same   ....d2.Node(..)..., etcend; //
the really popular trapvar d3: TXmlDocument; // classbegin   d3 :=
TXmlDocument.Create(nil); // refcounter = 0   try     
....d3.Node(..)...  // refcounter goes to +1, then to 0 and the object
is destroyed.      ....      ....d3.Node(..)...   // booom!  use after
free   finally       d3.Free;  // boom!  double-free   end;end; 
Delphi watches are prone to problems like this too...

----------------------------------------------------------------------
Martin_fr · 2022-09-09 18:49

Quote from: Arioch on September 09, 2022, 07:18:21 pmAn idea for
intrinsic - watch (non-destructively) references counter for objects,
interfaces, strings and dynarrays.
All with the same name, not type-specific accessors.
That is actually a good idea. (for strings: codepage too)

For Strings and dyn arrays it is possible to get them.

For interfaces, there is almost zero debug info delivered by FPC. The
debugger can't currently get any useful data for them.
"Objects" => Do you mean via the new smart pointers? That has still to
be investigated how it works with the debugger. (I.e. what info fpc
delivers to the debugger).

In any case, it can be started for any data for which it currently
works, and be extended as things improve.

QuoteNone-destructive... well...

The Delphi debugger seems to use RTL functions, properly incrementing
and decrementing counters.
Well, when those counters were correct from get go. But we usually
debug not-correct programs, right?

So, occasionally Delphi eval/watch tends to DESTROY the very object
being looked into.
Like when you are debugging the very process of creating a ref-counted
object.
The intrinsic would not modify the data. Not even call code in the
debugged app.
At least for strings and dyn array, the location of the data in memory
is known, and can be read.

When strings are part of "watch function eval" in the debugged app,
that differs. Then those low level functions are called (already in
place).


Quote
Also, the classic Delphi problem of being too smartie-flexible.
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i // correct
use with manual memory managementvar d1: TXmlDocument; // class 
Delphi watches are prone to problems like this too...

I am not sure of the exact inner workings of that class in Delphi.

But if this is about the refcount required for the interface => afaik
the object implements it as a normal field. The debugger doesn't touch
that. And also the debugger has no info that this field is special.

So That means however, if you were to eval a watch, that calls a
function, and that function returns an interface => then that refcount
goes up by one, and never down (because it waits for the caller to
"release" the interface).
So, yes in that case, after the debugger had evaled that function, the
debugger would have to call the correct method on the interface to
"release" it, and get the refcount reduced.

But that does not work now, because fpc does not provide enough info
for the debugger to do so.


For strings that is done, if the eval-ed function returns an
ansistring, then it is dec-refed as it should. (Mind that result is
not in any variable of the debugged exe).

----------------------------------------------------------------------
Arioch · 2022-09-09 19:14

> for strings: codepage too

Not so sure. It only exists for strings, not other types. And then on
can just call RTL function, which alreayd exists. Or typecast to
PStringRec. If there were many different data types with codepages -
then yes, bringing all them into one single function would make its
use easier. But currently with intrinsic or not - it would be the same
memorization of one function name for one special purpose

> "Objects" => Do you mean via the new smart pointers?

I meant TInterfaceOnject, it's internal fields. Just when accessed bu
object pointer not interface pointer.


> At least for strings and dyn array, the location of the data in
memory is known, and can be read.

Yes, but when Delphi/FPC passes managed type into a function as non-
const parameter, it already increases counter.
I exactly meant that this should be prevented.


> I am not sure of the exact inner workings of that class in Delphi.

You can easily find it. But internal details is not hat i pointed at,
i pointed exactly at very non-expected behavior, which only makes
sense when you draft possible implementations. And for programmer who
wants to just use the library - it is a hidden trap. And hidden
funciton calls by watches accessing properties can trigger it.

I once hoisted on a similar petard, debugging my own object creation -
and the Watches triggered refcounting way to early. Good thing i had a
breakpoint in destructor and when it suddenly was triggered - i was
royally confused, but at least i got alerted.

> afaik the object implements it as a normal field.

That was a really good question.
Indeed, stock TInterfacedObject does.

But it is not required (albeit it is a most common case) to be derived
from it.

Delphi just uplifted Microsoft API, which does not have
"GetCurrentValue" method, only icnrement and decrement, the rest is
incapsulated.
https://docs.microsoft.com/en-us/windows/win32/api/unknwn/nn-unknwn-
iunknown

In general case it is not possible to query current value without
changing data. Strictly speaking, no one says this counter even exists
at all. The methods should work, and implementation is black box.

Gladly, newer Delphi made interfaces expose original object classs, so
coming back to the class layout is possible. It was a problem with old
Delphi versions, and it seems it still is a problem (i remember mORMot
team complained a lot, when they decided to avoid Delphi/LLVM and go
FPC instead).

However, FPC is not Windows-targeted. Even in Delphi there is no
warranty that when making COM/Interface, a real ref-counter would be
implemented at all. With FPc it would be even less warranty.

> So That means however, if you were to eval a watch, that calls a
function, and that function returns an interface => then that refcount
goes up by one, and never down (because it waits for the caller to
"release" the interface).

But what is the caller here? i think the caller is exactly the
evaluating functions. As soon as the value was rendered to the screen
- the functin exist, and its local variables (including that nterface
pointer) are released. Evaluating naturally includes changing
refcounter forth and back.

> But that does not work now, because fpc does not provide enough info
for the debugger to do so.

I'll have to think about it. At least in Delphi that is a nonsense.
The ony thing needed ot know is "this is interface type", then COM
standard dictates how exactly references are released, ABI is fixed
forever.
The only thing debugger/evaluatoir has to know for being able to free
is "the data type is some interface", it would be enough.


For strings that is done, if the eval-ed function returns an
ansistring, then it is dec-refed as it should. (Mind that result is
not in any variable of the debugged exe).
[/quote]

----------------------------------------------------------------------
Martin_fr · 2022-09-09 20:34

Btw, the forum has [ quote ]... [ / quote ] blocks....
Quote> At least for strings and dyn array, the location of the data in
memory is known, and can be read.

Yes, but when Delphi/FPC passes managed type into a function as non-
const parameter, it already increases counter.
I exactly meant that this should be prevented.

IIRC, in FPC the receiving routine does the refcount handling.
So if you have a watch "pos(var1, var2)" then "pos" will inc and dec
the refcount. Nothing that can be done there.
If var2 has a refcount of 0 before the call (due to some severe bug)
then yes, such a watch will free the memory of var2.  But it's the
called function, not the debugger that does it.

QuoteYou can easily find it. But internal details is not hat i pointed
at, i pointed exactly at very non-expected behavior, which only makes
sense when you draft possible implementations. And for programmer who
wants to just use the library - it is a hidden trap. And hidden
funciton calls by watches accessing properties can trigger it.
Big first point on https://wiki.freepascal.org/FpDebug-Watches-
FunctionEval : function eval can have side effects.

I remember too well, my app always crashed. Except when I was
debugging it. Because back then I had no idea the watch was creating
the missing data in the target app.

A watch could even make calls to the OS. Teaching the debugger how to
revert the effects of every possible call to the kernel or any library
=> nope.


Quote> So That means however, if you were to eval a watch, that calls
a function, and that function returns an interface => then that
refcount goes up by one, and never down (because it waits for the
caller to "release" the interface).

But what is the caller here? i think the caller is exactly the
evaluating functions. As soon as the value was rendered to the screen
- the functin exist, and its local variables (including that nterface
pointer) are released. Evaluating naturally includes changing
refcounter forth and back.
The debugger is the caller, if the function result is of type
(managed) interface.

Quote> But that does not work now, because fpc does not provide enough
info for the debugger to do so.

I'll have to think about it. At least in Delphi that is a nonsense.
The ony thing needed ot know is "this is interface type", then COM
standard dictates how exactly references are released, ABI is fixed
forever.
The only thing debugger/evaluatoir has to know for being able to free
is "the data type is some interface", it would be enough.
If it is a com interface....
But that must be known.

And IIRC, fpc does not put that into the DWARF info. (sadly)

----------------------------------------------------------------------
Arioch · 2022-09-09 22:14

> Forum has [ quote ]... [ / quote ] blocks.

I know , but they are much longer to type (or require a third arm for
mouse, while other two arms are on keyboard.)

> If var2 has a refcount of 0 before the call (due to some severe bug)
then yes, such a watch will free the memory of var2.  But it's the
called function, not the debugger that does it.

Frankly, it is mostly an argument about definitions, funny but usually
meaning little. If the program died with division by zero error, was
it an error in the program or an error in the function but not in the
program? If debugger calls the function that then unexpectedly
destroys something - then from user point of view it was destroyed by
debugger. Just like a prey is killed by the hunter, not by the bullet
or by the rifle.

If we go really techy, then the function did nothing wrong, it was
inconsistent state of the passed parameter object that led to
disaster. And then who was the actor, who gave that inconsistent value
to the function? it was the watch window. On the next iteration it
would be unsavvy devleloper, who enabled function calls in watch
window without making sure the function works safely on incorrect
data. And so forth go blame games.

From appliaction programmer's point of view, function is the code he
types. Programmer does not do refcounting, it is "Delphi" or "Lazarus"
which does it automatically. Compiler developer then would shout
"learn your tools". AppDev would yell "fix your tools" and so it goes,
making the question "who did it" senseless.

> of 0 before the call (due to some severe bug)

The thing is, the programmer (the program he writes) did not do any
calls yet, he is tracing through the object nested constructors (bonus
points for foing it in assembler window not even Pascal one). It is
debugger then which suddenly calls a funciton (like property getter)
before the object is ready for it.

So there is "severe bug" in the program, there was not enough of
understanding of inner working of the debugger. Or not enough of
forethought and sharp memory.

> Teaching the debugger how to revert the effects of every possible
call to the kernel or any library => nope

Not in Pascal, yes. It would need a different language.

But implementing intrinsics in the way that would preclude side
effects, and then not extending in the ways breaking it  - is
possible. That is what i wrote. Special care has to be done that
intrinsics would be free from side effects by design and by
implementation.

Otherwise, debugger's :length is worse than RTL's Length. But if an
intrinsic gives warranty of never triggering side effects (even on
inconsistent program states and data structures), then it alone makes
a good reason to duplicate things. I think it shuld be requirement for
current and futue intrinsics to never ever have side effects, better
fail than this.

> If it is a com interface....  But that must be known.

In Delphi Classic it is by definition so. But as soon as we go cross-
platform - it can no more be. At most it cold have been Mozilla XPCOM,
but i guess it also does not exist on every FPC-supported platform. So
it is even less certainty in FPC than there is in Delphi/COM (where it
is lacking already).

> And IIRC, fpc does not put that into the DWARF info

I don't know DWARF, but i also would not be sure FPC even could put
that info there without breaking compatibility with some "old classic"
tools.
Not all formats are extensible enough, and when they are - practical
programs are not always forgiving...

----------------------------------------------------------------------
PascalDragon · 2022-09-11 15:43

Quote from: Arioch on September 09, 2022, 09:14:56 pm> But that does
not work now, because fpc does not provide enough info for the
debugger to do so.

I'll have to think about it. At least in Delphi that is a nonsense.
The ony thing needed ot know is "this is interface type", then COM
standard dictates how exactly references are released, ABI is fixed
forever.
The only thing debugger/evaluatoir has to know for being able to free
is "the data type is some interface", it would be enough.
For interfaces in general you can not know the reference count,
because IUnknown only provides methods to increment (_AddRef) and
decrement (_Release) the reference count, but not to retrieve it. And
doing a _AddRef followed by a _Release (both of which will return the
reference count) might be a rather involved operation depending on the
implementation so it might not be a good idea to do that inside the
debugger.

----------------------------------------------------------------------
Martin_fr · 2022-09-11 15:53

Quote from: PascalDragon on September 11, 2022, 05:43:14 pmAnd doing a
_AddRef followed by a _Release (both of which will return the
reference count) might be a rather involved operation depending on the
implementation so it might not be a good idea to do that inside the
debugger.

Also FPC currently doesn't even supply the addresses of those
routines.

The dwarf entry for any interface currently is an address, the
typename, and "this is a structure". And nothing more. No names or
addresses of routines. Nothing.

----------------------------------------------------------------------
Arioch · 2022-09-11 18:42

Quote from: PascalDragon on September 11, 2022, 05:43:14 pmFor
interfaces in general you can not know the reference count, because
IUnknown only provides methods...
Yes, and i spelled it above. Incapsulation agaisnt introspeciton :-)

Quote from: PascalDragon on September 11, 2022, 05:43:14 pmAnd doing a
_AddRef followed by a _Release

is exactly what we try to avoid introducing non-destructive
intrinsics.

So, yes, in general case it would be impossible, only if we can
somehow learn some well-known implementation like TInterfacedObject
was used

----------------------------------------------------------------------
Arioch · 2022-09-11 18:44

Quote from: Martin_fr on September 11, 2022, 05:53:54 pmThe dwarf
entry for any interface currently is an address, the typename, and
"this is a structure". And nothing more. No names or addresses of
routines. Nothing.

address of what?

COM ABI mandates fixed positions for those methods at fixed offsets
from every "interface pointer".
That is the only way interfaces could work, since inheritance concept
was damned by COM.

similar how TObject provides some virtual functions that are now
forever at fixed offsets of any TClass pointer.

----------------------------------------------------------------------
marcov · 2022-09-11 18:54

Quote from: Arioch on September 11, 2022, 08:44:45 pm
address of what?

COM ABI mandates fixed positions for those methods at fixed offsets
from every "interface pointer".
That is the only way interfaces could work, since inheritance concept
was damned by COM.

Freepascal also allows non-COM interfaces (as parent of  IUnknown)

----------------------------------------------------------------------
Arioch · 2022-09-11 19:02

Quote from: marcov on September 11, 2022, 08:54:31 pmQuote from:
Arioch on September 11, 2022, 08:44:45 pm
address of what?

COM ABI mandates fixed positions for those methods at fixed offsets
from every "interface pointer".
That is the only way interfaces could work, since inheritance concept
was damned by COM.

Freepascal also allows non-COM interfaces (as parent of  IUnknown)

Then perhaps FPC can extend DWARF with signalling if it was IUnknown
(wih children) or non-IUnknown interface?

P.S. there was a significant holy war within Firebird 3 and their new
API, if IUnknown is COM or on top of COM.
It seems that from rigorous point of view COM does not require
IUnknown and hence does not include those three key methods.
But in practical use of terms, COM and IUnknown are siamese brothers,

----------------------------------------------------------------------
Martin_fr · 2022-09-11 19:40

Quote from: Arioch on September 11, 2022, 09:02:03 pmThen perhaps FPC
can extend DWARF with signalling if it was IUnknown (wih children) or
non-IUnknown interface?

"IUnknown" simple means that there are Inc/Decref methods. It does not
tell us anything about where that data is. In fact I could implement
that interface in my own class, and since I plan one global instance
that will be for the lifetime of the app, I can get away with not even
storing the refcount.

And then even FPC wouldn't know where the ref count is stored (or if
it is actually stored at all). It couldn't give that info to the
debugger, and the debugger can't know either.

FPC should however add
- inheritance info
- names and location of methods

And because a debugger can be allowed to know more than the app, FPC
could add info how to find the underlaying object (if exists). Though
this part is particularly tricky...

Actually, the debugger could already find that object, but it would
have to disassemble code, and it would need to find exactly the code
it expects. I.e. it would need have code specific to different fpc
versions, or fpc on different platforms, ..... Would still not get the
refcount ....

----------------------------------------------------------------------
Arioch · 2022-09-11 20:48

Quote from: Martin_fr on September 11, 2022, 09:40:43 pmQuote from:
Arioch on September 11, 2022, 09:02:03 pmThen perhaps FPC can extend
DWARF with signalling if it was IUnknown (wih children) or non-
IUnknown interface?

"IUnknown" simple means that there are Inc/Decref methods. It does not
tell us anything about where that data is

Sure. I did not say you would get access to the counter. I said long
ago COM/IUnknown does not expose it and that in general case peeking
into it would not be possible. It is kind of becoming running joke, we
repeat it to one another again and again :-)

Here i was thinking just about enabling evaluation/inspection, which
is actually more dependent upon the 3rd methopd, QueryInterface.
Actually, AFAIR that was exactly how Delphi implemented typecasts back
to classes. Priorly all kinds of hacks were used, and eventualyl
Delphi just made a special interface to return TObject pointer.

So two topics - general detection of IUnknown and specifically non-
destructive intrinsics got mixed in and confused.

> In fact I could implement that interface in my own class,

I know. It is typical thing to, for example, disable actual reference
counting (then no counter at all), when the interface used to access,
for example, DOM nodes of XML or JSON tree.
And other purposes too.

QuoteActually, the debugger could already find that object, but it
would have to disassemble code, and it would need to find exactly the
code it expects. I.e. it would need have code specific to different
fpc versions, or fpc on different platforms, ..... Would still not get
the refcount ....

Veeery hackish way. Even Delphi was not going THAT deep to get from
interfaces back to objects AFAIr.

That said, is it practical scenario that Lazarus is used to debug a
program with no Pascal sources?
Because if you have sources - IDE would probably recompile them
compile with the same RTL as Lazarus itself and debugger use. And then
no need to have a database of different versions.

There are a number of stand-alone debuggers, and Lazarus is not one of
them.

However, this still seems badly fragile.
Like hackish functions trying to understand if a random pointer is
object or not. Most time they work good, other time they damage
program, like burning stack guard pages

----------------------------------------------------------------------
Martin_fr · 2022-09-18 22:08

Quote from: Arioch on September 09, 2022, 07:18:21 pmAn idea for
intrinsic - watch (non-destructively) references counter for objects,
interfaces, strings and dynarrays.

Done for AnsiString(dwarf-3) and DynArrays

https://wiki.freepascal.org/FpDebug-Watches-Intrinsic-Functions