Curly brackets in strings
lazarus / 62-General

----------------------------------------------------------------------
simsee · 2022-10-02 09:18

Consider the following trivial program:

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i program
Project1;begin  {  writeln('}');  }  writeln;end.
The compiler gives an error, as it is deceived by the presence of the
closing brace inside the string in writeln.

Is this a bug? How should curly brackets be handled in strings? Is it
mandatory to use #123 and #125?

In the documentation:

https://www.freepascal.org/docs-
html/current/ref/refse8.html#x19-180001.8)

there is no mention of this issue.

Shouldn't the compiler parser ignore what's inside the quotes?

Thanks.

----------------------------------------------------------------------
Thaddy · 2022-10-02 09:29

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i program
Project1;begin  { <---- this is a multiline comment  writeln('}'); //
this would compile if not surrounded by the multiline comment....  and
this is the end of a multiline comment --->} writeln;end.Code: Pascal
 [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i
beginwriteln('{');end.

----------------------------------------------------------------------
MarkMLl · 2022-10-02 09:31

Hoo boy :-(

Not strictly a bug: use (* *) for the outer comments.

MarkMLl

----------------------------------------------------------------------
Thaddy · 2022-10-02 09:32

(* and *) are equal to { and } only older...

----------------------------------------------------------------------
MarkMLl · 2022-10-02 09:34

Quote from: Thaddy on October 02, 2022, 11:32:38 am(* and *) are equal
to { and } only older...

No, they were introduced at the same time. Wirth's original comments
were /* */ but (I suspect) he realised that that style was also being
used by the B language.

Somewhat later: https://www.research-
collection.ethz.ch/bitstream/handle/20.500.11850/68910/eth-3059-01.pdf
page 50 which shows that comments were in braces with /* */ as
alternatives. I believe that at that time Wirth was working largely on
CDC kit which had a fairly comprehensive character set, but swathes of
the industry still relied on (rebadged) IBM 026 and 029 cardpunches
for code entry.

MarkMLl

----------------------------------------------------------------------
simsee · 2022-10-02 10:53

Thanks for the good explanations. But back to one of my initial
questions: Shouldn't the compiler ignore what is in quotes?

----------------------------------------------------------------------
Thaddy · 2022-10-02 10:55

Well, /* or divide then multiply has always been nonsense...  as is
multiply then divide...

----------------------------------------------------------------------
Thaddy · 2022-10-02 10:58

Quote from: simsee on October 02, 2022, 12:53:16 pmThanks for the good
explanations. But back to one of my initial questions: Shouldn't the
compiler ignore what is in quotes?
But it does ignore it.... Always has.... It translates to a string
literal and is stored as is. Except for escaped characters which are
first evaluated and then stored.

----------------------------------------------------------------------
MarkMLl · 2022-10-02 11:03

Short: No.

Longer: Any attempt to do this would mean that the lexer would have to
scan the content of the comment and decide that the closing brace was
inside a string. That is, broadly speaking, anathema since it implies
that the content of the comment would have to be valid and correct
Pascal source. I believe that the only situation in which anything
inside the comment is interpreted is if the opening brace (or
equivalent digraph) is followed by a $directive.

So it you want to exclude a block of code, either use a non-matching
multiline comment (i.e. (* *) if your comment contains braces, or vice
versa) or a {$ifdef ... $endif } pair. But even that can be fooled if
there is an embedded $endif, so the "comment of last recourse" is a //
at the start of every line.

MarkMLl

----------------------------------------------------------------------
simsee · 2022-10-02 11:04

Thanks Thaddy.  I rephrase my question: are curly brackets, as markers
for the start and end of comments, to be considered active also inside
the quotes?

----------------------------------------------------------------------
Thaddy · 2022-10-02 11:09

No. The quotes are parsed first and that is even if the contain itself
a curly bracket.
If it does not work, that would be a serious parser bug. (Even with
nested comments)

@Mark: you are wrong, see the docs about chicken and egg, (* was
first, {} is TP style.:
https://www.freepascal.org/docs-html/ref/refse2.html

----------------------------------------------------------------------
MarkMLl · 2022-10-02 11:15

Quote from: Thaddy on October 02, 2022, 01:09:42 pm@Mark: you are
wrong, see the docs about chicken and egg, (* was first, {} is TP
style.:
https://www.freepascal.org/docs-html/ref/refse2.html

No Thaddy, YOU ARE WRONG: I cited a 1973 Wirth paper. Page 50 of the
PDF, page 44 of the original.

MarkMLl

----------------------------------------------------------------------
simsee · 2022-10-02 11:16

I'm sorry, I don't want to make you lose your patience, but from my
example it would seem that the curly bracket inside the quotes is
active and pairs with the previous open one, closing the multiline
comment.

----------------------------------------------------------------------
Thaddy · 2022-10-02 11:32

The code inside the start of a comment is ignored until the compiler
sees a closing of the comment. The compiler does not evaluate complex
comments (well, it evaluates nested comments.) So the compiler thinks:
Oh start of a comment, just look for the closing.. Code inside
comments is never compiled, not even evaluated.
In your case you can use proper real old school comments as already
suggested: (*...*).

----------------------------------------------------------------------
Bart · 2022-10-02 11:36

FWIW: Delphi 7 also does not compile the example code.

Bart

----------------------------------------------------------------------
wp · 2022-10-02 11:39

Neither does Delphi XE 10.4 CE.

----------------------------------------------------------------------
Thaddy · 2022-10-02 11:39

For the same reason.

----------------------------------------------------------------------
wp · 2022-10-02 11:48

I think Thaddy is right. Suppose this:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i { This
comment contains a curly brace }. }Everybody would say that the
comment ends at the first brace (even the highligher of the forum
software does so). If the parser does not perform analysis of what is
inside the braces then there is no difference to the situation
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i { 
WriteLn('}');}'

----------------------------------------------------------------------
simsee · 2022-10-02 11:55

So we can conclude that the curly brackets are always active,
regardless of the surrounding context.

----------------------------------------------------------------------
ASerge · 2022-10-02 11:58

Quote from: simsee on October 02, 2022, 01:55:43 pmSo we can conclude
that the curly brackets are always active, regardless of the
surrounding context.
Yes
Compilers scan from '{' to the nearest '}', and only then scan tokens,
including quoted strings.
As result
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i { 
WriteLn('}');}' is then same as
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i ');}'and
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i { This
comment contains a curly brace }. } is
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i . }Both
produce error.

----------------------------------------------------------------------
Fred vS · 2022-10-02 12:02

fpc dont like this one too:

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i program
test; { I am a comment with this: { something... } beginend.
Result:

Quotefred@fred-80m0 ~> fpc test.pas
Free Pascal Compiler version 3.2.2 [2021/07/09] for x86_64
Copyright (c) 1993-2021 by Florian Klaempfl and others
Target OS: Linux for x86-64
Compiling test.pas
test.pas(3,29) Warning: Comment level 2 found
test.pas(6,5) Fatal: Unexpected end of file
Fatal: Compilation aborted
Error: /usr/bin/ppcx64 returned an error exitcode

----------------------------------------------------------------------
MarkMLl · 2022-10-02 12:07

Quote from: Thaddy on October 02, 2022, 01:32:52 pmIn your case you
can use proper real old school comments as already suggested: (*...*).

On that at least we agree. J&W-era Pascal had definitely moved from /*
*/ to (* *) as the digraph, I don't have a copy to hand but I suspect
that it used that form in the code examples rather than braces. The
fact that I habitually use (* *) for "long term" comments suggests
that it's a habit I got into early, despite most of the systems I've
used having the full ASCII character set.

I believe that braces got into ASCII circa 1965 and were definitely in
the definitive ASCII-68, but the fact that Xerox PARC's Smalltalk of
the late '70s and early '80s still had arrows in the character set
suggests that some manufacturers stayed on an older version (replacing
print drums in fast line printers was expensive). I believe that Wirth
was using a CDC when he worked on Pascal, a quick trawl in Bitsavers
suggests that their large-scale printers didn't have braces (but did
have some of the ALGOL-style "funnies") but I've seen some CDC Pascal
stuff in the past which implied that at least some of their terminals
etc. had something like a 12-bit character set which included e.g.
much if not all of the APL operators.

MarkMLl

----------------------------------------------------------------------
dseligo · 2022-10-02 12:16

Quote from: simsee on October 02, 2022, 01:55:43 pmSo we can conclude
that the curly brackets are always active, regardless of the
surrounding context.

Not only curly brackets. This also doesn't compile:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i     program
Project1;    begin      (*      writeln('*)');     *)     writeln;  
 end.
Thaddy's explanation in post #13 is correct.

----------------------------------------------------------------------
simsee · 2022-10-02 12:33

But this compiles:

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i program
Project1; begin  writeln('ok//');end.
So, for single line comments the rule the rule does not apply.

----------------------------------------------------------------------
Fred vS · 2022-10-02 12:38

Quote from: Thaddy on October 02, 2022, 01:32:52 pmThe code inside the
start of a comment is ignored until the compiler sees a closing of the
comment.
...

Hum, ok, but it seems that the compiler does care of other open
brackets.
See my post: https://forum.lazarus.freepascal.org/index.php/topic,6078
6.msg455827.html#msg455827

----------------------------------------------------------------------
dseligo · 2022-10-02 12:56

Quote from: simsee on October 02, 2022, 02:33:54 pm But this compiles:

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i program
Project1; begin  writeln('ok//');end.
So, for single line comments the rule the rule does not apply.

It does apply, your '//' isn't inside a comment.

Quote from: Thaddy on October 02, 2022, 01:32:52 pmThe code inside the
start of a comment is ignored until the compiler sees a closing of the
comment.
<snip>

----------------------------------------------------------------------
wp · 2022-10-02 13:56

It's always the same rule:
- Find a comment beginner '(*" --> search until the next '*)' is
found, ignore everything else in between.
- Find a comment beginner '{' --> Search until the next  '}' is found,
ignore everything else.
- Find a quote indicating that a string begins --> search until the
next quote character is found (unless duplicated) and ignore
everything else.

----------------------------------------------------------------------
Arioch · 2022-10-02 14:10

wp - no, not always.  In some dialects, more theoretican practical,
like ANSI Pascal the following are correct and complete comments:

(*  bla-bla-vla }
{ bla-vla-vla *)

Actually, TP (if not USCD) violated the standard and hardware history
when making them incompatible. But doing so they enables erzats nested
comments, so they did good

On the contrary.  (. 1..2 ] and [ 3 .. 4 .) are still valid
sets/ranges.  Inconsistent, but practical :-)

----------------------------------------------------------------------
MarkMLl · 2022-10-02 14:48

Quote from: Arioch on October 02, 2022, 04:10:57 pmwp - no, not
always.  In some dialects, more theoretican practical, like ANSI
Pascal the following are correct and complete comments:

(*  bla-bla-vla }
{ bla-vla-vla *)

Actually, TP (if not USCD) violated the standard and hardware history
when making them incompatible. But doing so they enables erzats nested
comments, so they did good

On the contrary.  (. 1..2 ] and [ 3 .. 4 .) are still valid
sets/ranges.  Inconsistent, but practical :-)

Yes, depending on whether they are processed as digraphs at the lexer
level or as separate elements.

I'd expect comments to always be discarded by the lexer, and that
includes /everything/ between an opening and closing marker. I'd
expect the (. and .) digraphs to be converted to single characters and
then passed to the parser with no further attempt at interpretation.

Part of the issue with matching comments is because in FPC a comment
marker closes a macro, and a macro definition, which could itself
potentially include a (non-matching) comment.

MarkMLl

----------------------------------------------------------------------
Arioch · 2022-10-02 15:24

My *blind guess* it that originally, in TP descending parser FSM, it
was just simpler to copy-paste {/} and (*/*) pairs verbatim, than
making special cases of "one closing another"

you meet '{' - search for the next '}'
you meet '(*' - search for the next '*)'

it is simple and large;y error-safe, just colling one single search
fuinciton, Pos :-)

REP SCANSB is all you need, well, almost

multivariant searcher is much more complex, and thus slow and error
prone.

This, by accident, enabled nested comments.  "And programemrs saw it,
and they saw it was good" and demanded it to forever be so :-)

----------------------------------------------------------------------
PascalDragon · 2022-10-03 12:38

Quote from: Fred vS on October 02, 2022, 02:02:02 pmfpc dont like this
one too:

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i program
test; { I am a comment with this: { something... } beginend.
Result:

Quotefred@fred-80m0 ~> fpc test.pas
Free Pascal Compiler version 3.2.2 [2021/07/09] for x86_64
Copyright (c) 1993-2021 by Florian Klaempfl and others
Target OS: Linux for x86-64
Compiling test.pas
test.pas(3,29) Warning: Comment level 2 found
test.pas(6,5) Fatal: Unexpected end of file
Fatal: Compilation aborted
Error: /usr/bin/ppcx64 returned an error exitcode
That depends upon the mode. In modes FPC and ObjFPC FPC allows nested
comments, so you can have { foo { bar } blubb }, while in modes TP and
Delphi you need to alternate the comment style for this to compile: {
foo (* bar *) blubb }. This is why your code fails as the compiler
detects a second level comment and expects as many closing brackets to
finish of the comment(s).

----------------------------------------------------------------------
MarkMLl · 2022-10-03 13:00

Quote from: PascalDragon on October 03, 2022, 02:38:02 pmThat depends
upon the mode. In modes FPC and ObjFPC FPC allows nested comments, so
you can have { foo { bar } blubb }, while in modes TP and Delphi you
need to alternate the comment style for this to compile: { foo (* bar
*) blubb }. This is why your code fails as the compiler detects a
second level comment and expects as many closing brackets to finish of
the comment(s).

When you say "alternate" do you literally mean /alternate/ or is there
still a two-deep limit?

MarkMLl

----------------------------------------------------------------------
PascalDragon · 2022-10-05 21:06

Quote from: MarkMLl on October 03, 2022, 03:00:01 pmQuote from:
PascalDragon on October 03, 2022, 02:38:02 pmThat depends upon the
mode. In modes FPC and ObjFPC FPC allows nested comments, so you can
have { foo { bar } blubb }, while in modes TP and Delphi you need to
alternate the comment style for this to compile: { foo (* bar *) blubb
}. This is why your code fails as the compiler detects a second level
comment and expects as many closing brackets to finish of the
comment(s).

When you say "alternate" do you literally mean /alternate/ or is there
still a two-deep limit?

For both Delphi and modes Delphi and TP only two levels are supported.
With the full blown nested comments of the other modes you can have
arbitrary levels (well, I think there is some limit in the compiler).

----------------------------------------------------------------------
MarkMLl · 2022-10-06 07:29

Quote from: PascalDragon on October 05, 2022, 11:06:35 pmFor both
Delphi and modes Delphi and TP only two levels are supported. With the
full blown nested comments of the other modes you can have arbitrary
levels (well, I think there is some limit in the compiler).

Thanks for that. I'd remark that I've always found coding for Wirth's
later style (only (* *) comments) a right pain in the neck,
particularly if they could contain pragmata. It's... amusing that
Modula-2 repurposed braces but mandated that comments be nestable,
while allowing both single- and double-quotes on strings.

ALGOL-60 had multi-line comments at the parser level (everything
between COMMENT and ; ignored) and possibly at the lexer level
(single-line starting with % ). I'm not sure but I think that Ada
(hence VHDL etc.) only has single-line comments.

MarkMLl