TregExpr and unicode/cyrillic
lazarus / 21-General

----------------------------------------------------------------------
Jungle · 2023-12-13 11:40

I want to manipulate the case of a cyrillic text via regex. IIUC
according to the info on this page, Lazarus uses TRegExpr by Sorokin
which supports Unicode categories like "\p{L}".

When I use "\p{L}*" in my expression, I get the following error:
QuoteTRegExpr compile: incorrect {} braces (pos 4).
If I change "\p{L}*" to "\w*", search fails. If I define WordChars to
'абвгдежзиклмнопрстуфхцчшщьыъэюя', search succeedes but case change
fails.

So the following code works ok

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i procedure
TForm1.Button2Click(Sender: TObject);var  r: TRegExpr;  s:
string; begin  r := TRegExpr.Create();  try    s := 'hello';   
r.InputString := s;    r.Expression  := '(\w*)';     if ( r.Exec() )
then    begin      ShowMessage(r.Match[0]);      s := r.Replace(s,
'\u$1', true);      ShowMessage(s);    end;  finally    r.Free(); 
end;end;
The following doesn't work as expected.
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i procedure
TForm1.Button3Click(Sender: TObject);var  r: TRegExpr;  s:
String; begin  r := TRegExpr.Create();  try    s := 'привет';   
r.InputString := s;    r.Expression  := '(\w*)';//    r.Expression  :=
'(\p{L}*)'; // TRegExpr compile: incorrect {} braces (pos 4)   
r.WordChars   := 'абвгдежзиклмнопрстуфхцчшщьыъэюя';     if ( r.Exec )
then    begin      ShowMessage(r.Match[0]);      s := r.Replace(s,
'\u$1', true);      ShowMessage(s);    end;  finally    r.Free(); 
end;end; 
What is the proper way?

Lazarus 2.2.6 x64 on Win11

----------------------------------------------------------------------
Martin_fr · 2023-12-13 12:54

\p{} is afaik only supported if the TRegExpr was compiled to use
WideChar.

There is a define in the unit <FPC>\packages\regexpr\src\regexpr.pas
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i { off
$DEFINE UnicodeRE} // Use WideChar for characters and
UnicodeString/WideString for strings
Since it is part of the FPC distribution, the best way is to make a
copy, change the define, and recompile.

You then also need to convert your strings to widestring
"Utf8ToUtf16()".

----------------------------------------------------------------------
Jungle · 2023-12-13 13:10

Quote from: Martin_fr on December 13, 2023, 01:54:00 pmThere is a
define in the unit <FPC>\packages\regexpr\src\regexpr.pas
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i { off
$DEFINE UnicodeRE} // Use WideChar for characters and
UnicodeString/WideString for strings
I can't find this define there. But there're a lot of
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i {$IFDEF
UniCode}

----------------------------------------------------------------------
Martin_fr · 2023-12-13 13:28

It seems to have changed in FPC 3.3.1. (and in the origin project).

I  just saw there is a file "uregexp.pas" that may already provide the
correct version (not sure though)

If in doubt, the latest version is here
https://github.com/andgineer/TRegExpr

----------------------------------------------------------------------
Thaddy · 2023-12-13 13:34

Correct, but contrary to your previous remark the string type is
unicodestring, not widestring.