Accented characters in concatenated strings
lazarus / 62-General

----------------------------------------------------------------------
Roland57 · 2016-07-19 18:58

Hello!

I try to display accented characters in a Windows console program.
When the string is concatenated to another, the accented characters
isn't displayed correctly. Why ?

Here is my code. I compiled it with FPC 3.0.0.

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i begin 
WriteLn('r'#130'pertoire');  WriteLn('r'#130'pertoir' + 'e');end.

----------------------------------------------------------------------
engkin · 2016-07-19 19:10

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i {$codepage
UTF8} begin  WriteLn('répertoire');  WriteLn('répertoir' + 'e');end.

----------------------------------------------------------------------
marcov · 2016-07-19 19:11

Prints two times the same here. (fpc 3.0 on the win32 console)

possible reasons:
- settings relating to sourcecode encoding
- some lazarus feature that changes the default encoding.

----------------------------------------------------------------------
engkin · 2016-07-19 20:01

The first one is written using fpc_write_text_shortstr, no codepage
conversion is involved. You'll see the correct letter if your console
output codepage has é for #130.

The second one is written using fpc_Write_Text_AnsiStr, a codepage
conversion happens from DefaultSystemCodePage to
TextRec(Output).CodePage:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i  
WriteLn('DefaultSystemCodePage: ', DefaultSystemCodePage); 
WriteLn('TextRec(Output).CodePage: ', TextRec(Output).CodePage);
This conversion could corrupt the letter based on the two codepages.

----------------------------------------------------------------------
Roland57 · 2016-07-19 20:12

@engkin, marcov

Thank you for your answers.

Indeed, with {$codepage UTF8} the result is correct. (The source code
encoding is UTF-8 without BOM.)

But I get a warning: Implicit string type conversion from "AnsiString"
to "UnicodeString". It isn't really a problem (since the result is
correct), but I wonder what I should write to avoid the warning.

Here is my full code:

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i program
touchdirectory;{$codepage UTF8} uses  SysUtils, DateUtils,
Process; const  TOUCH = 'C:\BCC101\bin\touch.exe';//
https://www.embarcadero.com/fr/free-tools/ccompiler var  year, month,
day, hour, minute, second, millisecond: word;  stamp, path:
string; begin  if (ParamCount = 1) and DirectoryExists(ParamStr(1))
then  begin    path := ParamStr(1) + '\';    WriteLn('Traitement du
répertoire "' + path + '".');  end else  begin    path := '';   
WriteLn('Traitement du répertoire courant.');  end;   
DecodeDateTime(Now(), year, month, day, hour, minute, second,
millisecond);  stamp := Format('%0.2d%0.2d%0.2d%0.2d%0.2d', [month,
day, hour, 0, year mod 100]);    with TProcess.Create(nil) do  begin 
 Executable := TOUCH;    Parameters.Add('-d' + stamp);   
Parameters.Add('-D');    Parameters.Add('-s');   
Parameters.Add('-v');    Parameters.Add(path + '*.*');    Options :=
Options + [poWaitOnExit];    Execute;    Free;  end;    Write('Appuyez
sur Entrée pour continuer... ');  ReadLn;end.

----------------------------------------------------------------------
Roland57 · 2016-07-19 20:16

Quote from: engkin on July 19, 2016, 10:01:03 pmThe first one is
written using fpc_write_text_shortstr, no codepage conversion is
involved. You'll see the correct letter if your console output
codepage has é for #130.

The second one is written using fpc_Write_Text_AnsiStr, a codepage
conversion happens from DefaultSystemCodePage to
TextRec(Output).CodePage:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i  
WriteLn('DefaultSystemCodePage: ', DefaultSystemCodePage); 
WriteLn('TextRec(Output).CodePage: ', TextRec(Output).CodePage);
This conversion could corrupt the letter based on the two codepages.

Thank you for the explanations and for the code.

----------------------------------------------------------------------
engkin · 2016-07-19 20:32

Quote from: RolandC on July 19, 2016, 10:12:13 pmBut I get a warning:
Implicit string type conversion from "AnsiString" to "UnicodeString".
It isn't really a problem (since the result is correct), but I wonder
what I should write to avoid the warning.

Assuming you are using Lazarus, I know of two possibilities:

1-Hide the warning in this specific instance, using Lazarus IDE
directive {%H-} at the location the warning is generated. In this case
exactly to the left side of the variable path without leaving a space:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i    
WriteLn('Traitement du répertoire "' + {%H-}path + '".');
2-Hide all instances by passing -vm4104 to the compiler in the project
options.

You'll see both if you right-click on that warning.

----------------------------------------------------------------------
Roland57 · 2016-07-19 20:45

@engkin

I learn many things today! Thank you.

----------------------------------------------------------------------
Tomas Hajny · 2016-08-04 08:04

Note that the warning about implicit conversions may be avoided also
by either using an explicit conversion (like: Ansistring
('répertoire')), or by not mixing UnicodeString (your constant) and
ansistring (your variables). Both are probably better solutions than
just hiding the warning.