UTF-8 lazarus / 61-Beginners ---------------------------------------------------------------------- baltas99 · 2014-11-30 15:01 I have this command in my code Writeln(UTF8ToConsole('Το πραγραμμα εκτελεστηκε επιτυχως!Πατηστε οποιοδηποτε κουμπι για εξοδο')); The language is greek.I have attached FileUtils and it runs with no errors but in cmd i see ?? ????????? i mean i see question marks instead of Greek letters.When i changed command to  Writeln(UTF8ToConsole('Компилируйся сцуко)); Russian language it run again smoothly and in cmd i saw normal Russian text.Why greek doesnt work? PS i use windows8.1 professional 64bit with amd fx6300 processor ---------------------------------------------------------------------- engkin · 2014-11-30 16:21 Try this: Code: [Select]program project5; {$mode objfpc}{$H+} {$APPTYPE CONSOLE} {$codepage utf-8} uses   Windows; var   PreviousValue: UINT;   s: String; begin   PreviousValue := GetConsoleOutputCP;   //Using UTF8   if not SetConsoleOutputCP(CP_UTF8) then   begin     WriteLn('SetConsoleOutputCP(CP_UTF8) Failed!');     exit;   end;   S := 'Το πραγραμμα εκτελεστηκε επιτυχως!Πατηστε οποιοδηποτε κουμπι για εξοδο';   {$ifdef FPC_HAS_CPSTRING}  //<--- Based on your compiler   if GetTextCodePage(Output)<>StringCodePage(s) then     SetTextCodePage(Output, StringCodePage(s));   {$endif FPC_HAS_CPSTRING}   WriteLn(s);   SetConsoleOutputCP(PreviousValue); end. Edit: Sorry, forgot to answer your question: Quote from: baltas99 on November 30, 2014, 04:01:26 pmWhy greek doesnt work? By default your Windows uses some Russian-supported ANSI code page in console windows. The previous code uses UTF8. Alternatively you can use Greek-supported ANSI code page, but you'll have to convert your strings from UTF8 to CP_DOSGreek before passing them to the console. ---------------------------------------------------------------------- baltas99 · 2014-11-30 16:50 Thank you for your help.However i had had the same problem http://postimg.org/image/st3pztoet/ If i wil simply make it with writeln('GREEK TEXT') and run it in linux will it work? ---------------------------------------------------------------------- engkin · 2014-11-30 17:02 Does this work for you: Code: [Select]program project6; {$mode objfpc}{$H+} {$APPTYPE CONSOLE} {$codepage utf-8} uses   Windows; var   PreviousValue: UINT;   s: String;   c: DWORD; begin   PreviousValue := GetConsoleOutputCP;   //Using UTF8   if not SetConsoleOutputCP(CP_UTF8) then   begin     WriteLn('SetConsoleOutputCP(CP_UTF8) Failed!');     exit;   end;   S := 'Το πραγραμμα εκτελεστηκε επιτυχως!Πατηστε οποιοδηποτε κουμπι για εξοδο';   if not WriteFile(StdOutputHandle,s[1],Length(s),c,nil) then      WriteLn('WriteFile Failed! ');   {$ifdef FPC_HAS_CPSTRING}  //<--- Based on your compiler   if GetTextCodePage(Output)<>StringCodePage(s) then     SetTextCodePage(Output, StringCodePage(s));   {$endif FPC_HAS_CPSTRING}   //WriteLn(s);   SetConsoleOutputCP(PreviousValue); end. ---------------------------------------------------------------------- Never · 2014-11-30 17:02 This will work Code: [Select]program project_greek2; {$mode objfpc}{$H+}     var s:WideString;         s1:String;     const c='a - α';       const c1='α'; begin   s:='a  -  α';   s1:='a  -  α';   WriteLn('a - α');   WriteLn(s);   WriteLn(s1);   WriteLn(c);   WriteLn(c1);   WriteLn( 'α' );   WriteLn('==========================');   WriteLn(Utf8ToAnsi('a - α'));   WriteLn(Utf8ToAnsi(s));   WriteLn(Utf8ToAnsi(s1));   WriteLn(Utf8ToAnsi(c));   WriteLn(Utf8ToAnsi(c1));   WriteLn( Utf8ToAnsi('α') );   ReadLn; end. Edit***:is working if file encoding is utf-8 @engkin: some kind of double/triple conversion is done when code page is used ---------------------------------------------------------------------- baltas99 · 2014-11-30 17:12 Thank you again but it didnt work. http://postimg.org/image/gmc63xd4r/ . ---------------------------------------------------------------------- Never · 2014-11-30 17:16 do you have Greek instaled as a second input language? from your regional settings? ---------------------------------------------------------------------- Never · 2014-11-30 17:33 @engkin here the results from two snipets why is not working? (http://s16.postimg.org/mvm0qkyfl/engkin1.png) (http://s16.postimg.org/jq1f0dftd/engkin2.png) ---------------------------------------------------------------------- engkin · 2014-11-30 20:18 @Never, did you change the font? First you have to choose a font like Lucida Console. ---------------------------------------------------------------------- Never · 2014-11-30 20:30 @engkin yes i changed the fonts also but the efect is the same do you want me to upload screenshots for these also? ---------------------------------------------------------------------- engkin · 2014-11-30 21:01 Quote from: Never on November 30, 2014, 09:30:13 pmdo you want me to upload screenshots for these also? @Never, no. You already did. Can you try this code? Code: [Select]program project5; {$mode objfpc}{$H+} {$APPTYPE CONSOLE} {$codepage utf-8} uses   Windows; var   PreviousValue: UINT;   s: String;   procedure Test;   begin     WriteLn('GetConsoleOutputCP: ', GetConsoleOutputCP);     WriteLn('GetTextCodePage(Output): ', GetTextCodePage(Output));     WriteLn('StringCodePage(s): ', StringCodePage(s));   end; begin   s := 'a - α';   Test;   WriteLn('WriteLn: ',s);   WriteLn('');   PreviousValue := GetConsoleOutputCP;   //Using UTF8   if not SetConsoleOutputCP(CP_UTF8) then   begin     WriteLn('SetConsoleOutputCP(CP_UTF8) Failed!');     exit;   end;   Test;   WriteLn('WriteFile: ');   if not WriteFile(StdOutputHandle,s[1],Length(s),c,nil) then      WriteLn('WriteFile Failed! ');   WriteLn('');   WriteLn('');   if GetTextCodePage(Output)<>StringCodePage(s) then     SetTextCodePage(Output, StringCodePage(s));   Test;   WriteLn('WriteLn: ', s);   SetConsoleOutputCP(PreviousValue); end. Try it on a new console. ---------------------------------------------------------------------- Never · 2014-11-30 21:11 @engkin used another example of yours with Code: [Select] {$calling stdcall}   function GetConsoleOutputCP:UINT; external 'kernel32' name 'GetConsoleOutputCP';   function SetConsoleOutputCP(wCodePageID:UINT):WINBOOL; external 'kernel32' name 'SetConsoleOutputCP';   function WriteFile(hFile: THandle; const Buffer; nNumberOfBytesToWrite: DWORD; var lpNumberOfBytesWritten: DWORD; lpOverlapped: pointer): BOOL; external 'kernel32' name 'WriteFile';and worked !!! Edit***:and now is working with windows in uses too!! ---------------------------------------------------------------------- engkin · 2014-11-30 21:25 Try it on a new console. Edit: The only explanation I have now is that: 1- SetConsoleOutputCP from unit Windows is not working. <-- does not make sense. OR 2- CP_UTF8 holds a wrong value, not 65001. <-- does not make sense either. ---------------------------------------------------------------------- Never · 2014-11-30 21:36 can't find [ GetTextCodePage ] added lazutils and lazutf8 but still can't find them added this Code: [Select] type TSystemCodePage=word; ....  function GetTextCodePage(var T: Text): TSystemCodePage;   begin   {$if defined(FPC_HAS_CPSTRING) and defined(FPC_HAS_FEATURE_ANSISTRINGS)}   GetTextCodePage:=TextRec(T).CodePage;   {$else}   GetTextCodePage:=0;   {$endif}   end;  and for [ StringCodePage ] didn't find something more except function StringCodePage(const S : RawByteString): TSystemCodePage; overload; StringCodePage<<--- is used in lazutils/lazutf8 missing files from my instalation? searched lazarus instalation directory with find in files app ---------------------------------------------------------------------- Bart · 2014-11-30 21:47 I gues it's in fpc trunk? Bart ---------------------------------------------------------------------- Never · 2014-11-30 21:55 fund them in CT  fpc sources and here is the result ---------------------------------------------------------------------- engkin · 2014-11-30 23:18 @Never Baltas99's system is using Russian CP, Your system is using Greek CP The image you posted above shows that FPC is converting UTF8 string to your Greek CP. The other day Marco mentioned DefaultSystemCodePage. I assume DefaultSystemCodePage on your system is 737. If you add: Code: [Select]  DefaultSystemCodePage := CP_UTF8; to the previous code it should work. Can you try? ---------------------------------------------------------------------- Never · 2014-11-30 23:25 i test it with CT and faild but  Code: [Select] DefaultSystemCodePage ; is not avaliable with lazarus ---------------------------------------------------------------------- engkin · 2014-12-01 17:22 @Never, What font did you use in your last two images you posted? On my system: when I use raster font the zero has a slash, as in my first attachment. when I use Lucida Console font there is no slash, as in my second attachement. ---------------------------------------------------------------------- baltas99 · 2014-12-04 14:07 Quote from: engkin on December 01, 2014, 12:18:30 am@Never Baltas99's system is using Russian CP, Your system is using Greek CP The image you posted above shows that FPC is converting UTF8 string to your Greek CP. The other day Marco mentioned DefaultSystemCodePage. I assume DefaultSystemCodePage on your system is 737. If you add: Code: [Select]  DefaultSystemCodePage := CP_UTF8; to the previous code it should work. Can you try? Thank you but why i have an Error:Identifier not found "GetTextCodePage".Do i have to add an reqirment to project inspector? ---------------------------------------------------------------------- baltas99 · 2014-12-04 14:16 Quote from: baltas99 on December 04, 2014, 03:07:58 pmQuote from: engkin on December 01, 2014, 12:18:30 am@Never Baltas99's system is using Russian CP, Your system is using Greek CP The image you posted above shows that FPC is converting UTF8 string to your Greek CP. The other day Marco mentioned DefaultSystemCodePage. I assume DefaultSystemCodePage on your system is 737. If you add: Code: [Select]  DefaultSystemCodePage := CP_UTF8; to the previous code it should work. Can you try? Thank you but why i have an Error:Identifier not found "GetTextCodePage".Do i have to add an reqirment to project inspector? It finally worked by just writing Writeln(UTF8ToConsole('Τα δεδομενα που εισαγατε ειναι λανθασμενα.')); when i chaneged my lanuage for non unicode programmes to greek. ---------------------------------------------------------------------- Never · 2014-12-04 16:37 @baltas99 glad you made it @engkin excuse me i didn't see your response earlier Quote@Never, What font did you use in your last two images you posted?i can't remember which one is in the picture but none of them was working the pictures are a screenshot of the last i tried. Usually i try fonts in the console from top to bottom so most probably is to be raster if this is important i can dig to re-execute ---------------------------------------------------------------------- engkin · 2014-12-04 21:02 @Never,  it's ok. I think we are done with this thread for now. @baltas99 Well done, just be aware that you can't mix languages if you stick to ANSI solution.