System codepage related Issue with Lazarus 1.6 RC1 lazarus / 16-Databases ---------------------------------------------------------------------- Mladen · 2015-12-13 11:15 When I updated from Lazarus 1.4.4/PFC 2.6.4 to Lazarus 1.6 RC1/FPC 3.0 i have experienced issues with DBGrid sorting.(LATIN characters) For example names starting with these letters were sorted: A B C Č Ć D Dž Đ E . . . V Z Ž And now they are like: A B C D E . . . Š Ž Č Ć Đ My database character set is ANSI with codepage WIN1250 I was using In memory database sorting with procedure below: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i     procedure TSQLForm.ResultsGridTitleClick(Column: TColumn);    const    ImageArrowUp=0; //should match image in imagelist      ImageArrowDown=1; //should match image in imagelist    var      ASC_IndexName, DESC_IndexName:string;      procedure UpdateIndexes;    begin        // Ensure index defs are up to date        OpenQuery.IndexDefs.Updated:=false; {<<<--This line is critical. IndexDefs.Update will not        update if already true, which will happen on the first column sorted.}        Openquery.IndexDefs.Update;      end;    begin      ASC_IndexName:='ASC_'+Column.FieldName;      DESC_IndexName:='DESC_'+Column.FieldName;      // indexes can't sort binary types such as ftMemo, ftBLOB      if (Column.Field.DataType in [ftBLOB,ftMemo,ftWideMemo]) then        exit;      // check if an ascending index already exists for this column.      // if not, create one      if OpenQuery.IndexDefs.IndexOf(ASC_IndexName) = -1 then      begin        OpenQuery.AddIndex(ASC_IndexName,column.FieldName,[]);      UpdateIndexes; //ensure index defs are up to date      end;      // Check if a descending index already exists for this column      // if not, create one      if OpenQuery.IndexDefs.IndexOf(DESC_IndexName) = -1 then      begin        OpenQuery.AddIndex(DESC_IndexName,column.FieldName,[ixDescending]);      UpdateIndexes; //ensure index defs are up to date      end;         // Use the column tag to toggle ASC/DESC      column.tag := not column.tag;      if boolean(column.tag) then      begin        Column.Title.ImageIndex:=ImageArrowUp;        Openquery.IndexName:=ASC_IndexName;      end      else      begin      Column.Title.ImageIndex:=ImageArrowDown;        OpenQuery.IndexName:=DESC_IndexName;      end;      // Remove the sort arrow from the previous column we sorted      if (FLastColumn <> nil) and (FlastColumn <> Column) then        FLastColumn.Title.ImageIndex:=-1;      FLastColumn:=column;    end; The procedure was being called by DbGridTitleOnClick event. Cheers! ---------------------------------------------------------------------- JuhaManninen · 2015-12-13 13:45 Your code depends on system codepage on Windows. This is a situation where our new UTF-8 system breaks old code. You can still use the FPC's default system codepage by defining "DisableUTF8RTL". This wiki page explains details:   http://wiki.freepascal.org/Lazarus_with_FPC3.0_without_UTF-8_mode User Michl and others have fixed bugs in LazUtils and LCL related to DisableUTF8RTL. The wiki page however would need more practical examples. The page mentions SysToUTF8(). You will need explicit conversions just like was needed earlier with LCL + FPC 2.6.x. There are 3 ways to go when your code depends heavily on Windows system codepage. 1. Isolate all codepage dependent input/output code into functions. All codepage conversions are done inside them and UTF-8 is used everywhere else. This is the best choice for most cases (I guess). 2. Use DisableUTF8RTL and do explicit conversions when needed around the code. 3. Continue using FPC 2.6.4. The plan is to keep latest lazarus 1.6 and trunk compatible with it. Then you still get the nice new IDE features when using FPC 2.6.4. This is a good choice for code which is in maintenance mode. For new development it is not as good because FPC 3.0 brings so many improvements. You code only sets DB indexes. Where is the actual string comparison done? If the external DB engine does it, your code should still work. I am not a DB expert, nor a Windows expert. I hope others can help here. I hope more examples are added to the wiki page, too. ---------------------------------------------------------------------- Mladen · 2015-12-13 14:42 Quote from: JuhaManninen on December 13, 2015, 02:45:03 pmYour code depends on system codepage on Windows. This is a situation where our new UTF-8 system breaks old code. You can still use the FPC's default system codepage by defining "DisableUTF8RTL". This wiki page explains details:   http://wiki.freepascal.org/Lazarus_with_FPC3.0_without_UTF-8_mode User Michl and others have fixed bugs in LazUtils and LCL related to DisableUTF8RTL. The wiki page however would need more practical examples. The page mentions SysToUTF8(). You will need explicit conversions just like was needed earlier with LCL + FPC 2.6.x. There are 3 ways to go when your code depends heavily on Windows system codepage. 1. Isolate all codepage dependent input/output code into functions. All codepage conversions are done inside them and UTF-8 is used everywhere else. This is the best choice for most cases (I guess). 2. Use DisableUTF8RTL and do explicit conversions when needed around the code. 3. Continue using FPC 2.6.4. The plan is to keep latest lazarus 1.6 and trunk compatible with it. Then you still get the nice new IDE features when using FPC 2.6.4. This is a good choice for code which is in maintenance mode. For new development it is not as good because FPC 3.0 brings so many improvements. You code only sets DB indexes. Where is the actual string comparison done? If the external DB engine does it, your code should still work. I am not a DB expert, nor a Windows expert. I hope others can help here. I hope more examples are added to the wiki page, too. Thank you very much for your help. I have used 2nd option as you suggested and it worked for me. activate the -dDisableUTF8RTL mode with Project Options ... -> Compiler Options -> Additions and Overrides -> click on Use system encoding I will keep exploring. Cheers!