The problem of splitting mixed strings from delphi to lazarus Chinese characters lazarus / 21-General ---------------------------------------------------------------------- jianwt · 2023-01-15 06:46 A piece of code transferred from delphi can work normally in delphi, but the code cannot work normally after being transferred to lazarus. When splitting a mixed string of Chinese characters, letters and numbers, there are problems such as inaccurate splitting and garbled code. Novice can't correct the error by themselves, please help us, thank you! I translated it with Google. The language is not necessarily accurate. For details, please see the simple DEMO and renderings. ---------------------------------------------------------------------- KodeZwerg · 2023-01-15 08:06 Is it this you was asking for? Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i function CreateStringSplit(const AString: string; const ACharsPerLine: Integer; var AOutput: TStringList): Boolean;var  i: Integer;  s: string;begin  if AOutput = nil then    AOutput := TStringList.Create;  AOutput.Clear;  s := '';  for i := 1 to Length(AString) do    begin    if ((Ord(AString[i]) > 31) and (Ord(AString[i]) <> 127)) then      s := s + AString[i]        else        s := s + ' ';      if (Length(s) = ACharsPerLine) then        begin          AOutput.Add(s);          s := '';        end;    end;  if (s <> '') then    AOutput.Add(s);  Result := (AOutput.Count > 0);end;Call it like: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i procedure TForm1.BitBtn1Click(Sender: TObject);var sl: TStringList; ST:string; i: Integer;begin  sl := nil;  ST:='你从字面上理解:1个字 节是8bit,每一个bit代表点阵中的一个点。16x16点阵,就是1行有16个点,共16行。那么,一行16个点,正好可以用16bit来表示, 正好是2个字节。16行x2=32个字节。'+#13#10+'一个汉字2个字节,这是编码,通过这个编码,可以计算出它的存储位置,告诉你某个汉字 的存储位置的。';  if CreateStringSplit(ST, 50, sl) then    for i := 0 to Pred(sl.Count) do      Memo1.Lines.Add(sl[i]);  sl.Free;end;I hope it helps. Good luck! ---------------------------------------------------------------------- PierceNg · 2023-01-15 08:21 This version of unit1.pas works on my computer. Changes: - mode set to DelphiUnicode - where you set sDest to a string of 4 spaces, I changed to setting to empty string - for sDest length test in line 72 and 81, I changed from "Length(AnsiString(sDest))" to "Length(sDest)" Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i unit Unit1; {$MODE DelphiUnicode} interface uses Classes, SysUtils, Forms, Controls, Graphics, Dialogs, StdCtrls,Buttons;type  TForm1 = class(TForm)    Memo1: TMemo;    BitBtn1: TBitBtn;    procedure BitBtn1Click(Sender: TObject);  private    { Private declarations }  public    { Public declarations }  end; var  Form1: TForm1; implementation {$R *.lfm}   {str:String to be split i_gs: Number of split     gbk Each Chinese character occupies 2 bytes  demo:length(utf8toansi(AnsiToUtf8('附件123')))=7; }function splittext(str: String; i_gs: Integer): TStrings; // Split into equal- width stringsvar  iPos: Integer;  sSource: String;  sDest: string;  iChar: Integer;  list: TStrings;  // iDestLen  : Integer;begin  sSource := Trim(str);  iPos := 1;  sDest := '';  // iDestLen  := 4;  list := TStringList.Create;  // list.Clear;  while iPos <= Length(sSource) do      //length(utf8toansi(AnsiToUtf8('附件123')))=7;  begin    iChar := Ord(sSource[iPos]);    if (iChar >= $4E00) then // // Judge whether it is Chinese characters  判断是汉字    begin      sDest := sDest + sSource[iPos];      iPos := iPos + 1;      // Inc(iDestLen,2);    end    else    begin      if iChar = 13 then      begin        list.Add(sDest);        sDest := '';        // iDestLen  := 4;        iPos := iPos + 2;      end      else      begin        sDest := sDest + sSource[iPos];        iPos := iPos + 1;        // Inc(iDestLen,1);      end;     end;    //if Length(AnsiString(sDest)) >= i_gs then    //Replacing Unicodestring doesn't work    if Length(sDest) >= i_gs then    //Replacing Unicodestring doesn't work  begin      list.Add(sDest);      sDest := '';      // iDestLen  := 0;    end;   end;   if Length(sDest) > 0 then   //Replacing Unicodestring doesn't work  //if Length(AnsiString(sDest)) > 0 then   //Replacing Unicodestring doesn't work  begin    list.Add(sDest);  end;  result := list;end;{A piece of code transferred from delphi can work normally in delphi, but the code cannot work normally after being transferred to lazarus.When splitting a mixed string of Chinese characters, letters and numbers, there are problems such as inaccurate splitting and garbled code. Novice can't correct the error by themselves, please help us, thank you!I translated it with Google. The language is not necessarily accurate. For details, please see the simple DEMO and renderings.}procedure TForm1.BitBtn1Click(Sender: TObject);var list: TStrings; ST:string;begin      ST:='你从字面上理解:1个字节是8b it,每一个bit代表点阵中的一个点。16x16点阵,就是1行有16个点,共16行。那么,一行16个点,正好可以用16bit来表示,正好是2 个字节。16行x2=32个字节。'+#13#10+'一个汉字2个字节,这是编码,通过这个编码,可以计算出它的存储位置,告诉你某个汉字的存储位 置的。';     list := TStringList.Create;    try     list:=splittext(st,50);   // 50 bytes per line, including empty bytes     Memo1.Lines:=list;     finally     list.Free;    end; end; end. ---------------------------------------------------------------------- PierceNg · 2023-01-15 08:37 Quote from: KodeZwerg on January 15, 2023, 09:06:26 amIs it this you was asking for? Code: [Select]  ST:='你从字面上理解:1个字节是8bit,每一个bit代表点阵中的一个点。16x16点阵,就是1行有16 个点,共16行。那么,一行16个点,正好可以用16bit来表示,正好 This ST assignment is actually 2 strings joined by #13#10. Scroll to the right to see. Your code does not appear to take care of the CRLF. ---------------------------------------------------------------------- KodeZwerg · 2023-01-15 09:20 Quote from: PierceNg on January 15, 2023, 09:37:32 amQuote from: KodeZwerg on January 15, 2023, 09:06:26 amIs it this you was asking for? Code: [Select]  ST:='你从字面上理解:1个字节是8bit,每一个bit代表点阵中的一个点。16x16点阵,就是1行有16 个点,共16行。那么,一行16个点,正好可以用16bit来表示,正好 This ST assignment is actually 2 strings joined by #13#10. Scroll to the right to see. Your code does not appear to take care of the CRLF. When asked for "I want to split a string at 50 chars", my solution gives 50 chars, no matter if there are linebreaks or whatever else. 50 chars is 50 chars. So I do filter out everything that is a control char and replace it with a space. If you want to have as result something that can be smaller than 50, just enable checking for linebreaks and react on them. ---------------------------------------------------------------------- jianwt · 2023-01-15 09:28 KodeZwerg @PierceNg @PierceNg Thank you for your answers. I have done all the experiments. The questions are shown as follows  1. Here are the renderings I asked for(Look at the arrow)。2 Also, do not change {$mode ObjFPC}{$H+} switch, can do code? ---------------------------------------------------------------------- KodeZwerg · 2023-01-15 10:06 I am not that familar with chinese letters so it is hard for me to count the real length. Can you check if the result is what you expect it should be? Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i uses  LazUTF8.... function CreateStringSplit(const AString: string; const ACharsPerLine: Integer; var AOutput: TStringList): Boolean;var  i: Integer;  s: string;begin  if AOutput = nil then    AOutput := TStringList.Create;  AOutput.Clear;  s := '';  for i := 1 to UTF8Length(AString) do    begin      s := s + UTF8Copy(AString, i, 1);      if (UTF8Length(s) = ACharsPerLine div SizeOf(WideChar)) then        begin          AOutput.Add(s);          s := '';        end;    end;  if (s <> '') then    AOutput.Add(s);  Result := (AOutput.Count > 0);end; (i turned off filtering) ---------------------------------------------------------------------- KodeZwerg · 2023-01-15 10:37 With the following modification, it should do what you asked for but I am still unsure about counting... Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i function CreateStringSplit(const AString: string; const ACharsPerLine: Integer; var AOutput: TStringList): Boolean;var  i: Integer;  s: string;  bytes, chars: Integer;begin  if AOutput = nil then    AOutput := TStringList.Create;  AOutput.Clear;  s := '';  bytes := 0;  chars := 0;  for i := 1 to UTF8Length(AString) do    begin      Inc(chars);    s := s + UTF8Copy(AString, i, 1);      bytes := bytes + UTF8CodepointSize(PChar(UTF8Copy(AString, i, 1)));      if (chars = ACharsPerLine) then        begin          AOutput.Add(s);          s := '';          bytes := 0;          chars := 0;        end;    end;  if (s <> '') then    AOutput.Add(s);  Result := (AOutput.Count > 0);end; ---------------------------------------------------------------------- KodeZwerg · 2023-01-15 10:54 I think your error is because of that inserted lineending and not outfiltering. I attached an image that also show me how many chars a line got, as you can see the size (50) is printed after each line, but because of your inserted lineending the counting continues on next line. ---------------------------------------------------------------------- jianwt · 2023-01-15 11:25 @KodeZwerg Thank you very much. This split is for printing the frxReport report. The effect of your segmentation is shown in Figure 1.(The red parts are all split incorrectly.) The correct printing effect after segmentation is shown in Figure 2. ---------------------------------------------------------------------- jianwt · 2023-01-15 11:40 Write a print DEMO. Take a look.  GBK code, a Chinese character accounts for two bytes. UTF-8 encoding is variable-length encoding, usually three bytes of kanji Question: 1. The width of each string is too different, so the split is not correct. 2. When a newline is encountered, there are not four blank characters before the first line. ---------------------------------------------------------------------- paweld · 2023-01-15 14:07 you need to check the width of the drawn text, not the number of characters in the text, because each glyph has a different width depending on the font used. ---------------------------------------------------------------------- jamie · 2023-01-15 14:53 Hmm, I don't fully understand the problem here? but here is something I just did, what am I missing? Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i procedure TForm1.Button1Click(Sender: TObject);Var  ST:String;  L:TStringList;  I,H:Integer;begin  L := TSTringList.Create;  ST:='你从字面上理解:1个字节是8bit,每一 个bit代表点阵中的一个点。16x16点阵,就是1行有16个点,共16行。那么,一行16个点,正好可以用16bit来表示,正好是2个字节。1 6行x2=32个字节。'+#13#10+'一个汉字2个字节,这是编码,通过这个编码,可以计算出它的存储位置,告诉你某个汉字的存储位置的。'; L.Text := St;  H:= 0;  For I := 0 To L.Count-1 do   Begin    Canvas.TextOut(0,H,L[I]);    H:= H+Canvas.Textheight(L[I]);   end;  L.Free;end; ---------------------------------------------------------------------- jianwt · 2023-01-15 15:18 @paweld Your approach is quite correct and the problem is solved. I wanted to think about using a canvas to hold strings, but it didn't work, you did it, thank you very much! I wish you peace!! ---------------------------------------------------------------------- wp · 2023-01-15 15:47 Since the string, in Lazarus, is utf8 encoded you must not iterate over it "char" by "char" (or byte by byte) because each UTF8 "character" can consist of up to 4 chars (bytes). Instead, use the nice string iterator of unit LazUnicode with which you can hop over the string in steps of UTF8 "characters": Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i var  ch: String;begin  for ch in ST do    ... Note that ch is not a "char" but a "string" since it represents all the bytes constituing each "character". While iterating over the string you add the current "ch" to a "line" string, and count the characters added. When the maximum count of characters per line is reached the "line" string is added to the result stringlist and reset to '' for the next line. A special handling may be required for the LineEnding (#13, #10, #13#10) which adds the currently accumulated line to stringlist even if the full line length has not yet been achieve. Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i procedure SplitText(Str:String; MaxCount: Integer; List: TStrings);var  ch, prevCh: String;  countInLine: Integer;  line: String;  i: Integer;   procedure StoreLine;  begin    List.Add(line);    line := '';    countInLine := 0;  end; begin  List.Clear;   line := '';  countInLine := 0;  prevCh := #0;  for ch in Str do  begin    if ch = #13 then    StoreLine()    else    if (ch = #10) and (prevCh <> #13) then      StoreLine()    else    begin      line := Line + ch;      inc(countInLine);      if CountInLine = MaxCount then        StoreLine();    end;    prevCh := ch;  end;  if line <> '' then    StoreLine();end; Please test the attached project. [P.S.] This code is essentially the same as the code posted by paweld. But pawelds code requires multiple scanning over the text to determine the utf8 codepoints while the LazUnicode string iterator does this on-the- fly. ---------------------------------------------------------------------- KodeZwerg · 2023-01-15 22:35 Here you get again my version and added a method for real linewidth calculations. This method you can call to get the biggest pixel width of your current string input. With that information you can easy calculate how much chars fit into one visual component by doing compontent.width div BiggestCharWidth = number of max chars per line, that is needed for the other method Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i function BiggestCharWidth(const AString: string; const AFont: TFont): Integer;var  Bmp: TBitmap;  i, LMax: Integer;  LChar: string;begin  Result := 0;  LMax := 0;  Bmp := TBitmap.Create;  try    Bmp.SetSize(1, 1);    Bmp.Canvas.Font.Assign(AFont);    for i := 1 to UTF8Length(AString) do      begin        LChar := UTF8Copy(AString, i, 1);        if Bmp.Canvas.TextWidth(LChar) > LMax then          LMax := Bmp.Canvas.TextWidth(LChar);      end;    Result := LMax;  finally    bmp.Free;  end;end;Here the main split method, with extended parameters. Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i function CreateStringSplit(const AString: string; const ACharsPerLine, ALeadingSpace: Integer; const AStripControlChars, AKeepLineBreaks: Boolean; var AOutput: TStringList): Boolean;var  i: Integer; // loop counter  s, sLead: string; // temp string that be inserted into output  LChar: string; // current char  chars: Integer; // those keep track about used space, in bytes and chars  CanAddChar, IsLineBreak: Boolean; // Bool that represents if we add a char or replace current with a spacebegin  // create object if non-existant, beware to nil it where-ever you have declared it(!)  if (AOutput = nil) then    AOutput := TStringList.Create;  // clear current list  AOutput.Clear;  // initialize basics  sLead := '';  for i := 0 to ALeadingSpace do    sLead := sLead + ' ';  s := sLead;  chars := Length(sLead);  // loop over full string  for i := 1 to UTF8Length(AString) do    begin      // increment current used chars      Inc(chars);      // initialize state      CanAddChar := True;      // get current char from string    LChar := UTF8Copy(AString, i, 1);      // check if its a control char      if (AStripControlChars or AKeepLineBreaks) then        begin          // initialize state          IsLineBreak := False;        // do we found a control char?          if (CanAddChar and ((Ord(LChar[1]) < 32) or (Ord(LChar[1]) = 127))) then            begin              // is it a carriage return (CR) ?              if (AKeepLineBreaks and (Ord(LChar[1]) = 13)) then                IsLineBreak := True;              // is it a linefeed (LF) ?  (if we already had CR, or one comes next, skip it)              if ((AKeepLineBreaks and (Ord(LChar[1]) = 10)) and ((Pred(i) > 0) and (Ord(UTF8Copy(AString, Pred(i), 1)[1]) <> 13)) and ((Succ(i) <= UTF8Length(AString)) and (Ord(UTF8Copy(AString, Succ(i), 1)[1]) <> 13))) then                IsLineBreak := True;              // forbid usage of current char              if AStripControlChars or IsLineBreak then                CanAddChar := False;            end;      end;      // we found a char to add      if CanAddChar then      begin          // add current char to the string          s := s + LChar;          // did we reached the limit?          if (chars >= ACharsPerLine) then            begin              // add current string to the string list              AOutput.Add(s);              // initialize basics              s := sLead;              chars := Length(s);            end;        end        // not allowed char found        else          // was the not allowed char a linebreak that we wanted to rescue?          if (AKeepLineBreaks and IsLineBreak) then            begin              // add current string to the string list              AOutput.Add(s);              // initialize basics              s := sLead;              chars := Length(s);            end            else              // replace control char with a space              s := s + ' ';    end;  // do not forget to append the last information to list  if (s <> '') then  AOutput.Add(s);  // exit method by telling a success state  Result := (AOutput.Count > 0);end; My new signature in short explained: AString = the main input string ACharsPerLine = how many chars per line ALeadingSpace = how much leading space AStripControlChars = remove all control characters AKeepLineBreaks = shall we keep linebreaks? AOutput: = a TStringList as output I hope you understand everything and be able to solve your problem now fully. ---------------------------------------------------------------------- jianwt · 2023-01-15 23:19 Thanks to the help of the above several netizens, the problem has been solved. Thank you very much.