TDBEdit and extended characters
lazarus / 16-Databases

----------------------------------------------------------------------
nikolaou · 2009-06-28 12:45

I use TDBEdit to edit an ftString with length 40 from a dbf. My
problem is the following:

When I input English (standard ascii) characters it shows all 40
characters on TDBGrid and saves them to my field. When I use the Greek
keyboard to input, TDBGrid shows only 20 characters  and saves
accordingly to my dbf field. I can understand what caused this, since
extended ascii characters needs 2 bytes.     

Is there any way to bypass this problem (this is to allow 40 char
input on TDBEdit and save the same on the ftstring field) and what is
the TDBEdit EditMask property for extended characters?

Sorry for my GREnglish
Nikos. (An old dog trying to learn new tricks...)

----------------------------------------------------------------------
clauslack · 2009-06-28 15:01

You can check the Length of the string before the post.

Example:
ÑAÑEZ: has 5 UTF8 characters, but saves 7 charactes in the DBF.

with
Code: [Select]SHOWMESSAGE(inttostr(Length(DBEdit1.text)));//show 7
characters

you know how many characters are in the DBEdit

About EditMask, I don't probe anything.

Regards

----------------------------------------------------------------------
nikolaou · 2009-07-03 06:49

Thanks Clauslack for your reply but this does not solve my problem
which is the following:

My (win32) program is creating a dbf file and use it to store names
and addresses in Greek (could also be Bulgarian or Russian,,, etc).
So, in order to store a string with a length of 50 extended ascii
chars I will need an ftstring field with a length of 100. (...and if i
want to use this field as an index field? Too long!). If i use TDBEdit
to edit this field it will allow 100 characters input. This will make
the user to believe that he can actually saves this input but in
reality he will save as much as the half of it. (And thats the reason
I wonder if there is an appropriate TDBEdit EditMask property)

Does the TDBEdit needs some more working to cover cases like this or
I'm missing something (...which is very possible)?

Thanks, Nikos

----------------------------------------------------------------------
theo · 2009-07-03 07:25

This is a general problem. It is not lazarus specific.
Either you double or triple the field length or you convert between
utf-8 and CP1253 in your case. See unit LConvEncoding.

----------------------------------------------------------------------
tbebekis · 2024-05-03 00:08

Perhaps too late, but..

If you're going to have Greek and English characters in a TDataset
field then, I think, you should use the ftWideString field-type.

I had a similar problem as yours and the solution was to use
ftWideString.

PS. Strings can drive you crazy
  S := 'καλημέρα';   // goodmorning in Greek
  Len := Length(S);  // 8 in Delphi 11, 16 in Lazarus 3 (any mode)

----------------------------------------------------------------------
wp · 2024-05-03 09:58

Quote from: tbebekis on May 03, 2024, 02:08:04 amPS. Strings can drive
you crazy
  S := 'καλημέρα';   // goodmorning in Greek
  Len := Length(S);  // 8 in Delphi 11, 16 in Lazarus 3 (any mode)
No, try to understand UTF8:
https://wiki.freepascal.org/UTF8_strings_and_characters,
https://utf8everywhere.org/
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i uses 
LazUTF8;...var  S: String;  Len, LenUTF8: Integer;begin  S :=
'καλημέρα';   // goodmorning in Greek  Len := Length(S);  // 8 in
Delphi 11, 16 in Lazarus 3 (any mode)  LenUTF8 := UTF8Length(S);   //
8 in Lazarus 
You must accept the fact that in Unicode a "character" is not
identical with a code unit (byte in utf8, word in Delphi's UTF16). The
function Length() does not return the number of characters - no, it
returns the number of code units, in Lazarus bytes, in Delpi words. A
greek character consists of two bytes in UTF8 (Lazarus), but of one
word in UTF16 (Delphi). Therefore, many believe that Delphi has
resolved the Unicode issues because "a character consists of one code
unit". But this is wrong as the following example with a smiley
character '😏'shows:
Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i var  s:
String;  Len: Integer;  LenUTF8: Integer;begin  s := '😏';   //
Sorry - the forum software replaces the similey character by this code
sequence... Paste in the smiley from the text if you want to test. 
Len := Length(s);   // 2 in Delphi 11, 4 in Lazarus  LenUTF8 :=
UTF8Length(s);  // 1 in Lazarus

----------------------------------------------------------------------
tbebekis · 2024-05-03 23:00

@wp

thanks for your answer.

I think I understand the difference between a character and a code
point/code unit, quite well.

I faced that length problem while testing the TBufDataset and TDbf
datasets.

If I define a string field as

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i
Table.FieldDefs.Add('Field_Name', ftString, 10);
I expect it to have enough space for 10 characters. Not bytes.

And if I try to pass a non-English character, then I should get an
error or something. And not end up with a truncated string.

The Size property is different from the DataSize property for a
reason. Or not?

Anyway, that's why I like programming: you always have a problem to
solve.

----------------------------------------------------------------------
tbebekis · 2024-05-03 23:31

Digging more, I think the problem is that the TFieldDef constructor
uses the CP_ACP CodePage for the ftString, ftFixedChar and ftMemo
fields.

Using CP_UTF8 CodePage the Size, in characters, is respected again.

So to answer the original, and very old question, a possible solution
is to use a function like the following, when it comes to add an
ftString field.

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i function
AddStringField(Dataset: TDataset; FieldName: string; Size: Integer;
Required: Boolean = False; ReadOnly: Boolean = False):
TFieldDef;begin  Result := Dataset.FieldDefs.Add(FieldName, ftString,
Size, 0, Required, ReadOnly, Dataset.FieldDefs.Count + 1, CP_UTF8
);end;

----------------------------------------------------------------------
cdbc · 2024-05-04 05:31

Hi
Wouldn't it be possible to use 'ftWideString' instead of 'ftString'?!?
Regards Benny