TDbf and Encoding
lazarus / 16-Databases

----------------------------------------------------------------------
bigeno · 2013-11-22 08:44

I fight for a long time with encoding in dbf, please help. I can't get
correct Polish characters.

I use TableLevel 4. I try LanguageID as $23,  $C8, $00 (null)
strings : MyDbf.Fields.AsString:=   as UTF8, UTF8ToCP1250,
UTF8ToCP852, UTF8ToANSI

I checked all combinations and nothing, (Windows 8 PL), and open files
in ESRI (arsgis) or OpenOffice.

The only progress is LanguageID:=$23 and strings as UTF8ToANSI, then
in OpenOffice with opened as UTF-8 I see correct chars in data, but
not in FiledsDef.

----------------------------------------------------------------------
clauslack · 2013-11-22 12:06

What charset use your dbf files?

Regards

----------------------------------------------------------------------
BigChimp · 2013-11-22 12:34

UTF-8 or any other unicode format isn't supported in TDBF (though you
might be able to do something with raw bytes or something... I vaguely
remember there is an encoding none or something).

What FPC version are you using? You might try trunk/2.7.1 as it has
some fixes (you could use e.g. fpcup to do a parallel independent
install).

I suppose running Code: [Select]chcp on the command line gives
QuoteActive code page: 852
for you? Could you check?

Language ID $23 would then make sense, yes, and if your input text is
in utf8, I suppose UTF8ToCP852 or UTF8ToANSI should work...

----------------------------------------------------------------------
bigeno · 2013-11-22 16:33

You saved me, Big thanks BigChimp.
My active code page is 852
I use fpc 2.6 fixes

Now, when I set lang to $23 and use UTF8ToCP852 then I see correct
chars in Excel (just open), OpenOffice (as DOS 852), ArcGis (just
open).

But not in FieldDefs, maybe I put strings in wrong way ?
for FieldDefs I use Code:
[Select]MyDbf.FieldDefs.Add(UTF8ToCP852(s),ftString,40,false)for data
I use Code: [Select]MyDbf.Fields[i].AsString:=UTF8ToCP852(s)When I put
same polish chars in Field names and data and open in hexeditor then
the hex values are different.

and other question, when I use UTF8ToAnsi then I don't get correct
results. Why ? If my codepage is 852 then it should be the same ?!

----------------------------------------------------------------------
howardpc · 2013-11-22 17:23

You have to distinguish between fieldnames and field data.
Field names are normal Pascal identifiers, and so restricted to Ansi
characters 'A'..'Z', 'a'..'z' '_' and (provided it is not the first
character in the name) '0'..'9'. You can have a fieldname 'girl' or
'fille', or 'dziewczyna', but not a fieldname 'Mädchen'. So your
problem with fieldnames may not be one of encoding but of illegal
characters in the code (characters which would be perfectly OK if they
were data).
Encoding is an issue then mainly for data, not for names used in
Pascal programs, which are completely inflexible in this regard (at
least as far as FPC is concerned, I do not know if there are other
Pascal compilers that allow non-ansi characters in identifiers).
The Lazarus editor may encode Pascal identifiers and the rest of your
program code as UTF8. However, this does not matter, since the lower
UTF8 character range is identical to the earlier ANSI encoding.

----------------------------------------------------------------------
bigeno · 2013-11-22 18:01

Ok, that is not big problem, I can replace national characters with
ASCII equivalent for field names.
And I think I understand the difference between UTF8ToAnsi UTF8To852,
852 is for console but Windows GUI is 1250.