[Solved] Zeos adding an extra #0 at the end of a UTF8 string
lazarus / 16-Databases

----------------------------------------------------------------------
JohnvdWaeter · 2024-02-13 09:10

Hi All,
Banging my head over this...
In my firebird 2.5 database all varchar fields are UTF-8.
The DB connection is UTF-8.
I use Zeos components in a console/server application in Free Pascal.
The ZConnection has all properties set to UTF-8.

At this application, a JSON arrives with a text containing UTF8 chars.
This should be posted into a FB table via a Zeos zquery that calls an
SP.
The text arrives properly at the application, checked by
writeln(atext);
Then I want to put it in the zQuery's parameter:

params.parambyname('atext').asString:=atext;
wrong result, garbage text where special characters are. But I thought
all Free Pascal strings {$H+} are now UTF8?

Ok, next try:

params.parambyname('atext').asUnicodeString:=atext; Allmost good, it
appears nicely in the table (checked by IBExpert) BUT it adds a chr(0)
at the end.

I can't figure out where the #0 comes from...

----------------------------------------------------------------------
zeljko · 2024-02-13 10:31

What is ZEOS version ? I don't have such problems here.

----------------------------------------------------------------------
marcov · 2024-02-13 10:35

OS?   On Windows UTF8 is only default for Lazarus GUI apps to my
knowledge.

----------------------------------------------------------------------
cdbc · 2024-02-13 10:57

Hi
The extra #0 comes from the double #0#0 terminator in PWideChar...
methinks...
Regards Benny

----------------------------------------------------------------------
Thaddy · 2024-02-13 12:15

Quote from: JohnvdWaeter on February 13, 2024, 10:10:12 amBut I
thought all Free Pascal strings {$H+} are now UTF8?
That is only true for Lazarus applications. In FPC the stringtype is
Ansi, not UTF8. (Is your app a console app?)
The extra zero comes from Widestring/PWideChar indeed.

----------------------------------------------------------------------
tonyw · 2024-02-13 12:37

You have fallen foul of this bug

https://gitlab.com/freepascal.org/fpc/source/-/issues/39746

I came across it when testing IBX and added the following workaround

Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i function
SafeAnsiUpperCase(const s: AnsiString): AnsiString;2194    var i:
integer;2195    begin2196      {Guard against string length being
increased by 1 and the trailing zero2197       being incorporated in
the string - see2198      
https://gitlab.com/freepascal.org/fpc/source/-/issues/39746}2199    
 Result := AnsiUpperCase(s);2200      i := length(Result);2201    
 while (i > 0) and (Result[i] = #0) do Dec(i);2202    
 SetLength(Result,i);2203    end; 
The line numbers are from IBUtils.pas. As the name suggests, it has to
be called instead of AnsiUpperCase and applies when international
character sets are enabled.

----------------------------------------------------------------------
JohnvdWaeter · 2024-02-14 02:44

It is a console/server application for Linux (Debian)
Lazarus version = 2.12, FPC 3.2.0, running on Linux
Zeos version 7.2.14

Your responses put me on the right track!
The solution was to know what to Google for...

Simply putting LazUTF8 in the main program units clause and getting
rid of all utf8strings, widestrings, ansistrings, unicodestrings etc
and replacing them with just String did the job!

Thanks