Extreme Slowdown in FPC 3.0.x versus 2.6.4, DBGrid Sqlite3 master- detail lazarus / 16-Databases ---------------------------------------------------------------------- jbthiel · 2020-11-15 03:49 I am working to migrate an app to newer FPC 3.0.x and Lazarus 2.0.x From prior FPC 2.6.4, and Lazarus versions 1.2.x, 1.4.x, 1.6.x, 1.8.x on Linux GTK2 Testing described below is all with Lazarus 1.8.4 which compiles with both FPC 2.6.4 and 3.0.4 App design is master-detail DBGrids, on SQLite v3.8.5 database, with associated SQLConnections, Datasets, etc. PROBLEM: With FPC 3.0.4, master-detail DBGrid is extremely much slower, the app is unusable. To illustrate: Cursor Next Line (Down arrow) in a master DBGrid, the cursor should move and detail grid update. With FPC 2.6.4 is instantaneous. With FPC 3.0.4 this takes about 1+ second. During the delay, it's 100% usage on a single cpu, and the whole app is frozen, menus don't work, etc. When it comes back, the cursor moves and detail grid updates, simultaneously (as expected), and correct data is displayed. Does not seem to be dependent on how many records are in the detail. Also, I noticed if you key Next Line 2x, the delay will be 2x, and the final update will show the 2nd line down. and likewise 3x, 4x, etc.   Also note PgDown takes same amount of time as NextLine. I have done some valgrind Profiling, which does not show obvious runaway functions that I can distinguish from normal operation. It's basically all idle loop and handle message, context dispatch, widget events, alloc, free, etc. I have 2 suspect areas so far: 1) Possible Bug(s) in FCL event handlers, spurious/repeated events, exceptions, etc. 2) Comparing execution profile of FPC 304 with 264, I see considerably more ANSISTRING manipulations. In particular, these stand out, are not in FPC 264: Quote  ( 4.06%)  ???:SYSUTILS_$$_INTERNALCHANGECASE$ANSISTRING$TSYSCHA RSET$LONGINT$$ANSISTRING   ( 1.55%)  ???:fpc_ansistr_compare_equal   ( 1.48%)  ???:CWSTRING_$$_UPPERANSISTRING$ANSISTRING$$ANSISTRING Instead of the last one above, FPC 264 has   ( 1.51%)  ???:SYSUTILS_UPPERCASE$ANSISTRING$$ANSISTRING I noticed there are many changes in the FCL Sqlite code for 3.0.4, like sqlite3conn.pp and sqlite3.inc, and all Sqlite external function declarations with pchar were changed to pansistring. Type PChar = PAnsiString, so it shouldn't matter, but maybe that reflects now ubiquitous Ansistrings in the FCL Sqlite internals, with lots more heap activity. Also AnsiStrings now have CodePage in FPC 3. Do I need to make some settings to account for this? I'm unfamiliar with codepage and basically ignoring it. Sqlite has the unusual feature of being string-based at some level (ie. dates are strings; you can store a string in a "numeric" field; etc), so if Sqlite FCL 3.0 is now using heap-based Codepage AnsiStrings for all these tiny field strings, (that were formerly stack-based bytestrings?) that would be quite some overhead (and distributed across the whole program, instead of one specific rogue function -- which is what I see). QUESTION is two-fold: 1) What is going wrong, what is the bug, what changed in FPC or FCL 3.0.x that could be causing such extreme performance drop?   (also, should I upgrade my Sqlite version, from 3.8.5, which is somewhat older) 2) What debugging techniques can be used for this kind of problem? How can I find and measure effects like spurious/repeated events, excessive ansistrings, etc? The results are all "correct". If I catch specific events and step thru the debugger, each step basically happens "fast". Just that overall it is 10x or 100x slower. How can I find what it's doing during these 1+ second freezes?  like profile exactly all function calls between key NextLine until it refreshes all grid displays. I am planning to post this in bugtracker as well, inquiring here to see if a known issue, or any ideas. Especially also any discussion in regards specialized techniques for debugging. Thanks for any suggestions. My app goes from high performance on FPC 2.6.4 -- you can NextLine scroll the master grid at key repeat rate, and it will live scroll with the detail grid keeping up -- to completely unusable on FPC 3.0.4. This also means I cannot upgrade to Lazarus 2.x series. ---------------------------------------------------------------------- PascalDragon · 2020-11-15 11:12 Quote from: jbthiel on November 15, 2020, 04:49:08 amI noticed there are many changes in the FCL Sqlite code for 3.0.4, like sqlite3conn.pp and sqlite3.inc, and all Sqlite external function declarations with pchar were changed to pansistring. Type PChar = PAnsiString, so it shouldn't matter, but maybe that reflects now ubiquitous Ansistrings in the FCL Sqlite internals, with lots more heap activity. Just to clarify: they were changed from PChar to PAnsiChar, not PAnsiString. The former is indeed the same as PChar for now (this would change if PChar would become an alias to PWideChar) while the latter would be a pointer to a AnsiString which is something that is much more than a PChar and would essentially lead to the import unit not working at all. ---------------------------------------------------------------------- marcov · 2020-11-15 11:49 The FPC 3 series has gained unicode support. If the code you use (either yours or in the database components that you use) does something with already a bad performance (like O(n^2) comparing of large amounts of strings) this could suddenly become noticeable. ---------------------------------------------------------------------- jbthiel · 2020-11-16 03:37 Thanks for this clarification, and note about Unicode.  Quite right, PChar = PAnsiChar, that's what I meant. I had seen a couple new AnsiStrings and all the pansi... (char) changed declarations flying by in my diffs and didn't register it was separate stuff. So FCL 3  Sqlite is probably still using Shortstrings when possible, just with AnsiChar elements instead of Char. That's good, then it's hopefully not a systemic AnsiString problem. I rather suspect some event(s) are unnecessarily repeated;  I have seen that kind of bug before. Or maybe a timeout because some signal is not arriving. The correct normal computations just cannot be this slow, even with widespread AnsiString/Codepage/Unicode string processing. I would think the extra "real-world" overhead from using AnsiString instead of Shortstring is maybe 2% or 5% or 10%,  certainly not 10x to 100x. I don't know how to trace/profile down the entire DBGrid- SQLConnection-Dataset event stack, especially when there is no wrong data to find. And it's not easy to recognize the kind of multiple- event bug I think it could be. If anyone has some profiler tips, I will be glad to try them out. I will update my report for the bugtracker.