ZEOSLIB increasing memory usage inside TThread lazarus / 16-Databases ---------------------------------------------------------------------- nmra · 2022-07-21 19:34 Hello all, I tried to run a SELECT SQL to retrieve some data from MySql database. For this, I had created a TZQuery inside a TThreadn to avoid freezing the GUI of the application. But, after the end of each consult (tested with 100,000+ rows, just to "force" a freeze) the memory was increasing instead of returning to the initial values (even after FreeAndNil(ZQuery) and FreeAndNil(TThread)) increasing ~2mb to each query executed (verified using Windows taskman.) The test: 1 - Click button1 to start thread and query 2 - Wait callback 3 - Click button2 to finish and free query Code: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i     { TForm }        procedure TForm1.Button1Click(Sender: TObject);    begin    //DBQuery: TSubThreadTest;      DBQuery:= TSubThreadTest.Create(SQLStatement, true, ZCon, @onQueryCallBack);    DBQuery.Start;    end;        procedure TForm1.Button2Click(Sender: TObject);    begin      FreeAndNil(DBQuery);      //Tried using WaitFor + Free, but not difference.    end;        procedure TForm1.onQueryCallBack(fail: boolean; query: TZQuery);    begin      lLastExec.Caption:= FormatDateTime('hh:mm:ss', Now);    end;         { TSubThreadTest }        constructor TSubThreadTest.Create(SQLStatement: string; toExec: boolean;      connector: TZConnection; EOnConclude: TSubThreadTestCallBack);    begin      inherited Create(true);          //main: TZQuery;          main:= TZQuery.Create(nil);      main.Connection:= connector;      main.SQL.Text:= SQLStatement;          isToExec:= toExec;          OnConclude:= EOnConclude;          FreeOnTerminate:= False;    end;          procedure TSubThreadTest.callback;    begin      if assigned(OnConclude) then        OnConclude(true, main);    end;      procedure TSubThreadTest.Execute;    begin      if isToExec then      main.ExecSQL      else        main.Open;          Synchronize(@callback);          FreeAndNil(main);    end;  First thread create and call: 24mb memory usage Second thread create and call: 26,5mb memory usage Etc... ---------------------------------------------------------------------- malcome · 2022-07-22 10:33 So what? Mysql is waiting your next order with the cache thing, i suppose. ---------------------------------------------------------------------- marcov · 2022-07-22 11:17 This is normal. If you allocate large blocks, the memory will be returned to the OS on deallocated. Smaller allocations are pooled and reused, and that is probably what tdataset/zeos does. ---------------------------------------------------------------------- nmra · 2022-07-22 11:59 Thanks for the replies. But, this don't cause memory leak after? ---------------------------------------------------------------------- malcome · 2022-07-22 19:37 You have some ways. Stop using TthreadStop using ZeosStop using MysqlStop using LazarusIf you find a person who caused memory leak and report it, then lots of users say you thanks! ---------------------------------------------------------------------- rvk · 2022-07-22 19:55 There isn't really a leak. There is only a leak if it is reported when you enable heaptrc in project options. Until then this is just normal behavior. Memory is just used and sometimes during a programs lifetime even more memory is used. At some point this should stabilize. All memory will be automatically released at end of program. Here is a similar discussion (about Delphi but the priciple is the same) https://stackoverflow.com/questions/2569661/why-does-my-delphi- programs-memory-continue-to-grow @nmra what happens if you click the button 10 times? And 60 times? Does it keep increasing with 24MB? Taskmanager isn't really the tool to check for memory leaks. BTW, I would use WaitFor + Free instead of directly freeing the thread. Also... I'm not entirely sure what you are doing (using a TZConnection from the main thread inside a thread) is thread safe. It could be that each thread would need it's own  TZConnection. But maybe someone with more ZEOS experience can tell us that. ---------------------------------------------------------------------- SymbolicFrank · 2022-07-25 07:23 The amount of memory your application allocates has no direct relation to the amount of memory it actually uses. The OS uses paging, which means that pages are only inserted when written. And blocks of memory that aren't used can be discarded or written to file, when needed. If you look in the task manager (Windows), under memory, you see that there are many different kinds of usage and that you generally have about twice as much memory committed (allocated) than is actually in use (paged in). ---------------------------------------------------------------------- Thaddy · 2022-07-25 11:01 The memory is free'd when you use SomeThread.waitfor in the main thread. Just before program ends. Don't kill a thread to soon. That gets you into trouble. Actually in your example it does not matter too much becase the OS will release the memory anyway. If you use waitfor, the application ends without leaks. (On a side note: I still not understand why people are using zeoslib, it has a sick/broken architecture and does not have any advantages above the core db components which have a very sound architecture. Asking for trouble. In my opinion it should be removed from the distribution to prevent users using it.) ---------------------------------------------------------------------- rvk · 2022-07-25 11:24 Quote from: Thaddy on July 25, 2022, 01:01:26 pm(On a side note: I still not understand why people are using zeoslib, it has a sick/broken architecture and does not have any advantages above the core db components which have a very sound architecture. Asking for trouble. In my opinion it should be removed from the distribution to prevent users using it.) Is Zeos in the distibution? I thought it was only downloadable in the OPM. (But I would also prefer SQLdb over Zeos) And if you want it off OPM, well, there might be more candidates for that ---------------------------------------------------------------------- Thaddy · 2022-07-25 14:00 Indeed, sorry, OPM. And your second point is also right: there are more packages that are remove/dissolve/disappear candidates. And some that - because of quality code but lack of maintenance - should be marked as legacy. ---------------------------------------------------------------------- Thaddy · 2022-07-25 17:25 Quote from: malcome on July 22, 2022, 09:37:27 pmYou have some ways. Stop using TthreadStop using ZeosStop using MysqlStop using LazarusIf you find a person who caused memory leak and report it, then lots of users say you thanks! The only valid entry is 2. ---------------------------------------------------------------------- marsupilami79 · 2022-07-26 20:47 Quote from: Thaddy on July 25, 2022, 04:00:06 pmAnd some that - because of quality code but lack of maintenance - should be marked as legacy. How come you are qualified to talk about Zeos maintenance or code quality? When did you even try to do someting there? What is your big contribution where to see how much better you are? ---------------------------------------------------------------------- marsupilami79 · 2022-07-26 20:52 Quote from: rvk on July 22, 2022, 09:55:47 pmAlso... I'm not entirely sure what you are doing (using a TZConnection from the main thread inside a thread) is thread safe. It could be that each thread would need it's own  TZConnection. But maybe someone with more ZEOS experience can tell us that. Zeos is thread safe as long as one TZConnection only gets used from one thread at a time. Multiple threads can each use their own TZConnection. We do have a user who does big operations in a background thread and connects GUI components to the data sets after fetching the data is finished. See https://zeoslib.sourceforge.io/view topic.php?f=40&t=83751&p=212402#p212086 for more information.