Multi thread: my bad design or bug? lazarus / 21-General ---------------------------------------------------------------------- torbente · 2021-02-12 10:12 Pseudo code: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i ExecuteThread : boolean = false;PointerThread : array of integer;NumberCpus : integer;condition : boolean; SetLength(PointerThread,0);NumberCpus:=GetPresentCPUs(); //Returns value of GetEnvironmentVariable('NUMBER_OF_PROCESSORS') or 1 Procedure VerifyForThreads();Beginif ((condition) and (not ExecuteThread)) then   begin   ExecuteThread := true;   for counter := 1 to NumberCpus do      begin      setlength(PointerThread,length(PointerThread)+1);      PointerThread[length(PointerThread)-1] := Beginthread(tthreadfunc(@FuncRunThread));      end;   end;If not condition then ExecuteThread  := false;End; function FuncRunThread(aParam:Pointer):PtrInt;  Beginwhile ExecuteThread  do   begin   {a long and semi infinite stuff}   end;result := 1;if length(PointerThread) >0 then length(PointerThread) ;= length(PointerThread)-1;End;  This code works "apparently" fine. If condition is true, then as many threads as detected cpus are open. And array of integer is filled to "control" the number of running threads. (checking the length of the array returns the number of running threads) When condition becomes false, ExecuteThread turns false to allow the threads end. When a thread finish, it decreases the length of the array by 1. After all the threads are over, the array PointerThread have a length of 0. I have 2 questions about this code, related to how FP handles threads: 1.- When ExecuteThread is turned false, the thread is really finished? Are there any other things that must be performed to avoid memory leaks? Can i check it? 2.- How i could finish the threads without change ExecuteThread value? ---------------------------------------------------------------------- MarkMLl · 2021-02-12 10:16 Quote from: torbente on February 12, 2021, 11:12:38 am1.- When ExecuteThread is turned false, the thread is really finished? Are there any other things that must be performed to avoid memory leaks? Can i check it? 2.- How i could finish the threads without change ExecuteThread value? 1: No, it could still be stuck in a tight loop. 2: Haven't a clue. You've not told us anything about the OS etc., and while some might have facilities to kill individual threads others don't. In any event it's probably not safe to try, since thread state including allocated memory etc. will be left dangling. MarkMLl ---------------------------------------------------------------------- Zvoni · 2021-02-12 10:21 And "PointerThread" should be "Array of TThreadID" ---------------------------------------------------------------------- Warfley · 2021-02-12 11:28 Quote from: torbente on February 12, 2021, 11:12:38 am1.- When ExecuteThread is turned false, the thread is really finished? Are there any other things that must be performed to avoid memory leaks? Can i check it? 2.- How i could finish the threads without change ExecuteThread value? 1. After the Execute method finishes, you give up control over the thread, the thread finishes, but the thread object might not necessarily be destroyed, this depends on the FreeOnTerminate property. If you don't need the Thread object afterwards, you can set it to automatically free the memory afterwards. 2. I don't think the RTL implementation has a method for this, but you can use the system API (pthread for Unix and winapi for windows) to archive this. For pthread it should be pthread_cancle or pthread_kill(SIGKILL) or something like this. PS: you can use the already existing property Terminated instead of ExecuteThread, it's a simple boolean exactly for that matter PPS: You should keep in mind that most standard types are not thread safe, so PointerThread better not be an array but rather a thread safe list implementation (or you manually use critical sections/locks) ---------------------------------------------------------------------- torbente · 2021-02-12 14:12 Sorry, i forgot : Using windows 8.1, lazarus 2.0.10 The threads should start/stop up to infinite times. But my app hangs sadly after few cycles. I will try with another variable to terminate the threads without change ExecuteThread : Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i var AbortAllThreads : boolean; function FuncRunThread(aParam:Pointer):PtrInt;  Beginwhile ExecuteThread  do   begin   {a long and semi infinite stuff}   if AbortAllThreads then break;   end;result := 1;if length(PointerThread) >0 then length(PointerThread) ;= length(PointerThread)-1;End; and before call the threads to start again... Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i Procedure VerifyForThreads();Beginif ((condition) and (not ExecuteThread)) then   begin   AbortAllThreads  := false;   ExecuteThread := true;   for counter := 1 to NumberCpus do      begin      setlength(PointerThread,length(PointerThread)+1);      PointerThread[length(PointerThread)-1] := Beginthread(tthreadfunc(@FuncRunThread));      end;   end;If not condition then ExecuteThread  := false;End; So now i should have an aditional way to finish the function. But i still do not know how to properly terminate the thread. ---------------------------------------------------------------------- MarkMLl · 2021-02-12 14:39 Quote from: Warfley on February 12, 2021, 12:28:54 pm1. After the Execute method finishes, you give up control over the thread, the thread finishes, but the thread object might not necessarily be destroyed, this depends on the FreeOnTerminate property. If you don't need the Thread object afterwards, you can set it to automatically free the memory afterwards. I got bitten by that one a few days ago, so would council caution. I had a background thread doing some comms stuff, set FreeOnTerminate (which is not something I normally do) with control falling out of the exec when the physical device stopped sending data. I also had a GUI menu entry to break the exec loop by doing a fcommsThread.Terminate, which also worked. Then I artificially clicked the stop button if the GUI noticed an error condition from the backend... which blew up because the TThread variable now referred to an object that had freed itself. So I told the thread where its TThread variable would be stored (a field in the main form) but was still left with Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i (* There's a very slight risk of a race condition here, if the thread releases  *)(* its own resources between the Assigned() and Terminate. For that reason I've *)(* put an exception handler around it.                                        *)   try    if Assigned(fcommsThread) then    fcommsThread.Terminate  except  end;  The bottom line is that there are pitfalls which need to be thought through very carefully. MarkMLl ---------------------------------------------------------------------- torbente · 2021-02-12 15:32 I have to add this: the problem/bug/missfunction started when i try to start the thread multiple times (up to cpu times). If only one thread is started, it works perfect infinite cycles. It should be something with assigning the result to an array ---------------------------------------------------------------------- AL · 2021-02-12 16:30 Quoteif length(PointerThread) >0 then length(PointerThread) ;= length(PointerThread)-1; Seems to me that shortening the array by one does not mean that the terminated thread is the last one in the array ! Also this:  length(PointerThread) ;= length(PointerThread)-1 should be  length(PointerThread) := length(PointerThread)-1 ---------------------------------------------------------------------- MarkMLl · 2021-02-12 16:39 Quote from: AL on February 12, 2021, 05:30:39 pmQuoteif length(PointerThread) >0 then length(PointerThread) ;= length(PointerThread)-1; Seems to me that shortening the array by one does not mean that the terminated thread is the last one in the array ! Also this:  length(PointerThread) ;= length(PointerThread)-1 should be  length(PointerThread) := length(PointerThread)-1 He did say that it was pseudocode, and rather more serious is that length() can't be assigned to. MarkMLl ---------------------------------------------------------------------- Warfley · 2021-02-12 19:22 Quote from: torbente on February 12, 2021, 04:32:00 pmI have to add this: the problem/bug/missfunction started when i try to start the thread multiple times (up to cpu times). If only one thread is started, it works perfect infinite cycles. It should be something with assigning the result to an array What datastructure do you use for PointerThread? Because if it is not thread safe the resulting race condition can lead to the decrement being lost. Example: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i program Project1; {$mode objfpc}{$H+} uses  Classes, SysUtils; type   { TTestThread }   TTestThread = class(TThread)  protected    procedure Execute; override;  end; var  Threads: array of TTestThread;  Running: Boolean; procedure TTestThread.Execute;begin  While Running do; // run until running is false  SetLength(Threads, Length(Threads) - 1);end; var i: Integer;begin  Running := True;  SetLength(Threads, 12);  for i:=0 to 11 do    Threads[i] := TTestThread.Create(False);  // Wait for threads to start  Sleep(500);  Running := False;  // wait for threads to finish  Sleep(500);  WriteLn(Length(Threads));  ReadLn;end. Because SetLength takes rather long, this results in at most 1 decrement of the array, because the threads race against eachother with outdated data. Using a thread safe datastructure, this works totally fine: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i program Project1; {$mode objfpc}{$H+} uses  Classes, SysUtils, Generics.Collections; type   { TTestThread }   TTestThread = class(TThread)  protected    procedure Execute; override;  end;   TThreadSafeList = specialize TThreadList; var  Threads: TThreadSafeList;  Running: Boolean; procedure TTestThread.Execute;begin  While Running do; // run until running is false  Threads.Remove(self);end; var i: Integer;begin  Running := True;  Threads := TThreadSafeList.Create;  for i:=0 to 11 do    Threads.Add(TTestThread.Create(False));  // Wait for threads to start  Sleep(500);  Running := False;  // wait for threads to finish  Sleep(500);  // no easy access to count, we therefore need to lock and unlock  WriteLn(Threads.LockList.Count);  Threads.UnlockList;  ReadLn;end. ---------------------------------------------------------------------- torbente · 2021-02-14 08:15 Looks like  a simple solution, suggested here, was the valid one. Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i And "PointerThread" should be "Array of TThreadID" I change it and now looks like it is working as expected.