[SOLVED]error when modifying main form from indy thread lazarus / 32-Linux ---------------------------------------------------------------------- Michael Collier · 2015-10-16 14:50 I've posted this here (linux forum) because this error only seems to happen on my linux machine and not on windows, but I'm not sure if it is related to linux per se. I'm using lazarus v1.0.10 wth fpc2.6.2 compililng with dwarf2 , not using cmem. I have an indy server object that uses sendmessage() to update a memo on the main form. Very simple but on linux I get SIG type errors, the call stack shows functions relating to gtk e.g. gtk_text_layout_xxxxxxx My understanding is that calls from threads to the main form are ok using sendmessage, there are no other updates to the main form (I don't even move the mouse over the application). I even tried using a criticalsection, just to see if it made any difference (it didn't). Could the problem be with gtk? Can I do anything about that? Is the problem with lazarus 1.1.10? ---------------------------------------------------------------------- JuhaManninen · 2015-10-16 14:55 Use TThread.Synchronize. ---------------------------------------------------------------------- balazsszekely · 2015-10-16 15:46 Always update visual controls with TIdSync or TIdMessage. The first one is synchronous the later on the other hand is asynchronous. Something like this: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i uses IdSync;type  TSync = class(TIdSync)  protected    procedure DoSynchronize; override;  public    FMessage: string;  end; procedure TSync.DoSynchronize;begin  Form1.Memo1.Lines.Add(FMessage)end; procedure TForm1.IdTCPServer1Execute(AContext: TIdContext);var  Sync: TSync;begin  //...  if (SomeCondition) then  begin    Sync := TSync.Create;    try      Sync.FMessage := 'This is a test';      Sync.Synchronize;    finally      Sync.Free;    end;  end;end; ---------------------------------------------------------------------- Michael Collier · 2015-10-16 16:24 Thank you, it is much better now ---------------------------------------------------------------------- taazz · 2015-10-16 17:18 what happens if the postmessage is used instead? Is this behavior a bug (probably) or by design on GTK? Just to clarify a bit of what I'm asking here I'm quoting the help file Quote from: http://lazarus- ccr.sourceforge.net/docs/lcl/lclintf/sendmessage.htmlThe difference between SendMessage and PostMessage is the way that they return control to the calling thread. With SendMessage control is not returned until the window that the message was sent to has completed processing the sent message, however with PostMessage control is returned immediately. ---------------------------------------------------------------------- Michael Collier · 2015-10-16 18:04 Just tried postmessage() to find out what would happen, same errors as sendmessage(). ---------------------------------------------------------------------- taazz · 2015-10-16 21:25 Thank you for testing. At this point I think that this issue worth a bug report but since I haven't looked close enough on the message implementation in GTK I'll probably stay back on this one and let others decide. One last think can you create a simple test application that shows the error something along the lines of a thread sending a message every 10 seconds up to a number of times eg 10 messages. I would like to run it on QT as well to see how it fairs. ---------------------------------------------------------------------- Michael Collier · 2015-10-17 10:39 Ok I'll get something done today, I can test on apple mac too if you like. Not sure what is meant by QT though, is it this http://wiki.freepascal.org/Qt_Interface ---------------------------------------------------------------------- zeljko · 2015-10-17 11:45 Postmessage works fine with Qt (multithreading app). ---------------------------------------------------------------------- Michael Collier · 2015-10-17 12:50 I have attached test project.. thread_sendmessage.zip You can choose the number of messages, on linux I can do 3 messages without crashing, but higher numbers cause the error to happen. You can also select a sleep period in milliseconds between messages, not sure if this has any affect on bug. ---------------------------------------------------------------------- Michael Collier · 2015-10-17 14:44 I extended the test project (attached thread_sendmessage2.zip )to offer the option of performing update(),refresh() & processmessages() procedures and retested linux/windows and this time apple mac too. The mac crashes if update(),refresh() & processmessages()  is called within sendmessage() but ok for postmessage() and  synchronise(). The mac is Version 10.5.8. Lazarus 1.0.12 fpc 2.6.2 Windows is fine, under all circumstances, so I guess an application tested on windows could fail on apple. thread_sendmessage2.zip ---------------------------------------------------------------------- taazz · 2015-10-17 14:49 I'll take a closer look on the demo later tonight. Thank you for your time, I appreciate it. ---------------------------------------------------------------------- Michael Collier · 2015-10-18 12:22 I'm happy to help, any improvements to lazarus benefits us all ---------------------------------------------------------------------- taazz · 2015-10-19 07:05 I'm still having problems installing lazarus 1.4.4 on my linux. Testing is impossible at this time for me I have the test application though so at some point in the future I will run my tests. Sorry for any and all delays on this . ---------------------------------------------------------------------- Michael Collier · 2015-10-19 10:14 No worries, thanks for trying ---------------------------------------------------------------------- rvk · 2015-10-19 10:51 F.Y.I. I tested your thread_sendmessage2 on Linux debian 3.16.0-4-586 #1 Debian 3.16.7-ckt9-3~deb8u1 (2015-04-24) i686 GNU/Linux with Lazarus 1.5 r48991M FPC 2.6.5 i386-linux-gtk 2 and Lazarus 1.0.10 through 1.2.4 All work fine (with all 3 methods) if you don't choose "update and refresh". However, when choosing "update and refresh" the sendmessage fails in all versions. (The other 2, synchronize and postmessage keep working, regardless the sleep) For sendmessage/"update and refresh", just the ProcessMessages doesn't give any problem. Update and Refresh both give problems. I did find this post: Quote from: skalogryz on March 26, 2008, 02:26:23 pmSendMessage works different on the gtk+ widgetset than win32. on Win32. SendMessage sends the message to the queue, and blocks sending thread, until executing thread process the message. If sending thread is the same as process thread, the message is processed immediatly. But on gtk+ widgetset Send Message is processed directly from the sending thread, instead of putting message to the queue and wait for processing thread to process it! And this cause bugs, especially with GUI applications. is it a bug or feature? If this is really true (i.e. the SendMessage is processed IN the sending thread) then using SendMessage is the same as directly calling handler_send_message from the thread (which is a big no no). So that could be the problem when using SendMessage on Linux. So policy should be... always use PostMessage or Synchronize. ---------------------------------------------------------------------- taazz · 2015-10-19 11:07 Quote from: rvk on October 19, 2015, 12:51:41 pmQuote from: skalogryz on March 26, 2008, 02:26:23 pmBut on gtk+ widgetset Send Message is processed directly from the sending thread, instead of putting message to the queue and wait for processing thread to process it! And this cause bugs, especially with GUI applications. is it a bug or feature? If it is true then it is a bug(incomplete implementation), not a feature and must be corrected.  Quote from: rvk on October 19, 2015, 12:51:41 pmIf this is really true (i.e. the SendMessage is processed IN the sending thread) then using SendMessage is the same as directly calling handler_send_message from the thread (which is a big no no). So that could be the problem when using SendMessage on Linux. So policy should be... always use PostMessage or Synchronize. Basically even if it did work as advertised it has no difference from using synchronize, so I would say that use synchronize in the 1st place instead of sendmessage so you have no illusions of what really goes on. ---------------------------------------------------------------------- Michael Collier · 2015-10-19 17:00 Quote from: rvk on October 19, 2015, 12:51:41 pmF.Y.I. I tested your thread_sendmessage2 on Linux debian 3.16.0-4-586 #1 Debian 3.16.7-ckt9-3~deb8u1 (2015-04-24) i686 GNU/Linux with Lazarus 1.5 r48991M FPC 2.6.5 i386-linux-gtk 2 and Lazarus 1.0.10 through 1.2.4 All work fine (with all 3 methods) if you don't choose "update and refresh". Just tried again, I've updated to lazaraus 1.4 since my previous posts, but anyway.. sendmessage() fails even without "update and refresh", you have to select a higher number of message count =100 , and execute the task several times before the application crashes, say 5 time ish. ... just tried postmessage() again.. yes it crashes for me, just takes more attempts ---------------------------------------------------------------------- rvk · 2015-10-19 17:43 Quote from: Michael Collier on October 19, 2015, 07:00:59 pmJust tried again, I've updated to lazaraus 1.4 since my previous posts, but anyway.. sendmessage() fails even without "update and refresh", you have to select a higher number of message count =100 , and execute the task several times before the application crashes, say 5 time ish. ... just tried postmessage() again.. yes it crashes for me, just takes more attempts I can understand why SendMessage fails. If the message is executed in the same thread as where you call it, it doesn't matter if you call update/refresh or not. It will eventually fail (because you're updating the main-thread-components in another thread). But the fact that PostMessage fails puzzles me. It should post a message to the message queue and continue (and the queue should be handled by the correct thread). What parameters did you use to make the PostMessage crash on Laz1.4? I'll try to do some tests tomorrow to see if I can identify if the SendMessage is executed in the same thread. (Does anybody have code- snippet suggestions for that ) I tried GetCurrentThreadId() but that doesn't seem to work in a thread on Linux. (It keeps giving me -1233851584, GetThreadId too.) Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i // in main.pas    Memo1.Lines.Add( 'sendmessage :'    + IntToStr( Message.lParam )    + ' thr: ' + IntToStr( Message.wParam ) // received via sendmessage from thread    + ' cur: ' + IntToStr( GetCurrentThreadId )...    Memo1.Lines.Add( 'postmessage :'    + IntToStr( Message.lParam )    + ' thr: ' + IntToStr( Message.wParam ) // received via postmessage from thread    + ' cur: ' + IntToStr( GetCurrentThreadId )...// in thread_classes.pas      mm_sendmessage : begin        SendMessage( form_thread_message.Handle                   , LM_SEND_MESSAGE_TEST                   ,  GetCurrentThreadId()                 , FCount                   )                        ;    end;      mm_postmessage : begin        PostMessage( form_thread_message.Handle                   , LM_POST_MESSAGE_TEST                 , GetCurrentThreadId()                   , FCount                 )                        ; ---------------------------------------------------------------------- Michael Collier · 2015-10-19 18:04 Quote from: rvk on October 19, 2015, 07:43:05 pmWhat parameters did you use to make the PostMessage crash on Laz1.4? I set the message count to 100, sleep to zero, then just clicked away at it, to be honest I nearly gave up thinking I had mis-remembered the original test, but in the end it did crash. I also forced it to fail via sendmessage() first - because I wasn't sure if previous failures had any affect on subsequent attempts. This was probably just superstition on my part but that's what I did. ---------------------------------------------------------------------- rvk · 2015-10-19 18:41 Even with setting 100 times and 0 sleep and update/refresh I couldn't get PostMessage to crash. I did however got Synchronize to hang with that  (see last paragraph because it could also be the problem with your PostMessage) With PostMessage you can see the thread is terminated at about the 2nd or 3rd iteration. All the messages are in queue and the thread has done its job. If I click really fast after each other the threads are already terminated while the messages are still being processed. I don't know how many times you did this? Is there a maximum of messages for a queue? I tested with 100.000 PostMessages and the thread ended when about message nr. 1.000 was being executed. The reason I got Synchronize to hang is probably due to your btn_free_server_threadClick procedure. It does a my_server_thread.Terminate but doesn't wait until it is really terminated before FreeAndNil. And I clicked free_server while the thread was still running !! And then it hangs. So if you did this same test with the PostMessage and you clicked btn_free_server_threadClick before the thread was terminated then it's logical that hangs too. You should change it to this: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i procedure Tform_thread_message.btn_free_server_threadClick(Sender: TObject);begin  my_server_thread.Terminate      ;  my_server_thread.waitfor        ;  FreeAndNil ( my_server_thread ) ;  configure_buttons ( True)       ;end; Another options would be to only activate the btn_free_server_thread when the thread has ended. Now that button is enabled when the thread is started. What happens with your tests if you build in the tthread.waitfor? ---------------------------------------------------------------------- Michael Collier · 2015-10-19 19:47 Quote from: rvk on October 19, 2015, 08:41:54 pmWhat happens with your tests if you build in the tthread.waitfor? Thanks for pointing that out, I have now added the waitfor statement. I tried again and postmessage() still crashed. At first I thought you had cured it, because I was clicking away and it didn't crash. I upped the qty to 1000 and also tried dragging the main form during the execution cycle. It fails much more quickly after failing once. It stopped at count = 220. ---------------------------------------------------------------------- rvk · 2015-10-19 20:07 Ok, now for me, on Linux, I got an SIGSEGV at message 11.427 (of the 100.000) after the thread was already finished a long time ago. I'm wondering if this is really a thread-problem or something else. If the thread is finished (terminated but not yet freed) it shouldn't have anything more to do with the message-queue, so why it gives a SIGSEGV is beyond me. For me it crashed in XRenderCompositeText8. I'll try to debug this some more, tomorrow. I'm wondering how stable the message-queue is under Linux. If I do 100.000/1ms/PostMessage the process slows down at about 5000 and hangs at about 6000 if I choose it without Update/Refresh. With 100.000/0ms/PostMessage/no update I even got a SIGABRT at #510. I don't even know what that is   (abort signal, heap corrupt/overflow??) ---------------------------------------------------------------------- rvk · 2015-10-20 10:30 Mmmm, I did some tests. I've stripped the threads from the test- project and wanted to only test PostMessage and SendMessage. On Windows, when using PostMessage, it doesn't go beyond 10.000. Apparently the maximum Posts you can do is 10.000. But it doesn't crash or hang. Then Linux. When doing the same test it does go beyond 10.000 but it crashes with a SIGSEGV at 11.624. Constantly at exactly the same message. Even if I do 7.000 and click again right away (posting 14.000 messages) it still runs only to 4.624 for the second run (so message 11.624). The fact it crashes seems a bug. If the number of messages is a problem it should stop at 10.000 like it does in Windows. In this test-project I couldn't get the SendMessage to crash but that's logical. No threads and SendMessage directly executed the message. @Michael, does this test-project crash for you sooner than 11.624? (100.000/0ms/PostMessage/no update & update) ---------------------------------------------------------------------- Michael Collier · 2015-10-20 15:03 Linux crashes  @11,586 every time (100.000/0ms/PostMessage/no update & update) I can reduce by 1 to 11, 585 and repeat quickly without crashing, so my machine behaves differently to yours, although the number at which it fails seems very close. My linux box has 2.0 GiB memory. A bit more.. I modified your test so that the loop counter Cnt is global (to make it accessible from do_updates() ), then only did processmessages() if (cnt mod 10 = 0). Thus reducing the number of processmessages() without reducing the number of postmessages(). It goes much further now, - is still running without crashing, count of postmessage() >190,000. Not sure if this helps but thought I'd let you know. ---------------------------------------------------------------------- rvk · 2015-10-20 15:44 Quote from: Michael Collier on October 20, 2015, 05:03:52 pmI modified your test so that the loop counter Cnt is global (to make it accessible from do_updates() ), then only did processmessages() if (cnt mod 10 = 0). Thus reducing the number of processmessages() without reducing the number of postmessages(). It goes much further now, - is still running without crashing, count of postmessage() >190,000. Not exactly sure how you did that but making cnt (from the btn_executeClick) a global variable doesn't work. Because the PostMessages are executed first, so cnt is always 100.000 (or max) when the messages are really executed. I did it like this. In handler_post_message I called do_updates(Message.lParam) and that way you can do the (cnt mod 10) correctly. But my guess is that you did something similar otherwise it wouldn't run so far. So it seems to be a combination of PostMessage and Application.ProcessMessages that's giving problems here. I did find that for Windows the limit is 10.000. So that's the reason in Windows it stops at 10.000. Why the crash happens in Linux, I don't know. Unfortunately, I don't have that much knowledge about the messaging system in Linux. I'm also not sure why the thread-example crashes. For me it sometimes hangs and sometimes crashes with a SIGSEGV (with PostMessage). But that could be the same cause as the PostMessage in the no-thread example. Sometimes I can see it slow down... and hang (at around 2000-3000) and sometimes it just hangs at 1500 (all with sleep(0)). With sleep(1) it goes to about 8000-9000. So my guess is there is really something "fishy" with the messaging-queue in Linux. I did find mention about the messaging-queue not being thread-safe in the past but that should have been solved. So if you have a need for lots of messages in your thread then stay away from PostMessage and use Synchronize. (Sendmessage was already a "no no" because it's executed in the same thread) Any Linux/Lazarus gurus out there (with knowledge about the messaging- queue in Lazarus on Linux) who want to find out why the PostMessage crashes at about 11.000? ---------------------------------------------------------------------- taazz · 2015-10-20 16:21 Quote from: rvk on October 20, 2015, 05:44:54 pmSo if you have a need for lots of messages in your thread then stay away from PostMessage and use Synchronize. (Sendmessage was already a "no no" because it's executed in the same thread) Wrong conclusion. If the message exchange is low then a simple synchronize should not be a problem if its to high then the use of synchronise is out of the question it will show down the thread speed to a crawl, you have to use postmessage to keep things speedy. There are a number of problems in the test case described here (I'll take a look on the code later tonight on windows) but the most important are 1) the number of messages is to high for normal operation 2) never ever call processmessages its the worst think you can do. although lcl makes it impossible you should realy try to avoid it at all costs. ---------------------------------------------------------------------- rvk · 2015-10-20 18:13 Quote from: taazz on October 20, 2015, 06:21:07 pm1) the number of messages is to high for normal operation I'll admit that the number of messages is indeed abnormal. But it does shows a flaw in the underlying messaging-system on Linux. And I'm not sure if that flaw show its head in other cases. For instance in the example with the threads where PostMessage method already hangs at about 1.500 messages (which might be a real-time number in normal operations). And that example even hangs without a ProcessMessages on Liinux. Quote from: taazz on October 20, 2015, 06:21:07 pm2) never ever call processmessages its the worst think you can do. although lcl makes it impossible you should realy try to avoid it at all costs. Yeah... lcl makes it impossible but I should avoid it at all costs Nice. And how would I avoid it in this example? I added a more simpler test-project. It just does 20.000 PostMessages in a loop (without ProcessMessages) and after the loop is done the messages are processed with a ProcessMessages. There is no chance of recursion here and it's a simple example so using ProcessMessage shouldn't be a problem. On Windows the maximum messages is 10.000 which shows quite clearly. And it does the handling perfectly. On Linux it crashes for me at around 13.000 with a SIGSEGV. I also noticed that on Linux the 20.000 PostMessages are done in an instance while on Windows it takes a while. (Maybe because the messaging on Windows is handled by Windows itself?) (note: On Windows this all runs perfectly so it really needs to be tested on Linux) ---------------------------------------------------------------------- taazz · 2015-10-21 00:42 Quote from: rvk on October 20, 2015, 08:13:38 pmQuote from: taazz on October 20, 2015, 06:21:07 pm1) the number of messages is to high for normal operation I'll admit that the number of messages is indeed abnormal. But it does shows a flaw in the underlying messaging-system on Linux. And I'm not sure if that flaw show its head in other cases. For instance in the example with the threads where PostMessage method already hangs at about 1.500 messages (which might be a real-time number in normal operations). And that example even hangs without a ProcessMessages on Liinux. Quote from: taazz on October 20, 2015, 06:21:07 pm2) never ever call processmessages its the worst think you can do. although lcl makes it impossible you should realy try to avoid it at all costs. Yeah... lcl makes it impossible but I should avoid it at all costs Nice. And how would I avoid it in this example? 1) what is processmessages? what it does and when is used? In this case you simple don't need to use it at all. the processing is in an other thread the messages will be processed in due time from the main thread stop disrupting the normal flow of things for no reason. ---------------------------------------------------------------------- Cyrax · 2015-10-21 04:22 Quote from: taazz on October 21, 2015, 02:42:28 amQuote from: rvk on October 20, 2015, 08:13:38 pmQuote from: taazz on October 20, 2015, 06:21:07 pm1) the number of messages is to high for normal operation I'll admit that the number of messages is indeed abnormal. But it does shows a flaw in the underlying messaging-system on Linux. And I'm not sure if that flaw show its head in other cases. For instance in the example with the threads where PostMessage method already hangs at about 1.500 messages (which might be a real-time number in normal operations). And that example even hangs without a ProcessMessages on Liinux. Quote from: taazz on October 20, 2015, 06:21:07 pm2) never ever call processmessages its the worst think you can do. although lcl makes it impossible you should realy try to avoid it at all costs. Yeah... lcl makes it impossible but I should avoid it at all costs Nice. And how would I avoid it in this example? 1) what is processmessages? what it does and when is used? In this case you simple don't need to use it at all. the processing is in an other thread the messages will be processed in due time from the main thread stop disrupting the normal flow of things for no reason. Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i procedure TApplication.ProcessMessages;begin  if Self=nil then begin    // when the programmer did a mistake, avoid getting strange errors    raise Exception.Create('Application=nil');  end;  WidgetSet.AppProcessMessages;  ProcessAsyncCallQueue;end; It calls widgeset specific message handling procedure and procedure which handles asynchronous call queues periodically.  Widgeset specific message handling procedure (in case win32 widgeset) calls CheckSynchronize and CheckPipeEvents procedures. CheckSynchronize procedure is important one and related to handling thread execution succesfully. If you are using threads, you need to call CheckSynchronize periodically in your main program loop. ---------------------------------------------------------------------- taazz · 2015-10-21 04:28 Quote from: Cyrax on October 21, 2015, 06:22:42 amCode: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i procedure TApplication.ProcessMessages;begin  if Self=nil then begin    // when the programmer did a mistake, avoid getting strange errors    raise Exception.Create('Application=nil');  end;  WidgetSet.AppProcessMessages;  ProcessAsyncCallQueue;end; It calls widgeset specific message handling procedure and procedure which handles asynchronous call queues periodically.  Widgeset specific message handling procedure (in case win32 widgeset) calls CheckSynchronize and CheckPipeEvents procedures. CheckSynchronize procedure is important one and related to handling thread execution succesfully. If you are using threads, you need to call CheckSynchronize periodically in your main program loop. good good, Can you point me to the main lcl application loop too? What it does? ---------------------------------------------------------------------- Cyrax · 2015-10-21 04:36 Quote from: taazz on October 21, 2015, 06:28:46 amQuote from: Cyrax on October 21, 2015, 06:22:42 amCode: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i procedure TApplication.ProcessMessages;begin  if Self=nil then begin    // when the programmer did a mistake, avoid getting strange errors    raise Exception.Create('Application=nil');  end;  WidgetSet.AppProcessMessages;  ProcessAsyncCallQueue;end; It calls widgeset specific message handling procedure and procedure which handles asynchronous call queues periodically.  Widgeset specific message handling procedure (in case win32 widgeset) calls CheckSynchronize and CheckPipeEvents procedures. CheckSynchronize procedure is important one and related to handling thread execution succesfully. If you are using threads, you need to call CheckSynchronize periodically in your main program loop. good good, Can you point me to the main lcl application loop too? What it does? Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i {---------- --------------------------------------------------------------------  TApplication Run  MainForm is loaded and control is passed to event pr ocessor.-------------------------------------------------------------- ----------------}procedure TApplication.Run;begin  if (FMainForm <> nil) and FShowMainForm then FMainForm.Show;  WidgetSet.AppRun(@RunLoop );end; {-------------------------------------------------------------- ----------------  TApplication RunLoop  control is passed to event pro cessor.--------------------------------------------------------------- ---------------}procedure TApplication.RunLoop;begin  repeat    if CaptureExceptions then      try // run with try..except        HandleMessage;      except        HandleException(Self);      end    else      HandleMessage; // run without try..except  until Terminated; end; {---------------------------------------------------------------- --------------  Method: TApplication.HandleMessage  Params: None  Returns:  Nothing   Handles all messages first then the Idle --------- ---------------------------------------------------------------------} procedure TApplication.HandleMessage;begin  WidgetSet.AppProcessMessages; // process all events  if not Terminated then Idle(true);end; procedure TWidgetSet.AppRun(const ALoop: TApplicationMainLoop);begin  if Assigned(ALoop) then ALoop;end;   It seems that it calls widgeset specific main loop. In win32, AppRun method isn't overloaded. ---------------------------------------------------------------------- rvk · 2015-10-21 07:34 Quote from: rvk on October 20, 2015, 08:13:38 pmQuote from: taazz on October 21, 2015, 02:42:28 amQuote from: taazz on October 20, 2015, 06:21:07 pm2) never ever call processmessages its the worst think you can do. although lcl makes it impossible you should realy try to avoid it at all costs. Yeah... lcl makes it impossible but I should avoid it at all costs Nice. And how would I avoid it in this example? 1) what is processmessages? what it does and when is used? In this case you simple don't need to use it at all. the processing is in an other thread the messages will be processed in due time from the main thread stop disrupting the normal flow of things for no reason. O wow, for Windows you can indeed remove the ProcessMessages in my simple_postmessage.zip. BUT... on Linux you DO need the ProcessMessages. Otherwise the lines in memo1 don't get displayed until ALL messages are handled !!! My simple_postmessage is a simple program without threads but apparently Windows sees the chance to update the memo1 but Linux does not. So the messaging handling in Linux is really really different I think I understand now why, on Linux, I get the error after 13.000 messages processed. I thought ProcessMessages shouldn't be called in events because you could click a button again but it should also not be called in a message-procedure. Here is why: Here is the essence of my simple_messages: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i procedure TForm1.Button1Click(Sender: TObject);var  p: integer;begin  Memo1.Clear; // BUG on Linux - Needs to be clear otherwise it doesn't scrol  for p := 1 to 20000 do  begin    PostMessage(Self.Handle, LM_POST_MESSAGE, p, p);    Button1.Caption := IntToStr(p);  end;end; procedure TForm1.post_message(var Msg: TLMessage);begin  Memo1.Lines.add('p = ' + IntToStr(Msg.wParam) + ' ' + IntToHex(Integer(sPtr),4));  Application.ProcessMessages;              // <------- danger of recursion here if there are more LM_POST_MESSAGE on queueend; When all the PostMessage are done the handling/processing begins. It runs post_message() and in that I call Application.ProcessMessages (note: on Linux this is necessary to update the memo!). What happens in Linux, I think, is that the App.ProcessMessages calls post_message() again for the second message. And because of the App.ProcessMessages it calls it again for the third. So we have recursion !! After 13.000 iterations/recursion it crashes with a SIGSEV. My question is twofold: 1) How can I update the layout/form without calling ProcessMessages (because that recurses into all the other messages) ? 2) Shouldn't ProcessMessages (under Linux) have a mechanisme to prevent such recursion ?? Edit: You can see the recursion happen on Linux when I print out the sPtr (stack pointer) (nibbling 640 bytes from the stack each time). Without the App.ProcessMessages that number stays the same for all calls and with it it changes (until it crashes). On Windows it always stays the same regardless of the App.ProcessMessages. So, somehow Windows avoids the recursion (while the messaging on Linux does not). ---------------------------------------------------------------------- Cyrax · 2015-10-21 09:34 Quote from: rvk on October 21, 2015, 09:34:52 am... Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i procedure TForm1.Button1Click(Sender: TObject);var  p: integer;begin  Memo1.Clear; // BUG on Linux - Needs to be clear otherwise it doesn't scrol  for p := 1 to 20000 do  begin    PostMessage(Self.Handle, LM_POST_MESSAGE, p, p);    Button1.Caption := IntToStr(p);  end;end; procedure TForm1.post_message(var Msg: TLMessage);begin  Memo1.Lines.add('p = ' + IntToStr(Msg.wParam) + ' ' + IntToHex(Integer(sPtr),4));  Application.ProcessMessages;              // <------- danger of recursion here if there are more LM_POST_MESSAGE on queueend;... Move Application.ProcessMessages out of your message handling method, put it in Button1Click method and call it only in certain periods. Like this : Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i procedure TForm1.Button1Click(Sender: TObject);var  p: integer;begin  Memo1.Clear; // BUG on Linux - Needs to be clear otherwise it doesn't scrol  for p := 1 to 20000 do  begin    PostMessage(Self.Handle, LM_POST_MESSAGE, p, p);    Button1.Caption := IntToStr(p);    if p mod 100 = 0  then       Application.ProcessMessages;   end;end;... ---------------------------------------------------------------------- rvk · 2015-10-21 09:59 Quote from: Cyrax on October 21, 2015, 11:34:41 amMove Application.ProcessMessages out of your message handling method, put it in Button1Click method and call it only in certain periods. Like this : Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i procedure TForm1.Button1Click(Sender: TObject);var  p: integer;begin  Memo1.Clear; // BUG on Linux - Needs to be clear otherwise it doesn't scroll  for p := 1 to 20000 do  begin    PostMessage(Self.Handle, LM_POST_MESSAGE, p, p);    Button1.Caption := IntToStr(p);    if p mod 100 = 0  then       Application.ProcessMessages;   end;end;...That doesn't quite work. The PostMessage returns immediately (after putting the message in the queue). It's not like a SendMessage, which waits for the message to be processed. So using App.ProcessMessages inside the Button1Click doesn't work because the messages are still being processed long after Button1Click is finished. And the ProcessMessages was needed in post_message() for the update of the memo1 (on Linux). Executing App.ProcessMessages every 10 times in the post_message() should work to alleviate the problem somewhat, but the fact remains that recursion occurs on the Linux-platform (nibbling that 640 bytes from the stack each time). For me it's not a problem (I'm not actively developing for Linux) but for those who do, it should be noted that multiple PostMessage in combination with Application.ProcessMessages might be a cause for problems because of the recursion. ---------------------------------------------------------------------- Michael Collier · 2015-10-22 15:58 Quote from: rvk on October 21, 2015, 11:59:43 amFor me it's not a problem (I'm not actively developing for Linux) but for those who do, it should be noted that multiple PostMessage in combination with Application.ProcessMessages might be a cause for problems because of the recursion. For multithreading: Also for linux it should be noted that even without using processmessages() or exceeding the thread limit (of 11,000 ish), both postmessage() and sendmessage() cause the application to crash (even when using try/except I believe) And for Mac developers (PowerPC Darwin), sendmessage() crashes, even without processmessages() or exceeding thread limit. I have attached thread_sendmessage3.zip , it has a few more features for selecting processmessages() and for enabling a background timer. I also find it helpful to write to the first line of the output memo, rather than adding lines, so that scrolling isn't required. ---------------------------------------------------------------------- rvk · 2015-10-22 16:23 Yes, for me under Linux the PostMessage stops at around 1500. It slows down and stops (sometimes with a SIGSEGV). I didn't test your latest project yet. Regarding the SendMessage. I think that will never work because the message is somehow executed in the same thread as where SendMessage is called from (on Linux). I tested this and as far as I could see this is still the case. So SendMessage should never be used if you're going to update something from the main thread. That is probably also the problem with SendMessage on the Mac. ---------------------------------------------------------------------- rvk · 2015-10-22 16:46 There is something I don't understand about the new sendmessage3 example. When I do 10.000 postmessages the thread should end relative quickly and the messages should still be processed. Could you make it visible when the thread is really finished? At this moment I only get the "Max reached" after all the postmessage-messages but I don't think that is correct. Even after the Max is reached there should still be messages being processed. With update=none I can't get the example to crash anymore (or ) ---------------------------------------------------------------------- Michael Collier · 2015-10-22 18:39 Quote from: rvk on October 22, 2015, 06:46:28 pmWhen I do 10.000 postmessages the thread should end relative quickly and the messages should still be processed. Could you make it visible when the thread is really finished? At this moment I only get the "Max reached" after all the postmessage-messages but I don't think that is correct. Even after the Max is reached there should still be messages being processed. It's just an optical illusion of the list I think. I have added gettickcount() to the output, here is what I get: postmessage :10 @291811315 OK. Thread reached max. @291811303 attached is thread-Sendmessage4.zip ---------------------------------------------------------------------- Michael Collier · 2015-10-22 19:16 ..I think I understand a bit better what you meant by the max should be displayed long before messages have been processed. On my slow Mac the same program does as you describe, it shows the max has been reached, then continues to count up the postmessages(). Can't seem to get it to do same on linux..? ---------------------------------------------------------------------- rvk · 2015-10-22 21:07 Yes, exactly. And when you cancel the thread by clicking the free button, the thread is terminated but there should still be lots of messages (from PostMessage) in the queue. So you should see that these are still being processed. I did manage it, a few days ago, to change the button to "ended" and then I could see the end and the messages were still being processed but I can't seem to recreate this with your last example. I'll try again tomorrow. Does anyone know what queuing mechanism is used on the Mac? As I understand it, on Windows the messaging-queue from Windows is used and on Linux there is a self-made list-queue mechanism used. (Am I right about that?) What about Mac? ---------------------------------------------------------------------- Michael Collier · 2015-10-22 22:42 If you change the post_message routine so that it goes back to adding lines instead of modifying line[0]  then you get the output you expect: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i     {    Memo1.Lines[0] := 'postmessage :'                    + IntToStr ( Message.lParam )                    + ' @'                    + IntToStr ( GetTickCount   )                    ;    }  add_line_to_memo( 'postmessage :'                   + IntToStr( Message.lParam )                   )                                  ; OK. Thread reached max. @306623698 postmessage :1 @306623705 postmessage :2 @306623705 postmessage :3 @306623705 Not sure why the memo couldn't sequence the output such that max is added to the list, whilst updates are made on line[0], but thought I'd let you know ---------------------------------------------------------------------- rvk · 2015-10-23 13:14 In the first image you see that @gettick for the postmessage is older than the end-thread. But... that should be logical. You did the GetTick in the process_message. You should save the GetTick at the moment of the PostMessage. I changed it here to show them both. I did it like this: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i         2 : begin  // PostMessage          PostMessage( form_thread_message.Handle                     , LM_POST_MESSAGE_TEST                     , GetTickCount                     , FCount+1                   )                                      ;and Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i     Memo1.Lines.Add( 'postmessage :'                   + IntToStr( Message.lParam )                   + ' @'                   + IntToStr ( GetTickCount   )                   + ' posted @' + inttostr(Message.wParam)                   + ' diff = ' + inttostr(GetTickCount - Message.wParam)                   )                                  ;In that case I get my second image. Now you can see the PostMessage is really done before the real processing. Somewhere around 5000 or 6000 the process slows down (or stops) and then, at once, all lines appear. But according to the log they where all added gradually. But at least it runs correctly for me. I couldn't get it to crash with the setting 1ms. When using 0ms it crashes at 1000 with a SIGABRT. When using 2ms it crashes at 7000. With what message does it crash for you? I still have the feeling that the message-queue under Linux isn't a 100% thread-safe. (But I lack the skills to check this) Sometimes it seems to hang and then after a few seconds all the lines appear immediately. Other times you get the SIGABRT crash. F.Y.I. I trace the SIGABRT back to TGtkMessageQueue.Lock which is called from TGtk2WidgetSet.AppProcessMessages: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i ... in Gtk2MsgQueueprocedure TGtkMessageQueue.Lock;begin  inc(fLock);  if fLock=1 then    {$IFDEF USE_GTK_MAIN_OLD_ITERATION}    EnterCriticalsection(FCritSec);    {$ELSE}    g_main_context_acquire(FMainContext);  // <-- no check is lock succeeds ??    {$ENDIF}end; ... in gtk2widgetset.inc   // then handle our own messages    while not Application.Terminated do begin      fMessageQueue.Lock;      // <--------- SIGABRT ---------------------------------------------------------------------- Michael Collier · 2015-10-27 12:51 Thanks for the help on this, I have opted to use synchronize for my own application and it is able to run 24+ hours without error. The lazarus wiki multithreading page mentions/promotes postmessage() & sendmessage() Quote one good option being the usage of SendMessage or PostMessage http://wiki.freepascal.org/Multithreaded_Application_Tutorial#Using_Se ndMessage.2FPostMessage_to_communicate_between_threads Maybe a caveat needs adding for linux/mac users? It would have helped me when I first encountered this (bug?) ---------------------------------------------------------------------- taazz · 2015-10-27 13:24 Quote from: rvk on October 21, 2015, 09:34:52 amO wow, for Windows you can indeed remove the ProcessMessages in my simple_postmessage.zip. BUT... on Linux you DO need the ProcessMessages. Otherwise the lines in memo1 don't get displayed until ALL messages are handled !!! erm that explanation doesn't make sense to me. Either the messages are not processed or they are processed. IF they are processed but not showing on screen try adding a call to invalidate in the process procedure instead of calling the processMessages and see if that helps at all. If it does help then the problem can be in any part of the framework. If it doesn't help and still you get all the lines at the end then there is probably a deadlock somewhere, not so easy to debug but far smaller area to check. Quote from: rvk on October 21, 2015, 09:34:52 amMy simple_postmessage is a simple program without threads but apparently Windows sees the chance to update the memo1 but Linux does not. So the messaging handling in Linux is really really different Is it linux or gtk? If compiled as a QT application does it have the same problem? I'm asking because I think that QT hsa its own message queue implementation that is similar with the windows message queue. Oh wait I have a 32bit linux now I can test my self!! Let me do that and get back to you. ---------------------------------------------------------------------- rvk · 2015-10-28 13:00 Quote from: taazz on October 27, 2015, 02:24:39 pmQuote from: rvk on October 21, 2015, 09:34:52 amO wow, for Windows you can indeed remove the ProcessMessages in my simple_postmessage.zip. BUT... on Linux you DO need the ProcessMessages. Otherwise the lines in memo1 don't get displayed until ALL messages are handled !!! erm that explanation doesn't make sense to me.It wasn't an explanation. It was an observation. It also puzzles me why this is happening. This is what's going on... (with my simple-example and no threads) When running the loop with 20.000 postmessages Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i   for p := 1 to 20000 do  begin    PostMessage(Self.Handle, LM_POST_MESSAGE, p, p);    Button1.Caption := IntToStr(p);  end;In Windows you see the button-caption running up. In Linux it directly displays 20.000. So the Windows-messaging somehow seems to keep doing the Paint for your form (even in a tight loop, maybe because of the PostMessage). In Linux paint-handling is only performed AFTER this procedure. Now for the post_message() procedure: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i procedure TForm1.post_message(var Msg: TLMessage);begin  Memo1.Lines.add('p = ' + IntToStr(Msg.wParam) + ' ' + IntToHex(Integer(sPtr),4));  //Application.ProcessMessages;end;In Windows the same happens. Without the ProcessMessages it can still paint the Memo1. So in Windows you see all the lines appear after each other. In Linux nothing seems to happen and the memo1 is only updated when ALL the post_messages are processed. Only if you use the ProcessMessages you'll see them appear sequentially. So somehow under Windows there is a separate process which handles the messaging for handing the painting and it runs besides the "normal" PostMessages-handling. Under Linux all messages (paint and post etc) are handled sequentially so the painting of the form is only performed at the end when all post_messages are processed (or when you insert a ProcessMessages). I hope I explained this clearly. But it demonstrates that the messaging-handling under Linux is way less advanced (or flexible) (?). QuoteIs it linux or gtk? If compiled as a QT application does it have the same problem? I'm asking because I think that QT hsa its own message queue implementation that is similar with the windows message queue.I used gtk (Debian, Lazarus 1.5 r48991M FPC 2.6.5 i386-linux-gtk 2) QuoteOh wait I have a 32bit linux now I can test my self!! Let me do that and get back to you. You'll see for yourself what I meant by above "observation" Note: B.T.W. All this is (probably) unrelated to the problem we're having when using PostMessage in a thread under Linux. I understand that SendMessage is a problem (because it is still executed in the thread which calls is, I checked this) but PostMessage should be handled differently and in the correct thread (I also checked this). But there is still a problem when using PostMessage in threads (when you use about 1000 to 1500 it crashes). ---------------------------------------------------------------------- taazz · 2015-10-31 05:28 Quote from: rvk on October 28, 2015, 02:00:58 pmIt wasn't an explanation. It was an observation. It also puzzles me why this is happening. OK. here is what I see on my memo when I run the application in opensuse 13 32bit. Code: [Select]3067414656:2956725056 :: 2956725056.SendMessage :596 3067414656 | Terminating... @165564669 3067414656 | Waiting... @165564669 3067414656 | [No Result] @165564681 3067414656 | Freeing... @165564686 Click "Create Server Thread" to start.. what you see on the first line, from left to right, :My_Server_Thread.ThreadID :: .SendMessage etc etc etc. This is proof that the message handling procedure is invoked from the thread that send the message instead of the application main thread. Any access to lcl components inside that handler is unsafe and should be avoided at all costs. In fact I think its some strange luck that allowed us to see anything in the memo so far or simple the QT library has better multi threading support. I'm going to run the same test on GTK and see the differences there. After I'm done testing I'll attach some screenshots too and if I manage it a gif showing the application updating it self with the update level set to none. By the way I made a couple of changes on the application namely I removed all access to lcl controls from the thread class and ofcourse removed all the calls to ProcessMessages in the main form too, except the one in the Do_updates procedure. Overall it seems that SendMessage does not do what advertises under QT and I'm afraid under GTK too. Edit: Just in case I get side tracked again here is the test with all the changes I made. Make sure that you choose none for update method before pressing the create server button.Added a screenshot from a windows test too to show the difference in execution. Code: [Select]3067414656: :: 3067414656.PostMessage :36283 @173038895 3067414656 | Terminating... @173038875 3067414656 | Waiting... @173038875 3067414656 | [No Result] @173038875 Click "Create Server Thread" to start.. 3067414656 | Freeing... @173038877 Click "Create Server Thread" to start.. here are the results from the post message test it seems that post message handler is executed in the main thread both in QT and GTK.