Threading - no idea what im doing lazarus / 38-Networking and Web Programming ---------------------------------------------------------------------- nugax · 2022-03-08 15:29 So I am trying to write a IRC like client. Command line based (actually over telnet connection). I have the read data TCP connection running fine in a separate thread that someone helped me with. My issue is that the write functions (from the read) will block at the get input portion of code. So, someone suggested that I put the read and write functions into separate threads, so as it stop blocking on the main program. The read is currently running in a thread, and I attempted to write the read functions in a  thread but Ill admit i have little knowledge of what I am doing and how to return the input back to the main program for writing purposes (as I believe you cannot write to the UI, even over telnet, from within a thread.) Ill post my entire code, its super messy as I have been experimenting, but if anyone can point me the right way to figure out threading I would be appreciative. Keep in mind, the reads and writes for writing to console are over a telnet connection, not direct to console, using Indy telnet server components. The collection of IRC messages is done over a SEPARATE TCPclient connection, and is contained in the thread ReadingThread. Im sure im all over the place, but a point in the right direction would be helpful! https://pastebin.com/YTWpzRvT ---------------------------------------------------------------------- alpine · 2022-03-08 17:23 Quote from: nugax on March 08, 2022, 04:29:05 pm*snip* Ill post my entire code, its super messy as I have been experimenting, but if anyone can point me the right way to figure out threading I would be appreciative. Keep in mind, the reads and writes for writing to console are over a telnet connection, not direct to console, using Indy telnet server components. *snip* If you're making a GUI application, I'll suggest you to use the TThread.Synchronize method. The main (GUI) thread spends it's entire time in processing window messages in a loop, so that method will execute the given procedure argument in the same loop/thread. You can read this Lazarus tutorial. BTW, it is not mandatory to use separate threads to achieve this, you can use non-blocking sockets to check whether there is something to read and to write when necessary in a periodic TTimer. It is much simplier. ---------------------------------------------------------------------- nugax · 2022-03-08 19:08 I am not writing a GUI program. This simply a IRC component in a telnet application, Linux, console-based. Quote from: y.ivanov on March 08, 2022, 06:23:07 pmQuote from: nugax on March 08, 2022, 04:29:05 pm*snip* Ill post my entire code, its super messy as I have been experimenting, but if anyone can point me the right way to figure out threading I would be appreciative. Keep in mind, the reads and writes for writing to console are over a telnet connection, not direct to console, using Indy telnet server components. *snip* If you're making a GUI application, I'll suggest you to use the TThread.Synchronize method. The main (GUI) thread spends it's entire time in processing window messages in a loop, so that method will execute the given procedure argument in the same loop/thread. You can read this Lazarus tutorial. BTW, it is not mandatory to use separate threads to achieve this, you can use non-blocking sockets to check whether there is something to read and to write when necessary in a periodic TTimer. It is much simplier. ---------------------------------------------------------------------- nugax · 2022-03-08 19:11 Is there a non blocking socket with Indy? I used Indy components for the telnet connection portion. Quote from: y.ivanov on March 08, 2022, 06:23:07 pmQuote from: nugax on March 08, 2022, 04:29:05 pm*snip* Ill post my entire code, its super messy as I have been experimenting, but if anyone can point me the right way to figure out threading I would be appreciative. Keep in mind, the reads and writes for writing to console are over a telnet connection, not direct to console, using Indy telnet server components. *snip* If you're making a GUI application, I'll suggest you to use the TThread.Synchronize method. The main (GUI) thread spends it's entire time in processing window messages in a loop, so that method will execute the given procedure argument in the same loop/thread. You can read this Lazarus tutorial. BTW, it is not mandatory to use separate threads to achieve this, you can use non-blocking sockets to check whether there is something to read and to write when necessary in a periodic TTimer. It is much simplier. ---------------------------------------------------------------------- SymbolicFrank · 2022-03-08 21:06 There are different kinds of threads. Blocking threads are totally fine, operating systems deploy thousands of them. If you have a service, you have a thread waiting for input. And while you could wake it up every 10 milliseconds or so, to look if there is data waiting, it's much more efficient to just suspend it until that data enters the queue / socket and it is needed. That thread will probably spawn another thread to handle the request, if data enters its socket. Simply because otherwise the socket wouldn't be available while it is handling that request. The new thread has a single task: handle the request. This can be calculating something or gathering data from a database and/or sending a response. And when it is done, it terminates. It isn't going to say that it is done. The thread that spawned it isn't going to wait for that completion. It is done and disappears. This might be a problem if you want a report. Has the request been handled in a satisfactory way? So, you could send a message to a queue to signal completion. That's basically the difference between UDP and TCP: the first one is fast but there is no guarantee. The second one does guarantee delivery, but can be very slow. Simply, because you have to keep a list, and when it takes too long (timeout), you have to start over. And the last type of thread is part of a thread pool: you give it a task (an execute function), which runs until completion. After that, it is ready for the next task. The problem with those threads is, that they require inter-thread communication to coordinate the actions. Which is, in general, a Bad Thing. Most of the time, it requires that threads wait for completion of the other threads, and there is additional overhead for handling those communications. Sometimes, this is the best option. Like, if you spawn multiple threads to calculate things from a GUI thread. But most of the time, it is easier and faster to simply do those calculations in the main thread, sequentially. So, to recap: blocking threads that wait for data are good. One-shot threads that post their result to a queue are good. Everything else is circumstantial, but probably not optimal. ---------------------------------------------------------------------- Warfley · 2022-03-08 23:22 First, a general thing I found when looking at your code, you have a Write and a Read thread, but both of them perform writing and reading. So I don't really see what the purpose of these threads is anyway. Also the WriteThread constructor does nothing (the context isn't defined anyway if I see correctly), meaning context will never be initialized, so whatever you intend to do in it, in this form it does nothing. Maybe you can explain further. On a side note, you could give my AsyncNet library a try (requires STAX). This allows to use non blocking sockets in async-await style tasks: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i procedure DoReceive(AExecutor: TExecutor; ASocket: TSocket);var  Message: String;begin  while True do  begin    Message := Await(AsyncReceiveLn(ASocket));    WriteLn('Message from server: ', Message);  end;end; procedure DoSend(AExecutor: TExecutor; ASocket: TSocket);var  UserInput: String;begin  While True do  begin  UserInput := Await(AsyncConsoleReadLn);    WriteLn('Sending to server: ', UserInput);    Await(AsyncSendLn(Socket, UserInput));  end;end; ...var  Connection: TSocket;begin  Connection:= TCPSocket(atIN4);  try    Await(AsyncConnect(Connection, IN4Address('127.0.0.1'), 1337));    AwaitAll([      AsyncProcedure(@DoReceive, Connection)),      AsyncProcedure(@DoSend, Connection))    ], ebRaiseAndTerminate);  finally    CloseSocket(Sock);  end;end;See the TCPTest, Pong and tcpexample for some further information (Note the latter two examples use the stax.asynctcp unit which is a stripped down version of asyncnet.sockets, but you should use asyncnet as it can do more) No synchronization issues, no race conditions, just read whenever you like, write whenever you like using the Async functions, and it will not block the other tasks. That said, STAX and AsyncNet are still rather experimental and haven't been tested on any large scale projects yet, but would be interesting if, as this is intendet to be easier than threads, this would actually make it easier for someone like you who, by your own admission, has some trouble getting into using threads right now ---------------------------------------------------------------------- alpine · 2022-03-09 08:52 Quote from: nugax on March 08, 2022, 08:08:54 pmI am not writing a GUI program. This simply a IRC component in a telnet application, Linux, console-based. It is still feasible to use TThread.Synchronize as long as you call regularly CheckSynchronize in your main loop. See Non LCL program into the multi-threading tutorial. Quote from: nugax on March 08, 2022, 08:11:49 pmIs there a non blocking socket with Indy? I used Indy components for the telnet connection portion. It seems that there isn't Indy non-blocking sockets, You should stick with the thread wrappers. Quote from: SymbolicFrank on March 08, 2022, 10:06:08 pm*snip* So, to recap: blocking threads that wait for data are good. One-shot threads that post their result to a queue are good. Everything else is circumstantial, but probably not optimal. Generally agreed with you, my point was the non-blocking sequential approach is much easier. For a simple client there is no need for more sophisticated ways, e.g. I/O completion ports, select constructs, thread pools, etc. ---------------------------------------------------------------------- MarkMLl · 2022-03-09 10:00 Quote from: nugax on March 08, 2022, 08:11:49 pmIs there a non blocking socket with Indy? I used Indy components for the telnet connection portion. You need to ask yourself what benefit something like Indy confers if you're implementing a a protocol at a lower level than anything it supports. I can't remember whether you've said what OS you're writing for- which you should have done, even if /you/ don't see its relevance- but generally speaking once you're dealing directly with a sockets API there are facilities for controlling blocking and timeouts, e.g. fpSelect() on unix platforms. MarkMLl ---------------------------------------------------------------------- nugax · 2022-03-09 15:13 I specified in the third messages what OS i was using this on: "I am not writing a GUI program. This simply a IRC component in a telnet application, Linux, console-based" Quote from: MarkMLl on March 09, 2022, 11:00:30 amQuote from: nugax on March 08, 2022, 08:11:49 pmIs there a non blocking socket with Indy? I used Indy components for the telnet connection portion. You need to ask yourself what benefit something like Indy confers if you're implementing a a protocol at a lower level than anything it supports. I can't remember whether you've said what OS you're writing for- which you should have done, even if /you/ don't see its relevance- but generally speaking once you're dealing directly with a sockets API there are facilities for controlling blocking and timeouts, e.g. fpSelect() on unix platforms. MarkMLl ---------------------------------------------------------------------- MarkMLl · 2022-03-09 15:42 Quote from: nugax on March 09, 2022, 04:13:39 pmI specified in the third messages what OS i was using this on: "I am not writing a GUI program. This simply a IRC component in a telnet application, Linux, console-based" ents for the telnet connection portion. Sorry, I'd missed the Linux bit. In that case whether or not you continue using Indy (or equivalent) you would probably benefit from a nodding acquaintance with the standard (Berkeley) sockets API, the non-blocking socket option, and basic use of fpSelect(). MarkMLl ---------------------------------------------------------------------- piola · 2022-03-09 16:14 Quote from: Warfley on March 09, 2022, 12:22:14 amOn a side note, you could give my AsyncNet library a try (requires STAX). This allows to use non blocking sockets in async-await style tasks This sounds interesting. I'm currently using Indy as well, but I'll give it a try. Maybe it can overcome some problems I have with Indy.