Need multi-Client / Server help lazarus / 38-Networking and Web Programming ---------------------------------------------------------------------- WalterK · 2011-10-10 17:09 I guess I'm experiencing brain freeze or something, having gotten bogged down in the various facets of the task, so I'm asking for suggestions / experiences / pitfalls to avoid / etc. in writing the server side of a client-server application for Linux. I had written a simple server in this fashion:    Sock := fpSocket (AF_INET,SOCK_STREAM,0);    SAddr.sin_family := AF_INET;    SAddr.sin_port := htons(SolvPort);    SAddr.sin_addr.s_addr := 0;    if (fpBind (Sock,@SAddr,sizeof(saddr)) = -1) then       begin       (exit);       end;    if (fpListen (Sock,1) = -1) then       PError ('Solver : Listen : ');    while (running) do       begin          if (not Accept (Sock,SrcAddr,Sin,Sout)) then             PError ('Solver : Accept : '+NetAddrToStr(SrcAddr.sin_addr));          SRD[ind].FIn := Sin;          SRD[ind].FOut := Sout;          SRD[ind].SrcAddy := SrcAddr;          ptr := @SRD[ind];          BeginThread (@ClientRun,ptr);          ...        end (various details omitted) I had never stress tested it, but it seemed to work fine.  Then I discovered that a socket error (e.g. the client side program exiting prematurely) would blow out the server program. (Small twinge of cold in the brain here.) Then I realized that I ought to spawn a process rather than a thread because of the various/unknown errors that might creep into the ClientRun routines - a small but growing set of code. So now the local databases (not shown) which I was going to access to keep client data and IP-data for filtering now needs to be multi- process capable.  (Brain got chilly.) I then learned that instead of knowing the ClientRun routines being done in several seconds, they instead ~might~ take an hour or more in some cases, so I had to prepare to disconnect nicely and reconnect later, picking up the results after re-login. (Brain starting to numb up.) (I can figure out the data pick-up code, after I get my bearings with the rest of it.) I likewise cannot pass network handles through a calling sequence hack, but have to pass them some other way.  (Brain froze here.) I'll have MySQL available on the server this code is destined for, but I was hoping to use a local (on machine) database of sorts (I have my own I had planned to use) to keep things fast. Can anyone give me pointers, suggested design, examples, etc.? Thank you. Walter ---------------------------------------------------------------------- jarto · 2011-10-10 19:23 On Linux you need to trap the SIGPIPE signal and handle it. Otherwise your FPC program bombs out. Set the handler: fpSignal(SIGPIPE, SignalHandler(@Handle_Linux_Signal)); handle the signal: procedure Handle_Linux_Signal(sig: LongInt); cdecl; begin   case sig of      SIGPIPE: //Ignore or handle any way you want.   end; end; AFAIK, Kylix used to ignore SIGPIPE. Edit: You need to add baseunix to your uses line. Other signals worth handling are SIGINT, SIGHUP and SIGQUIT. Edit2: And you don't need to spawn processes. Threads work very well. You might want to have a look at synapse for an example of a small server socket implementation. ---------------------------------------------------------------------- marcov · 2011-10-10 19:32 Do you get any warnings while compiling? ---------------------------------------------------------------------- mdalacu · 2011-10-10 19:39 Use Sinapse SVN version. ---------------------------------------------------------------------- jarto · 2011-10-10 19:55 And while I'm at it, let me help you with problems you may run into later ;-) 1. Trap that SIGPIPE. It is THE reason your program bombs out on Linux. 2. When you create the client thread during a connect, give the thread a smaller stack size. Otherwise you won't be able to create more than a few hundred simultaneous threads. 3. Make sure you monitor all your connections. Add new threads to a list and remove them from the list when they disconnect. Sometimes you won't get a disconnect from a dead socket causing extra threads to hang around. 4. Handle exceptions and do all locks well as you're writing multithreaded code. ---------------------------------------------------------------------- WalterK · 2011-10-12 15:26 Sorry for not replying sooner... jarto: Thank you for pointing out the need to handle signals - will do (have done).  And the other things, too. marcov: No there are no warnings from those sections of code. mdalacu: I'll look at Synapse; I know I need some help in handling the grittier points of net interfacing. Thanks. Walter ---------------------------------------------------------------------- WalterK · 2011-10-23 16:55 About handling signals... (To recap) In my daemon design, the main program waits for a network connect with Accept, and then spins off a thread to handle the request(s). I added a signal handler in the main program... this catches a network connection breaking (SigPipe), but - since it's a bit hard for me to construct a sure-fire test using more than one client being active - can someone tell me whether putting the handler inside the thread will isolate the signal handler to that one thread or not?  Obviously, I need to catch errors off of particular connections and leave the others alone. Also, rather than me introducing a flag as to whether a network pipe is good (connected) or not, is there a system flag I should test instead? TIA. Walter ---------------------------------------------------------------------- jarto · 2011-11-09 20:37 Quote from: WalterK on October 23, 2011, 06:55:09 pmAbout handling signals... (To recap) In my daemon design, the main program waits for a network connect with Accept, and then spins off a thread to handle the request(s). I added a signal handler in the main program... this catches a network connection breaking (SigPipe), but - since it's a bit hard for me to construct a sure-fire test using more than one client being active - can someone tell me whether putting the handler inside the thread will isolate the signal handler to that one thread or not?  Obviously, I need to catch errors off of particular connections and leave the others alone. Also, rather than me introducing a flag as to whether a network pipe is good (connected) or not, is there a system flag I should test instead? I use Synapse, where I'll just try to read from the socket with SOcket.RecvPacket. If Socket.LastError is anything other, than 10060 or 110, I consider the socket connection is alive any longer. Check Synapse's httpserver demo for a nice and small example of a server that does much of what you're doing. At least with Synapse and Indy, it's safe to ignore the SIGPIPE- signal. You need to catch it, but there's no need to try to find the right thread and handle it any futher.