Subj : Re: Can long blocking tasks scale? To : comp.programming.threads From : dj3vande Date : Wed Aug 24 2005 09:20 pm In article <1124911577.598295.297800@o13g2000cwo.googlegroups.com>, Hans Malherbe wrote: >What if your Windows server has to call a service that takes "long" and >you need to make every effort to make it scale to as many concurrent >requests as possible? > >The problem is that the called service is a "black box" (mainframe >program) which cannot be divided into smaller bits. Is there an easy way for a single thread to handle multiple concurrent calls to the black box? F'rexample: fill request buffer with live requests loop: wait for any pending request to complete process completed request + enqueue results for output thread dequeue new request + send to black box Handling not-always-full input queues will make this a bit more complicated, but that will be mostly bookkeeping details and not a fundamental change to what's happening. (One possible extra problem is that if your buffer is partially full, you'll want to wait for either a request to complete or a new request to come in; this could require some care to set up.) Whether this is possible or easy will depend on how you're communicating with the black box. F'rexample, if it's through TCP with a socket connection for each request, you can select on the set of sockets (or even just poll them with non-blocking reads and see if you get data); if the interface on the caller side is a single blocking call, it gets a lot harder. Even if there's a finite upper bound on the number of requests a single thread can handle, for a given upper bound on the number of processing threads, you're increasing the total number of concurrent requests that you can handle by that factor. If you pass the client connection information along with the request, you can make the inputs and outputs fairly simple: input main loop: get client connection take request enqueue request + connection output main loop: dequeue results + connection send results clean up connection and localize the complexity of keeping multiple black box requests in-flight to just the processing thread. >One proposed solution was to have the client make the request and drop >the connection after which the client will poll to retrieve the result. >The server thread will request from the "black box" and exit. The >problem with this is that the "black box" must change significantly. >Complexity aside, how does this solution stack up against the >alternative of one thread per request? Since it should be possible to pass connections around between threads, forcing the client to poll for results is probably making life unnecessarily difficult when you can just let the connection sit in a queue on your end until you have results to send through it. If the black box interface is (or can be) set up to allow a single thread to have multiple requests pending, one or several threads per processing stage and passing the connection between them is probably a cleaner design than one thread per connection. dave -- Dave Vandervies dj3vande@csclub.uwaterloo.ca >It doesn't last long, but if that works OK on standard explosives, it ought to >work OK on antimatter explosions, too. --Dan Holdsworth and Mike Andrews Remind me to keep you _happy_ with me. in the scary devil monastery .