Subj : Can long blocking tasks scale?
To   : comp.programming.threads
From : Hans Malherbe
Date : Wed Aug 24 2005 01:26 pm

(Apologies if this is a double post)

What if your Windows server has to call a service that takes "long" and
you need to make every effort to make it scale to as many concurrent
requests as possible?

The problem is that the called service is a "black box" (mainframe
program) which cannot be divided into smaller bits.

Currently every request consumes a thread which is mostly busy doing
nothing but wait.

Assuming the called service takes 40 seconds to respond with a request
rate of 20 per second, we'll need to create 800 threads before the
first ones start to return.

It seems silly to hit the wall at 50 requests per second (2000 threads
with 1 MB stack reserved per thread on Windows 2000 Server) when
essentially they are all just waiting instead of doing.

One proposed solution was to have the client make the request and drop
the connection after which the client will poll to retrieve the result.
The server thread will request from the "black box" and exit. The
problem with this is that the "black box" must change significantly.

Complexity aside, how does this solution stack up against the
alternative of one thread per request?

Am I even asking the right questions?

Groete
Hans Malherbe

.