Subj : pthread_cond_wait locks everything up before other threads are created.
To   : comp.programming.threads
From : marcthenarc
Date : Sun Jan 16 2005 11:35 pm

Hello.

I've built this multithreaded application in C++ and pthreads, with a
boss thread and workers threads: boss waits for TCP connection and once
a connection is made, it pushes the socket of the incoming connection
in a job queue, where workers take turn looking in the job queue to
pick one up and deal with it.

My problem is during thread creation. I've noticed that the program
frequently locks on startup.  I must add that before any thread
creation, I create a socket, bind() and listen(), then I create the
worker threads, like so:

<code>
// Thread class wrapping a pthread_t member
class Thread() {
int no;
pthread_t pth;
} thread[MAX];

// Worker threads
for (i=1; i<TH_MAX; i++) {

th[i].no = i;

if ((ret = pthread_create(&(th[i].pth), NULL, Worker, (void
*)&th[i])))
BailOut("Thread creation error", ret);
}

// Boss thread
pthread_create(&(th[0].pth), NULL, Boss, (void *)&th[0])))

</code>

In the Worker function, one thread will lock all the workers and if
will wait until it is waken up by the boss thread that a job is in the
queue, like so:

<code>
void * Worker(void *foo)
{
Thread *th = (Thread *)foo;

// Tell me who you are ...
cout << th->no << endl;

for (;;)
{
// Lock workers
if ((ret = pthread_mutex_lock(&lock_workers)) != 0)
BailOut("pthread_mutex_lock", ret);

// Check for queue
while (q.empty() && !shutdown_called)
{
// Wait until boss broadcasts a wake_up call to this
worker, signaling that queue
// isn't empty anymore.
if ((ret = pthread_cond_wait(&queue_not_empty,
&lock_workers)) != 0)
BailOut("pthread_cond_wait", ret);
}

// Lock Boss out using a mutex specifically for the queue,
called lock_boss,
// fetch job and unlock lock_boss and then unlock lock_workers.

(...)
}

</code>

Sometimes worker-thread number 1 seems to lock everyting up when it
reaches pthread_cond_wait() as other threads don't even enter the
Worker function. The boss thread, created after the workers threads, is
obviously locked out too, never having the time to listen() to a
connection.

But when the program does successfully go beyond that lock, and is
ready to go, I never experience any more strange locking and the server
holds very well multiple connections.

Can anyone think of was could be going wrong at this point, early in
the program ?

NOTE: I didn't want to post the whole code on this post, but can be
provided in its entierety to any one who wants to take a peek at it, if
the GPL is acceptable to you.

Thanks.

---

Marc (the narc).

.