From nobody@FreeBSD.ORG  Fri Oct  6 06:41:42 2000
Return-Path: <nobody@FreeBSD.ORG>
Received: by hub.freebsd.org (Postfix, from userid 32767)
	id 41BDB37B672; Fri,  6 Oct 2000 06:41:42 -0700 (PDT)
Message-Id: <20001006134142.41BDB37B672@hub.freebsd.org>
Date: Fri,  6 Oct 2000 06:41:42 -0700 (PDT)
From: wilco.oelen@cmg.nl
Sender: nobody@FreeBSD.ORG
To: freebsd-gnats-submit@FreeBSD.org
Subject: When msgrcv() blocks, it blocks ALL threads in multi-threaded application
X-Send-Pr-Version: www-1.0

>Number:         21783
>Category:       kern
>Synopsis:       When msgrcv() blocks, it blocks ALL threads in multi-threaded application
>Confidential:   no
>Severity:       critical
>Priority:       high
>Responsible:    freebsd-bugs
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Fri Oct 06 06:50:01 PDT 2000
>Closed-Date:    Thu Jan 31 02:27:04 PST 2002
>Last-Modified:  Thu Jan 31 02:27:49 PST 2002
>Originator:     Wilco Oelen
>Release:        3.5.1
>Organization:
CMG
>Environment:
FreeBSD bsd1.cmg.nl 3.5.1-RELEASE FreeBSD 3.5.1-RELEASE #1: Thu Oct 5 16:16:53 GMT 2000  root@bsd1:/usr/src/sys/compile/LOCAL i386
>Description:
When an application uses multiple threads (created by pthread_create()), and one of the threads performs a blocking msgrcv() call, then all threads are blocked, not just the thread which does the msgrcv() call.
>How-To-Repeat:
We have created a little test program, which creates a thread, in which a little loop just prints a line of text every 200 milliseconds. The main thread reads from a message queue, which is created by the program, just before the thread is created.

The program can be compiled using the command

cc -D_THREAD_SAFE -o tst tst.c -pthread

Here it is assumed that the program source is called tst.c.

When the program is started, without arguments, then you see that it prints a line of text every 200 milliseconds. After 5 seconds this stops. The program creates a message queue with key 12345678. When this key is removed (using ipcrm -q <ID>, where <ID> is the queue id for the queue with KEY==12345678), then the program appears to continue and it mixes messages about read errors every second through the output.

We expect the program to print messages every 200 milliseconds, also if the message queue is not yet broken and the main thread is still waiting in msgrcv(). 

Here follows the C-program:



/**************** START OF PROGRAM **********************/

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <pthread.h>

#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/msg.h>


static void *thread_func(void *dummy);




int main(void)
{
  int msqid_in;
  key_t key = (key_t)12345678;
  pthread_t thr1;
  char msg[100];

  /* Create a message queue. */
  msqid_in = msgget(key, 0644 | IPC_CREAT);

  /* Create a new thread. */
  pthread_create(&thr1, NULL, thread_func, NULL);

  /* Sleep a while, in the meantime the other thread runs. */
  sleep(5);

  while (1)
   {
    int status;

    /* Read from the message queue. This read will block.   */
    /* We expect this thread to block, but the other thread */
    /* should continue, however, it doesn't.                */
    status = msgrcv(msqid_in,
               (struct msgbuf *)&msg, 10, 0, MSG_NOERROR);
    if (status == -1)
     {
      /* This code is reached if one removes the message */
      /* queue, while this program is running. If this   */
      /* code is reached, then the other thread also     */
      /* runs again.                                     */
      printf("ERROR receiving message\n");
      sleep(1);
     }
    else
     {
      printf("Received message\n");
     }
   }
  return 0;
}




static void *thread_func(void *dummy)
{
  while (1)
   {
    usleep(200000);
    printf("Thread function\n");
   }
  return NULL;
}

/********************* END OF PROGRAM **************************/



The problem also exists for programs, where the msgrcv() call is not in the main thread but in another thread. If a program is created with 3 or 4 threads, then also ALL threads block, if the msgrcv() call blocks.

We compared the behaviour with LINUX and Digital OSF4.0f. On these platforms the other threads continue if one of them blocks in a msgrcv() call.
>Fix:


>Release-Note:
>Audit-Trail:

From: "David Schwartz" <davids@webmaster.com>
To: <freebsd-gnats-submit@FreeBSD.org>, <wilco.oelen@cmg.nl>
Cc:  
Subject: Re: kern/21783: When msgrcv() blocks, it blocks ALL threads in multi-threaded application
Date: Sun, 26 Nov 2000 12:50:55 -0800

 	This is extremely unlikely to get fixed in FreeBSD3. For FreeBSD4, the best
 solution is for programs that need this functionality to compile/link with
 the LinuxThreads port (ports/devel/linuxthreads). The problem is that
 FreeBSD's POSIX threads library (libc_r) actually only has a single kernel
 execution vehicle. So anything that stalls that vehicle stalls all threads.
 If you need multiple kernel execution vehicles, you should be using a
 threads library that provides it.
 
 	David Schwartz
 
 
State-Changed-From-To: open->closed 
State-Changed-By: sheldonh 
State-Changed-When: Thu Jan 31 02:27:04 PST 2002 
State-Changed-Why:  
This will be addressed in 5.0-RELEASE later this year, with the advent 
of KSEs.  For now, the suggested work-around is to use linuxthreads. 

http://www.FreeBSD.org/cgi/query-pr.cgi?pr=21783 
>Unformatted:
