From nobody@FreeBSD.org  Thu Jan 25 09:48:08 2001
Return-Path: <nobody@FreeBSD.org>
Received: from freefall.freebsd.org (freefall.FreeBSD.org [216.136.204.21])
	by hub.freebsd.org (Postfix) with ESMTP id 78ECA37B69F
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 25 Jan 2001 09:48:08 -0800 (PST)
Received: (from nobody@localhost)
	by freefall.freebsd.org (8.11.1/8.11.1) id f0PHm8B52012;
	Thu, 25 Jan 2001 09:48:08 -0800 (PST)
	(envelope-from nobody)
Message-Id: <200101251748.f0PHm8B52012@freefall.freebsd.org>
Date: Thu, 25 Jan 2001 09:48:08 -0800 (PST)
From: earl_chew@agilent.com
To: freebsd-gnats-submit@FreeBSD.org
Subject: pthread_rwlock_rdlock can deadlock
X-Send-Pr-Version: www-1.0

>Number:         24641
>Category:       kern
>Synopsis:       pthread_rwlock_rdlock can deadlock
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    freebsd-threads
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Thu Jan 25 09:50:00 PST 2001
>Closed-Date:    Mon Jun 05 00:30:47 GMT 2006
>Last-Modified:  Mon Jun 05 00:30:47 GMT 2006
>Originator:     Earl Chew
>Release:        3.3-RELEASE
>Organization:
Agilent Technologies
>Environment:
FreeBSD qar1 3.3-RELEASE FreeBSD 3.3-RELEASE #0: Tue Oct 17 16:21:01 EDT 2000
>Description:
The man page for pthread_rwlock_rdlock() says:

   A thread may hold multiple concurrent read locks.  If so,
   pthread_rwlock_unlock() must be called once for each lock obtained.

I tried this out after reading Bill Lewis' article on Usenet:
http://x64.deja.com/[ST_rn=ps]/getdoc.xp?AN=573590892&search=thread&CONTEXT=980444523.1181483044&HIT_CONTEXT=980444500.1181483040&HIT_NUM=11&hitnum=3

Bill wrote:

    Consider: T1 gets a read lock. T2 wants a write
    lock & blocks, waiting for T1. Now T1 wants a read
    lock again. Because writers have priority, T1 will
    now block, waiting for T2.

My test program prints the following, then gets stuck:

    > ./a.out
    Attempt to acquire read lock first
    Acquired read lock first
    Attempt to acquire write lock
    Attempt to acquire read lock second

Examining the source code (uthread_rwlock.c,v 1.5) it is pretty
clear why this deadlock occurs (as Bill Lewis predicts).

>How-To-Repeat:
#include <pthread.h>
#include <stdio.h>
#include <unistd.h>
#include <assert.h>

static pthread_rwlock_t rwlock1 = PTHREAD_RWLOCK_INITIALIZER;

static volatile int wrStarted;

void * wrfunc(void *)
{
  printf("Attempt to acquire write lock\n");
  assert(pthread_rwlock_wrlock(&rwlock1) == 0);
  printf("Acquired write lock\n");

  assert(pthread_rwlock_unlock(&rwlock1) == 0);

  return 0;
}

static volatile int rdStarted;

void * rdfunc(void *)
{
  printf("Attempt to acquire read lock first\n");
  assert(pthread_rwlock_rdlock(&rwlock1) == 0);
  printf("Acquired read lock first\n");

  rdStarted = 1;

  while (wrStarted == 0)
      sleep(1);

  printf("Attempt to acquire read lock second\n");
  assert(pthread_rwlock_rdlock(&rwlock1) == 0);
  printf("Acquired read lock second\n");

  assert(pthread_rwlock_unlock(&rwlock1) == 0);
  assert(pthread_rwlock_unlock(&rwlock1) == 0);

  return 0;
}

int
main(int, char**)
{
  pthread_t wrt;
  pthread_t rdt;
  pthread_attr_t a;

  assert(pthread_rwlock_init(&rwlock1, 0) == 0);

  assert(pthread_attr_init(&a) == 0);

  assert(pthread_create(&rdt, &a, rdfunc, NULL) == 0);

  while (rdStarted == 0)
      sleep(1);

  assert(pthread_create(&wrt, &a, wrfunc, NULL) == 0);

  assert(pthread_join(wrt, 0) == 0);
  assert(pthread_join(rdt, 0) == 0);

  assert(pthread_rwlock_destroy(&rwlock1) == 0);

  assert(pthread_detach(wrt) == 0);
  assert(pthread_detach(rdt) == 0);

  return 0;
}

>Fix:


>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->jasone 
Responsible-Changed-By: jasone 
Responsible-Changed-When: Mon Dec 17 11:33:22 PST 2001 
Responsible-Changed-Why:  
I'll take a look at this. 

http://www.FreeBSD.org/cgi/query-pr.cgi?pr=24641 
Responsible-Changed-From-To: jasone->freebsd-bugs 
Responsible-Changed-By: jasone 
Responsible-Changed-When: Sat May 11 15:22:34 PDT 2002 
Responsible-Changed-Why:  


http://www.freebsd.org/cgi/query-pr.cgi?pr=24641 
Responsible-Changed-From-To: freebsd-bugs->freebsd-threads 
Responsible-Changed-By: kris 
Responsible-Changed-When: Sat Jul 12 18:36:26 PDT 2003 
Responsible-Changed-Why:  
Assign to threads mailing list  

http://www.freebsd.org/cgi/query-pr.cgi?pr=24641 

From: Earl Chew <earl_chew@agilent.com>
To: The Thodes <aspiesrule@mcleodusa.net>
Cc: freebsd-gnats-submit@FreeBSD.org
Subject: Re: misc/24641: pthread_rwlock_rdlock can deadlock
Date: Mon, 05 Jan 2004 11:11:40 -0800

 The Thodes wrote:
 > First, Earl, GCC objected to your test program.
 
 Apologies. I compiled this as a C++ program (g++) hence the missing
 parameter names.
 
 > The patch given is needed to make it compile (in my case, at least).
 
 Yes, your patch will fill in the "junk" names, and allow the C
 compiler to parse the source successfully.
 
 > Also, it (your test program)
 > doesn't even get to trying to acquire the second read lock.  It hangs 
 > trying to acquire the write lock.  The output is provided in this (long) 
 > follow-up.
 
 Oops... there was a cut-and-paste error. Please look at the amended
 program below.
 
 Earl
 --
 #include <pthread.h>
 #include <stdio.h>
 #include <unistd.h>
 #include <assert.h>
 
 static pthread_rwlock_t rwlock1 = PTHREAD_RWLOCK_INITIALIZER;
 
 static volatile int wrStarted;
 
 void * wrfunc(void *unused)
 {
    printf("Attempt to acquire write lock\n");
    assert(pthread_rwlock_wrlock(&rwlock1) == 0);
    printf("Acquired write lock\n");
 
    assert(pthread_rwlock_unlock(&rwlock1) == 0);
 
    return 0;
 }
 
 static volatile int rdStarted;
 
 void * rdfunc(void *unused)
 {
    printf("Attempt to acquire read lock first\n");
    assert(pthread_rwlock_rdlock(&rwlock1) == 0);
    printf("Acquired read lock first\n");
 
    rdStarted = 1;
 
    while (wrStarted == 0)
        sleep(1);
 
    printf("Attempt to acquire read lock second\n");
    assert(pthread_rwlock_rdlock(&rwlock1) == 0);
    printf("Acquired read lock second\n");
 
    assert(pthread_rwlock_unlock(&rwlock1) == 0);
    assert(pthread_rwlock_unlock(&rwlock1) == 0);
 
    return 0;
 }
 
 int
 main(int argc, char **argv)
 {
    pthread_t wrt;
    pthread_t rdt;
    pthread_attr_t a;
 
    assert(pthread_rwlock_init(&rwlock1, 0) == 0);
 
    assert(pthread_attr_init(&a) == 0);
 
    assert(pthread_create(&rdt, &a, rdfunc, NULL) == 0);
 
    while (rdStarted == 0)
        sleep(1);
 
    assert(pthread_create(&wrt, &a, wrfunc, NULL) == 0);
 
    assert(pthread_join(wrt, 0) == 0);
    assert(pthread_join(rdt, 0) == 0);
 
    assert(pthread_rwlock_destroy(&rwlock1) == 0);
 
    assert(pthread_detach(wrt) == 0);
    assert(pthread_detach(rdt) == 0);
 
    return 0;
 }
 

From: Earl Chew <earl_chew@agilent.com>
To: aspiesrule@mcleodusa.net
Cc: freebsd-gnats-submit@freebsd.org
Subject: Re: misc/24641: pthread_rwlock_rdlock can deadlock
Date: Mon, 05 Jan 2004 14:21:00 -0800

 aspiesrule@mcleodusa.net wrote:
 > Same results, except for the fact that stepping thru your code in GDB yields 
 > a segfault in KERNEL32!IsBadWritePtr.
 
 It must be the holidays! Ok, I've added the missing line at the
 beginning of wrfunc().
 
 Earl
 --
 #include <pthread.h>
 #include <stdio.h>
 #include <unistd.h>
 #include <assert.h>
 
 static pthread_rwlock_t rwlock1 = PTHREAD_RWLOCK_INITIALIZER;
 
 static volatile int wrStarted;
 
 void * wrfunc(void *unused)
 {
    wrStarted = 1;
 
    printf("Attempt to acquire write lock\n");
    assert(pthread_rwlock_wrlock(&rwlock1) == 0);
    printf("Acquired write lock\n");
 
    assert(pthread_rwlock_unlock(&rwlock1) == 0);
 
    return 0;
 }
 
 static volatile int rdStarted;
 
 void * rdfunc(void *unused)
 {
    printf("Attempt to acquire read lock first\n");
    assert(pthread_rwlock_rdlock(&rwlock1) == 0);
    printf("Acquired read lock first\n");
 
    rdStarted = 1;
 
    while (wrStarted == 0)
        sleep(1);
 
    printf("Attempt to acquire read lock second\n");
    assert(pthread_rwlock_rdlock(&rwlock1) == 0);
    printf("Acquired read lock second\n");
 
    assert(pthread_rwlock_unlock(&rwlock1) == 0);
    assert(pthread_rwlock_unlock(&rwlock1) == 0);
 
    return 0;
 }
 
 int
 main(int argc, char **argv)
 {
    pthread_t wrt;
    pthread_t rdt;
    pthread_attr_t a;
 
    assert(pthread_rwlock_init(&rwlock1, 0) == 0);
 
    assert(pthread_attr_init(&a) == 0);
 
    assert(pthread_create(&rdt, &a, rdfunc, NULL) == 0);
 
    while (rdStarted == 0)
        sleep(1);
 
    assert(pthread_create(&wrt, &a, wrfunc, NULL) == 0);
 
    assert(pthread_join(wrt, 0) == 0);
    assert(pthread_join(rdt, 0) == 0);
 
    assert(pthread_rwlock_destroy(&rwlock1) == 0);
 
    assert(pthread_detach(wrt) == 0);
    assert(pthread_detach(rdt) == 0);
 
    return 0;
 }
 

From: Earl Chew <earl_chew@agilent.com>
To: aspiesrule@mcleodusa.net
Cc: freebsd-gnats-submit@freebsd.org
Subject: Re: misc/24641: pthread_rwlock_rdlock can deadlock
Date: Mon, 05 Jan 2004 15:26:25 -0800

 aspiesrule@mcleodusa.net wrote:
 > Oops, running your new program yields an assertion failure at line 37 and a 
 > core dump.  The output is as follows:
 > 
 > --snip--
 > Attempt to acquire read lock first
 > Acquired read lock first
 > Attempt to acquire write lock
 > Attempt to acquire read lock second
 > assertion "pthread_rwlock_rdlock(&rwlock1) == 0" failed: file "test.c", line 
 > 37
 > Aborted (core dumped)
 
 Right. The test is (correctly) showing an inadequacy in the rwlock
 implementation.
 
 I refer you to the original bug report:
 
 http://www.freebsd.org/cgi/query-pr.cgi?pr=24641
 
 and the referenced articles:
 
 http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&oe=UTF-8&selm=slrn87nusa.rsv.kaz%40ashi.FootPrints.net
 http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&oe=UTF-8&selm=38828D22.7A98%40LambdaCS.com
 
 Earl
 
State-Changed-From-To: open->closed 
State-Changed-By: rodrigc 
State-Changed-When: Mon Jun 5 00:28:27 UTC 2006 
State-Changed-Why:  
Problem does not occur in libpthread in FreeBSD 6.x. 

Your testcase needs to be modified slightly. 
If you call pthread_join(), 
then if you call pthread_detach() later on, pthread_detach 
may return EINVAL, because the thread may already be 
terminated and detached, so you can't detach it. 


#include <pthread.h> 
#include <stdio.h> 
#include <unistd.h> 
#include <assert.h> 
#include <errno.h> 

static pthread_rwlock_t rwlock1 = PTHREAD_RWLOCK_INITIALIZER; 

static volatile int wrStarted; 

void * wrfunc(void *unused) 
{ 
wrStarted = 1; 

printf("Attempt to acquire write lockn"); 
assert(pthread_rwlock_wrlock(&rwlock1) == 0); 
printf("Acquired write lockn"); 

assert(pthread_rwlock_unlock(&rwlock1) == 0); 

return 0; 
} 

static volatile int rdStarted; 

void * rdfunc(void *unused) 
{ 
printf("Attempt to acquire read lock firstn"); 

assert(pthread_rwlock_rdlock(&rwlock1) == 0); 
printf("Acquired read lock firstn"); 

rdStarted = 1; 

while (wrStarted == 0) 
sleep(1); 

printf("Attempt to acquire read lock secondn"); 
assert(pthread_rwlock_rdlock(&rwlock1) == 0); 
printf("Acquired read lock secondn"); 

assert(pthread_rwlock_unlock(&rwlock1) == 0); 
assert(pthread_rwlock_unlock(&rwlock1) == 0); 

return 0; 
} 

int 
main(int argc, char **argv) 
{ 
pthread_t wrt; 
pthread_t rdt; 
pthread_attr_t a; 
int ret; 

assert(pthread_rwlock_init(&rwlock1, 0) == 0); 

assert(pthread_attr_init(&a) == 0); 

assert(pthread_create(&rdt, &a, rdfunc, NULL) == 0); 

while (rdStarted == 0) 
sleep(1); 

assert(pthread_create(&wrt, &a, wrfunc, NULL) == 0); 

assert(pthread_join(wrt, 0) == 0); 
assert(pthread_join(rdt, 0) == 0); 

assert(pthread_rwlock_destroy(&rwlock1) == 0); 

if ((ret = pthread_detach(wrt)) != 0) { 
if (ret == EINVAL) { 
printf("wrt already detachedn"); 
} 
} 

if ((ret = pthread_detach(rdt)) != 0) { 
if (errno == EINVAL) { 
printf("rdt already detachedn"); 
} 
}  
return 0; 
} 


http://www.freebsd.org/cgi/query-pr.cgi?pr=24641 
>Unformatted:
