From nobody@FreeBSD.org  Fri May 16 11:21:17 2008
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 246961065671
	for <freebsd-gnats-submit@FreeBSD.org>; Fri, 16 May 2008 11:21:17 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (www.freebsd.org [IPv6:2001:4f8:fff6::21])
	by mx1.freebsd.org (Postfix) with ESMTP id 138F28FC0A
	for <freebsd-gnats-submit@FreeBSD.org>; Fri, 16 May 2008 11:21:17 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by www.freebsd.org (8.14.2/8.14.2) with ESMTP id m4GBJxq1085206
	for <freebsd-gnats-submit@FreeBSD.org>; Fri, 16 May 2008 11:19:59 GMT
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.14.2/8.14.1/Submit) id m4GBJxwO085205;
	Fri, 16 May 2008 11:19:59 GMT
	(envelope-from nobody)
Message-Id: <200805161119.m4GBJxwO085205@www.freebsd.org>
Date: Fri, 16 May 2008 11:19:59 GMT
From: Ivan Shcheklein <shcheklein@gmail.com>
To: freebsd-gnats-submit@FreeBSD.org
Subject: [patch][fix][sysvsem] System V semaphores cause unexpected hanging
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         123731
>Category:       kern
>Synopsis:       [patch][fix][sysvsem] System V semaphores cause unexpected hanging
>Confidential:   no
>Severity:       critical
>Priority:       high
>Responsible:    gonzo
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Fri May 16 11:30:02 UTC 2008
>Closed-Date:    Fri Jul 18 13:57:33 UTC 2008
>Last-Modified:  Fri Jul 18 14:00:09 UTC 2008
>Originator:     Ivan Shcheklein
>Release:        FREEBSD-6.2
>Organization:
ISPRAS
>Environment:
FreeBSD bs. 6.2-RELEASE FreeBSD 6.2-RELEASE #0: Fri Dec 14 20:38:20 MSK 2007     root@bsd.:/usr/obj/usr/src/sys/GENERIC  i386
>Description:
It seems that my previous PR (http://www.freebsd.org/cgi/query-pr.cgi?pr=118153) was too hard to understand. There is no any feedback for a long time and no one was assigned to consider the problem. Moreover it was tagged with [sysvshm] while [sysvsem] is quite better here.

I want to try once again since I suppose it is really critical bug.

The bug itself is the following:

in src/sys/kern/sysv_sem.c,v 1.78 2005/06/07 we have:

1. in line 1066 we get semptr through sem_base
  
    semptr = &semakptr->u.sem_base[sopptr->sem_num];  

2. in lines 1129-1333 we adjust counters using this semptr:

    if (sopptr->sem_op == 0)
        semptr->semzcnt++;
    else
        semptr->semncnt++;

3. in line 1135 we age going to sleep:

    error = msleep(semakptr, sema_mtxp, (PZERO - 4) | PCATCH, "semwait", 0);

4. in lines 1153-1156 after wake up we again adjust counters using the same semptr:

    if (sopptr->sem_op == 0)
         semptr->semzcnt--;
    else
         semptr->semncnt--;

During "msleep" in step 3 "sem_base" of the semaphore can be moved (if other process drops semaphore which was allocated before this one). Therefore, I believe, in step 4 "semptr" value can be invalid and MUST be renewed before use. 

As a circumstantial proof you can also consider the same Darwin's code:

----------------------------------------------------
		/*
		 * IMPORTANT: while we were asleep, the semaphore array might
		 * have been reallocated somewhere else (see grow_sema_array()).
		 * When we wake up, we have to re-lookup the semaphore 
		 * structures and re-validate them.
		 */

		suptr = NULL;	/* sem_undo may have been reallocated */
	 	semaptr = &sema[semid];	   /* sema may have been reallocated */
----------------------------------------------------

This bug can cause unexpected processes hanging in the multiprocess environment which extensively use System V semaphores.

What else do you need to consider this bug?


>How-To-Repeat:

>Fix:
Patch is attached which renew sempth after wakeup:

*** sysv_sem_old.c	Fri Dec 14 18:40:09 2007
--- sysv_sem.c	Fri Dec 14 18:23:46 2007
***************
*** 1145,1154 ****
--- 1145,1159 ----
  			error = EIDRM;
  			goto done2;
  		}
  
  		/*
+ 		 * Renew the semaphore's pointer after wakeup
+ 		 */
+ 		semptr = &semakptr->u.sem_base[sopptr->sem_num];
+ 
+ 		/*
  		 * The semaphore is still alive.  Readjust the count of
  		 * waiting processes.
  		 */
  		if (sopptr->sem_op == 0)
  			semptr->semzcnt--;

Note! Possibly some other strucutures should be also re-validates like in Darwin.

Patch attached with submission follows:

*** sysv_sem_old.c	Fri Dec 14 18:40:09 2007
--- sysv_sem.c	Fri Dec 14 18:23:46 2007
***************
*** 1145,1154 ****
--- 1145,1159 ----
  			error = EIDRM;
  			goto done2;
  		}
  
  		/*
+ 		 * Renew the semaphore's pointer after wakeup
+ 		 */
+ 		semptr = &semakptr->u.sem_base[sopptr->sem_num];
+ 
+ 		/*
  		 * The semaphore is still alive.  Readjust the count of
  		 * waiting processes.
  		 */
  		if (sopptr->sem_op == 0)
  			semptr->semzcnt--;


>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-i386->freebsd-bugs 
Responsible-Changed-By: gavin 
Responsible-Changed-When: Fri May 16 14:17:14 UTC 2008 
Responsible-Changed-Why:  
Not i386 specific 

http://www.freebsd.org/cgi/query-pr.cgi?pr=123731 
State-Changed-From-To: open->analyzed 
State-Changed-By: gonzo 
State-Changed-When: Wed Jun 18 16:58:16 UTC 2008 
State-Changed-Why:  
I was able to reproduce this bug. Will take care of it. 


Responsible-Changed-From-To: freebsd-bugs->gonzo 
Responsible-Changed-By: gonzo 
Responsible-Changed-When: Wed Jun 18 16:58:16 UTC 2008 
Responsible-Changed-Why:  
I was able to reproduce this bug. Will take care of it. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=123731 
State-Changed-From-To: analyzed->patched 
State-Changed-By: gonzo 
State-Changed-When: Thu Jun 19 18:09:06 UTC 2008 
State-Changed-Why:  
Fix commited 

http://www.freebsd.org/cgi/query-pr.cgi?pr=123731 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/123731: commit references a PR
Date: Thu, 19 Jun 2008 18:09:03 +0000 (UTC)

 gonzo       2008-06-19 18:08:42 UTC
 
   FreeBSD src repository
 
   Modified files:
     sys/kern             sysv_sem.c 
   Log:
   SVN rev 179879 on 2008-06-19 18:08:42Z by gonzo
   
   Renew semaphore's pointer after wakeup since during msleep
   sem_base may have been modified by destroying one of semaphores
   and semptr would not be valid in this case.
   
   PR: kern/123731
   
   Revision  Changes    Path
   1.91      +7 -0      src/sys/kern/sysv_sem.c
 _______________________________________________
 cvs-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/cvs-all
 To unsubscribe, send any mail to "cvs-all-unsubscribe@freebsd.org"
 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/123731: commit references a PR
Date: Fri, 18 Jul 2008 13:25:25 +0000 (UTC)

 gonzo       2008-07-18 13:24:56 UTC
 
   FreeBSD src repository
 
   Modified files:        (Branch: RELENG_7)
     sys/kern             sysv_sem.c 
   Log:
   SVN rev 180590 on 2008-07-18 13:24:56Z by gonzo
   
   MFC r179879:
   Renew semaphore's pointer after wakeup since during msleep
   sem_base may have been modified by destroying one of semaphores
   and semptr would not be valid in this case.
   
   PR:             kern/123731
   MFC after:      1 month
   
   Revision  Changes    Path
   1.89.2.1  +7 -0      src/sys/kern/sysv_sem.c
 _______________________________________________
 cvs-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/cvs-all
 To unsubscribe, send any mail to "cvs-all-unsubscribe@freebsd.org"
 
State-Changed-From-To: patched->closed 
State-Changed-By: gonzo 
State-Changed-When: Fri Jul 18 13:57:15 UTC 2008 
State-Changed-Why:  
fix has been MFCed 

http://www.freebsd.org/cgi/query-pr.cgi?pr=123731 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/123731: commit references a PR
Date: Fri, 18 Jul 2008 13:57:24 +0000 (UTC)

 gonzo       2008-07-18 13:57:05 UTC
 
   FreeBSD src repository
 
   Modified files:        (Branch: RELENG_6)
     sys/kern             sysv_sem.c 
   Log:
   SVN rev 180591 on 2008-07-18 13:57:05Z by gonzo
   
   MFC r179879:
   Renew semaphore's pointer after wakeup since during msleep
   sem_base may have been modified by destroying one of semaphores
   and semptr would not be valid in this case.
   
   PR:             kern/123731
   MFC after:      1 month
   
   Revision  Changes    Path
   1.78.2.3  +7 -0      src/sys/kern/sysv_sem.c
 _______________________________________________
 cvs-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/cvs-all
 To unsubscribe, send any mail to "cvs-all-unsubscribe@freebsd.org"
 
>Unformatted:
