From mikko@shiba.d.home.dynas.se  Fri Mar 17 10:29:23 2000
Return-Path: <mikko@shiba.d.home.dynas.se>
Received: from shiba.d.home.dynas.se (cns9-224-242.cm.starport.se [193.150.224.242])
	by hub.freebsd.org (Postfix) with ESMTP id F11F037BBEF
	for <FreeBSD-gnats-submit@freebsd.org>; Fri, 17 Mar 2000 10:29:17 -0800 (PST)
	(envelope-from mikko@shiba.d.home.dynas.se)
Received: (from mikko@localhost)
	by shiba.d.home.dynas.se (8.9.3/8.9.3) id TAA17749;
	Fri, 17 Mar 2000 19:29:32 +0100 (CET)
	(envelope-from mikko)
Message-Id: <200003171829.TAA17749@shiba.d.home.dynas.se>
Date: Fri, 17 Mar 2000 19:29:32 +0100 (CET)
From: mikko@dynas.se
Sender: mikko@shiba.d.home.dynas.se
Reply-To: mikko@dynas.se
To: FreeBSD-gnats-submit@freebsd.org
Subject: pthread_atfork() missing from libc_r
X-Send-Pr-Version: 3.2

>Number:         17437
>Category:       bin
>Synopsis:       pthread_atfork() missing from libc_r
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    jasone
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Fri Mar 17 10:30:02 PST 2000
>Closed-Date:    Tue May 23 10:03:29 PDT 2000
>Last-Modified:  Tue May 23 10:05:04 PDT 2000
>Originator:     Mikko Tyljrvi
>Release:        FreeBSD 5.0-CURRENT i386
>Organization:
>Environment:

FreeBSD [345].x

>Description:

While porting some code using pthreads to FreeBSD, I noticed to my
surprise that pthread_atfork() was nowhere to be found, and there
did not seem to be any PRs related to the function.

So either I am the only one using pthread_atfork(), or it has fallen
out of the standard...  Most other systems (Solaris, AIX, HP-UX, IRIX
and Linux) has the function, and it is documented at:

<http://www.opengroup.org/onlinepubs/007908799/xsh/pthread_atfork.html>

Enough complaining.  Unless I have seriously misunderstood the
workings of libc_r, the patch below should bring pthread_atfork() to
FreeBSD (applies against -CURRENT, March 16).  It works for me.

Someone may want to turn the man-page into proper English too...

	Regards,
	/Mikko


>How-To-Repeat:

Compile anything that uses pthread_atfork()...

>Fix:


--- lib/libc_r/uthread/Makefile.inc.org	Thu Mar 16 22:31:06 2000
+++ lib/libc_r/uthread/Makefile.inc	Thu Mar 16 22:31:42 2000
@@ -6,6 +6,7 @@
 SRCS+= \
 	uthread_accept.c \
 	uthread_aio_suspend.c \
+	uthread_atfork.c \
 	uthread_attr_destroy.c \
 	uthread_attr_init.c \
 	uthread_attr_getdetachstate.c \
--- lib/libc_r/man/pthread_atfork.3.orig	Fri Mar 17 19:01:53 2000
+++ lib/libc_r/man/pthread_atfork.3	Fri Mar 17 19:01:47 2000
@@ -0,0 +1,73 @@
+.\"
+.\" "THE BEER-WARE LICENSE" (Revision 42m):
+.\" <mikko@dynas.se> wrote this file.  As long as you retain this notice you
+.\" can do whatever you want with this stuff. If we meet some day, and you think
+.\" this stuff is worth it, you can buy me a beer in return.  Mikko Tyljrvi
+.\"
+.\" $FreeBSD$
+.\"
+.Dd March 16, 2000
+.Dt PTHREAD_ATFORK 3
+.Os BSD 4
+.Sh NAME
+.Nm pthread_atfork
+.Nd register fork handlers
+.Sh SYNOPSIS
+.Fd #include <pthread.h>
+.Ft int
+.Fn pthread_atfork "void (*prepare)(void)" "void (*parent)(void)" "void (*child)(void)"
+.Sh DESCRIPTION
+The
+.Fn pthread_atfork
+function registers fork handlers to be called before and after
+.Fn fork .
+Handlers are run in the context of the thread that calls
+.Fn fork .
+.Pp
+The
+.Fa prepare
+handlers are invoked before fork processing. The
+.Fa parent
+and
+.Fa child
+handlers are invoked after fork processing, in the parent and child
+processes respectively.
+Handlers will be called in the parent process even when the
+.Fn fork
+operation fails.
+.Pp
+Any number of handlers can be registered by
+.Fn pthread_atfork .
+Function pointers that are NULL will be ignored.
+The 
+.Fa Parent
+and
+.Fa child
+handlers are called in the order that they were registered. The
+.Fn prepare
+handlers are called in the reverse order.
+.Pp
+.Sh RETURN VALUES
+If successful, the
+.Fn pthread_atfork
+function will return zero.
+Otherwise an error number will be returned to
+indicate the error.
+.Sh ERRORS
+.Fn pthread_atfork
+will fail if:
+.Bl -tag -width Er
+.It Bq Er ENOMEM
+The process cannot allocate enough memory to register another set of
+handlers.
+.El
+.Pp
+.Sh SEE ALSO
+.Xr fork 2 ,
+.Xr atexit 3
+.Sh STANDARDS
+.Fn pthread_atfork
+conforms to ISO/IEC 9945-1 ANSI/IEEE
+.Pq Dq Tn POSIX
+.\" XXX: Dunno -- someone will have to check the exact details:
+Std 1003.1 Second Edition 1996-07-12.
--- lib/libc_r/man/Makefile.inc.org	Fri Mar 17 18:42:54 2000
+++ lib/libc_r/man/Makefile.inc	Fri Mar 17 18:42:31 2000
@@ -4,7 +4,8 @@
 
 .PATH: ${.CURDIR}/man
 
-MAN3+=	pthread_cancel.3 \
+MAN3+=	pthread_atfork.3 \
+	pthread_cancel.3 \
 	pthread_cleanup_pop.3 \
 	pthread_cleanup_push.3 \
 	pthread_cond_broadcast.3 \
--- include/pthread.h.org	Thu Mar 16 22:29:54 2000
+++ include/pthread.h	Thu Mar 16 22:37:37 2000
@@ -303,6 +303,9 @@
 
 int		pthread_attr_setfloatstate __P((pthread_attr_t *, int));
 int		pthread_attr_getfloatstate __P((pthread_attr_t *, int *));
+
+int		pthread_atfork __P((void (*prepare)(void),
+				    void (*parent)(void), void (*child)(void)));
 __END_DECLS
 
 #endif
--- lib/libc_r/uthread/pthread_private.h.org	Thu Mar 16 22:30:22 2000
+++ lib/libc_r/uthread/pthread_private.h	Fri Mar 17 16:26:21 2000
@@ -999,6 +999,16 @@
 #define _FD_UNLOCK(_fd,_type)		_thread_fd_unlock(_fd, _type)
 #endif
 
+/* Atfork handlers */
+#define ATFORK_PREPARE	0
+#define ATFORK_PARENT	1
+#define ATFORK_CHILD	2
+
+typedef struct atfork_data {
+	struct atfork_data *next, *prev;
+	void (*funcs[3])(void);
+} atfork_data;
+
 /*
  * Function prototype definitions.
  */
@@ -1060,6 +1070,7 @@
 void	_thread_enter_cancellation_point(void);
 void	_thread_leave_cancellation_point(void);
 void	_thread_cancellation_point(void);
+atfork_data *_thread_atfork_list(void);
 
 /* #include <signal.h> */
 int     _thread_sys_sigaction(int, const struct sigaction *, struct sigaction *);
--- lib/libc_r/uthread/uthread_atfork.c.org	Fri Mar 17 18:37:07 2000
+++ lib/libc_r/uthread/uthread_atfork.c	Fri Mar 17 18:27:19 2000
@@ -0,0 +1,56 @@
+/*
+ * "THE BEER-WARE LICENSE" (Revision 42m):
+ * <mikko@dynas.se> wrote this file.  As long as you retain this notice you
+ * can do whatever you want with this stuff. If we meet some day, and you think
+ * this stuff is worth it, you can buy me a beer in return.  Mikko Tyljrvi
+ *
+ * $FreeBSD$
+ */
+
+#include <errno.h>
+#include <stdlib.h>
+#ifdef _THREAD_SAFE
+#include <pthread.h>
+#include "pthread_private.h"
+
+static atfork_data *atfork_list = NULL;
+static spinlock_t atfork_lock	= _SPINLOCK_INITIALIZER;
+
+int
+pthread_atfork(void (*prepare)(void),
+	       void (*parent)(void),
+	       void (*child)(void))
+{
+	atfork_data *ap;
+
+	if (prepare == NULL && parent == NULL && child == NULL)
+		return 0;
+
+	if ((ap = malloc(sizeof(atfork_data))) == NULL)
+		return ENOMEM;
+
+	ap->funcs[ATFORK_PREPARE] = prepare;
+	ap->funcs[ATFORK_PARENT] = parent;
+	ap->funcs[ATFORK_CHILD] = child;
+
+	_SPINLOCK(&atfork_lock);
+	if ((ap->next = atfork_list) != NULL)
+		atfork_list->prev = ap;
+	atfork_list = ap;
+	_SPINUNLOCK(&atfork_lock);
+
+	return 0;
+}
+
+atfork_data *
+_thread_atfork_list(void)
+{
+	atfork_data *ap;
+
+	_SPINLOCK(&atfork_lock);
+	ap = atfork_list;
+	_SPINUNLOCK(&atfork_lock);
+
+	return ap;
+}
+#endif
--- lib/libc_r/uthread/uthread_fork.c.org	Fri Mar 17 18:12:22 2000
+++ lib/libc_r/uthread/uthread_fork.c	Fri Mar 17 18:10:23 2000
@@ -47,6 +47,16 @@
 	pid_t		ret;
 	pthread_t	pthread;
 	pthread_t	pthread_save;
+	atfork_data	*ap, *alist;
+
+	/* Run any registered "prepare" callbacks: */
+	alist = _thread_atfork_list();
+	for (ap = alist; ap != NULL; ap = ap->next) {
+		if (ap->funcs[ATFORK_PREPARE] != NULL)
+			ap->funcs[ATFORK_PREPARE]();
+		if (ap->next == NULL)
+			break;
+	}
 
 	/*
 	 * Defer signals to protect the scheduling queues from access
@@ -216,6 +226,15 @@
 	 * Undefer and handle pending signals, yielding if necessary:
 	 */
 	_thread_kern_sig_undefer();
+
+	/*
+	 * Run any parent/child callbacks matching the "prepare" callbacks
+	 * from function entry, in reverse order:
+	 */
+	i = ret ? ATFORK_PARENT : ATFORK_CHILD;
+	for (; ap != NULL; ap = (ap != alist) ? ap->prev : NULL)
+		if (ap->funcs[i] != NULL)
+			ap->funcs[i]();
 
 	/* Return the process ID: */
 	return (ret);



>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->jasone 
Responsible-Changed-By: jasone 
Responsible-Changed-When: Fri Mar 17 14:21:22 PST 2000 
Responsible-Changed-Why:  
Over to maintainer. 

From: =?ISO-8859-1?Q?Mikko_Ty=F6l=E4j=E4rvi?= <mikko@rsasecurity.com>
To: freebsd-gnats-submit@freebsd.org
Cc:  
Subject: Re: bin/17437: pthread_atfork() missing from libc_r
Date: Sat, 18 Mar 2000 10:36:51 +0100 (CET)

 (Following up on myself after further investigations):
 
 Nope, the above patch is insufficient, it only serves to uncover
 other bugs.
 
 The problem is that libc_r does not do complete cleanup of the other
 threads after the fork.  All deleted threads must be quietly removed
 from any queues they happen to be waiting in, or they may later be
 subject to "revival" after they have been freed.
 
 Scenario (typical use of pthread_atfork()):
 	 - forking thread aquires a number of mutexes.
 	 - another thread waits on one of the mutexes
 	 - after forking, the mutexes are released,
 	   which will make any waiting threads
 	   runnable -- but they have been deallocated!
 	 - Bad Things(tm) happen...
 
 All mutexes where this may happen can be conveniently located via the
 "mutexq" field of the running thread, so this particular scenario is
 easily fixed.
 
 There is no similar way to handle threads queued on, for example,
 condition variables, as there is no way to find the head of the "qe"
 and "pqe" queue entries.  The "join_queue" should be emptied too.
 
 Basically, right now libc_r does not work at all after a fork().
 
       Regards,
       /Mikko
 
 
State-Changed-From-To: open->closed 
State-Changed-By: jasone 
State-Changed-When: Tue May 23 10:03:29 PDT 2000 
State-Changed-Why:  
There is no plan to add pthread_atfork(), due to the difficulty and limited 
usefulness of doing so. 
>Unformatted:
