Subj : Re: A pity that there is no forkall() which clones threads To : comp.programming.threads From : David Hopwood Date : Sun Mar 06 2005 04:04 am Marcin 'Qrczak' Kowalczyk wrote: > My language Kogut has three variants of fork: > > * ForkProcess - other threads are stopped at a point where signals > are unblocked; the process is forked; in the child process threads > are canceled, and in the parent they are resumed > (the safest default) > > * ForkProcessCloneThreads - all threads continue running in both > processes > (the simplest to describe semantics) > > * ForkProcessKillThreads - other threads are killed immediately with > no cleanup > (used before ExecProgram) > > Meanwhile I added support for using pthreads for certain background > computations and for ensuring that callbacks from C run in their own > pthreads, based on the design described in a paper by Simon Marlow, > Simon Peyton Jones, and Wolfgang Thaller "Extending the Haskell > Foreign Function Interface with Concurrency". > > A Kogut thread can be bound to an OS thread or unbound; conversely, > an OS thread can be bound to a Kogut thread or be one of worker > threads for executing unbound Kogut threads. The main thread and > threads which currently run callbacks are bound. > > This design allows to create thousands of threads, doesn't require the > runtime and GC to be reentrant which would complicate and slow down it > too much, and at the same time those threads which must be run in > separate pthreads do so and there is no problem with concurrent > callbacks returning out of order on the same system stack. > > Since it's impossible to fork a process together with other pthreads, > in the child process after ForkProcess and ForkProcessCloneThreads > all threads other than the one doing the fork become unbound. > If such orphaned thread returns from a callback, or if it was > executing foreign code at the time of the fork, it is killed as with > ForkProcessKillThreads, because the code it was supposed to return to > was executing in another pthread which doesn't exist here. ForkProcess can work with sensible semantics if foreign code is uninterruptible by default. In that case only foreign code that has explicitly asked to be interruptible will be killed. Add a function that foreign code can use to set the interruptible flag for the current thread. The flag is always unset (i.e. uninterruptible) on entry to a foreign function, and setting it only lasts only until the code returns or next makes a call into the Kogut runtime. (This ensures that there can be no problems with forgetting to reset the flag or failing to do so because of a C++ exception.) Note that foreign code only has to set the interruptible flag when it does blocking I/O or a lengthy computation that does not call back into the runtime. On a ForkProcess call in thread T, 1. set a global flag that causes all threads except T to suspend if and when they reach an interruptible point. 2. wait until all threads have suspended except T, and possibly some threads that were already running interruptible foreign code at the time of the ForkProcess call. 3. unset the global flag. 4. call fork(). 5. in the parent, resume all threads that were suspended. The threads other than T that were left running at step 3 will just evaporate in the child, which is fine. Although not necessary to support ForkProcess, you can also have the function that sets the interruptible flag call "pthread_setcancelstate(PTHREAD_CANCEL_ENABLE, ...)", so that it is possible to cancel them using other Kogut APIs. In that case code that executes with the flag set must be POSIX async-cancel-safe, which is reasonable. As for ForkProcessCloneThreads, it seems to me that you're trying to implement an abstraction that simply can't be supported by the underlying OS in most cases. Just drop this API. -- David Hopwood .