Subj : Re: A pity that there is no forkall() which clones threads To : comp.programming.threads From : David Hopwood Date : Sun Mar 06 2005 10:23 pm First, there was an error in my previous post. It should have said: On a ForkProcess call in thread T, 1. set a global flag that causes all threads except T to suspend if and when they reach an interruptible point. 2. wait until all threads have suspended except T, and possibly some threads that were already running interruptible foreign code at the time of the ForkProcess call. 3. call fork(). 4. unset the global flag in the parent and child. 5. in the parent, resume all threads that were suspended. I.e. unset the global flag after the fork, not before. This closes a timing window where a thread could change from running interruptible foreign code to running uninterruptible code, and make unwanted modifications to the process state immediately before the fork. Marcin 'Qrczak' Kowalczyk wrote: > David Hopwood writes: > >>ForkProcess can work with sensible semantics if foreign code is >>uninterruptible by default. In that case only foreign code that has >>explicitly asked to be interruptible will be killed. > > Indeed it is uninterruptible. When it invokes the C function which > gives it access to the runtime again after a foreign call or before > a callback, and this thread had Kogut signals unblocked, then any > signals sent to this thread in the meantime are processed. > > This is unfortunate if the foreign code blocks for long, because there > is no good way to interrupt it. That's why I suggested the interruptible flag. > If this thread was designed to handle Unix signals, then they are > unblocked in this thread (and blocked in others) and thus they can > interrupt syscalls (though the actual Kogut signal handler will be run > when the thread goes back to the Kogut runtime). This is not enough > for ForkProcess - the suspending signal is plain Kogut signal, not a > Unix signal. Not a problem with the design I suggested. > Note that plain blocking I/O and timeouts don't make the thread bound > and just cause my scheduler to do epoll/poll/select, and thus they > pose no problems for processing Kogut signals. The issue is with > functions like waitpid and getaddrinfo which don't have asynchronous > equivalents in Unix API and the only way to support them without > blocking all threads is to make the current thread bound to a pthread > (and spawn a worker for the rest if it's needed and if there is no > idle worker). Yes. This is a common problem that comes up with essentially all concurrent language runtimes. > Maybe I should introduce something which says "when you want to > interrupt me while I'm making this foreign call, send me SIGUSR1". Unix signals and threads don't mix very well; you probably want to use pthread_cancel instead. The man page for that (or the POSIX spec) gives all the information you need, I think. >>Add a function that foreign code can use to set the interruptible >>flag for the current thread. The flag is always unset (i.e. >>uninterruptible) on entry to a foreign function, and setting it only >>lasts only until the code returns or next makes a call into the Kogut >>runtime. (This ensures that there can be no problems with forgetting >>to reset the flag or failing to do so because of a C++ exception.) > > Hmm. An interesting idea. This would be specific to ForkProcess: > "don't wait for me, I agree to be evaporated at fork". I was originally thinking that it would indicate both that the thread agrees to be evaporated on fork, and that it can be interrupted using pthread_cancel with cancellation type PTHREAD_CANCEL_ASYNCHRONOUS (see the man page). The disadvantage in combining these is that it would mean that code in this state can only call async-cancel-safe functions. In principle there are four "evaporability"/interruptibility states that make sense for foreign code: - not evaporable or interruptible - only evaporable on fork - evaporable on fork and interruptible at pthreads cancellation points (using pthread_setcanceltype(PTHREAD_CANCEL_DEFERRED, ...)) - evaporable on fork and interruptible anywhere (using pthread_setcanceltype(PTHREAD_CANCEL_ASYNCHRONOUS, ...)) Whether you actually need to distinguish all these is debatable. >>Note that foreign code only has to set the interruptible flag when it >>does blocking I/O or a lengthy computation that does not call back >>into the runtime. > > If it calls back to the runtime, it must use a C function which among > other things processes pending signals. This flag would be used before > lengthy computations which don't cause effects on the world, only read > its state, like getaddrinfo. > > The implementation would be very ugly though. I'm not sure whether > it's doable at all without rewriting the whole forking logic. I doubt it is that bad. You already have the logic to wait for threads to be suspended, IIUC; this is just a change to the criterion for whether a thread should be waited for when it is running foreign code. I did try to persuade you earlier that mixing pthreads and fork would be complicated ;-) -- David Hopwood .