Subj : Re: A pity that there is no forkall() which clones threads To : comp.programming.threads From : Marcin 'Qrczak' Kowalczyk Date : Sun Mar 06 2005 05:26 pm Giancarlo Niccolai writes: >> Maybe I should introduce something which says "when you want to >> interrupt me while I'm making this foreign call, send me SIGUSR1". > > This would be the nicest way to do it; however, this is not portable > to i.e. windows... Yes, and I don't know what should be used in similar cases on Windows, i.e. what means of interrupting a blocking API call are typically available. I haven't yet ported the compiler to Windows because I haven't programmed for Windows. This issue will wait at least a few months. There are many areas which would have to be virtualized and implemented separately for Windows (unless I rely on cygwin emulation of Unix API): file handles instead of file descriptors, WaitForMultipleObjects instead of epoll/poll/select, Windows timers instead of setitimer, Windows threads instead of pthreads, something instead of iconv, and the hardest thing: interface to various Windows libraries. Some of them are not essential and there is a working but less functional fallback if they are not available. For example OS threads are optional. Without them all long-lasting C calls block other threads, and all callbacks are executed by the same OS thread, which means that they might be delayed or even deadlock if they return out of order. > Oh well but fork is not portable to windows too, AFAIK ... Yes, but interruption of a foreign call by a Unix signal is not specific to fork. For example when one thread waits for a child process to finish, and another sends it a Kogut signal (e.g. to cancel it), currently the signal will be processed only after the threads finishes waiting. Unless it doesn't use WaitForProcess but is waiting for SIGCHLD to come. Then it is of course interruptible, but this is less modular: threads must agree who handles SIGCHLD and must find out whose process it was, instead of relying on waitpid to wait for a particular process only. Paradoxically, WaitForProcess is interruptible when pthreads are not used, because in this configuration the thread leaves the timer signal unblocked and waitpid fails with EINTR 100 times a second, allowing other threads to run. The thread waiting for a process eats its timeslices as if it was doing some computation. This is unfortunate but without OS threads it seems to be the only way to avoid stopping other threads. With OS threads it does the magic calls which turn this thread bound to an OS thread, which improves parallelism, but makes the thread uninterruptible (except for OS signals if this is the thread which handles them)... > That's right, but be carefull: the space between the flag setting in > the foreign api and the return to the VM (or the call to something > inside the VM) must be async-cancel-safe too. And that's a problem: > > mutex_lock(); > can_cancel_me = false; > mutex_unlock(); > > would not do as you can get cancelled before releasing the lock. > An atomic operation, or preventing cancellation right after the > calls/returns (with pthread_set_cancel_state), is in order. This should not be a problem: only one thread at a time has access to the Kogut runtime, so it can be stored in a place which is assumed to be touched only from a thread having access to the runtime. The requirement is that this flag is reset after this thread gains access to the Kogut runtime, but before we return to the scheduler - i.e. also before any pending signals are processed. This unfortunately means that the API of gaining access to the runtime must be polluted with the issue of evaporable foreign threads. The thread performing the fork will send the suspending signal to threads which are currently executing evaporable foreign calls too, but they will be excluded from the set of threads which must reach the suspension point before the fork can be made. It is possible that some of them will finish the foreign call before forking has caught all other threads, and then it's fine too. If they did finish but didn't yet notify the forker that they are suspending, it's still OK - either their thread was bound even before the foreign call and they will be evaporated, or it was unbound and they will be forked in some point near the suspension and they will be treated appropriately in both processes. I must only move storing threads to be continued/canceled from the handler of the suspension signal to the forker thread, so they are continued/cancelled if fork found them between returning from the foreign call and handling the suspension signal. For the curious, the current source of ForkProcess is at http://cvs.sourceforge.net/viewcvs.py/kokogut/kokogut/lib/Unix/System/Processes.ko?view=markup > IMHO, you should just say "if you use FORK with the kogut VM being > in MT, the state of the child is undefined. Either stop every thread > before calling fork() or call it before starting threads". This is a last resort, and it's very unfortunate if it's chosen because it prevents either threads or forking being invisible implementation details of libraries, and prevents combining libraries which use them. The current design guarantees (assuming that they will react sensibly to the suspending signal - uncooperating threads can mess things up; the suspending signal is actually a signal which means "please execute this function"): - ForkProcessKillThreads will succeed immediately, but you should not use data accessed by other threads before exec. All thread objects in the child will look as if they failed with ThreadKilled exception. - ForkProcess will do the fork only after all threads reached points where signals are unblocked. Foreign calls currently behave as if signals were temporarily blocked, and I want to change this. - Threads in the child process afer ForkProcess will be canceled and will have a chance to clean up resources, but only up to the nearest callback from C when they become killed. This means that one has to be aware which functions make callbacks from C if he cares about cleanup. - Resource leaks may occur only when a thread is killed. This is well-defined relative to the knowledge about C callbacks and about blocking signals for a long time (including C calls), and of course relative to the knowledge about the impact of fork on libraries. For example an internally used pipe might have to be created afresh after forking; there are AtForkProcess handlers to help with making this modular. This is better than being completely undefined. I will still keep ForkProcessCloneThreads. Callbacks from C are actually rare, and in the absence of them it works seamlessly. It will conditionally work even in their presence when I add optional support for forkall. forkall will also be used for ForkProcess when available, to reduce the need of killing instead of proper cleanup. -- __("< Marcin Kowalczyk \__/ qrczak@knm.org.pl ^^ http://qrnik.knm.org.pl/~qrczak/ .