[HN Gopher] Dark Side of Posix APIs
___________________________________________________________________
Dark Side of Posix APIs
Author : ingve
Score : 89 points
Date : 2021-05-17 11:00 UTC (12 hours ago)
(HTM) web link (vorner.github.io)
(TXT) w3m dump (vorner.github.io)
| pjmlp wrote:
| Just try to do the same across more UNIX implementations for more
| POSIX joy.
|
| Like most design by committee standards there is plenty of write
| once debug everywhere.
| emteycz wrote:
| > Like most design by committee standards there is plenty of
| write once debug everywhere.
|
| Are there any non-committee counter examples?
| pjmlp wrote:
| Proprietary APIs.
| forgotmypw17 wrote:
| write once debug everywhere is how the web works too, and i
| think it is just a fun fact of life that when you speak, each
| listener may understand you differently, and you have to think
| about it and account for it?
| pjmlp wrote:
| Anyone that doesn't think Web === ChromeOS needs to think and
| account for it.
|
| However in about 10 years that might not matter anymore.
| forgotmypw17 wrote:
| I think retro-Web will only continue growing, as will sites
| which care about accessibility by the long tail of older
| devices.
|
| If anything, I think Chrome will eventually split off into
| its own AOL, while the Web will continue on.
| matheusmoreira wrote:
| I clicked expecting to read about signals and was not
| disappointed. Dark corners of POSIX is putting it lightly. There
| just seems to be no correct way to deal with them.
|
| The best solution I know is signalfd, a Linux feature. Block all
| the traditional signal handlers then epoll a signal file
| descriptor in a perfectly normal event loop. When a signal is
| received, it's possible to read instances of signalfd_siginfo
| from it. Even this has limitations such as being unable to handle
| SIGSEGV or SIGFPE.
|
| https://man7.org/linux/man-pages/man2/signalfd.2.html
| kstenerud wrote:
| Except that once you do this, no one else can listen for that
| signal anymore. Pending signals go to the first to register for
| them, and then are no longer pending, so no one else gets
| notified. Aaaand you don't get notified of this problem - your
| registration call succeeds but you get nothing when a signal
| comes.
|
| The Android runtime library uses similar tricks for ANRs, with
| similar problems.
|
| Signals are probably the most broken of all POSIX APIs.
| matheusmoreira wrote:
| > Except that once you do this, no one else can listen for
| that signal anymore.
|
| What do you mean by that? Who else is there other than the
| process being signaled?
|
| > Signals are probably the most broken of all POSIX APIs.
|
| I agree completely.
| kstenerud wrote:
| >> Except that once you do this, no one else can listen for
| that signal anymore.
|
| > What do you mean by that? Who else is there other than
| the process being signaled?
|
| A library that needs to be notified of certain signals and
| perhaps also their contents.
| matheusmoreira wrote:
| Yeah, that's going to be a huge problem. Libraries
| shouldn't be doing that. They should leave signal
| handling to the caller and provide the functions that
| should be called in those cases. Just like memory
| allocation, initialization...
| kstenerud wrote:
| Which is great in theory, except that signal handlers are
| such an esoteric body of knowledge that very few people
| can do it at all, let alone do it properly. So, many
| libraries do it for them (which works so long as nobody
| uses the blocking or blocking-dependent APIs - which is
| generally true). Even the JVM taps SIGQUIT for dumping
| process info.
| R0b0t1 wrote:
| The workaround for this you can find in SDL2 and some
| other libraries with event loops is having a default,
| library-provided setup routine and event loop.
|
| Know what you're doing? Set up signals yourself. Don't?
| Library sets it up for you.
| scottlamb wrote:
| > There just seems to be no correct way to deal with them.
|
| No, you can definitely do it correctly.
|
| Signals are more than one thing. They can be divided into three
| buckets:
|
| * asynchronous, process-directed. When you press ctrl-C, the
| terminal driver generates a SIGINT to the process to handle at
| its leisure. It can be delivered to any thread of the process
| and isn't necessarily delivered immediately.
|
| * asynchronous, thread-directed. pthread_sigkill sends this.
|
| * synchronous, thread-directed. A null-pointer dereference (or
| other access to unmapped memory) causes a SIGSEGV to that
| specific thread. It's delivered immediately; letting the thread
| continue for a while first doesn't make sense.
|
| * (synchronous, process-directed isn't a thing IIRC and
| wouldn't make sense)
|
| Let's just talk about asynchronous, process-directed because
| that's what most people mean when they say signals.
|
| There are low-level techniques to handle them correctly. If you
| have an event loop, my favorite is the "self-pipe trick". At
| process start time, create a pipe and set the O_NONBLOCK flag.
| Hold onto both ends. From the signal handler, write into the
| pipe (ignoring EAGAIN). In your event loop, read from the pipe.
| When there's something to read, a signal is pending. This gives
| you roughly the same thing as signalfd but is cross-platform.
|
| The best high-level technique IMHO is to write your code in a
| language that has a common library for this stuff. The article
| is about signal-hook, which is commonly used in Rust.
| matheusmoreira wrote:
| > If you have an event loop, my favorite is the "self-pipe
| trick".
|
| That's nice. Seems good to me, similar to signalfd.
|
| What if you don't have an event loop though? When I tried to
| learn this I couldn't find any sane way to use signal handler
| functions. There's just too many caveats. With event loops I
| can dedicate a thread to the task, the code executes in a
| normal context and everything works as expected.
|
| > This gives you roughly the same thing as signalfd but is
| cross-platform.
|
| Roughly? What are the differences? The most obvious to me is
| the fact I can read signalfd_siginfo structures from the file
| descriptor.
| scottlamb wrote:
| > What if you don't have an event loop though?
|
| You can use my second-favorite technique: dedicate a thread
| to signals. Block the signals of interest, keeping them
| queued rather than running signal handlers. Do this early
| in main(), before spawning any threads, so that they'll
| inherit the block. Spawn a thread which dequeues in a loop
| via sigwaitinfo or sigtimedwait. Signal handling without a
| signal handler!
|
| > Roughly? What are the differences [between the self-pipe
| trick and signalfd]? The most obvious to me is the fact I
| can read siginfo structures from the file descriptor.
|
| That's the difference I mean. If you want to get those from
| the self-pipe trick, you need to do it yourself, dealing
| with the platform difference crap this article describes.
| Additionally, if you write more than one byte per signal,
| you'll have to be careful of half-written messages, since
| your signal handler can't block for capacity. (Why? It may
| be blocking the thread that does the reading, so it'd
| deadlock.) Maybe you'd have a separate buffer for the
| leftover part that doesn't quite fit, or ensure your your
| message size divides evenly into the pipe buffer capacity
| (see F_GETPIPE_SZ), or drop messages if there's
| insufficient capacity (use FIONREAD; the capacity shouldn't
| go down unless you're using the same pipe for multiple
| signals and don't have them masking each other out). Stuff
| like this can get complex but the basic technique of just
| ensuring ctrl-C reaches your event loop isn't that bad. You
| have to be very careful in the signal handler, but it's
| like five lines long, so life goes on.
|
| Another difference is that installing a self-pipe handler
| lets you see the previous handler (if any). I've heard of
| chaining handlers to address complaints like kstenerud's
| about libraries. I've never actually wanted to share
| signals like that, but it's something one could try.
| Caveat: if you also want to uninstall signals, doing that
| out of order wouldn't go well.
| jart wrote:
| > Long story short, while Rust's libc bindings give one access to
| the si_code field, it doesn't export the actual constants to know
| what that value means. While the maintainers are generally open
| to adding these constants, I don't have access to all the large
| number of platforms to figure out what values the constants have
| on each (yes, they are not the same) and what constants are even
| available on each of them
|
| https://github.com/jart/cosmopolitan/blob/7cbc2bc0838a88ff46...
| dragontamer wrote:
| Hmm. This seems like a good use of lock-free programming. Its not
| a technique you want to use very often (lock-free is known by a
| small minority of programmers, and is difficult to do), but... if
| mutexes can't work and you only can rely upon memory-level
| assumptions... that's the sort of stuff that load_acquire() and
| store_release() was made for. (As well as atomics, with
| associated memory orderings)
| jcranmer wrote:
| Signals are one of the things that I think UNIX (and later POSIX)
| got horribly, horribly wrong. POSIX makes it hard to do things
| that people would love to do with signals. And their safety
| issues are hard to encapsulate even in a language like Rust
| (because you can only call async-signal-safe functions from a
| signal handler, and those are pretty far and few between).
|
| And sadly, Linux manages to do some of its own custom things here
| that makes it even worse. One thing I find helpful sometimes is
| being able to inspect the actual hardware trap number, which can
| help distinguish between a SIGSEGV caused by a pointer to memory
| that doesn't exist (which is the normal cause) or an unaligned
| memory access in an instruction that requires aligned access [1].
| There's a field for it in siginfo_t, si_trapno... but Linux
| doesn't fill in si_trapno for x86 nor even provide it as a field
| to be provided, unlike the BSDs in both regards. However, Linux
| does helpfully provide the trap number in the mccontext_t
| struct... which is _only_ accessible via the third argument of a
| signal handler, and not via any ptrace access for a debugger.
|
| If I were to redesign signals, I'd start by dividing them up into
| two categories: synchronous signals and asynchronous signals.
| This division actually already exists, but only in terms of how a
| signal is dispatched; I'd extend it so that the details of what
| the signals look like and how they are handled are completely
| different.
|
| Synchronous signals are those that are caused by processor
| interrupts (think a page fault), or some set of user-custom
| dispatchable signals (e.g., being able to request thread
| cancellation). These would be handled like a regular try/catch
| scope, although with additional opportunities for first-ditch and
| last-ditch handling--Windows SEH is the kind of programming model
| I'd go for. What this allows is being able to do something like
| "I'm going to call some custom code, and if it causes a
| segmentation fault, I can report an error and unwind the stack to
| code that's unaffected" (and possibly report a full stacktrace in
| the meantime, maybe even as the default handler if uninstalled).
| This kind of handling allows it to be scoped on a per-thread
| basis, unlike the current per-process signal handlers.
|
| Asynchronous signals would instead be handled by inserting them
| into a per-process queue of unhandled signals. There would be
| some syscall for getting the next unhandled signal, and naturally
| something that could feed into whatever syscall you use for "wait
| until something interesting happens" (which on Linux would mean
| something akin to signalfd). At this point, dequeuing signals is
| now a relatively benign step, and you can handle this with a
| simple thread that does nothing but dequeues signals and sticks
| them into a thread-safe queue for other threads to check at their
| leisure. No more need to worry about async-signal-safety!
|
| [1] Side side rant: On x86, Linux maps the #AC (alignment check)
| exception to SIGBUS. But #AC isn't used for SSE aligned vector
| move instructions, instead #GP is used. #GP is instead mapped to
| SIGSEGV, but Linux doesn't fill in many of the fields. (Not that
| I think x86 fills in the faulted address for a #GP exception, but
| still, no si_trapno field to easily distinguish a #PF SIGSEGV
| versus #GP SIGSEGV). It took me several hours of pulling my hair
| out before I realized I was getting #GP and not #PF...
| wmanley wrote:
| On the subject of signals and other problems with UNIX I can
| recommend Neil Brown's Ghosts of Unix past, part 3: Unfixable
| designs[1]
|
| [1]: https://lwn.net/Articles/414618/
| ansible wrote:
| Is there a full and complete example of how to properly handle
| signals in a multi-threaded program that does the usual stuff
| (talk to disk, talk on the network)? Written in plain C?
|
| I _think_ I know everything that I 'm supposed to do (on Linux,
| for example). But I wouldn't mind looking through an example that
| handles everything correctly. Something that is designed as an
| example, as opposed to reading through some much larger and older
| project.
| pjmlp wrote:
| It is very hard to do it properly in correct way, given how OS
| specific and underspecified their behaviour is.
|
| The easiest way is just to set a variable to mark the
| occurrence of a signal and nothing else. Then check for changes
| from other parts of the program.
|
| Also be sure to check every C library error result and the
| value that it was interrupted by signal occurrence, and
| eventually retry if needed.
___________________________________________________________________
(page generated 2021-05-17 23:01 UTC)