[HN Gopher] Some Reflections on Writing Unix Daemon
       ___________________________________________________________________
        
       Some Reflections on Writing Unix Daemon
        
       Author : ltratt
       Score  : 75 points
       Date   : 2024-02-29 08:05 UTC (1 days ago)
        
 (HTM) web link (tratt.net)
 (TXT) w3m dump (tratt.net)
        
       | bluetomcat wrote:
       | It's Unix parlance for a forked child process that detaches from
       | its interactive terminal with the setsid() syscall, becomes a
       | child of init, possibly closes or redirects the 0/1/2 file
       | descriptors to logging facilities, possibly changes its CWD, and
       | no longer receives terminal-originating signals like SIGHUP,
       | SIGTTIN, SIGTTOU, etc.
       | 
       | Apart from that, a robust non-daemon program should use the same
       | defensive techniques. In a short-running command-line utility, it
       | might seem "acceptable" to leak memory, not close resources and
       | not do adequate error-checking, but that's just sloppy
       | programming.
        
         | ksherlock wrote:
         | Well...
         | 
         | "Since the missile will explode when it hits it's target or at
         | the end of it's flight, the ultimate in garbage collection is
         | performed without programmer intervention."
         | 
         | https://groups.google.com/g/comp.lang.ada/c/E9bNCvDQ12k/m/1t...
         | 
         | "to my surprise the cp command didn't exit. Looking at the
         | source again, I found that cp disassembles its hash table data
         | structures nicely after copying (the forget_all call). Since
         | the virtual size of the cp process was now more than 17 GB and
         | the server only had 10 GB of RAM, it did a lot of swapping."
         | 
         | (I believe GNU cp was updated to not free up structures at
         | exiting due after this).
         | 
         | https://lists.gnu.org/archive/html/coreutils/2014-08/msg0001...
        
       | Joker_vD wrote:
       | > Rather than having processes detach themselves from the
       | terminal, these managers run daemons as if they were normal
       | (albeit long-running) programs. This slightly simplifies the
       | daemons themselves and provides a more homogeneous experience for
       | the user.
       | 
       | It tremendously simplifies the daemons themselves and indeed does
       | provide a more homogeneous experience for the user. Remember: do
       | one thing only, and do it well. The "daemonizing" part is is a
       | second thing and it belongs in a separate utility. If the user
       | wants to run your daemon "interactively" (e.g. being able to
       | press Ctrl-C to stop it), they should be able to do so by simply
       | running your daemon from the shell. If the user wants to run your
       | daemon "in the background", _whatever it means to the user_ ,
       | they can arrange so by themselves. Why is it such a difficult
       | idea for the "daemon" writers to accept is beyond me: there is
       | negative value in a program forcefully detaching itself from
       | every single way to supervise it (double- and triple-forking
       | defeats supervising via wait(), closing FDs defeats watching for
       | the closing of an injected pipe, etc).
        
         | ninkendo wrote:
         | Not to mention that in order to properly start and stop such
         | double-forked daemons, you need everyone to agree on how
         | pidfiles work, where to store them, how to garbage collect
         | them, where to write logs, their permissions, how to rotate
         | them, etc etc etc.
         | 
         | With the systemd/"just stay in the foreground" approach you
         | just stay in the foreground and the process monitor can just
         | wait() on you. And for logging you can just write to stdout and
         | let the process monitor unify all the logging. Do one thing and
         | do it well indeed.
        
         | matheusmoreira wrote:
         | > Why is it such a difficult idea for the "daemon" writers to
         | accept is beyond me
         | 
         | There's plenty of old documentation out there saying things
         | like "a proper daemon completely detaches from its controlling
         | terminal, opens log files, etc."
        
           | Joker_vD wrote:
           | There is also plenty of old documentation that strongly
           | advises against doing that, see [0] for references (e.g. the
           | aside with an excerpt from the 1995-era AIX docs).
           | 
           | [0] https://jdebp.uk/FGA/unix-daemon-design-mistakes-to-
           | avoid.ht...
        
         | bluetomcat wrote:
         | > Why is it such a difficult idea for the "daemon" writers to
         | accept is beyond me
         | 
         | Because the basic OS interface is a 50-year-old design with
         | "terminals" being the central I/O entity.
         | 
         | Differentiating between "interactive" and "daemon" mode becomes
         | tricky for a daemon writer. In interactive mode, SIGHUP must
         | terminate your process. In daemon mode, you may interpret it
         | any way you like. Same for SIGINT, SIGQUIT, SIGPIPE, etc.
        
           | zare_st wrote:
           | Tricky? Checking if you're bound to TTY is trivial, checking
           | whether you have a parent is trivial, checking whether
           | interactive shell is somewhere in the tree is trivial, having
           | some if's in signal handlers is trivial, etc.
           | 
           | Besides "daemon writers" could just default to foreground,
           | because if it's ran as a daemon it is ran from a script hence
           | nobody cares for extra option typing such as -d --daemon,
           | it's written once in a file, as opposed to doing "program
           | --foreground" every time you want it to run in front of you
        
         | smcameron wrote:
         | > Why is it such a difficult idea for the "daemon" writers to
         | accept is beyond me
         | 
         | Because the old documentation was good, and the new
         | documentation is awful.
        
           | mjw1007 wrote:
           | The old documentation often wasn't good.
           | 
           | But if you're trying to replace an existing system, it's a
           | mistake to think "the old system had poor documentation so
           | it's OK for the new system's documentation to be poor too",
           | because people have already learned to cope with the old
           | system.
           | 
           | I've seen people make that mistake distressingly often in
           | recent years.
        
       | Mave83 wrote:
       | SystemD took away the pain. You no longer have to think and
       | reinvent the demonize, logging and so on. Just just start what
       | ever you want in a while(1) loop and write to stdout and stderr.
       | No log rotate nightmare, etc.
       | 
       | This makes daemons easier to debug, as they run just in
       | foreground so that you can start them from the cli to thinker
       | with them.
       | 
       | SystemD is highly discussed and some love and some hate it. But
       | such improvements are unmatched and extremely helpful.
        
         | simoncion wrote:
         | OpenRC, daemontools, and many other service managers provide
         | daemonizing wrappers, and all substantially predate systemd.
         | 
         | The Systemd Cabal are far from the first to have noticed and
         | attempted to resolve most of the problems they tackled... they
         | do have the most effective "developer relations" team of all of
         | the projects in this space, though.
         | 
         | (I do continue to love their old "It's so much better than
         | System V Init!" talking point... as well as their "You'll never
         | have to write a shell script to start or manage a service
         | again! Declarative, INI-formatted Unit Files For Everyone!"
         | one.)
        
           | WesolyKubeczek wrote:
           | The systemd cabal one-upped all other efforts that pre-dated
           | them by leveraging cgroups such that what rlimits your
           | services run with is dictated by the unit file alone, and not
           | by whatever your user session had when you issued the start
           | or restart command.
        
             | stonogo wrote:
             | And they did so by sweeping the entire UNIX landscape into
             | the dustbin, only supporting linux, and in return giving us
             | a management system that can't be sure anything is
             | running[1].
             | 
             | systemd is leaps and bounds better than most of the init
             | systems that came before it, but it's not a good daemon
             | supervisor, and the developers have basically declared that
             | it's "good enough" so improvements are unwelcome.
             | 
             | 1 - http://ewontfix.com/15/
        
               | WesolyKubeczek wrote:
               | Well, do systems like, say, FreeBSD even support
               | facilities functionally equivalent to control groups?
               | Even if they do (which they don't), it would be a whole
               | new separate implementation, and there's not enough
               | interest from anyone who can do it to roll their sleeves
               | and work on it.
        
               | stonogo wrote:
               | FreeBSD, specifically, has had jails and Capsicum longer
               | than systemd has existed, and rctl (aka Resource Limits)
               | for about a decade now.
               | 
               | Like I said, systemd is a great init system but it really
               | did not bring anything to the table as far as service
               | management. Apple's abdication of the problem makes sense
               | since they're tightly focused on workstation ever since
               | killing the Xserve line in 2011.
        
               | WesolyKubeczek wrote:
               | Also, do you happen to know how, say, macOS manages
               | service dependencies with launchd and makes sure
               | everything is running?
               | 
               | It doesn't.
               | 
               | It explicitly says in the scarce documentation that it
               | doesn't care, and that interdependent services should use
               | IPC and in general figure it out between themselves.
               | launchd only launches the processes as soon as it can,
               | and restarts them based on a boolean flag if they die.
               | That's all.
               | 
               | systemd at least gives you something to base your
               | expectations on.
        
           | Izkata wrote:
           | And I think "daemonize" might have been the first? One of the
           | earliest at least, it turns a single process into daemon but
           | doesn't have an overall system command to handle them.
        
       | matheusmoreira wrote:
       | Related:
       | 
       | https://mywiki.wooledge.org/ProcessManagement
        
       | habibur wrote:
       | > There are use-cases for async/await, particularly for single-
       | threaded languages, or for people writing network servers that
       | have to have to deal with vast numbers of queries. Rust is multi-
       | threaded - indeed, its type system forbids most classic multi-
       | threading errors - and very few of us write servers that deal
       | with vast numbers of queries.
       | 
       | This is the portion that I was seeking his feedback on. Rust
       | originally was designed to work with OS native threads. Later
       | Nginx like software emulated threads were proven far more
       | efficient and fast. Here a single thread jumps from connection to
       | connection, instead of waiting for the other side to respond. In
       | C you can do it but you need the ability for a function to resume
       | from the mid of its body where it left off. That asks for "co-
       | routines" which even though possible makes the code complex.
       | 
       | Rust solution is async/await. But now they have two solutions,
       | the more integrated multi threads and then this newly introduced
       | single threaded async/await. It's better to get feedback from
       | people that have worked on it and their good or bad experience.
        
         | zokier wrote:
         | The most common Rust async runtime, tokio, is multithreaded by
         | default.
        
       | asa400 wrote:
       | > My experience with snare and pizauth is that Rust is a viable
       | language for writing daemons in. Rust isn't a perfect language
       | (e.g. unsafe Rust currently has no meaningful semantics so any
       | code which uses unsafe comes with fewer guarantees than C)
       | 
       | What exactly does the author mean when they say that unsafe Rust
       | has "no meaningful semantics"? Is this a term of art in language
       | analysis or is the author just saying "it's weird"?
        
         | zokier wrote:
         | Rust as a language is in practice defined as whatever rustc
         | does, there is no authorative specification like ISO C standard
        
         | ltratt wrote:
         | As an example, I like to point people at https://doc.rust-
         | lang.org/std/cell/struct.UnsafeCell.html which for many years
         | now has contained this line:
         | 
         | > The precise Rust aliasing rules are somewhat in flux, but the
         | main points are not contentious
         | 
         | I've sometimes found myself in situations where the only way
         | I've been able to deal with this is to check the compiler's
         | output and trawl forums for hints by Rust's developers about
         | what they think/hope the semantics are/will be.
         | 
         | Historically speaking, this situation isn't uncommon: working
         | out _exactly_ what a language 's semantics should be is hard,
         | particularly when it has many novel aspects. Most major
         | languages go through this sort of sequence. Some sooner or
         | later than others --- and some end up addressing it more
         | thoroughly than others). Eventually I expect Rust to develop
         | something similar to the modern C spec, but we're not there
         | yet.
        
           | asa400 wrote:
           | Excellent - thank you for the example and the clarification.
           | This is exactly what I was looking for.
        
       | eadmund wrote:
       | > Ever since I have tried to think "can I solve this problem in a
       | way that reuses known conventions or do I really, really have to
       | do something different?" The answer, once I can control my ego,
       | is nearly always "I don't need to do something different".
       | 
       | This really is the way of wisdom. Not that there's not room for
       | improvement -- there is -- but it is normally more effective to
       | attack the problem at hand than try to solve ancillary problems
       | as well.
        
       | tmoertel wrote:
       | To make daemons easier to write, more uniform, and secure, djb
       | created https://cr.yp.to/daemontools.html and used it for many of
       | his projects
       | 
       | In the corresponding FAQ list, he says this about daemons that
       | detach themselves:
       | 
       | > How can I supervise a daemon that puts itself into the
       | background? When I run inetd, my shell script exits immediately,
       | so supervise keeps trying to restart it.
       | 
       | > _Answer: The best answer is to fix the daemon. Having every
       | daemon put itself into the background is bad software design._
        
       ___________________________________________________________________
       (page generated 2024-03-01 23:01 UTC)