[HN Gopher] Systemd, 10 years later (2020)
       ___________________________________________________________________
        
       Systemd, 10 years later (2020)
        
       Author : sysoleg
       Score  : 136 points
       Date   : 2022-01-08 19:12 UTC (3 hours ago)
        
 (HTM) web link (blog.darknedgy.net)
 (TXT) w3m dump (blog.darknedgy.net)
        
       | [deleted]
        
       | mongol wrote:
       | How does it work in a Kubernetes world? I found today that
       | "systemctl stop k3s" actually did not stop all k3s dependent
       | processes. A separate script was supplied to do that. So on a
       | Kubernetes box there are processes running in namespaces that
       | systemd does not manage?
        
         | fuzzy2 wrote:
         | Not sure what you're asking about. If Systemd didn't start it,
         | Systemd doesn't manage it. Nothing strange about that...?
         | Systemd does not start all the Kubernetes stuff.
         | 
         | k3s could stop everything, but it doesn't. It's a design
         | decision.
        
           | mongol wrote:
           | It did start it. For example the k3s process did start a
           | container registry, but it did not stop it. I thought one of
           | the points of systemd was that it would reap parentless
           | processes
        
             | c0l0 wrote:
             | Which processes systemd allows to live on after stopping an
             | associated service depends on the kind of KillMode= (see
             | systemd.kill(5)) that was specified for the service unit.
             | The default ('control-group') is usually what you want,
             | esp. if you are looking to not recreate the kind of chaos
             | improper process management SysV-style init scripts
             | frequently caused.
             | 
             | Chances are the k3s service unit specifies something else
             | (like 'none') - whether or not that is actually wise would
             | probably require some research.
        
               | mongol wrote:
               | Thanks for explaining that. My comment was not a
               | criticism against systemd, more a surprise that k3s did
               | something that was not expected by me when I interacted
               | with it through systemd. For example, httpd does not
               | supply a kill-all-httpd.sh script as far as I know.
               | Perhaps there are systemctl verbs that can be added to
               | support what k3s devs apparently thought neccessary to
               | implement in a separate script.
        
             | justinsaccount wrote:
             | It does exactly what the configuration tells it to do.
             | 
             | At one point some distributions accidentally configured the
             | sshd or cron unit to do that. People were a bit surprised
             | when restarting sshd or cron killed every single background
             | child process.
             | 
             | See https://www.freedesktop.org/software/systemd/man/system
             | d.kil...
             | 
             | Not to mention if k3s is just managing containers running
             | under docker or containerd, those processes are not
             | parentless just because k3s stopped.
        
               | mongol wrote:
               | My point is, this feels like an area where there is more
               | work to do. Not surprising, the container landscape is a
               | relatively new and rapidly evolving field that hardly
               | existed when systemd was invented. But the fact that a
               | piece of software supplies a killall.sh script reminds me
               | of the old situation before systemd, when every software
               | provided their own lifecycle scripts.
        
             | smarterclayton wrote:
             | Depending on the container runtime in use (containerd,
             | crio), a separate process is managing and running those
             | container processes (so k3s is telling that process what to
             | do, but doesn't "own" the containers). Also, modern
             | container runtimes interface directly with systemd to
             | launch transient units (think systemd services but without
             | on disk config), so restarting a cri-o for example won't
             | restart those processes. That allows systemd to manage the
             | process and for k3s/kubelet/cri-o to be upgraded / crash
             | and recover.
             | 
             | Under crio to stop all containers you can (well, _should_
             | but YMMV depending on any changes we've made recently)
             | 
             | systemctl stop kubepods.slice
             | 
             | Note that doing that a subsequent start shouldn't bring the
             | containers back unless k3s and crio were both running (the
             | transient units don't survive restart).
             | 
             | Note: there's tons of complexity here that has changed over
             | the last five or so years as container runtimes evolve, so
             | if anything I said is wrong now it was right at one point
             | and might actually be a bug.
             | 
             | Kube for instance didn't use to gracefully shut down
             | containers on node reboot, and over last few years we've
             | gradually fixed all of the complexities so that you _do_
             | get the correct shutdown behavior between systemd (which
             | takes the "power down" input) to the unit (which encodes
             | how long graceful shutdown should take based on the pod
             | termination grace period seconds) to cri-o (which has to
             | set the correct behavior in systemd) to kubelet (which has
             | to stay running to keep updating the API) to the other
             | components on the node (like container networking, which
             | needs to stay up until all pod network pods are done).
             | 
             | systemd makes a lot of that a lot less painful than it
             | would without.
        
               | mongol wrote:
               | Thank you for enlightening me on this topic!
        
         | xyzzy_plugh wrote:
         | Define manage?
         | 
         | If systemd is your init, then it manages everything. It may not
         | manage them at a _service_ level ala systemctl stop, but that
         | 's true of pretty much everything. Whether or not k3s brings
         | down everything with it is on k3s. I certainly see why stopping
         | k3s might not stop all dependent processes: it's useful for
         | live upgrades.
        
           | mongol wrote:
           | Yes the live upgrade argument was mentioned when I searched
           | for it. So it makes sense in some ways but it surprised me,
           | and I think probably this is an area that can be improved.
        
       | zh3 wrote:
       | It's not as bad as at first feared. Views here are along the
       | usual lines; for many when there's an issue either it's something
       | that's fixable inhouse (init scripts) or needs a support
       | contract.
       | 
       | Having said that, personally (old school) I lost 2 or 3 hours
       | yesterday due to not realising systemd was changing things I
       | changed for a reason.
        
         | na85 wrote:
         | It's not so bad _now_.
         | 
         | You might be forgetting how bad it was at first. In fact I'd
         | argue it was worse than initially feared, and the developer's
         | supreme arrogance did little to help.
        
         | opan wrote:
         | >I lost 2 or 3 hours yesterday due to not realising systemd was
         | changing things I changed for a reason.
         | 
         | This is one of my go-to reasons when people ask me why systemd
         | is bad. Once it overrode power settings I had configured in
         | xfce-power-manager. Mainly I wanted the lid close to do nothing
         | on AC power and to sleep on battery. It started sleeping for
         | everything after an update once. I think in the end I couldn't
         | figure out how to get the exact desired behavior with systemd
         | and I disabled sleep entirely.
         | 
         | The other reason I often mention is the ridiculous shutdown
         | timer thing that makes you wait for _minutes_ sometimes for
         | some service to end.
         | 
         | The main advantage to systemd is that a lot of unit files were
         | already written and packaged with stuff, including --user
         | variants, and also that it has a user mode at all (OpenRC
         | doesn't yet). Fairly minor advantage, imo. In general I prefer
         | shepherd, runit, or openrc. My needs are pretty simple, though.
         | Most of the time I just need to start/stop/restart a service,
         | which is pretty simple in every init system I've used.
        
           | blueflow wrote:
           | When waiting for units to shut down, have you tried pressing
           | Ctrl+Alt+Delete 7 times? It forces a reboot, and you can turn
           | the machine off before linux has taken over again.
        
             | zozbot234 wrote:
             | > have you tried pressing Ctrl+Alt+Delete 7 times? It
             | forces a reboot
             | 
             | Not sure if serious? At that point, how is that any better
             | than just using Windows? "Press Ctrl+Alt+Delete 7 times"
             | has long been SOP there.
        
               | ReactiveJelly wrote:
               | Ah, someone scooped me! Just confirmed it in the docs htt
               | ps://www.freedesktop.org/software/systemd/man/systemd.htm
               | ...
               | 
               | I didn't even know Windows did this. This is going to
               | save my butt some day.
        
               | blueflow wrote:
               | See https://cgit.freedesktop.org/systemd/systemd/tree/NEW
               | S?id=2d...
               | 
               | Color me as part of the anti-systemd crowd, but i still
               | got to support it on machines where i didn't get to
               | choose.
        
       | an_d_rew wrote:
       | This was an amazing review, and love systemd or hate it, this
       | article gives a fantastic history and exposition of what systemd
       | _actually is_.
       | 
       | My favorite quote (and when taken in context is much less tongue-
       | in-cheek thank you might think):
       | 
       |  _systemd is an event-driven object manager with dependency-like
       | side effects which 'boxes' primitive kernel resources and
       | userspace subsystems into a generic object type called Unit.
       | These Unit objects are scheduled through the state propagation
       | mechanism of 'jobs' and dynamically dispatched via a singleton
       | object called Manager, responsible for launching jobs in
       | 'transactions' which do merging, cyclic ordering and consistency
       | checks and serve as the main point at which unit dependencies are
       | pulled in. Unit startup is executed as a non-indempotent parallel
       | dataflow with weak ordering guarantees on the job level, mostly
       | independent of the active state of dependent units._
       | 
       | Any night you have insomnia, just read that a few times and
       | you'll be out like a light! :-)
        
         | ratorx wrote:
         | I'm a fan of this description. It might not be the easiest to
         | understand or the first one you should read, but I wish more
         | things had this kind of description somewhere (but not
         | necessarily on the introductory materials or front page).
        
           | rlonn wrote:
           | It's buzzword bingo with the aim to inflate the writer's ego
           | and intimidate the reader. This stuff happens all the time
           | when insecure people get to write things.
        
             | ratorx wrote:
             | Which words do you think are buzzwords in the description?
             | What I think of by buzzwords are things like "Blockchain",
             | "Machine Learning" etc. which are generic topics used to
             | build hype whilst not really explaining anything.
             | 
             | In this case however, the words are all fairly precise and
             | accurate (at least from my basic understanding of systemd).
             | They might not be words used in everyday conversation, but
             | they don't need to be unless the document is targeted as
             | introductory material (which I don't think is the case).
             | 
             | That's not to say this is the only way to explain it, but
             | it is a very concise way, which I at appreciate.
        
               | [deleted]
        
             | Sebb767 wrote:
             | I highly disagree. Sure, it contains a lot of buzzwords,
             | but it clearly describes systemd and its philosophy in a
             | single paragraph. It's not the first thing you should throw
             | at someone unprepared (because then it would really be
             | intimidation), but it prevents you from reading 20 pages of
             | documentation and not getting the gist.
             | 
             | Sometimes, a concise, high-level overview is what you need.
        
               | justin66 wrote:
               | > it clearly describes systemd
               | 
               | It really doesn't.
        
             | coldtea wrote:
             | That's what you've got from a very clear technical
             | description, in a very detailed history and breakdown?
        
             | Spooky23 wrote:
             | Exactly.
             | 
             | "Idempotent" is one of the key words used as a cudgel in
             | this way. It's great for the bullshit artist, as to a
             | layman the dictionary definition is completely inscrutable.
        
               | bollu wrote:
               | Idempotence is a desirable property of systems whereby
               | repeating an action has no effect. Mathematically, an
               | operator is idempotent if f(f(x)) = f(x).
               | 
               | Idempotence is handy because it implies that one can
               | retry safely if one isn't sure whether something
               | succeeded or failed. Not sure if your file was saved?
               | Save again! If it was indeed saved, the file continues to
               | be saved. If the file was not saved, it is now. Yay.
               | 
               | I would kill for such descriptions of every system I work
               | with.
        
               | lifeisstillgood wrote:
               | Idempotent is surely a valuable piece of software jargon.
               | I know exactly what the user intends, and have good ideas
               | of how hard it is and what likely trade offs they will be
               | making behind the scenes
               | 
               | surely it is the very opposite of bullshit.
        
               | diegocg wrote:
               | Idempotent is a perfectly fine word. There is absolutely
               | no reason why CS shouldn't have it's own jargon. I found
               | the description very descriptive.
        
               | ray__ wrote:
               | _" I found the description very descriptive"_ - an
               | idempotent system in action.
        
               | SkyMarshal wrote:
               | _> as to a layman the dictionary definition is completely
               | inscrutable._
               | 
               | This isn't written for laymen, who would obviously never
               | be reading about what SystemD is. It's written for
               | specialists who do engineering work on or with Linux, and
               | who actually need to know the technical details of their
               | tools.
               | 
               | And anyway, I suspect plenty of laymen, and highly
               | trained professionals in other fields like law, medicine,
               | and philosophy among others, would have no trouble
               | understanding what _" idempotent"_ is. It's not
               | inscrutable at all, but rather is one of the easiest
               | concepts in CS and software engineering to understand.
        
               | CogitoCogito wrote:
               | > Exactly.
               | 
               | > "Idempotent" is one of the key words used as a cudgel
               | in this way. It's great for the bullshit artist, as to a
               | layman the dictionary definition is completely
               | inscrutable.
               | 
               | Are you saying that being "idempotent" is completely
               | inscrutable? Is it not a fairly straight-forward concept?
        
               | [deleted]
        
               | tehbeard wrote:
               | Idempotent is a perfectly cromulent word for a technical
               | article not aimed at complete newbies.
        
               | coldtea wrote:
               | Idempotent is a very clear and quite basic term: being
               | able to run something multiple times with no different
               | outcome than running it once. A very valuable property
               | for certain systems (even for simple shell scripts).
               | 
               | And the description of systemd just uses bog standard
               | terms related to what systemd does and the resources it
               | manages.
               | 
               | But I get the feeling that both this and the parent
               | comment it agrees with were more about the putdown and
               | talking bad at someone, than about a technical or other
               | objection.
        
               | arendtio wrote:
               | Actually, it is just a word to describe the property that
               | if you do something repeatedly, it leads to the same
               | result as doing it just once. So the latter executions
               | don't change anything.
               | 
               | And because it is such a common property when talking
               | about algorithms and APIs the word is pretty common among
               | CS people.
               | 
               | One very popular example are REST APIs because some HTTP
               | verbs are idempotent and others are not [1]. This is also
               | the reason why your browser asks you, if you want to
               | submit a form again when refreshing a page that has been
               | reached by submitting a POST-form.
               | 
               | [1] https://en.wikipedia.org/wiki/Representational_state_
               | transfe...
        
             | arendtio wrote:
             | It uses many well defined technical terms like:
             | 
             | - event-driven
             | 
             | - object
             | 
             | - side effects
             | 
             | - kernel resources
             | 
             | - userspace
             | 
             | - subsystems
             | 
             | - unit
             | 
             | - scheduled
             | 
             | - state propagation
             | 
             | - dispatched
             | 
             | - singleton
             | 
             | - merging
             | 
             | - cyclic ordering
             | 
             | - consistency checks
             | 
             | - dependencies
             | 
             | - non-idempotent
             | 
             | - weak ordering guarantees
             | 
             | And while this list is good for BS Bingo, as these terms
             | are being used in other contexts too, the overall
             | description of what systemd does is pretty precise for just
             | three sentences.
        
         | zozbot234 wrote:
         | ...so, how does a systemd unit definition differ from a
         | Makefile (other than using a different syntax, obviously)? What
         | features would you need to add to UNIX make (and PID 1, for
         | that matter) in order to be able to separate out this whole
         | dependency management business from the other stuff PID 1 has
         | to deal with already?
        
           | eternityforest wrote:
           | For one thing make is imperative and doesn't actually know
           | what it's doing. Systemd is declarative. You'd need to
           | replace all the options service files give you.
           | 
           | Make doesn't do dependencies between unrelated projects. A
           | makefile can't say "I want to run if X runs, but I should run
           | before them".
           | 
           | Make doesn't have supervision, you'd need some kind of
           | declaratively configured poller.
           | 
           | Systemd is also parallel, there's probably some complexity
           | there that make might or might not handle well.
           | 
           | I think the most important problems would only be obvious
           | once you started using a replacement system though. Systemd
           | almost totally replaces the need for any imperative code
           | whatsoever for process autostart, configuration, and
           | supervision, and that might be hard to do in a more modular
           | setup.
           | 
           | You don't need extra stuff besides the service file.
           | Everything is extremely consistent and done the same way for
           | every service.
        
             | hedora wrote:
             | Everything statement that comment makes about make is
             | incorrect, except the bit about polling.
             | 
             | Polling is better-addressed by having each service
             | implement its own error handling. Restart loops are the
             | wrong policy for most services. In the rare cases where
             | they are actually the correct behavior, they're trivial for
             | the service author to implement.
             | 
             | Those that don't understand make are doomed to reimplement
             | it poorly.
        
           | hedora wrote:
           | My devuan linux desktop says something like this on each
           | boot:
           | 
           | INIT: version 2.88 booting
           | 
           | Using makefile-style concurrent boot in runlevel S.
        
             | TacticalCoder wrote:
             | I've tried Devuan once then it some roadblock because
             | Docker wouldn't run easily on it, but that was a long time
             | ago.
             | 
             | I may try it again one of these days now that the latest
             | Devuan stable is based on Debian Bullseye.
             | 
             | The "why" is simple: to restore some balance in the force.
             | 
             | I do prefer my Linux world to be one where I still have a
             | _choice_ as to which init system and PID 1 my system is
             | running.
        
         | throwawayboise wrote:
         | It reads like an over-eager developer read _Design Patterns_
         | and then was tasked with writing a new init system for Linux.
        
           | throwaway984393 wrote:
           | His name is Lennart Poettering, and apparently he was tasked
           | with reinventing the entire operating system
        
             | bbarnett wrote:
             | Quite true, and he was coerced!
             | 
             | http://systemd-the-porno.com
        
               | kinghajj wrote:
               | ... I don't get it? Who is "Madeline" supposed to
               | represent?
        
               | bbarnett wrote:
               | Your Mama?
               | 
               | (Sorry, _had_ to say that)
        
       | The_rationalist wrote:
       | good read
        
       | usrbinbash wrote:
       | https://www.reddit.com/r/archlinux/comments/4lzxs3/why_did_a...
       | 
       | quote:
       | 
       |  _" What most systemd critics consider "bloat", I consider
       | necessary complexity to solve a complex problem generically._"
       | 
       | endquote
        
         | zozbot234 wrote:
         | For what it's worth, the OP argues the opposite - that systemd
         | is _insufficiently_ generic. Much of its complexity is a matter
         | of conflating mechanisms and policy.
         | 
         | This is par for the course for what's essentially prototype-
         | quality software that's trying to address a number of non-
         | trivial issues - but let's not pretend that real improvement is
         | not possible here, or that systemd-as-it-currently-is does not
         | stray from what's often called 'the UNIX philosophy' or 'the
         | art of UNIX programming', of which separating mechanism from
         | policy is a key component.
        
         | rjzzleep wrote:
         | I use archlinux as a daily driver, but it breaks far more
         | randomly than gentoo ever did for me. Archlinux's bugzilla is
         | full of random breakages that happen at random system updates.
         | Here's a random python bug that been happening, cost me a
         | couple of hours to debug and for now I don't update python
         | until they get around fixing it[1]. The fix? Download an older
         | python version(not too old since that would break everything
         | else) from the arch package archive.
         | 
         | I don't want to hear anything about stable or production
         | systems or debugging from arch maintainers. Every so often I
         | forget the journalctl and coredumpctl command line flags and
         | have to google how to do basic tasks.
         | 
         | [1] https://bugs.archlinux.org/task/73002
         | 
         | EDIT: and before someone asks me why i'm using it, it's because
         | i lke tinkering and for the most part i can deal with these
         | "outages" on my laptop.
        
           | ReactiveJelly wrote:
           | The official Arch stance seems to be "You should have read
           | the blog before updating", and I enforce that by updating
           | maybe 3 times a year at most.
           | 
           | I think I'm just used to the installer. I like doing
           | everything by CLI. Ubuntu is always trying to push something
           | odd like Snap, and I think Xubuntu is defunct. The Ubuntu
           | installers are very opinionated on what kind of desktop they
           | want you to have.
           | 
           | One of these days I'll just buy all the ThinkPenguin stuff
           | and go to FOSS-only Debian with xfce. Debian has a great
           | installer, too. Then I'll turn my desktops into cattle-style
           | systems where I blow up the rootfs once a year for full
           | system updates.
        
             | mrtweetyhack wrote:
        
             | yonrg wrote:
             | Arch user here since 2013 on my mobile systems. I do -Suy
             | nearly every day, which causes me nearly null troubles. My
             | experiences with any supported enterprise distro are much
             | worse.
        
             | rjzzleep wrote:
             | Did you check the linked bug and its suggested workaround?
             | It's nothing an update note would address. It's just random
             | breakage. There are a lot of these issues scattered across
             | their bugzilla.
        
           | Sebb767 wrote:
           | > Download an older python version(not too old since that
           | would break everything else) from the arch package archive.
           | 
           | Unless you explicitly disabled it, you should have old
           | packages archived in `/var/cache/pacman/pkg`. So you can
           | simply downgrade via `pacman -U
           | /var/cache/pacman/pkg/python-x.y.z`; no need to go to an
           | (possibly untrusted) archive.
        
           | zozbot234 wrote:
           | Arch is not appropriate for production use. LTS distros (two
           | years of stability or more) exist for a reason.
        
         | dredmorbius wrote:
         | https://old.reddit.com/r/archlinux/comments/4lzxs3/why_did_a...
        
       | mnd999 wrote:
       | It's tolerated because it was forced upon people by distros. I
       | don't know anybody who likes it, and I would happily rip it out
       | if it wasn't so much work to do so.
        
         | Sebb767 wrote:
         | Count me in as a big fan. I've worked with a lot of init
         | systems and systemd has warts, but it's my favorite by far.
        
       | akagusu wrote:
       | I am old enough to remember that problems with systemd were less
       | technical and more political.
       | 
       | People didn't like the way systemd developers pushed the
       | community to adopt systemd, specially when they asked for 3rd
       | party developers to make systemd a hard dependency.
       | 
       | Unfortunately,people don't remember this today, and think users
       | resisted to systemd adoption only because they didn't like
       | systemd.
        
         | jandrese wrote:
         | SystemD is kind of like MacOS, if you have a typical use case
         | then it makes everything very easy. If you are doing something
         | weird it can make your life a living hell.
        
           | Gigachad wrote:
           | Can you give some examples?
        
             | hedora wrote:
             | Not the person you're responding to, but try getting it to
             | mount home directories over NFS.
             | 
             | I'm sure there are many other examples.
             | 
             | Honestly, even after 10 years, I still have't had a single
             | positive interaction with it vs. traditional init. From
             | what I've seen, old-school init handled (and continues to
             | handle) the problems systemd "solved" more robustly and
             | elegantly. I don't get it.
        
             | jandrese wrote:
             | Trying to set up network booting with NFS mounted root with
             | SystemD is a challenge to say the least.
        
           | Denvercoder9 wrote:
           | _> If you are doing something weird it can make your life a
           | living hell._
           | 
           | I find it much easier to reliably accomplish non-standard
           | things with systemd unit definitions, than it used to be with
           | the ill-defined complex set of shell scripts we had before.
        
             | jcrawfordor wrote:
             | This issue is a lot bigger outside of services... I think
             | the systemd service architecture is a clear improvement
             | over SysV. The bigger issues I run into with systemd
             | flexibility are related to the many other aspects of it.
             | systemd-resolved, for example, has a very "opinionated" (to
             | be polite) set of expectations about the environment and
             | the systemd project tends to view any complaints about it
             | not working outside of that environment as being the fault
             | of the complainant. This leads to, well, DNSSEC is
             | basically just broken and split horizon DNS is extremely
             | unreliable. The former is sad but mostly doesn't matter,
             | the latter leads to a lot of corporate networks having to
             | disable resolved (I think configuring it to remove the
             | semi-hardcoded "backup" resolvers reliably fixes this
             | problem but honestly I find it completely ridiculous that
             | resolved has a hardcoded list of DNS servers it just uses
             | instead sometimes. It's unclear to me whether or not that's
             | a bug at this point and I got tired of trying to follow the
             | issues and mailing list threads where the developers were,
             | uh, not amazingly helpful).
             | 
             | systemd management of mounts can be similarly narrow about
             | the types of configurations it supports, and systemd-
             | firewalld just sort of openly only claims it can support
             | simple use-cases.
             | 
             | systemd-journald is also a net loss of flexibility compared
             | to rsyslog but, on the other hand, rsyslog could quickly
             | turn into an inscrutable mess if you used any of the
             | advanced features (rainerscript...), so this may not be an
             | entirely bad thing.
             | 
             | And in general systemd is, well, opinionated. I hate to
             | bang on resolved too much but I just happen to have spent
             | quite a few hours last week figuring out resolved problems.
             | Resolved does not handle "dotless" domains correctly in a
             | lot of existing environments (it's very particular about
             | exactly how the search domain is set up). Poettering has
             | basically responded that it's because dotless domains are
             | stupid and no one should use them, so it won't be changed.
             | I don't necessarily disagree that dotless domains are not a
             | good idea today but it does mean that resolved breaks a lot
             | of older corporate and institutional environments that have
             | been using them successfully for decades. This manifests as
             | "I updated my distro and the intranet stopped working."
             | That kind of breaking change is not very common with core
             | Linux services and isn't going to make many friends in the
             | IT crowd.
        
               | mianos wrote:
               | I think systemd is OK for a lot of things, I don't even
               | hate journald. The API pretty good.
               | 
               | But, I am 100% agreement on the whole resolvd thing. It
               | is a complete fiasco for anything but someone's idea of a
               | standard network.
               | 
               | Even the most basic things, like waiting for DNS to be up
               | before mounting a network filesystem has to be done by
               | writing your own units if you don't want to just try and
               | mount and hope for the best.
        
         | throwaway984393 wrote:
         | Part of it was political, but it was also just practical.
         | Poettering's designs are some of the worst for usability I've
         | ever seen. They _technically_ work, but you need a million
         | StackOverflow Q &As to figure out how to get anything done or
         | solve the many common problems.
         | 
         | It's like somebody decided that the problem with Linux was it
         | wasn't complicated enough. And then everbody just went with it.
        
         | imoverclocked wrote:
         | Network configuration (on say, Debian) is still a pain and has
         | a lot of different ways of accomplishing similar tasks.
         | Anything beyond the plain vanilla singular network device that
         | is either dhcp/static is often painful. Systemd has some sauce,
         | there is legacy /etc/network* stuff, there are daemons which
         | can be configured graphically... it's still a mess. In this
         | arena, I think systemd has failed for the last decade. It
         | introduced yet another way of doing things without considering
         | the whole scope of what it needed to replace.
         | 
         | I'm old enough to remember the culture wars and even older to
         | remember a well thought out approach to overall configuration
         | in Debian given the available tools of the time.
        
           | jasonjayr wrote:
           | Networking on linux is wildly powerful. Most desktop/SOHO
           | installations would just be Wifi + DHCP. Maybe some VPN in
           | the mix. But on the other side of the scale, there are
           | routers with complex netfiltering, vlans, crazy GRE Tunnels,
           | Tun/Tap, Wireguard, macvlans split between namespaces,
           | bridges, etc, you name it.
           | 
           | Defining a straightforward configuration structure for that
           | complexity is going to be insane. NetworkManager + Debian's
           | /etc/network/interfaces (IMHO) seems to be a good fit: auto
           | config w/ a GUI in NM in the simple case, and if the
           | interface is listed in /etc/network/interfaces, you can do it
           | anyway you want with Debian's system, with pre-up, up, post-
           | up, post-down, etc scripts to your heart's content to setup
           | whatever complexity you need.
        
           | taftster wrote:
           | Yes, this. The problem with systemd isn't systemd. It's that
           | it never really replaced the other "legacy" parts
           | effectively. Networking is a fantastic example. I'd be OK if
           | there was one-definitive-place to configure networking, but
           | when I read an article or follow a stack-overflow post, I'm
           | not sure if what I'm changing is the most correct change or
           | if its deprecated in favor of something newer. Systemd, I
           | thought, would help settle this problem, but instead has only
           | contributed to it.
        
         | fartcannon wrote:
         | I still don't trust it, given how aggressively it was pushed.
        
         | goodpoint wrote:
         | > enough to remember
         | 
         | We are talking 10 years ago, not 1975 here. Unless everybody is
         | 25 years old on HN...
        
       | beefee wrote:
       | This is an important document and the best piece of writing I've
       | ever seen on the topic of systemd.
        
       | NotAWorkNick wrote:
       | ".... and found that someone was about to release TCP/IP for
       | Linux" (Fred van Kempen, linked interview 1992)
       | 
       | Wow. I had forgotten how ropey early Linux was. I mean, I rip on
       | Windows folk about early versions of Windows prior to 98SE or NT4
       | (and can recall having to install Winsock on a home Win95 install
       | and thinking why the hell????) but finding out 30 years later
       | that my early struggles with Linux were not entirely down to my
       | own incompetence is ... refreshing. I mean how do you Google (ok
       | in those days Altavista!) a "how to" if the network stack is
       | borked? How do you even download the drivers or updates if the
       | interface fails to come up?
       | 
       | It sounds silly but I'm so glad to know it wasn't something I was
       | doing wrong on my end. 30 years later and dayamn!
        
       | tarkin2 wrote:
       | The hatred for Systemd seems to come from those who see the unix
       | philosophy of small, single use programs being thrown away in
       | favour of a monolith.
       | 
       | Back when I had linux as a desktop (around the day of slackware
       | 10) there were plenty of problems with simple stuff like mounting
       | usbs, connecting to wifi etc--possible but a pain. I've hardly
       | looked at systemd or linux in such a way for a long time,
       | however.
        
         | Gigachad wrote:
         | Linux users tend to come up with whatever "philosophy" suits
         | their agenda of complaining about all change. The unix
         | philosophy gets thrown out the window as soon as it comes to
         | Wayland which took a huge, bloated, and buggy Xorg and broke it
         | down to something more sane and manageable.
        
           | hedora wrote:
           | Doesn't wayland force window managers to be integrated into
           | some behemoth bloated monolith?
           | 
           | My preferred WM is evilwm. I read the entire source to it in
           | a few hours, and occasionally edit the source when I want to
           | customize it for my odd workflows. I don't think that's
           | possible with wayland.
           | 
           | Also, wayland forced a bunch of questionable security
           | policies around screen capture and keyboard handling, and
           | didn't provide escape hatches required for basic
           | functionality. Is that fixed yet?
           | 
           | I stopped paying attention years ago, to be honest.
        
         | kzrdude wrote:
         | I'm not sure, it seems to me that the hatred for systemd seems
         | to come from those that learned something else and don't want
         | to learn systemd. My analysis would be that the "unix
         | philosophy" is just a convenient argument against it.
        
         | dijit wrote:
         | I don't care about philosophy; I don't like being forced and I
         | don't like opaque systems where they aren't required.
         | 
         | SystemD is essentially forced (for better or worse, not gonna
         | argue) because it is a hard dependency for certain software and
         | tightly integrated with other tools in the systemd ecosystem
         | 
         | It is opaque because I cannot read, modify or understand what
         | it's doing without breaking out gdb, heaven forbid I want to
         | alter its behavior in a less than approved way.
         | 
         | This leads to weird situations like the MySQL unit executing a
         | bash script to run: because systemd has limited preflight
         | checking compared to alternatives.
         | 
         | Before someone claims I want sysvinit back: No, I don't want
         | that. I would simply prefer systemd to be smaller so that we
         | can replace it with something better or more tailored should
         | the need arise.
         | 
         | We're locking ourselves in to prototype level software (which
         | we best into working semi decently) with obstinate leadership
         | team who deny any faults.
        
       | marcodiego wrote:
       | Systemd pissed off many people. For me and most users, the
       | transition was fully imperceptible. With the difference that
       | systemctl tools seem to work better for what it does than the
       | many sparse different tools it replaced.
        
         | klysm wrote:
         | I'm young enough to have missed the transition and really like
         | systemd, I rely on it for as much as I can
        
           | MuffinFlavored wrote:
           | Like what if you don't mind me asking, and what did people
           | used to use as an alternative?
        
             | hedora wrote:
             | Take a look at OpenBSD. They managed to sidestep all this
             | drama, and still ship a sane, well-documented init.
             | 
             | If you spend an afternoon installing it and configuring a
             | few network services, you'll already know most of what
             | there is to know about how init used to work.
             | 
             | It used to be extremely elegant and robust.
             | 
             | The main flaw in the design was that the "service" files
             | were written in a Turing complete language, which meant
             | that distribution maintainers could ship things that were
             | arbitrarily terrible, and many chose to do so.
             | 
             | (Edit: Also, the organizations that somehow couldn't figure
             | out how to use and maintain traditional init systems were
             | the ones that produced the current generation replacements
             | and pushed them so hard.)
        
             | diegocg wrote:
             | A bunch of very large bash scripts that were entirely
             | different in each distro, had tons of races and were much
             | slower to boot (one of the reasons why systemd became so
             | popular and there was popular pressure to switch to it is
             | that it would speed up booting by several seconds).
             | 
             | Also upstart, which was actually pretty good, but systemd
             | was more ambitious.
        
             | dharmab wrote:
             | For the init system, the most common predecessor was a
             | combination of sysvnit and "initscripts" (a shell script
             | for each daemon on your system). Writing the init scripts
             | was simple for the most basic cases but quite complex for
             | complete, bug free case coverage. A big appeal of systemd
             | for packagers and sysadmins was the relative simplicity of
             | unit files. Now we just have to fill out a simple
             | boilerplate INI style file for most services.
        
               | kzrdude wrote:
               | And less distro divergence in how to write the service
               | scripts, or how to start services etc.
        
               | Blikkentrekker wrote:
               | It was not.
               | 
               | There is strange false history that systemd replaced
               | sysvinit, which occurred only on Debian, because Debian
               | was the last major system to discard sysvinit.
               | 
               | Gentoo had replaced sysvinit with OpenRC in 2006; Fedora
               | and Ubuntu were using Upstart in 2005. systemd primarily
               | replaced Upstart and systems that used Runit or OpenRC
               | tended to have stayed with them.
               | 
               | Frankness be, this false history seems to be crafted in
               | order to compare it with very outdated technology to make
               | it look good. Obviously systemd is better than sysvinit,
               | technology that almost all systems but Debian had
               | discarded long before systemd was even started. The issue
               | is whether it compares favorably to Upstart, runit, and
               | OpenRC, and to the first I would say yes, because Upstart
               | was full of very ugly hacks and didn't deliver it's
               | ambition. In the second two cases, their mode of
               | operation is so different that it's hard to compare.
               | runit was inspired by daemontools and is very simple and
               | effective and works in a very different way from most
               | models of service management; OpenRC is backwards
               | compatible with sysvinit and is essentially a large
               | library so that init scripts typically need nothing more
               | than something such as:                  #!/sbin/openrc-
               | run        command=sshd        args="-D"        daemon=no
               | provides=ssh        after=net
               | 
               | Or something similar to work. I forgot the exact syntax
               | but it works similar to simply assigning some variables
               | and it handles the rest, but it still gives one full
               | access to the shell to use whatever shell functionality
               | one wants.
        
         | api wrote:
         | It's a functional improvement over old init, but the thing I
         | hate about it is how unnecessarily arcane it is. It seems
         | "designed for job security" or more likely designed for
         | consulting revenue.
         | 
         | There is absolutely no need for its command names and
         | everything else about it to be so counterintuitive.
        
           | rnd0 wrote:
           | >It seems "designed for job security" or more likely designed
           | for consulting revenue.
           | 
           | Exactly this, yes; and that's why it was pushed the way it
           | was.
           | 
           | I dread the day when systemd or an init system just like it
           | consumes the BSD world.
        
         | a-dub wrote:
         | it's kind of like locked bootloaders and pentalobe screwdrivers
         | for organizing a unix system where most users were used to easy
         | to inspect and modify scripts and single purpose programs .
         | 
         | yes, it's still open source, but modifications and inspection
         | are far more complicated creating a much higher barrier to
         | entry.
         | 
         | in the earlier days when it was still buggy, even linus himself
         | was screaming about not being able to easily take the case off
         | of pid 1 when it misbehaved.
        
           | wrycoder wrote:
           | Right. I have a simple rule: don't use a GNU/Linux system
           | that uses systemd. It doesn't respect the Unix philosophy.
           | It's trying to turn it into Windows.
           | 
           | I'm waiting for Microsoft to buy Red Hat. Failing that, IBM.
        
             | gtirloni wrote:
             | Uh?
             | 
             | https://www.redhat.com/en/about/press-releases/ibm-closes-
             | la...
        
             | cfors wrote:
             | IBM already purchased red hat for $34b 3 years ago, so no
             | need to wait any longer :)
        
             | [deleted]
        
             | zkldi wrote:
             | I mean, emacs doesn't respect the UNIX philosophy either,
             | but that's one of the most praised pieces of software to
             | come out of the UNIX space.
             | 
             | The UNIX philosophy isn't meant to be taken as gospel -
             | hell, it's very hard to define what "doing one thing" even
             | means.
        
               | elcritch wrote:
               | Didn't emacs get developed on ITS or Multics? Seems Unix
               | just imported it, but your point still holds.
        
               | lizknope wrote:
               | https://en.wikipedia.org/wiki/Emacs#History
               | 
               | Yes.
               | 
               | Emacs development began during the 1970s at the MIT AI
               | Lab, whose PDP-6 and PDP-10 computers used the
               | Incompatible Timesharing System (ITS) operating system
               | that featured a default line editor known as Tape Editor
               | and Corrector (TECO)
        
               | Breazy wrote:
        
             | rnd0 wrote:
             | I hate the feature and mission creep of systemd and try to
             | use other unixes where I am able; but what you wrote is
             | downright insane.
             | 
             | There were several init replacements in the mainstream
             | before systemd. OSX and Solaris both had their own, and
             | they were as UNIX (officially certified) as you can get.
             | 
             | Also, as long as we're doing non-sequitors; Microsoft is
             | more likely to buy Canonical than they are to buy RedHat or
             | IBM.
        
             | jcelerier wrote:
             | that's great because the unix philosophy is pretty much a
             | PITA as soon as you get out of grep | cut | tail | tee. I
             | definitely don't want "unix philosophy" for operating
             | system management, that is the wrong tool for the job.
        
           | VTimofeenko wrote:
           | Anecdata: as someone not engaged in systemd development and
           | one that likes tinkering with hardware tokens and luks, I
           | found that at least the fido2/cryptsetup part of systemd was
           | fairly easy and logical to follow and debug.
           | 
           | I would argue that locked and pentalobe metaphor is not very
           | accurate - systemd is no more locked than most other open
           | source software.
        
           | pengaru wrote:
           | It's not like systemd is written in deliberately obfuscated C
           | or something inscrutable altogether like Brainfuck.
           | 
           | A monorepo containing well-organized boring C, developed on
           | github of all places, is the polar opposite of raising the
           | barrier to contributing vs. what was replaced.
           | 
           | Full-disclosure: I contribute to systemd.
        
             | Blikkentrekker wrote:
             | Of course, but I would not like to have to patch it to
             | modify something that on other systems can be modified with
             | either a configuration option, or a shell script.
             | 
             | To me, the most poisonous example that contrasts the
             | traditional design with Freedesktop's design is acpid
             | contrasting logind. In the former case, an acpi event is
             | sent, and acpid simply executes a file such as
             | `/etc/acpi/action/lid_down.sh`. That file can contain
             | anything the system administrator wills, from a simple
             | script that does nothing more than `systemctl sleep` to
             | whatever complex task he could desire.
             | 
             | The last time I checked the situation, logind had replaced
             | this powerful, and simple mechanism with a configuration
             | that offers nine options to execute when the lid is
             | pressed, none of which involve executing a simple file. Of
             | course I can edit the source code and recompile to allow it
             | to do what I want it to, but I would far rather simply
             | quickly alter a simple shell script to do so.
             | 
             | And luckily I can, because I do not use logind but acpid,
             | but that does not change the ire I experience when dealing
             | with many Redhat and Freedesktop people who make it very
             | clear that I should be using logind, that it's better, and
             | that they create artificial dependencies on it to as
             | Poettering calls it "gently puh" me in that direction.
        
               | vanviegen wrote:
               | Although I like the acpid approach, the logind way is
               | probably safer to abstract behind a user friendly GUI.
               | That's what most people will be using. :-/
        
               | Blikkentrekker wrote:
               | And that is probably the motivation for this design
               | philosophy of limiting options that Redhat and
               | Freedesktop champion where more and more, simple, clean,
               | turing complete solutions are replaced by a discrete list
               | of simple options.
               | 
               | But it does not change at all why I dislike it, and why I
               | am annoyed by Freedesktop's pushing and insistence that
               | many keep to these more custoizable solutions supposedly
               | simply because they are "stuck in their ways" and that
               | their philosophy is the future and "modern" whereas all I
               | see is more hurdles that stop me from effectively being
               | the master of my own machine.
        
             | ReactiveJelly wrote:
             | There is a difference between the best C, and scripts that
             | you can modify in-place.
             | 
             | But, like some other comments have said, I was young enough
             | to dodge the drama. I basically grew up with systemd and I
             | don't feel like I'm missing anything. The biggest problem I
             | have is that Ubuntu Server defaults (or used to default) to
             | having the system wait on network interfaces, which is
             | annoying when I want to build a desktop off of it. And
             | sometimes on shutdown it waits for some daemon I don't care
             | about to quit. Which is probably just a bug in the daemon.
        
               | eternityforest wrote:
               | The transition period of any new tech is usually horrid.
               | 
               | But now that things aren't buggy, being unable to modify
               | them in place is a good thing. It sets a clear boundary
               | that this is not how you do things on a modern system, so
               | you can always expect systemd to be the same, consistent,
               | non customized thing it always is.
        
               | a-dub wrote:
               | just like pentalobe screws!
               | 
               | i'm not going to argue it's a bad thing, i actually
               | really like macs and think they hide a lot of details
               | making time and space for other pursuits, although since
               | you cede user serviceability it's important to purchase a
               | service plan and that you don't try anything that wasn't
               | provided for in the design.
               | 
               | maybe simple composable parts are past their prime. i
               | always loved the simplicity of inetd. it was both self
               | describing and inclusive. you could write an internet
               | service without any knowledge at all of sockets and learn
               | about the system by simply looking at it. but i suppose
               | that's given way to large monolithic daemons these days.
               | 
               | personally though, i always thought the simplicity and
               | the composability of unix philosophy was what bred
               | systems that outperformed their competition in terms of
               | security and extensibility. so many commercial systems
               | reflected apparent power grabs within the corporate
               | structures that produced them... i suppose that when you
               | see foss monoliths like this, it's the same game playing
               | out on a different field.
        
               | shawnz wrote:
               | > i always loved the simplicity of inetd. it was both
               | self describing and inclusive. you could write an
               | internet service without any knowledge at all of sockets
               | and learn about the system by simply looking at it.
               | 
               | You could achieve the same thing with systemd socket
               | activation :)
        
           | kzrdude wrote:
           | systemd made it possible for me to write new service scripts
           | - system and user kind - in a way I couldn't do before. It
           | seems to have empowered me, because it's actually quite easy
           | to use.
        
           | eslaught wrote:
           | I have to hard disagree on this one. As someone who had half
           | a foot in sysvinit, half in upstart, and half in systemd
           | (don't try to do math with those numbers), sysvinit was
           | always the hardest to use by _wide_ margin. It was
           | ridiculously complicated to get anything nontrivial to work.
           | It was error prone, and the number of corner cases you had to
           | think about was extreme. And the worst part was I kept having
           | to relearn it each time because I did it infrequently enough
           | to forget what I 'd learned the last time.
           | 
           | Personally, I was more of a fan of upstart then systemd, but
           | both were a massive improvement over sysvinit in terms of
           | offering an actual interface with abstractions for the things
           | you wanted to do. This is again from the perspective of a
           | user, but that's where (in my opinion) most of the
           | cost/benefit was anyway.
        
             | [deleted]
        
         | coldtea wrote:
         | I like systemd, but I abhor the arrogance and manouvering with
         | which is was pushed.
        
         | INTPenis wrote:
         | This is the comment I came to upvote and agree with.
         | 
         | I am considered a Linux expert in my field but the truth is
         | that I really don't know how systemd internals work well enough
         | to read that review and fully understand it. The review did
         | teach me some things about what Units really are, more than
         | just services and timers, but other than that it was too deep
         | for me.
         | 
         | And yet I work full time with Linux systems, building services
         | on Linux systems for major clients and government contracts.
         | 
         | From my perspective as a user and operator of Linux, systemd
         | has been nothing but good things.
        
       | xarthna wrote:
       | I used systemd for a time, and after looking into all the
       | documentation I found it hard to understand and it's scope
       | overreaching. It doesn't stand by the unix philosophy of
       | mechanism not policy and is prescriptive in it's inplementation.
       | As another comment stated, the politics behind how it was pushed
       | never sat right with me and I believe it is leading the Linux
       | community to vendor lock in.
       | 
       | I have been using s6 on Artix and find it a great init system and
       | service manager with a clear cut design goal of knowing where to
       | stop. It does what it is supposed to do and no more - by design.
       | It doesn't have the user friendly interface of systemctl but that
       | is being currently worked on. If one is looking for an
       | alternative to systemd, I think it is an excellent option to
       | consider.
        
       | pkrumins wrote:
       | The first thing that I do when I setup a new server is rip
       | systemd out completely and replace it with regular sysv init
       | scripts.
        
       | AceJohnny2 wrote:
       | (2020)
       | 
       | See lively discussion from last time:
       | https://news.ycombinator.com/item?id=23062072
        
         | cf100clunk wrote:
         | Of all the topics for which we do not need dupes, this one is a
         | perfect example. Don't people use HN's search tool?
        
           | dang wrote:
           | On HN, reposts only count as dupes if the story has had
           | significant attention in the last year or so. This is in the
           | FAQ: https://news.ycombinator.com/newsfaq.html.
           | 
           | (but also, it's true, people tend not to use HN search before
           | posting)
        
         | dang wrote:
         | Thanks! Macroexpanded:
         | 
         |  _Systemd, ten years later: a historical and technical
         | retrospective_ - https://news.ycombinator.com/item?id=23062072
         | - May 2020 (432 comments)
         | 
         | and:
         | 
         |  _Systemd, 10 years later: a historical and technical
         | retrospective_ - https://news.ycombinator.com/item?id=23204666
         | - May 2020 (59 comments)
        
       | LAC-Tech wrote:
       | I don't actively hate systemd, but the distros I use these days
       | (Void and Alpine) don't use it, so neither do.
       | 
       | I'm generally a fan of simple systems that don't do much
       | controlled by simple text files. Maybe systemd is way better at
       | runit for more complex workloads than a desktop or a simple
       | server. But maybe it isn't. The intersection of people who like
       | systemd and have tried runit is probably quite a small one!
        
       | cmurf wrote:
       | If something were to be proposed as a systemd replacement today,
       | I'm pretty sure I'd treat it with a lot of skepticism. I likely
       | would use arguments the sysv folks made like "why fix what isn't
       | broken" etc.
       | 
       | Excluding the kooks angered enough to send hate mail to the
       | systemd developers, I can understand the old criticisms and
       | aggravation from sysv folks. I was front and center as a
       | relatively new user of Fedora at the time systemd was adopted.
       | While I was familiar with sysv, I wasn't attached to it. I also
       | found sysv a lot more confounding and obscure than systemd in the
       | early days, even though the transition did have rough spots. But
       | I'm definitely attached to systemd now.
        
       | generalizations wrote:
       | It would be interesting to see Systemd compared to other modern
       | init systems, like runit and the FreeBSD init. Seems like 2021
       | Systemd is being mostly compared to 2000's Upstart and Sysvinit.
        
       | zodiakzz wrote:
       | I miss Systemd inside Docker containers so much. IMO, for an app,
       | "one-process per container" is a huge pain and very messy. I have
       | a massively popular software to back up that claim:
       | https://github.com/discourse/discourse_docker
       | 
       | I use phusion/baseimage-docker[1] right now, love every bit of
       | it. Having the database, web server(nginx) and the java backend
       | all in one place works beautifully.
       | 
       | The only pain point is having to write `runit` scripts for
       | startup and shutdown for each service, hence the wish for Systemd
       | inside containers.
       | 
       | 1: https://github.com/phusion/baseimage-docker
        
         | cesarb wrote:
         | You can use systemd inside Docker containers. There are images
         | (like RedHat's ubi8-init) which start systemd by default, and
         | there are others which can be easily adapted to start systemd
         | (for instance, the following 3 lines for a Debian image which
         | starts systemd: "FROM debian:10", "RUN apt -y update && apt -y
         | install systemd-sysv", "CMD ["/sbin/init"]").
        
         | tracnar wrote:
         | Podman supports running systemd units inside containers:
         | https://developers.redhat.com/blog/2019/04/24/how-to-run-sys...
         | 
         | One process per container is indeed strange in a lot of use
         | cases, you lose the self-contained advantage of a container...
        
           | topspin wrote:
           | Thank you for that link. That is eye opening.
        
       ___________________________________________________________________
       (page generated 2022-01-08 23:00 UTC)