Thought for init replacement reddit / BSD ---------------------------------------------------------------------- jamhob · 2021-05-29 22:38 I had a thought that I want to run past the bsd people before doing. The init rc system in bsd is super simple and as a result easily usable by pretty much anyone, no matter how long they have been using bsd. Systemd for Linux (and launchd) has traded this simplicity for lazy loading of services and parallelisation. So here is my thought. I wondered if I could write an init rc program that used something like makefile syntax. It could be one file again, but you could easily notate parallelism in much the same way that make does. You name a service (or whatever), say what it depends on and in the body you will find exactly what we are currently writing in our init scripts. What do you think? ---------------------------------------------------------------------- segfaulting · 2021-05-29 22:48 https://xkcd.com/927 ---------------------------------------------------------------------- jamhob · 2021-05-29 22:56 Ouch ---------------------------------------------------------------------- [deleted] · 2021-05-30 01:07 I think this would be a cool idea. you should try it. segfaulting's xkcd comic is not really relevant because you are not trying to make a new standard for everyone to use, you're just trying something new ---------------------------------------------------------------------- S-A-R · 2021-05-30 01:13 The comic makes a point, but don't let that deter you from trying a good idea. ---------------------------------------------------------------------- vivekkhera · 2021-05-30 01:37 The provides and requires lines in the rc scripts already define the dependencies. They are used by the rcorder program to identify the proper order to start services. Moving such data into a separate file just makes it likely to be forgotten when a service changes something. A nice update would be to let rcorder spit out the independent dependency graphs so they could be run in parallel as well. ---------------------------------------------------------------------- dnabre · 2021-05-30 02:05 BSD doesn't need a new init system. It wouldn't hurt, but it's not problem. The current scripts work fine for an init system. The problem is that modern computing really requires a service management system. The current init system brings the system up fine, but it really doesn't do much to handle services afterwards. These are the sort of tasks that systemd has taken on. BSD will need to step up and provide a solution to those tasks in the long run. There's a lot more to it than just bringing the system up. I doubt something Makefile-based could really handle the task. Benno Rice has a number of good talks on the subject. Addressing the history of systemd, where it came from, and why it's solving a problem (regardless of one's opinion on how well). Many of them address how BSD doesn't provide this. I'd recommend starting with [The Tragedy of systemd from BSDCan 2018](https://www.youtube.com/watch?v=6AeWu1fZ7bY) . Great to see people stepping up and trying to tackle these sort of problems, get a feel for the problem space. There is a lot more to do than just re-implement the current init system. ---------------------------------------------------------------------- bumblebritches57 · 2021-05-30 02:29 god no, anything but make syntax ---------------------------------------------------------------------- kraileth · 2021-05-30 04:55 While I'm fairly sure that I've read about an experimental make-based init system quite a while ago, I haven't been able to dig it up quickly. Didn't try it out back then but thought that it was an interesting idea. Personally I not really fond of make syntax, but make certainly is pretty capable of e.g. resolving the order something needs to be executed in. My main concern regarding any possible rc.d replacement is that most candidates (e.g. OpenRC) adopted that SysV folly of abusing the filesystem for startup configuration (i.e. configuration via *symlinks*). I like central configuration files like rc.conf (that can be version controlled easily) and hate it when additional files dictate whether a service is enabled or disabled. A major concern with most replacement projects is to eliminate the shell involvement from init. Do you intend to do something in this regard, too? ---------------------------------------------------------------------- paprok · 2021-05-30 06:31 > eliminate the shell involvement from init care to elaborate why it's undesirable? ---------------------------------------------------------------------- jamhob · 2021-05-30 07:03 I don't think I will be eliminating shell? But I hadn't thought Of it? Why would I? Security? ---------------------------------------------------------------------- jamhob · 2021-05-30 07:14 Do most programs use these pragmas properly? ---------------------------------------------------------------------- kraileth · 2021-05-30 08:50 Sure: Simplicity, security, efficiency. Shell scripts are simple and convenient only from the user perspective. Even though shells meant primarily for interactive use (e.g. bash) are much worse, plain old `sh` is already more complex than it has to be for a task like that. Linux distros like Debian acknowledged the problem by switching to a simpler shell for non-interactive use (i.e. the Debian Almquist Shell or `dash`) when they were still using SysV init. There's some interesting info on the net regarding execution speed. While the `dash` example is a step in the right direction, other people wanted more. One benefit of `systemd` (it certainly has its very own issues and defects, but in this special regard it's actually a change for the better) is that it uses *unit files* instead of shell scripts. Init replacements as `nosh` even have their main goal in the name. For a very good reading I recommend [this page](https://www.skarnet.org/software/execline/dieshdiedie.html). It's a great summary of the problems with using `sh` for something that should probably use a better tool. The author created his own shell-like scripting language that's *much* simpler and gets rid of many if not all of the known security-related problems. ---------------------------------------------------------------------- kraileth · 2021-05-30 09:04 Security for one thing. But also execution speed (which is certainly a nice bonus if we're talking *init*) and in general "using the right tool for the job". `Sh` can do it, no doubts, and did it for decades. But that doesn't mean that something else wouldn't be a better fit. After taking a brief look at `s6`, I must say that I like the approach in general. People are building things around it like `66` and I'd love to explore this a bit more. Learning `lang/execline` is slowly moving to the top of my todo list and I plan on trying to use that together with rc.d to (partially) eliminate the need for `sh`. Since you're also planning on tinkering with init, I thought I'd just ask about that aspect. ---------------------------------------------------------------------- paprok · 2021-05-30 09:08 [ouch](https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_c hap02.html) ---------------------------------------------------------------------- jamhob · 2021-05-30 09:15 I'll definitely add it to the aims list. When I wrote the post it was a 2:00am light bulb, not a fully formed idea which has the benefit of it being mailable. So far the aims are: - make inspired language to simplify dependency graphs and simplify scripts in general - formal methods to eliminate bugs - some way of lazy loading services (I like the socket ideas in systemd) - a service manager for auto restarting and alarming and monitoring that doesn't cause repetitive strain disorder to use, who's output can be easily piped If you want to help with design, message me. You clearly know more about the current state of things than I do and it's more likely I'll see a project through if it isn't just me. ---------------------------------------------------------------------- jamhob · 2021-05-30 09:20 What things about make do you hate? I was only actually going to keep one part of it: rule : dep1 dep2 ... depN body ---------------------------------------------------------------------- kraileth · 2021-05-30 09:21 Yes, exactly. This is why the author of `execline` came to the conclusion that not a certain shell implementation is the culprit here but that the problem is actually unfixable with *any* shell if you don't want to violate the POSIX specs. And in fact it all makes sense: A shell needs to do many more things than the subset of its capabilities that is being used in the context of init. There are only two conclusions to draw from this: a) decide to live with that and ignore the implications of completely unnecessary complexity b) go for radical simplicity and try out another approach. After a little bit of reading, I'm very much in favor of b) (after being in the a) camp for years). ---------------------------------------------------------------------- paprok · 2021-05-30 09:23 thanks for the insight! ---------------------------------------------------------------------- jamhob · 2021-05-30 09:24 It's a good talk! I'd definitely attempt some service management. But I think it's unlikely that what I wrote would catch on. But maybe it will inspire something better. ---------------------------------------------------------------------- bumblebritches57 · 2021-05-30 09:59 it's been years since I switched to cmake and I wasn't great with make to begin with. but the whitespace sensitivity, and tabs/spaces being relevant is just a killer. ---------------------------------------------------------------------- paprok · 2021-05-30 10:08 > ...attempt some service management. if you want to look at a working implementation (that is not systemd) - check out Solaris' SMF (if you haven't/don't know it already). ---------------------------------------------------------------------- jamhob · 2021-05-30 10:09 You can guarantee that my dsl will not be whitespace sensitive for ethical reasons. Whitespace sensitivity is why devs needs unions... ---------------------------------------------------------------------- jamhob · 2021-05-30 10:10 Thanks! I'll check it out ---------------------------------------------------------------------- paprok · 2021-05-30 10:14 i think you can even look at the source code... OpenIndiana/Illumos most likely inherited SMF from Solaris 10/11. ---------------------------------------------------------------------- tangomikey · 2021-05-30 13:14 Agreed. It may not get adopted , but it is worthwhile exploring new ideas. ---------------------------------------------------------------------- tcmart14 · 2021-05-30 14:12 I say give it a shot. It doesnt hurt, as this is FOSS, so your free to try new ideas. It may end up as something many people will want to adopt, or a new BSD OS, or it could go no where. Even if it goes nowhere, its not exactly a zero sum game. It would expand knowledge of init system implementation in the space and even if no one adopts it wholly, there is a possibility you write a few features that others may want to adopt into their own systems. At the end of the day, I only see it as a positive benefit to the BSD community and FOSS in general. ---------------------------------------------------------------------- daemonpenguin · 2021-05-30 14:37 It sounds like you're basically describing SysV init. SysV init will (when run with startpar) allow you to set up service dependency trees in a makefile and then startpar will load services as close to as parallel as possible using the makefile syntax. Most of the work has been done for you as I think startpar and insserv (the tool which automates writing the makefile) have already been ported to FreeBSD. All you'd need to do is make a few tweaks to SysV init's accounting code to make it work with your preferred BSD. ---------------------------------------------------------------------- ElimGarakTheSpyGuy · 2021-05-30 15:09 That talk is great I watched it a while back. Super informative. ---------------------------------------------------------------------- losthalo7 · 2021-05-30 15:25 Having multiple drop-in modular init systems as options isn't the same as having multiple competing *standards* for init systems. ---------------------------------------------------------------------- segfaulting · 2021-05-30 15:27 It certainly is the same when it comes to package maintenance and technical support. You have the option to use some 3rd party crap if that's what floats your goat but it won't be supported because it's not the standard. ---------------------------------------------------------------------- losthalo7 · 2021-05-30 17:01 *You have the option to use some 3rd party crap if that's what floats your goat but it won't be supported because it's not the standard.* Why assume it will be crap or tell OP 'no will use it anyway'? Why the need to be so fractious about it when OP started by politely asking whether it was a good idea to pursue? ---------------------------------------------------------------------- ctisred · 2021-05-30 20:02 came here to say just this, at least for netbsd - not sure how freebsd integrated rcng, but openbsd didn't pull in rcorder and hardcodes the start sequence ---------------------------------------------------------------------- ctisred · 2021-05-30 20:03 they should - if not, it would be a bug in the rc script for not properly defining it's requirements.. see other comment in thread about rcorder not being everywhere. ---------------------------------------------------------------------- crocflamingo · 2021-06-06 22:30 well it relies on tabs and spaces to actually run. I don't want essential processes to fail because I forgot a space. ---------------------------------------------------------------------- RogerLeigh · 2021-06-10 06:19 You just described "insserv" and "startpar" which were used by SuSE and Debian to parallelise startup, or (when parallelisation was disabled) to strictly-order the startup sequence in dependency order. "insserv" generated a dependency graph and serialised it as a Makefile-style file. "startpar" evaluated the file and did the parallelised service startup. These relied upon extracting dependency information from comments in the script headers, so-called "LSB headers". (Ex Debian sysvinit maintainer.) ---------------------------------------------------------------------- jamhob · 2021-06-10 16:05 I've had a look and it actually looks amazing! Why did it get overtaken by systemd? ---------------------------------------------------------------------- RogerLeigh · 2021-06-10 17:59 Well, that's a good question. I'd love to give you a concise and unbiased answer, but I'm afraid that isn't possible. There are many reasons why systemd won out, both for technical and non-technical reasons some of which are valid, and others which are not so valid. There are discussions and flamewars spanning over half a decade which I'm loath to revisit. If you want to investigate, there is ample discussion to be found on the topic. It's fair to say that dependency-based boot works robustly, but it's also fair to say that systemd does far more than that and solves many problems which sysvinit and insserv/startpar considered out of their scope. Possibly it goes too far; its service activation actually introduces indeterminism to the boot process while startpar is deterministic in how it walks the dependency graph (though it's fair to say timing differences in parallel service startup can potentially alter the ordering, but not the correctness). startpar is completely deterministic in a non-parallelised startup since it's effectively a linear sequence at that point. Let's just say that this whole topic is one (of several) reasons I'm no longer a Debian developer or user, and switched to using FreeBSD about five years ago. I was massively burnt out by the stress of the multi-year-long horrible flamewars, and my enthusiasm for free software development has never really recovered. Even my full-time job was less stressful, and this was something I did for personal enjoyment... I tinker around with bits and pieces, but my days of a second full-time job doing distribution maintenance are long over at this point. It's a mug's game.