[HN Gopher] There are only 12 binaries in Talos Linux
___________________________________________________________________
There are only 12 binaries in Talos Linux
Author : JustinGarrison
Score : 94 points
Date : 2024-03-04 18:52 UTC (4 hours ago)
(HTM) web link (www.siderolabs.com)
(TXT) w3m dump (www.siderolabs.com)
| cedws wrote:
| >As opposed to systemd which is over 3000 lines of C code I'll
| never comprehend.
|
| Well, technically true, but systemd is a whole lot more than 3000
| lines...
|
| I can see another binary in the demo video called apid, does that
| one not count?
|
| Any comparison with Bottlerocket OS?
| JustinGarrison wrote:
| Many of the binaries you see in the demo video are showing
| processes running from inside containers. There will be a lot
| more processes once you start pulling containers and starting
| containerized services.
|
| Notably the kubelet is also missing from the list because it's
| not built into the OS but pulled as needed from the correct
| version of Kubernetes requested.
|
| Bottlerocket runs systemd and also runs 2 versions of
| containerd. One for the system and one for workloads. This (in
| theory) hardens the OS more, but in practice makes things
| extremely annoying to manage because you have to get a shell on
| the host to access the API.
|
| disclaimer, I used to work at AWS on EKS and closely with the
| Bottlerocket team.
| bantunes wrote:
| The systemd hate is getting long in the tooth now. It's not
| like it doesn't do anything with its line count, or that the
| code is obfuscated.
| JustinGarrison wrote:
| I'm actually a big fan of systemd. It's an awesome, general
| purpose, and flexible init system.
|
| I don't think the complexity it brings is required for
| Kubernetes.
| tehbeard wrote:
| it could have been said without the x lines of code
| comment. "lines of code" is so often used as a
| "disparagement" about software rather than a metric for
| understandability.
|
| Something along the lines of "...not needing a general
| purpose init system that integrates with logging, network
| and mounting, when all we are running is Kubernetes."
| hosh wrote:
| There's work in a new kublet replacement that moves things
| that would normally go into daemonset into systemd (or
| something like systemd).
|
| There's also a neat feature of podman that runs pods as
| systemd units, which is a nice intermediate step between a
| more traditional pet server and a full kubernetes cluster.
| LAC-Tech wrote:
| I'm not reading that as hate, I read that as criticising
| systemd in the context of a stripped down system designed to
| do one thing.
|
| I have systemd on the laptop I am typing on right now. Do I
| want it on some tiny embedded linux device? probably not.
| JustinGarrison wrote:
| Thanks for clarifying because that was the vibe I was going
| for (not hate).
|
| I'm using projectbluefin.io for all my laptops/desktops and
| love it. Wouldn't want the same on single-purpose,
| production servers.
| shrubble wrote:
| So there is a quantum of criticism/observations about systemd
| that can be made but after that, no more is acceptable?
| jchw wrote:
| In retrospect, it would've saved a lot of trouble and
| misunderstandings if systemd had called the init daemon
| "systemd-init" to make it clear that not literally everything
| that is under the umbrella is part of the init daemon.
| yjftsjthsd-h wrote:
| Eh... Most of the other components have a hard dependency on
| the init part; I'm not convinced that they're all that
| separate.
| Jumziey wrote:
| SystemD bashing aside :p Talos is pretty awesome for setting up
| clusters. At home I just run talos with matchbox for PXE
| bootstrapping it works like a charm. Been really easy to maintain
| too. I normally just update matchbox and then reset a machine at
| a time with talos ctl for a clean install. It's something very
| reassuring with completely reset your machines so you know you
| could reinstall or replace them easily.
|
| Granted just used in a home setting running smaller workloads for
| backups, private projects, git etc.
| wmf wrote:
| Where is networking configured? I assume the system has to have
| an IP address before containerd can fetch images.
| JustinGarrison wrote:
| Everything is API driven and static networking can be
| configured via kernel args
|
| https://www.talos.dev/latest/reference/configuration/v1alpha...
| burnte wrote:
| DHCP I would assume.
| wmf wrote:
| Yeah but where is the DHCP client? In the kernel?
| andrewrynhard wrote:
| In machined (PID1 of Talos).
| E39M5S62 wrote:
| It's disingenuous to say that /sbin/init (machined/main.go) is
| less than 400 lines of code. Sure, that file is. What about all
| of the in-tree modules that are being imported? A super lazy
| summing of Go lines in the master branch of the repo:
|
| $ find . -name *.go | xargs wc -l | tail -1
| 354085 total
|
| Heck, there are almost 100k lines under internal/app!
|
| $ find internal/app -name *.go | xargs wc -l | tail -1
| 96885 total
|
| I'm curious what argument you are making here with regards to the
| number of lines in a single file.
| JustinGarrison wrote:
| I know there are a lot more lines and I didn't count any of the
| imports from systemd either. 300 loc (machined) vs 3000 loc
| (systemd) was the closest comparison I could think of without
| crawling all imports and deps.
|
| Would be happy to update with a different comparison you think
| is more fair.
| raziel2p wrote:
| How about we just don't compare lines of code at all, as if
| it's a useful metric of anything?
| Karellen wrote:
| "Measuring programming progress by lines of code is like
| measuring aircraft building progress by weight."
|
| -- attributed to Bill Gates
| coldtea wrote:
| Kind of like TextAdept claiming that text editor is "just 2K
| lines of C/C++ and 4K lines of Lua", when it's in fact wrapping
| a well known third party text editor widget.
|
| At least here it makes more sense, since it's not like "our
| init system is just 200 lines" when those lines are wrapping a
| third party init system library.
|
| It's more like "our init system logic is just 200 lines, not
| including third party dependencies". That's legit, provided
| that those deps are stuff like parsers, some library for
| dealing with strings or running processes, and things like
| that.
| abound wrote:
| Big fan of Talos, have used it in some homelab + cloud clusters
| over the years, currently powers all my self-hosting. The
| `talosctl` command is great, and any time you need to do node-
| level debugging, there's always something like node-shell [1].
|
| [1] https://github.com/kvaps/kubectl-node-shell
| cperciva wrote:
| _The /sbin/init binary is hard linked to /sbin/dashboard,
| /sbin/poweroff, /sbin/shutdown, and /sbin/wrapperd. While this
| technically is 5 files, it's a single file hard linked 4 times to
| provide convenience commands._
|
| Err, that's definitely 1 file with 5 directory entries, not 5
| files.
| titanomachy wrote:
| Wonder why they would use hard links instead of symlinks.
|
| Edit: interesting, seems like there's a mild performance
| benefit.
|
| https://unix.stackexchange.com/a/20716
| 0xbadcafebee wrote:
| There's a lot of benefits actually, as symbolic links often
| need to be handled as a completely different type of file
| with different semantics which often leads to bugs. Hard
| links are always better when you know you are dealing with a
| single static host & filesystem. Symbolic links in such a
| case are only better for indicating quickly to the user which
| files are linked to which others.
| vundercind wrote:
| Reverse question: why use symlinks when you can get away with
| hard links?
| AtlasBarfed wrote:
| Main thing I've seen with hard links is that deletions
| delete the source file which about 90% of the time isn't
| what an end user wants
| whartung wrote:
| Only if its the last hard link. If nothing else, its a
| wee bit of insurance from deletion since no single link
| removal should remove the file.
|
| Anecdote, eons ago, we had a problem where the vendor
| needed to log in to the machine with the intent that they
| were going to upload some utilities, fix a problem, and
| then delete them.
|
| Before I let them in, I set up a script that constantly
| scanned the directory tree they were in, and hard linked
| everything so I could look at what they were using later.
| rcxdude wrote:
| that isn't generally how hardlinks work: you need to
| delete all instances of it to delete the file, as opposed
| to having one 'real' file like with symlinks.
| wizzwizz4 wrote:
| That's a problem with directory hardlinks: a recursive
| deletion will wipe the directory (in the same way that
| truncate will destroy a file regardless of how many
| hardlinks it has). But we don't usually allow directory
| hardlinks these days.
| rwiggins wrote:
| Super cool. I always enjoy reading about systems that challenge,
| well, "ossified" assumptions. An OS not providing a shell, for
| example? Madness! ... or is it genius, if the OS has a specific
| purpose...? It's thought-provoking, if nothing else.
|
| I'm a bit skeptical of parts. For instance, the "init" binary
| being less than 400 lines of golang - wow! And sure, main.go [1]
| is less than 400 lines and very readable. Then you squint at the
| list of imported packages, or look to the left at the directory
| list and realize main.go isn't nearly the entire init binary.
|
| That `talosctl list` invocation [2] didn't escape my notice
| either. Sure, the base OS may have only a handful of binaries -
| how many of those traditional utilities have been stuffed into
| the API server? Not that I disagree with the approach! I think
| every company eventually replaces direct shell access with a
| daemon like this. It's just that "binary footprint" can get a bit
| funny if you have a really sophisticated API server sitting
| somewhere.
|
| [1]:
| https://github.com/siderolabs/talos/blob/main/internal/app/m...
|
| [2]: https://www.talos.dev/v1.6/reference/cli/#talosctl-list
| sspiff wrote:
| Exactly this. I was thinking of making a similar comment but
| you made it far better than I could.
|
| Number of binaries is kind of a meaningless metric, especially
| for a system that historically follows the UNIX philosphy of
| each program doing one thing.
|
| Sure, a shell is complicated and a potential risk, and perhaps
| it's a good idea to exclude from the base system in this
| context.
|
| But I'd rather have ls, tr and wc on my system than some
| bespoke, all-encompassing API service that has been far less
| battle tested providing similar functionality.
|
| And like you rightly pointed out, these new binaries all
| contain their own list of dependencies which are pulled in at
| build time and need to be taken into scope as well.
|
| That's not to say Talos or its approach doesn't hold merit, but
| I think it's a little disengenious to simply point at the
| number of binaries.
| JustinGarrison wrote:
| I agree number of binaries is an arbitrary metric but also an
| indicator that things work differently with Talos. You have
| to use the declarative API for management which some people
| could see as a bad thing.
|
| I'd also like to point out that the system API is designed to
| be extendable and adaptable to different operating systems.
| We'd love for more vendors to create adapters/shims to get
| the benefits of API managed Linux
|
| https://github.com/cosi-project/community
| 0xbadcafebee wrote:
| 20 years ago, I used to make custom Linux distros for fun. Floppy
| distros, CDROM distros, RAM-resident distros, network-boot
| distros. In a few of them, I custom-made my own binary that was
| both the init system, and a few applications, stripped it down,
| and shipped just that as the distro (basically just a few files
| and my static binary).
|
| A lot of people downloaded them, and it was great fun - to start.
| Problem is when you want to do more things. You have to start
| finding workarounds to bolt-on additional tools, or maybe you
| just throw one or two extra tools in there by default. Over time
| you find more and more missing things or incompatibilities with
| other systems, which make it harder to cover more use cases. And
| finally you realize that "the tiniest system" is a lot more
| effort than it's worth, and what you really want is "a slim yet
| compatible system". The system you end up with is a lot fatter,
| but a lot less headache.
|
| (The security benefits of fewer files are overblown, too. If you
| audit and harden the system, it doesn't matter how many binaries
| you have, because the attack vectors they use will be mitigated)
| andrewrynhard wrote:
| In the case of Talos, Kubernetes can provide the flexibility
| you want from a more traditional Linux distribution.
| AtlasBarfed wrote:
| That sounds like Kool aid. Can you expound more on this?
| andrewrynhard wrote:
| With Kubernetes you can schedule workloads in a number of
| different ways. Let's say you insisted on having a shell
| and package manager. Run your favorite distro's container
| as a DaemonSet. With the proper mounts and permissions you
| can do a lot. In other words use Kubernetes to do the
| things need to do. Then what role does the OS really play?
| Well in the case of Talos it's only there to run
| Kubernetes.
| JustinGarrison wrote:
| We use "system extensions" to give you flexibility while
| keeping the base small.
|
| Want GPU drivers? Add the extension. Need Tailscale? Extension.
|
| https://www.talos.dev/v1.6/talos-guides/configuration/system...
| Dwedit wrote:
| MX Linux would be that "slim-yet-compatible" system. It can
| load itself completely into RAM, and has many distro-specific
| components such as Frugal Install and Remastering, yet also
| lets you install standard Debian packages.
| miki123211 wrote:
| I wonder how a really slimmed down distro like Alpine would
| compare here, particularly in terms of image size.
|
| It offers most of the standard Linux utilities we know and love,
| but most of them are actually just symlinks to Busybox, which is
| ~900K on my (ARM64) system. That's less than a hello world in Go,
| for a program that can replace most common Linux utilities in
| daily usage.
| JustinGarrison wrote:
| Busybox is only 1 binary symlinked hundreds of times
___________________________________________________________________
(page generated 2024-03-04 23:01 UTC)