[HN Gopher] Nix is a better Docker image builder than Docker's i...
___________________________________________________________________
Nix is a better Docker image builder than Docker's image builder
Author : tosh
Score : 65 points
Date : 2024-03-15 19:56 UTC (3 hours ago)
(HTM) web link (xeiaso.net)
(TXT) w3m dump (xeiaso.net)
| madjam002 wrote:
| Does anyone here have any experience using
| https://github.com/pdtpartners/nix-snapshotter ?
|
| I build a lot of Docker images using Nix, and while yes it's
| generally more pleasant than using Dockerfiles, the 128 layer
| limit is really annoying and easy to hit when you start building
| images with Nix. The workaround of grouping store paths makes
| poor use of storage and bandwidth.
| mikepurvis wrote:
| I haven't tried it yet as I need to produce containers that can
| work on public cloud k8s, but it definitely looks like the way
| to go. All the existing methods for grouping store paths into
| layers are finnicky, brittle, and non-optimal.
| mikepurvis wrote:
| No discussion about Nix-built containers is complete without
| mentioning nix2container:
|
| https://github.com/nlewo/nix2container
|
| It is _truly magical_ for handling large, multi-layered
| containers. Instead of building the container archives themselves
| and storing them in the nix store, it builds a JSON manifest that
| is consumed by a lightly patched version of skopeo that streams
| the layers directly to either your local container engine or the
| registry.
|
| This means you never rebuild or reupload a container layer that
| is unchanged.
|
| Disclosure: I contributed a change in nix2container that allows
| cheaply pulling non-Nix layers into the build, just using the
| content hashes from their registry manifests.
| tetris11 wrote:
| Guix is also pretty good at this, only lacking up to date
| packages that one would want to build an image with
| djaouen wrote:
| This is my one gripe with Guix. Alas, we (apparently) can't
| have it all!
| operator-name wrote:
| This is great if you've already adopted Nix, and I'd love for
| nothing than more declarative package management solutions like
| Nix or Guix to take off.
|
| If you're already using Docker but want to gradually adopt Nix,
| there is an alternative approach outlined by this talk:
| https://youtu.be/l17oRkhgqHE. Instead of migrating both the
| configuration AND container building to Nix straight away, you
| can keep the Dockerfile to build the nix configuration.
|
| The biggest downside is that you don't take advantage of layers
| at all, but the upside is that you can gradually adapt your
| Dockerfiles, and reuse any Docker infrastructure or automation
| you already use.
| debuggerpk wrote:
| horrible, horrible font on the website!
| mmh0000 wrote:
| What a weird thing to complain about.
|
| Just override the font if you dislike it that much. That's the
| great thing about the interwebs, it's all just text, you can
| format it however you want.
|
| Assuming you're using something based off chrome:
| https://chromewebstore.google.com/detail/font-changer/obgkji...
|
| Or Firefox: https://addons.mozilla.org/en-
| US/firefox/addon/refont/
| djaouen wrote:
| I just wanted to chime in here and say that Guix also has a nice
| and easy-to-use Docker option with "guix pack -f docker" [1].
| Guix also has the advantage of using an already-used language
| (Guile/Scheme) rather than its own bespoke one. :)
|
| [1] https://guix.gnu.org/manual/en/html_node/Invoking-guix-
| pack....
| Nonoyesnoyes wrote:
| The article lost me somewhere. Long intro and than just assuming
| too much.
|
| That's just possible due to nix being more granular? Is that
| right?
|
| Can I really build nix from 5 years ago? No src gone? No cache
| server gone? Nothing?
|
| I mean yeah Ubuntu as a base is shitty.
| xena wrote:
| The text is written to be spoken, it works better when I
| present it. I'll have the video edited next week, that may flow
| better for you.
| j-bos wrote:
| I like the article but had a hard time following the specifics of
| the configs and commamds. Feels like it's more meant for people
| already familiar with nix, or sufficiently interested to study up
| while reading
| febed wrote:
| I didn't fully grok how this works - what is the base image for
| the generated image? Also wouldn't the image size be large if the
| glibc is copied over again
| aidenn0 wrote:
| > what is the base image for the generated image?
|
| Default is none (i.e. like "FROM scratch" in a Dockerfile); you
| can specify a baseImage if needed, but I haven't had to yet. It
| works by copying parts of the nix store into the image as
| needed, but see also below.
|
| > wouldn't the image size be large if the glibc is copied over
| again
|
| The original Nix docker-tools buildImage did suffer from poor
| reuse of common dependencies. Docker already has a way to reuse
| parts of images (e.g. if you build 7 images where the first N
| lines of a Dockerfile are the same, the 7 images will use a
| shared store for the results of running the first N lines).
| There are several backends for Docker storage that accomplish
| this in various ways (e.g. FS overlays, tricks with ZFS/btrfs
| snapshots).
|
| Nix docker-tools now has a "buildLayeredImage" that uses this
| ability of Docker to share much of the storage for the
| dependencies, so if you build several images that all rely on
| glibc, you only pay the cost of storing glibc in docker once.
| febed wrote:
| Thanks, that made the article clearer for me
| xlii wrote:
| I spent last 2-3 days trying to get Docker images built on Darwin
| and I feel that this article is a universe making fun of me.
|
| Nix is absolutely the best tool for what I want to achieve but it
| has those dark forsaken corners that just suck your soul out dry.
|
| I love it but sometimes it feels like being a Morty on Rick's
| adventure to the compilerland.
| whazor wrote:
| This blog post is missing the reasoning on why shared docker
| layers are useful. It is because of caching. The more images are
| sharing the same layers the better, as it allows you to cache
| more stuff. Better caching means faster startup of containers.
|
| Why is docker bad at this? In order to enjoy the caching benefit,
| each time you build a docker image you want it to output as much
| existing layers as possible. So running apt-get install python3
| today should result in the exact same layer as yesterday, if
| there are no new updates. But this requires the all the files to
| be exactly the same, including the metadata like creation time.
| As docker layers are cached by hashing the files.
|
| Now, Nix already does storing dependencies by hash. So the layers
| will always be the same with the same version and same
| configuration.
___________________________________________________________________
(page generated 2024-03-15 23:02 UTC)