[HN Gopher] BuildKit in depth: Docker's build engine explained
___________________________________________________________________
BuildKit in depth: Docker's build engine explained
Author : kylegalbraith
Score : 79 points
Date : 2024-02-06 17:31 UTC (5 hours ago)
(HTM) web link (depot.dev)
(TXT) w3m dump (depot.dev)
| inetknght wrote:
| Buildx also has a completely different build output and it's
| completely useless for anything. Use `--output=plain`.
|
| Also, buildx won't commit the intermediate layers during the
| build. So if something fails, you can't just grab the previous
| intermediate layer and do `docker run --entrypoint /bin/bash` on
| that layer to poke around.
|
| I've had cases where BuildKit will get stuck and `docker buildx
| prune -a` will leave things around. The way to fix it is with
| `systemctl restart containerd.io`. This seems to happen _a lot_
| when I interrupt the build (eg, I know it will fail and need to
| restart it with a fix), but it 's also happened on successful
| builds.
|
| All of that is on top of docker itself having extremely poor
| error messages.
|
| I really do not like docker or buildx.
| adamgordonbell wrote:
| If you build your dockerfile with earthly, we have a -i flag
| that will pop you into a shell on a failure, and we are using
| buildkit, so it's totally possible.
| 4oo4 wrote:
| Well said, these are the reasons I use podman/buildah now.
| EdwardDiego wrote:
| Was it a drop in replacement or did you need to give things a
| jiggle to get it going?
| Narushia wrote:
| I've had a good experience with Podman as a Docker
| alternative. The Podman CLI is designed to be API-
| compatible with Docker's CLI, so most things should just
| work. The only thing I've had to change with my Dockerfiles
| (or Containerfiles) is the `SHELL` instruction, since
| Podman by default builds OCI images, and that instruction
| is not supported by the OCI standard. But you can still
| specify the shell individually for the `RUN` instructions,
| or even just use `--format=docker` if you don't want OCI
| images.
| cpuguy83 wrote:
| "BUILDX_EXPERIMENTAL=1 docker buildx build <args> --invoke
| /bin/sh"
|
| There's actually a whole debugging client available.
| mdaniel wrote:
| and its "oh, you want multi-arch, do you?" friend. While
| prosecuting this <https://gitlab.com/gitlab-
| org/gitlab/-/issues/339567> I learned that they bundle binfmt
| into the buildkit docker image <https://github.com/moby/buildki
| t/blob/v0.12.5/Dockerfile#L11...> which itself is compiled
| against a static copy of qemu <https://github.com/tonistiigi/bi
| nfmt/blob/deploy/v6.2.0-26/D...> with various
| /proc/sys/fs/binfmt_misc wizardry <https://github.com/tonistiig
| i/binfmt/blob/deploy/v6.2.0-26/c...>
|
| We ended up switching to using multiarch/qemu-user-
| static:7.2.0-1 <https://hub.docker.com/layers/multiarch/qemu-
| user-static/7.2...> which also mutates binfmt_misc in buildx's
| context in order to exec the static copy of qemu in it but I
| find their shell script much more legible about what's going
| on: <https://github.com/multiarch/qemu-user-
| static/blob/v7.2.0-1/...> not to mention the fact that it more
| clearly represents the recent qemu version versus secreting it
| away inside the moby/buildkit:heheh-good-luck-friend
|
| They have an entire GH label for that:
| https://github.com/docker/buildx/labels/area%2Fqemu and my
| journey into the buildx sewers started from
| https://github.com/docker/buildx/issues/1170
| adamgordonbell wrote:
| I did not know about the CargoWharf frontend!
|
| Writing your own front end to BuildKit can be pretty simple. At
| my work, this is sort of a starter task for everyone who joins
| the team.
|
| My frontend was based on intercal and I don't recommend anyone
| use it but it was fun to play around with[1].
|
| [1]: https://github.com/adamgordonbell/compiling-
| containers/tree/...
| notso411 wrote:
| Really? Just busy work? Senior engineers incapable of thinking
| of good onboarding tasks?
| adamgordonbell wrote:
| Nope, i mean i can see how youd think that, but familarity
| with BuildKit is just part of the job.
|
| We are building on top of it, and sometimes adding to it.
| nicornk wrote:
| So how did you ,,further optimized [buildkit] to build Docker
| images up to 40x faster"
| maxmcd wrote:
| hopefully depot will reply, but from my perspective it is
| mostly laid out on their homepage. they are comparing against
| builds in other CI products that use network-backed disks,
| virtualized hardware, and don't keep a layer cache around.
| Depot provides fast hardware and disks and is good at making
| the layer cache available for subsequent builds.
|
| You could likely get very similar performance by provisioning a
| single host with good hardware and simply leverage the on-host
| cache.
___________________________________________________________________
(page generated 2024-02-06 23:00 UTC)