[HN Gopher] Dagger: Define software delivery workflows and dev e...
___________________________________________________________________
Dagger: Define software delivery workflows and dev environments
Author : ahamez
Score : 70 points
Date : 2025-12-09 08:32 UTC (5 days ago)
(HTM) web link (dagger.io)
(TXT) w3m dump (dagger.io)
| oulipo2 wrote:
| I was interested in the beginning for CI/CD, but then they tried
| to take a kind of "AI-oriented" view in order to ride the AI
| wave, and the value prop of their tool was completely muddied
| up...
| jiehong wrote:
| Same, the marketing was even worse a few months ago.
| junon wrote:
| This seemed cool as it looked like a new CI/CD tool or IaC
| system.
|
| Then... it wasn't. The more I read the less I ever want to see
| this again. The LLM train has got to end at some point.
| tajd wrote:
| This looks interesting but I'm trying to understand it in more
| layman's terms. Is it more about providing abstractions for llms
| to work within to do things?
| jiehong wrote:
| It's actually a CI/CD as code tool, where some pieces can be
| LLM agents.
|
| But, the marketing heavily focuses on LLM stuff to the point of
| making everyone confused.
| lowmagnet wrote:
| It does a pretty good job of caching and that does help speed
| up builds. I also run all of my end to end tests from it
| because I can coordinate secrets and clusters of containers
| through it.
| mayhemducks wrote:
| I've never tried it. My first impression based purely on
| reading the homepage is it adds complexity to something I can
| already do with a Dockerfile and bash. What can it do that I
| can't already do more simply?
| esafak wrote:
| Instead of YAML workflows you write code, and there is an
| analog of the Github Actions Marketplace (the Daggerverse).
| Havoc wrote:
| Anybody using it extensively? It doesn't seem to have made the
| splash I expected it to at launch
| pxc wrote:
| I used the old CUE-based version when it came out, and was
| really excited about it. I liked it, and enjoyed working with
| CUE, but the API was clunky and incomplete.
|
| Then they completely abandoned not just the CUE frontend, but
| CUE altogether (while strenuously denying that they were doing
| so) for a GraphQL-based rewrite that focused on letting people
| use popular general-purpose languages to construct their
| workflows. The initial rollout of this was not feature complete
| and only supported imperative languages (Python and TypeScript,
| IIRC), which I didn't like.
|
| Instead of porting everything over to all their new interfaces,
| I hopped off the train and rewrote all of our portable pipeline
| scripts in Nix, via Devenv. At the time, I'd never used Devenv
| before, but getting the work done that time still took maybe a
| tenth of the time or less. More than anything else, this was
| due to not having to fuck around with the additional overhead
| Docker entails (fussing with mount points, passing files from
| one stage to another, rebuilding images, setting up VMs... all
| of it). I got the reproducibility without the extra bullshit,
| and got to work with interfaces that have proven much more
| stable.
|
| I still think there's a place for something like Dagger,
| focused just on CI, perhaps even still using Docker as a
| deployment/distribution strategy. But I no longer trust Dagger
| to execute on that. I think a proper external DSL (probably
| special-purposw but still Turing-complete, e.g., Nickel) is the
| right fit for this domain, and that it should support multiple
| means of achieving repeatability rather than just Docker (e.g.,
| Nix on bare metal and Podman, to start). An option to work on
| bare metal via reproducible environment management tools like
| Nix, Guix, or Spack is a valuable alternative to burdensome
| approaches like containers.
|
| I haven't looked at Dagger in several months, but the other big
| piece that is missing for portable CI workflows is a library
| that abstracts over popular CI platforms so you can easily
| configure pull/merge request pipelines without worrying about
| the implementation details like what environment variables each
| platform exposes to indicate source and target branch.
|
| Idk anything about all the AI horseshit; I was off the Dagger
| bandwagon before they took that turn. I don't know if it's
| serious or a nominal play to court investors. But that kind of
| pivot is another reason not to build core infra on top of the
| work of startups imo. If the product is 70% of what you want,
| you have no way of knowing whether filling that 30% gap is
| something the maintainers will suddenly pivot away from, even
| if their current direction looks aligned with yours.
|
| I'd recommend considering tools in this space only if (a)
| they're already close to 100% of what you need and (b) they're
| open-source. Maybe you can relax (a) if it's really easy to
| extend the codebase (I find this to be true for Devenv's Nix
| modules, for example.)
| digdugdirk wrote:
| Do you have any examples of your Devenv workflow you can
| share? I took a look at Dagger and really like the concept,
| but I'm trying to figure out the limitations/why there's so
| much negativity in this thread.
|
| I currently manage my development environments via NixOS and
| Devenv, so if I could just keep that and achieve the same
| functionality, that sounds good to me.
| jiehong wrote:
| Anybody used it?
|
| Without the LLM bits, this is basically like Bazel or buck2,
| right?
| esafak wrote:
| No, it's code based CI/CD not a build system.
| ndrpnt wrote:
| I'd say it's somewhere in between. Sure it's marketed in the
| CI space, but to me the selling point of Dagger is not so
| much "write your GitHub workflows/GitLab CI in JavaScript"
| but "local exec, sandboxing, determinism, and fine-grained
| (remote) caching for mere mortals". So comparing it to
| Bazel/Buck2 is reasonable.
| esafak wrote:
| I wouldn't because build systems like Bazel are declarative
| and dagger is imperative. I accidentally created a build
| system in dagger and saw the difference; the code based way
| was highly branched, and thus unwieldy. I think you would
| want to call bazel _from_ dagger to handle the build step.
| mxey wrote:
| Dagger uses BuildKit. all your "imperative" code does it
| assemble a graph that's gonna be executed by BuildKit in
| dependency order.
| esafak wrote:
| Yes, but that does not help you build declaratively in
| dagger, does it? I would like to see an example of how to
| do so, if you have one.
| Too wrote:
| But that dependency order is usually just one big blob of
| "COPY src/ . + RUN make", within that block you have none
| of the benefits. Bazel/Buck has much finer awareness down
| to every individual file.
|
| Out of curiosity, would it be feasible to take a big
| cmake project and generate thousands of compile rules
| into dagger and use it as a substitute for make with
| sandboxing? I've never seen builkit used with such many
| nodes, how would it fare?
| shykes wrote:
| Dagger is a declarative DAG engine. So, yes, you can do
| that.
| mxey wrote:
| There is an overhead per container launched so it would
| probably not be worth it.
| justincormack wrote:
| Its not as fine grained as bazel/buck. That doesnt
| necessarily matter.
| elryry wrote:
| Yeah, it's more like a .NET Aspire
| lowmagnet wrote:
| I use it to do builds in our monorepo. We got onboard before
| the LLM trash features. The base design is ok but there's
| things I'd do different today if I knew the build stuff would
| fade away for the LLM push.
| dilyevsky wrote:
| Works fine for us for glueing a bunch of CI steps together
| which would've been a pile of bash otherwise. Works well with
| depot.dev caches. We don't use it for anything AI either.
| kylegalbraith wrote:
| Founder of Depot here! Glad to hear it's working for you all.
| Always happy to help expand things or make things better if
| you ever have ideas.
| usrme wrote:
| Dagger was something I looked into two or so years ago before
| they got consumed by the LLM and AI agent hype, and while the
| promise of being able to run the exact CI workflows locally
| seemed excellent, it seemed that there's basically no way be a
| Dagger user without buying into their Dagger Cloud product.
|
| I ended up opting for CUE and GitHub Actions, and I'm glad I did
| as it made everything much, much simpler.
| Xiol wrote:
| Do you have any more details on using Cue with GHA? I've also
| looked at Dagger and been quite disappointed with it (and their
| terrible documentation).
| usrme wrote:
| When I got started it was much more difficult as you had to
| do a lot of manual work to get things started, and you really
| had to believe the promises that CUE offered (which I
| did...), but nowadays they've made so many steps in the right
| direction that getting something going is far quicker!
|
| Here are a few links to whet your appetite:
|
| - https://cue.dev/docs/getting-started-with-github-actions-
| cue...
|
| - https://cue.dev/docs/drying-up-github-actions-workflows/
|
| - https://cue.dev/docs/spotting-errors-earlier-github-
| actions-...
|
| Definitely read through the CUE documentation
| (https://cuelang.org/docs/), watch their YouTube videos
| (https://www.youtube.com/@cuelang/videos), and join the
| community Slack channel (https://cuelang.org/community/).
| I've gotten a lot of help in the Slack from both enthusiastic
| community members and from the developers themselves whenever
| I've gotten stuck.
| 9dev wrote:
| Maybe it's just me, but these sample workflows don't look
| less complicated, just another kind of complex? If you're
| already heavily using CUE in your project this lateral
| complexity shift might make sense, but I don't see why I
| would start using it...
| diarrhea wrote:
| > just another kind of complex?
|
| To some extent yes. If all you have is 2 GitHub Actions
| YAML files you are not going to reap massive benefits.
|
| I'm a big fan of CUE myself. The benefits compound as you
| need to output more and more artifacts (= YAML config).
| Think of several k8s manifests, several GitHub Actions
| files, e.g. for building across several combinations of
| OSes, settings, etc.
|
| CUE strikes a really nice balance between being primarily
| data description and not a Turing-complete language (e.g.
| cdk8s can get arbitrarily complex and abstract), reducing
| boilerplate (having you spell out the common bits _once_
| only, and each non-commit bit _once_ only) and being
| type-safe (validation at build /export time, with native
| import of Go types, JSON schema and more).
|
| They recently added an LSP which helps close the gap to
| other ecosystems. For example, cdk8s being TS means it
| naturally has fantastic IDE support, which CUE has been
| lacking in. CUE error messages can also be very verbose
| and unhelpful.
|
| At work, we generate a couple thousand lines of k8s YAML
| from ~0.1x of that in CUE. The CUE is commented
| liberally, and validation imported from native k8s types,
| and sprinkled in where needed otherwise (e.g. we know for
| our application the FOO setting needs to be between 5 and
| 10). The generated YAML is clean, without any indentation
| and quoting worries. We also generate YAML-in-YAML, i.e.
| our application takes YAML config, which itself is in an
| outer k8s YAML ConfigMap. YAML-in-YAML is normally an
| enormous pain and easy to get wrong. In CUE it's just
| `yaml.Marshal`.
|
| You get a lot of benefit for a comparatively simple
| mental model: all your CUE files form just one large
| document, and for export to YAML it's merged. Any
| conflicting values and any missing values fail the
| export. That's it. The mental model of e.g. cdk8s is
| massively more complex and has unbounded potential for
| abstraction footguns (being TypeScript). Not to mention
| CUE is Go and shipped as a single binary; the CUE v0.15.0
| you use today will still compile and work 10 years from
| now.
|
| You can start very simple, with CUE looking not unlike
| JSON, and add CUE-specific bits from there. You can
| always rip out the CUE and just keep the generated YAML,
| or replace CUE with e.g. cdk8s. It's not a one-way door.
|
| The cherry on top are CUE scripts/tasks. In our case we
| use a CUE script to split the one-large-document (10s of
| thousands of lines) into separate files, according to
| some criteria. This is all defined in CUE as well,
| meaning I can write ~40 lines of CUE (this has a bit of a
| learning curve) instead of ~200 lines of cursed, buggy
| bash.
| Kinrany wrote:
| Same: the promise of defining CI/CD in code is good, but the
| implementation didn't make sense to me even before the LLM
| stuff
| tom1337 wrote:
| Same - we began the migration to Dagger but then switched to
| just Docker-In-Docker and custom scripts which run vendor-
| agnostic
| digdugdirk wrote:
| Can you explain/link to why you can't really use this without
| their cloud product? I'm not seeing anything at a glance, and
| this looks useful for a project of mine, but I don't want to be
| trapped by limitations that I only find out about after putting
| in weeks of work
| themgt wrote:
| Overall I like Dagger conceptually, but I wish they'd start
| focusing more on API stability and documentation (tbf it's
| not v1.0). v0.19 broke our Dockerfile builds and I don't feel
| like figuring out the new syntax atm. Having to commit dev
| time to the upgrade treadmill to keep CI/CD working was not
| the dream.
|
| re: the cloud specifically see these GitHub issues:
|
| https://github.com/dagger/dagger/issues/6486
|
| https://github.com/dagger/dagger/issues/8004
|
| Basically if you want consistently fast cached builds it's a
| PITA and/or not possible without the cloud product, depending
| on how you set things up. We do run it self-hosted though,
| YMMV.
| pxc wrote:
| One thing that I liked about switching from a Docker-based
| solution like Dagger to Nix is that it relaxed the
| infrastructure requirements to getting good caching
| properties.
|
| We used Dagger, and later Nix, mostly to implement various
| kinds of security scans on our codebases using a mix of
| open-source tools and clients for proprietary ones that my
| employer purchases. We've been using Nix for years now, and
| still haven't set up any of our own binary cache. But we
| still have mostly-cached builds thanks to the public NixOS
| binary cache, and we hit that relatively sparingly because
| we run those jobs on bare metal in self-hosted CI runners.
| Each scan job typically finishes in less than 15 seconds
| once the cache is warm, and takes up to 3 minutes when the
| local cache is cold (in case we build a custom dependency).
|
| Some time in the next quarter or two I'll finish our
| containerization effort for this so that all the jobs on a
| runner will share a /nix/store and Nix daemon socket bind-
| mounted from the host, so we can have relatively safe
| "multi-tenant" runners where all jobs run under different
| users in rootless Podman containers while still sharing a
| global cache for all Nix-provided dependencies. Then we get
| a bit more isolation and free cleanup for all our jobs but
| we can still keep our pipelines running fast.
|
| We only have a few thousand codebases, so a few big CI
| boxes should be fine, but if we ever want to autoscale
| down, it should be possible to convert such EC2 boxes into
| Kubernetes nodes, which would be a fun learning project for
| me. Maybe we could get wider sharing that way and stand up
| fewer runner VMs.
|
| Somewhere on my backlog is experimenting with Cachix, so we
| should get per-derivation caching as well, which is finer-
| grained than Docker's layers.
| shykes wrote:
| Hi, I'm the founder of Dagger. It's not true that you can't
| use Dagger without our cloud offering. At the moment our only
| commercial product is observability for your Dagger
| pipelines. It's based on standard otel telemetry emitted by
| our open source engine. It's completely optional.
|
| If you have questions about Dagger, I encourage you to join
| our Discord server, we will be happy to answer them!
| flanked-evergl wrote:
| What I don't get is why would someone code in the terrible
| GitHub actions dsl which only runs on GitHub actions and
| nowhere else when there are so many other options that run
| perfectly fine if you just run it from GitHub actions.
| esafak wrote:
| dagger was originally CUE-based, but there was not enough
| demand so it was dropped. https://dagger.io/blog/ending-cue-
| support
| pxc wrote:
| > If you've been active in the Dagger community, this news
| will come as no surprise. Since we released multi-language
| support, we have seen a steep decline in usage of our
| original CUE configuration syntax, and have made it clear
| that feature parity with newer SDKs would not be a priority.
|
| That is, of course, a self-fulfilling prophecy (or, perhaps,
| a self-inflicted wound). As soon as Dagger's "multi-language
| support" came out (actually a bit before), the CUE SDK was
| rendered abandonware. Development only happened on the new
| backend, and CUE support was never ported over to the new
| one.
| shykes wrote:
| Dagger founder here. We moved away from CUE because the
| number one complaint from our early users was having to
| learn CUE. The number two complaint was bugs in the
| language that we diligently escalated upstream, but would
| never get fixed, including crippling memory leaks.
|
| We shipped multi-language support because we had no choice.
| It was a major engineering effort that we hadn't originally
| planned for, but it was painfully obvious that remaining a
| CUE-only platform was suicide.
| sontek wrote:
| Do you have examples of your CUE and Github Actions setup?
| sontek wrote:
| I see someone else asked below:
|
| https://news.ycombinator.com/item?id=46262846
| LeBit wrote:
| Wow, the comments are generally negative. That's sad. I was about
| to create a POC to present to my team.
|
| What else could be used to abstract away your CICD from the
| launcher (Jenkins, Argo Workflows, GitHub Actions, etc.)?
| shykes wrote:
| Hi, I'm the founder of Dagger. I can't speak to the negativity,
| but if you're looking for a way to make your CI more portable,
| I recommend joining our Discord and asking our community
| directly about the pros and cons of using Dagger. Even if you
| don't end up using it, there are a lot of people there who are
| passionate about CI and can recommend other alternatives, in a
| more constructive and pragmatic way than you are getting here.
| bflesch wrote:
| I might be getting old but the videos are too fast for me to
| understand. Why can't they just put the full command text and the
| output of it instead of a video.
| leetrout wrote:
| Curious if anyone in the thread has / is using windmill?
|
| They don't seem to have jumped for AI hype (yet?)...
|
| https://www.windmill.dev/
| someguy101010 wrote:
| have used it, and i do like it, but the licensing situation is
| not great. It open source but its not free software by any
| means.
| oulipo2 wrote:
| I've tried it, but there's too much "sorry not in the open-
| source edition, please buy the entreprise edition" stuff all
| around, which makes it quite unusable
| michaelbuckbee wrote:
| A lot of the comments here feel like they're disappointed that
| this is a "Docker with unnecessary LLM crap thrown in" when I
| think what they're really going for is more "LLM workflows with a
| higher degree of observability and sanity".
|
| I think a more interesting point of comparison is the Claude Code
| Github Action, Co-Pilot code reviews, etc.
| oulipo2 wrote:
| I think it started as some kind of CI/CD tools, then they jumped
| on the AI hype and they started to use it to make it possible to
| run agents in containers easily... perhaps to do automated
| actions on CI/CD pipelines that use agents (eg try to solve some
| minor bugs automatically when you push on a branch, etc)
|
| Although I'm not sure if that's so much a value-added? It's not
| so hard to just create a container and launch an agent in it.
|
| The whole interesting thing was to use actual programming
| languages for Docker build, which I think was what they initially
| tried to do, but now it's a bit incomprehensible... I guess
| conceptually Dagger relates to Dockerfile a bit like Pulumi
| related to Terraform?
| stephen wrote:
| I thought Dagger had/has a lot of potential to be "AWS-CDK for CI
| pipelines".
|
| I.e. declaratively setup a web of CI / deployment tasks, based on
| docker, with a code-first DSL, instead of the morass of copy-
| pasted (and yes orbs) CircleCI yaml files we have strewn about
| our internals repos.
|
| But their DSL for defining your pipelines is ... golang? Like who
| would pick golang as "a friendly language for setting up
| configs".
|
| The underlying tech is technically language-agnostic, just as
| aws-cdk's is (you can share cdk constructs across
| TypeScript/Python), but it's rooted in golang as the
| originating/first-class language, so imo will never hit aws-cdk
| levels of ergonomics.
|
| That technical nit aside, I love the idea; ran a few examples of
| it a year or so ago and was really impressed with the speed; just
| couldn't wrap my around "how can I make this look like cdk".
| esafak wrote:
| They have SDKs in many languages, not just Go. I use the python
| one. And they use code, not a DSL.
| sgammon wrote:
| Dagger is already a very popular thing -- a DI framework
| isuckatcoding wrote:
| Yeah I initially thought of this coming from android
| nozzlegear wrote:
| It's also a type of blade, been around for hundreds of years!
|
| /s I've never heard of Dagger the DI framework but I have heard
| of this Dagger. Names will overlap sometimes and it's not a big
| deal.
| Too wrote:
| If I understand correctly, this is essentially a more composable
| way to write Dockerfiles? That alone is a very welcome
| improvement. They would do themselves a big favor if they were
| more clear on that in their marketing, instead of boasting around
| the bush with all kinds of other terminology and claims of
| redefining foundations.
|
| If I already have a Dockerfile that doesn't need composition, how
| does this help me vs being a small cosmetic improvement over
| "docker build" command line?
| vivzkestrel wrote:
| why do i need this when we got docker? anyone mind explaining?
| moltar wrote:
| I loved the original promise of Dagger and it's still 90% great.
|
| But one flaw (IMO) that it can't export artifacts and import into
| other steps without breaking the cache.
|
| Eg if you provide monorepo as input, and then on some step narrow
| your build to one specific dir, then even when files change
| outside of that dir then caching still is invalidated.
|
| Which makes it extremely verbose and maintenance nightmare to
| keep multiple narrow inputs and keep all those paths up to date.
| shykes wrote:
| You can filter input directories before they are loaded, to
| avoid this. There's no reason you shouldn't be able to get
| precise cache invalidation in a large monorepo. If you want, DM
| me on the Dagger Discord and I'll help you out!
| moltar wrote:
| And for those who may not know the founder of Dagger is the same
| guy who founded Docker - Solomon Hykes.
| techn00 wrote:
| The title says "dev environments". Can dagger be used for creatin
| dev environments, like devcontainers or
| https://www.jetify.com/docs/devbox/quickstart ?
___________________________________________________________________
(page generated 2025-12-14 20:02 UTC)