[HN Gopher] The CD Pipeline Manifesto
___________________________________________________________________
The CD Pipeline Manifesto
Author : bullcitydev
Score : 79 points
Date : 2024-12-20 16:19 UTC (3 days ago)
(HTM) web link (manifesto.getglu.dev)
(TXT) w3m dump (manifesto.getglu.dev)
| mdaniel wrote:
| I'm guessing this is relevant:
| https://news.ycombinator.com/item?id=42267316 _Show HN: Glu -
| Deployment pipeline framework as code_ - Nov, 2024 - 2 comments
|
| And, tellingly, it seems they still haven't provided a "why not
| ${other tool}" anywhere that I can readily spot
| esafak wrote:
| You beat me to it: why not Dagger?
| jiggawatts wrote:
| I had a look at the example glu deployment pipeline and I'm
| decidedly unimpressed.
|
| Admittedly most of my criticism is related to the choice of Go as
| an implementation language: more than 80% of the code volume is
| error handling boilerplate!
|
| Before the lovers of Go start making the usual arguments consider
| that in a high-level pipeline script _every_ step is expected to
| fail in novel and interesting ways! This isn't "normal code"
| where fallible external I /O interactions are few and far
| between, so error handling overhead is amortised over many lines
| of logic! Instead the code becomes _all_ error handling with
| logic... in there... somewhere. Good luck even spotting it.
|
| Second, I don't see the benefit of glu (specifically) over
| established IaC systems such as Pulumi -- which is polyglot and
| allows the use of languages that aren't mostly repetitive error
| handling ceremony.
|
| This seems like an internally developed tool that suits the
| purposes of a single org "thrown over the fence" in the hope that
| the open source community will contribute to their private tool.
| cdaringe wrote:
| ocamlci is an OCaml Platform offered canned recipe, a la glu,
| and they really cut that boilerplate down. I almost never use,
| but i had the same vibes as you did and it made me think of an
| impl i thought glu may have something to learng from.
|
| https://github.com/ocurrent/ocaml-ci?tab=readme-ov-file
| paweladamczuk wrote:
| > Without types, it is difficult to compose pipelines together.
|
| I would gladly hear this argument expanded. It's really not
| obvious to me that that's the case.
| wesselbindt wrote:
| Suppose I give you two functions f, and g. Can you run f(g())
| without breaking things? The honest answer is you don't know
| until you read the functions, which is a slow and difficult
| thing to do.
|
| Suppose I give you functions f and g of respective types int ->
| str and Nothing -> str. Can you compose them? No, and you see
| this immediately from the types. Types make reasoning about
| composability a lot easier.
|
| Of course, it's not a panacea, and it's less helpful the more
| side effects a function has. Can we compose pure int->int
| functions? Of course! Can we compose two of them where the
| second expects some image to exist in some docker registry?
| You'll need to read the first to be able to tell.
|
| Given the highly side effectful nature of pipelines, I'd think
| the applicability of types would be limited. But maybe that's
| just a lack of imagination on my part.
|
| Certainly information like "this pipeline expects these
| variables" and "this pipeline sets these variables" are
| susceptible to a typed approach, and it would make things
| easier. By how much, I don't know.
| joshAg wrote:
| What are side-effects but undocumented arguments and returns?
|
| Firstly, you want to ensure your functions are pure with
| respect to input. That is to say, they might reference a
| configuration or context object that is passed to them as an
| argument, but they'll never reference some global
| object/variable.
|
| So then the docker image inside some docker registry? Both
| the image and the registry are values in the config/context
| argument at the least. Maybe they're their own separate
| arguments depending on whether you prefer a single big object
| argument or a bunch of smaller more primitive arguments.
|
| So then the pure function that expects the docker image to
| exist in some registry is no longer Int ->
| Int
|
| It's now String -> String -> Int -> Int
|
| because it needs a registry and an image. Maybe it's
| String -> String -> String -> String -> Int -> Int
|
| because there's a username and password required to access
| the registry. Icky, but if we make a few types like
| data Registry { user :: String, password ::
| String, url :: String }
|
| that becomes Registry -> String -> Int
|
| But we could make it better by doing something like
| data Foo { reg:: Registry, image :: String
| }
|
| and now the function can be Foo -> Int -> Int
|
| This doesn't fix the image not actually existing in the
| registry, but at least now we know that the two functions
| aren't composable, and when it fails because the image
| doesn't exist we can hopefully trace through to see who the
| caller is that's giving it incorrect data.
|
| PS: sorry if i got the haskell typing wrong. I don't know
| haskell so that's the result of what i could cobble together
| from googling about haskell type syntax
| sourishkrout wrote:
| I commend anyone who's taking a hard look at our current CI/CD
| practices. Good work! Succinctly stating the problems is easier
| said than done.
|
| I believe https://dagger.io checks all these manifesto boxes and
| more. At least that's where I'm focusing my attention.
| GrumpyCat42 wrote:
| My company is currently adopting this and I don't see the
| appeal yet - likely from a lack of knowing much about it.
|
| I added it to a side project just to get familiar and it added
| quite a few sdk files and folders to my project, and lots of
| decorators. It also required Docker and yadda yadda yadda.
|
| I just could not justify using it compared to just running some
| regular Typescript file with Bun (or, in a different project,
| `go run cmd/ci/main.go`)
| sourishkrout wrote:
| Simple problems require simple solutions. If Makefile, NPM
| run, or Rake gets the job done, stick with it. That's great.
|
| The problem that Dagger and similar efforts solve is for
| pipelines at scale, whether that's a sea of microservices
| maintained by an armada of teams (which never work the same)
| or your massive pipelines that should be decomposed into a
| more atomic pipeline that fed into one.
|
| I believe the latter is a big productivity hurdle even
| without org-scale. My release pipeline runs for 25min with a
| team <5 because it's multi-staged (testing pyramid) and
| includes end-to-end tests. I love my pipeline because it
| makes me feel safe releasing my software upon success.
|
| However, god forbid it fails with a non-obvious error of 20
| minutes into exec. Lack of portability (Hi GHA vendor lock-
| in) and reproducibility (local-run = impossible) will make
| this feedback-loop hell.
|
| Now, wiseguys might tell me that pipelines shouldn't run for
| multiple minutes and only unit tests blah. That's divorced
| from reality. This sentiment won't not solve automation
| problems and won't optimize for velocity. It merely throws it
| over the fence to somebody else. If you have "the luxury" a
| QE/QA/Release team which I feel bad for.
|
| So the question to ask yourself is: how do I know I have
| outgrown `go run cmd/ci/main.go`?
| moltar wrote:
| Best pipeline I've had the pleasure to design is AWS CodePipeline
| via AWS CDK. Ticks all boxes. Uses pure TypeScript code.
| clvx wrote:
| I have a hot take on this. I don't care how you build and deploy
| as long as it's reproducible and the whole process can be tracked
| in their metadata. I'd rather have a process validating CI/CD
| stages and artifacts metadata in a central db than unifying
| pipelines that won't get standardized due communication
| complexity. This way I can have a conversation on visibility
| rather than code edge cases.
| ttyprintk wrote:
| This is important for SBOM (software bill-of-materials) which
| will soon be mandatory in regulated domains.
| eddsolves wrote:
| What will SBOM require in regulated domains?
| jas39 wrote:
| I'm thinking before we build a CI/CI pipeline, make sure there is
| a Makefile. Why have they gone out of style?
| ericyd wrote:
| Not sure but my guess is because they aren't a good fit for
| many languages. If you need a task runner then often languages
| will have a built in option or there are better alternatives
| than Make. If you need a build system then Make isn't a good
| fit for a lot of modern languages.
| SOLAR_FIELDS wrote:
| I've been using earthly a lot lately and its general value
| prop is simple: it turns out that if Buildkit is your primary
| build tool that Make targets can almost always be represented
| as OCI image layers. The killer feature IMO is that its
| syntax is familiar enough to end users of both Make and
| Dockerfiles that engineers tend to be willing to onboard to
| it. A lot of these other solutions that use proprietary DSL's
| struggle to cover every use case, and the implementations in
| turing complete typical language SDK approach often forces
| you into analysis paralysis if there is no existing pattern.
| maccard wrote:
| If I want to build and test a golang app and push it to a
| container repository, what value does a makefile provide over
| go build && docker push?
|
| All the tools do their own dependency tracking already
| (unfortunately).
| drowsspa wrote:
| Honestly I like it just to keep it as a command runner with
| the needed flags. Then in the unholy YAML there's just make
| build, make test, etc
| pdimitar wrote:
| The `just` tool is a better and much easier to understand
| command runner than `make`, however. Much less feature
| surface, too, which eliminates nasty surprises coming from
| the unnecessary complexity of `make`.
| drowsspa wrote:
| There are a pletora of tools better than Make. But it's a
| standard tool, everyone is familiar with it, you probably
| don't even have to install it.
| pdimitar wrote:
| `just` is 90% similar to `make` in syntax, only it has
| 100x less foot guns. :)
|
| Also I'll never understand the appeal of "not having to
| install a tool". We're not in the 1980s anymore when that
| was an actual chore. You run a command, the tool is there
| (including in CI/CD), boom, done. What am I missing here?
|
| The advantages you list are flimsy at best.
| maccard wrote:
| Bootstrapping can be painful in some languages or
| frameworks. Not everyone is running containerised builds
| where there are ephemeral environments that you just
| install a tool (and pay the 30+ second cost per build to
| run apt-get update). There's certainly value in having a
| front door entry point. But I think it should be a shell
| script, not a makefile.
| pdimitar wrote:
| Yes to your last. Either sh/bash script or a precompiled
| Golang program. If installing a tool is really such a
| problem then having a precompiled strongly typed program
| doing various tasks should be a no-brainer.
|
| I started openly hating `make` because I re-learned its
| specifics and quirks several times over the course of
| 10-ish years and then figured that I want to learn stuff
| with a staying power in my brain. I don't use `make`
| every work day so eventually any quirks disappear --
| that's how our brains work.
|
| So that's why I learned most of `just` and it hasn't
| betrayed me so far, not once. Though I did write a few
| Elixir and Golang programs for running various tasks in
| production environment, too.
| maccard wrote:
| At that point you end up with a makefile that has a 1:1
| mapping with targets in my experience. At a previous job,
| we had an enormous makefile, most of which was defining
| phone targets and translating make arguments into maven
| arguments. All the actual targets were calling maven. Make
| provided no value at all in that, other than requiring you
| to know Make and maven to modify anything in the build.
|
| Personally I'd rather a shell script for a command runner
| in most cases
| drowsspa wrote:
| Yeah, mine often just defers back to some shell scripts.
| But it's useful to enumerate them in the Makefile.
| tom_ wrote:
| If you have a telltale prefix for any internal phony
| targets (I use "_"), then you can have the Makefile list
| all the interesting targets itself. Cat the Makefile,
| print every line matching "^\\.PHONY:[[:space:]]*[^_]",
| then strip out the prefix. Leave any suffix, as you can
| put a trailing comment in, e.g.,
| .PHONY:build_windows # build for Windows. Supply DEBUG=1
| for a debug build
|
| I find this super useful. Even if you remember exactly
| what the target was called, it still gives you a nice
| list of words you can double click on to pull into the
| command line.
| jas39 wrote:
| Maybe none, but at some point you may want to do other things
| at buildtime, such as generating an sqlite db or generate
| code stubs for protobuf. Having a universal, and highly
| refined, tool like Make will help developers without domain
| knowledge. It also does not exclude the use of other tool
| like Just and Docker. A Makefile is also an easy jump-off
| point for a build pipeline.
| maccard wrote:
| Then introduce it for those things. But a makefile to call
| go build, go test, docker push, Ecs update-services
| provides no value other than all of a sudden not working on
| windows without another tool.
| tom_ wrote:
| You want to use GNU Make, and then you can ignore the Make
| dependency tracking. GNU Make is much easier when you only
| use so-called phony targets (consult the manual), which
| always execute without doing any dependency tracking.
|
| As for the advantage, a makefile will definitely perform both
| go build _and_ docker push, rather than just (say) docker
| push, an ever-present risk if you have to rely on your
| fingers to type these things in, or rely on your eyes to
| check that you recalled the right command from the history.
| It will also explicitly tell you the build failed rather than
| relying on you to do echo $? or for the tools to have some
| obvious output in the error case.
|
| A shell script is also an option. Makefiles have some helpful
| extra features: by default, commands are echoed; by default,
| the build fails if any command exits with a non-0 exit code
| (please consult local Unix person for the details); a
| Makefile inherently has multiple entry points; and, a
| Makefile can also be easier to get working on Windows than a
| shell script, though if you can demand that people run it
| from Git Bash then maybe there's not much in this.
|
| If you're still not convinced: that's fine! This is not a
| sales pitch.
|
| (I've more recently switching to using a do-everything Python
| script for this stuff, which is more annoying when it comes
| to invoking commands but has obvious advantages in terms of
| how easy it is to add extra logic and checks and some UI
| creature comforts and so on.)
| d0mine wrote:
| fabfile (Fabric/invoke: https://www.pyinvoke.org) may be an
| alternative to .PHONY-heavy Makefile (shell commands glued
| together by Python code)
| GrumpyCat42 wrote:
| My struggle with Make and bash is that they're not very
| expressive - maybe that's something we want in our CIs, but
| I've always preferred writing an actual program in that
| program's native language for CI/CD, even if it has to shell
| out some commands every now and again.
| ttyprintk wrote:
| I prefer that, too. I've heard (less-experienced) tech leads
| forbid Makefiles because they're not declarative enough
| compared to yaml.
| a1o wrote:
| If you stick with what is in common between ninja build and
| Makefiles, and comment any usage you do of what isn't, the
| file will mostly mention a series of inputs -> box ->
| outputs. What happens is make will dispatch in a way that
| the inputs are all satisfied. It works fine afaict, the
| only issue is make doesn't contain by itself the tools that
| it expects to be available in your environment, so you will
| still need something else to solve that.
| gwbas1c wrote:
| I touched make once in 1999, in school. The syntax was arcane,
| even by 1999 standards.
|
| > Why have they gone out of style?
|
| Because no modern toolchain uses make. Its syntax is so arcane
| that it's been replaced with various tools that are designed
| for the specific stack. Otherwise, more generic build systems
| use modern languages / markup.
| azeirah wrote:
| None of these are a problem anymore since the advent of Nix.
| aarmenaa wrote:
| FTA:
|
| > The Fix: Use a full modern programming language, with its
| existing testing frameworks and tooling.
|
| I was reading the article and thinking myself "a lot of this is
| fixed if the pipeline is just a Python script." And really, if I
| was to start building a new CI/CD tool today the "user facing"
| portion would be a Python library that contains helper functions
| for interfacing with with the larger CI/CD system. Not because I
| like Python (I'd rather Ruby) but because it is ubiquitous and
| completely sufficient for describing a CI/CD pipeline.
|
| I'm firmly of the opinion that once we start implementing "the
| power of real code: loops, conditionals, runtime logic, standard
| libraries, and more" in YAML then YAML was the wrong choice. I
| absolutely despise Ansible for the same reason and wish I could
| still write Chef cookbooks.
| rat87 wrote:
| > Pipeline definitions are scattered across multiple tools--
| GitHub Actions, Jenkins, ArgoCD, Kubernetes--and environments.
| This fragmentation leads to confusion, configuration drift, and
| duplicated effort.
|
| So are they talking about some sort of meta language compiling
| into multiple yaml configs for the different environments or a
| single separate CI tool that has plugins and integrates with
| GitHub/gitlab/etc?
|
| I do agree with them about the need for a real programming
| language. I hate yaml in gitlabs config, it is very hard to
| interpret how it will be interpreted. Things were much easier
| when I was scripting Jenkins even though I didn't know or like
| groovy then with gitlab
| cdaringe wrote:
| > So are they talking about ... meta language
|
| Said kindly, no, they're not. They're just stating values here,
| imho, not impl detail
| vergessenmir wrote:
| Is any of this reproducible? Not sure why that requirement has
| been quietly overlooked.
|
| I've worked in this space for a long time and can't make head or
| tail of what glue is.
|
| A motivating examplt would be help which I might have missed?
| a1o wrote:
| I don't think this is reasonable if you have a cross platform
| (web, android, iOS, macOS, Windows, Linux, FreeBSD, ...) app,
| things won't be that clean, that only works if whatever you do is
| very simple, otherwise there will be some patchwork to build and
| test across all platforms - there's just no way to run all of
| them local in your single platform computer whatever that is.
| Honestly a lot of what is there is not that useful, I don't need
| types in the pipeline when scripting python for build integration
| logic, there's nothing that types brings that are a must in that
| case.
| cdaringe wrote:
| Well written. Ive long held these values, and never could express
| them so concisely. Kudos!
|
| I look forward to seeing some matrix eval of impl strategies
| against these values
___________________________________________________________________
(page generated 2024-12-23 23:01 UTC)