[HN Gopher] GitHub Actions Are a Problem
___________________________________________________________________
GitHub Actions Are a Problem
Author : benrutter
Score : 137 points
Date : 2023-11-12 14:22 UTC (8 hours ago)
(HTM) web link (felix-knorr.net)
(TXT) w3m dump (felix-knorr.net)
| polotics wrote:
| What I always guessed: if you don't want to bite the bullet of
| trusting SRE scripters (with an s) to the role of building the CD
| that actually makes sense for your use case, why do you expect
| some random company will have more of your interest at heart? At
| least with Joe's python deployment scripts, Jane can do a code
| review. So much of cloud CI/CD lingo translates to "believe in
| magic, and also our bigness-based authority" when actually proper
| software deploys itself properly without making a fuss or glueing
| your fingers with sticky Yaml...
| CptKriechstrom wrote:
| I've been migrating some Workflows to dagger.io. So far it's
| amazing.
| redhale wrote:
| Thanks for sharing, I was unaware of dagger.io!
|
| I'm curious now -- are there any other alternatives with
| similar philosophy/benefits?
| maestrae wrote:
| There's earthly.dev but like previous commenters, I went with
| dagger because i prefer its Apache 2.0 license over earthly's
| BSL.
| adamgordonbell wrote:
| Earthly is MPL 2.0 now. I work for earthly and people
| didn't like BSL so we listened and changed.
| paradox460 wrote:
| Earthly is legitimately one of the coolest projects I
| wish I could use. But entrenched velocity in existing
| systems makes it a difficult pitch
| adamgordonbell wrote:
| That's great to here!
|
| Sometimes people find just starting with a messy part of
| the build process, or starting to replace dockerfiles
| here and there can add value, without having to change
| the world.
| tao_at_garden wrote:
| Yes, there's us over at https://garden.io! We're big
| believers in pipelines that run anywhere. I even made a short
| little video that should give you the gist. [1]
|
| Some of the short-list of differences: we use YAML for our
| configuration language, Dagger can use full-fat languages to
| define its pipelines. Our feature scope is broader: you can
| use us to vend IDP-like stacks to your developers if you're a
| Platform Team; we make development with remote Kubernetes
| clusters very easy, including all the remote image builds;
| and we have a number of integrations so you can bring your
| IaC tool of choice (Pulumi, Terraform) into your pipeline and
| set up service -> infra dependencies.
|
| [1] https://www.youtube.com/watch?v=JFnan6s2cDg
| andthat wrote:
| Same here, it's so liberating! I'm also using a Magefile to run
| the pipelines (written in Go) locally.
| ahofmann wrote:
| This are the reasons I don't use GitHub actions. I don't want to
| add another hard dependency into my development environment and
| testing actions looks like a very slow feedback loop, very time
| consuming.
| simonw wrote:
| If you want the ability to run as much of your GitHub Actions
| locally as possible (which you should! Testing via pushing
| commits is painful) you can arrange your actions so the majority
| of the logic lives in shell scripts (or Python scripts or
| whatever) that are called by the workflow.
|
| Then you can run those scripts locally and iterate in them before
| you try them on GitHub Actions.
|
| I agree: Actions would be more useful if I could run the entire
| stack locally (an official implementation, not a community-
| maintained clone). But it's not a big enough inconvenience for me
| to care very much.
| mplewis wrote:
| Incidentally, this is sort of how the Earthly runner serves to
| provide you local actions that also work in CI.
| simcop2387 wrote:
| Isn't that exactly what the self hosted runners are?
|
| https://docs.github.com/en/actions/hosting-your-own-runners/...
|
| Then you configure the repo settings to only use the self
| hosted ones.
| jahsome wrote:
| If you're suggesting the self-hosted runners are equivalent
| to running locally, then no, self-hosted runners still
| require a commit + push and runs remotely with the added
| overhead of the job being queued, reported, etc.
|
| The dream is to have something like a container _locally_ and
| be able to run something like: > gh actions
| run
| dinkleberg wrote:
| This is the way. It also lets you pretty much seamlessly switch
| CI tools at any time.
| jahnu wrote:
| I third this opinion: another reason it's important is it
| allows you to do your build in an emergency when CI is down
| for whatever reason.
|
| CI automation is for me the thing that replaces me running
| scripts one by one (and reporting/deploying results)
|
| It is not the thing that does the building/testing/deploying.
| That's the scripts that are hopefully written in a
| debuggable, portable, language.
| simonw wrote:
| This is a really good point. Most of my deploys go via
| GitHub Actions these days and it's not easy for me to
| deploy manually should I need to when Actions is
| unavailable - I should fix that!
| jahnu wrote:
| Like aviation, every regulation/rule is written in blood
| ;)
| leetrout wrote:
| This is why the three of us should record a podcast
| episode talking about this in the context of the points
| made in the Ironies of Automation
|
| https://en.wikipedia.org/wiki/Ironies_of_Automation
| notnmeyer wrote:
| yes! what we did was use a task runner to encapsulate the work
| in a "ci" task. you can run it locally, and then the GHA runner
| does little more than clone, install a couple depa and run the
| task.
| remram wrote:
| The problem is that GitHub Action offers a large ecosystem of
| reusable actions that you can't easily call from a script.
| xeromal wrote:
| You, you sacrifice that convenience for something purely in
| shell that is more portable.
| remram wrote:
| you you?
| xeromal wrote:
| It's like a yo-yo but faster
| philipswood wrote:
| Like yo-yo, but slower due to the extra W.
| simonw wrote:
| I mainly ignore those - I stick to the actions that install
| things (setup-python) and run scripts.
| remram wrote:
| You can get around the setup ones by using Docker images
| (e.g. run in python:3.11 rather than using setup-python).
| ddorian43 wrote:
| Can you provide a link to an example?
| slenk wrote:
| You should check out: https://github.com/nektos/act
|
| Works pretty well for the actions I have used it on.
| maccard wrote:
| Doing this also means that your build scripts are version
| controlled. You can iterate on the build scripts in different
| branches, for example.
| twic wrote:
| GitHub actions are configured by version-controlled files too
| though. I have an actions-dev branch on my project, so I can
| iterate on actions config without having to clutter up
| master.
|
| Would still be better if I could run locally to iterate. But
| the local run would have to have very high fidelity to what
| happens on GitHub to be useful.
| philihp wrote:
| Running actions triggered from a non-main branch does it
| for me, either on pushes to the branch, or
| 'workflow_dispatch'. You'll otherwise never get that level
| of fidelity to what happens in production.
|
| Keep the logic in your YAML "dumb". Avoid variables and
| subroutines in the file, if you want to DRY something,
| create your own custom action. You can have unit tests on
| an action, you can't have unit tests on a GHA workflow.
| mdaniel wrote:
| That cuts both ways, in my experience: a bug fix needs a
| metric boatload of cherry-pick, possibly across a bunch of
| repos, to fix any bugs. Centralization is great until it's
| not
|
| We have gotten a lot of good mileage out of GitLab's include:
| feature <https://docs.gitlab.com/ee/ci/yaml/#include>, which
| for clarity absolutely carries this same hazard, but can mean
| centralized fixes are less onoz than trying to patch every
| active branch of every active repo's build scripts
|
| I am also aware this cuts against the thrust of the thread
| about "build scripts should be locally executable," but until
| they (they: GitHub, they: GitLab) straighten out their runner
| binary to be sane, it is currently a better use of our
| engineering effort maintaining CI yaml than teaching devs how
| to run CI locally
| SethMLarson wrote:
| This is the way.
| marcosdumay wrote:
| Building Java and C# on the command line nowadays is a hard
| undertaking, requiring weeks of learning and debugging, just to
| have your code change in a way that makes the scripts outdated
| at any time (oh, and it's never portable).
|
| I really don't get why people use those complex CI/CD tools for
| other languages, but at least on the enterprise ones, I can
| understand people moving from the IDE into some huge
| centralized mess that can import it.
| neonsunset wrote:
| Building C# is quite literally 'dotnet build -c release'.
| hypeatei wrote:
| Exactly. I don't know where the newer .NET versions get the
| bad rap from.
|
| You can criticize Microsoft for other things but .NET is
| not one of them. They've done a great job with it.
| Kwpolska wrote:
| (And building code that targets the classic .NET Framework
| can usually be accomplished by a single call to
| msbuild.exe.)
| twic wrote:
| Why does Java take weeks?
|
| For a straightforward application or library project, you can
| fill in this form and get a shovel-ready build:
| https://gradle-initializr.cleverapps.io/
|
| I usually tweak it a bit, but that's minutes, not weeks.
|
| There's often more stuff to add as a project grows, but
| again, minutes to hours for the common stuff. Setting up
| publishing to a Maven repository is more work than I'd like,
| but it's still not weeks.
|
| So what are you doing that it takes so long? And how long
| would that take if done any other way?
| Timon3 wrote:
| I wish I didn't have to circumvent the UI with a strategy like
| this. Part of the reason I like CI/CD tools is the
| visualization factor - I can create specific steps, see which
| step is currently active during execution, and only care for
| the output of a specific step during debugging.
|
| A platform with support for visual control from the scripts
| (implemented as no-ops during local execution) would be
| perfect.
| truculent wrote:
| If you split your scripts into small tasks (i.e. one task per
| script), and call each one from your GA yaml, you would seem
| to have the best of both?
| Timon3 wrote:
| That still makes running it a bit annoying, since you have
| to call each script in order. You could of course create a
| script that covers this, but that's duplication that can
| drift apart. I guess you could generate the script from
| your YAML or use a linter or something, but it all feels a
| bit annoying. It would be awesome to e.g. just call a
| command like "gh-success" or "gh-failure" to manually
| indicate the success/failure, maybe with an optional ID to
| prevent accidental double calls etc.
| snnn wrote:
| I'm not quite familiar with GitHub Actions, but Azure DevOps
| Pipeline has a nice Preview API:
| https://learn.microsoft.com/en-us/rest/api/azure/devops/pipe...
| which runs the preprocessing steps(like running the C
| preprocessor on a C source file) then gives you the processed
| yaml file. Then if you'd like, you can write your own local
| runner based on that if your yaml files do not use too many
| different kinds Azure DevOps pipeline steps.
| mb7733 wrote:
| My strategy is just to keep the yaml to a minimum and call python
| scripts within the action to do anything complicated. The GHA
| yaml is just used for defining when the action runs, and on what
| architectures, etc. It's worked well for me.
| sametmax wrote:
| Same. My local astk runner does the heavy lifting.
| mst wrote:
| Was that a typo for task or is 'astk' a punnily named task
| runner in its own right?
|
| (sorry for the dumbass question but it could go either way)
| hardwaresofton wrote:
| GHA is "good enough" (GitLab CI was/is much better IMO), and
| it'll likely never go away because of how dominant GitHub is.
|
| Also, a simpler solution that takes a bit more discipline is
| writing all your CI tasks as scripts/makefile|justfile
| targets/etc and using containers to run them in GHA.
|
| IIRC containers are a little slower than running scripts in the
| VM directly (more than it must necessarily be), and also cross-
| platform builds might be slightly more complicated...
| XorNot wrote:
| Gitlab CI desperately needs backwards dependencies. A huge
| number of my remaining problems with it would be solved if I
| could subscribe to upstream outputs from my own pipelines.
| mdaniel wrote:
| I'm aware it says Premium but FWIW it does seem that exists
| since 12.8: https://docs.gitlab.com/ee/ci/pipelines/#trigger-
| a-pipeline-...
|
| I can also appreciate that the devil's in the details, so all
| its current constraints may not cover your use case
| XorNot wrote:
| So this seems to be enabled on the edition I'm using at
| work, but the limit is bizarrely "two pipelines". Which I
| already exceed since I have projects using git-dependencies
| in Python with 3+ repo dependencies.
|
| But it's also a weird project setting not a .gitlab-ci one.
| gavinhoward wrote:
| It's scary how closed-source our development platforms are.
|
| That's why building a fully-self-contained VCS, one that includes
| CI, is on my roadmap, but I fear the open part of our industry
| will fail before I finish.
| bogwog wrote:
| Github is pretty much the only major one that's closed afaik.
| Gitlab, Gitea/Forgejo, and SourceHut are all great existing
| alternatives that are open source. It would probably be better
| to volunteer your time to one of those than to build another
| one.
| gavinhoward wrote:
| Isn't GitLab partially closed?
|
| Didn't Gitea suffer a hostile takeover recently?
|
| SourceHut seems to be alright, but the founder hates me.
| bogwog wrote:
| * Gitlab EE (enterprise edition) is closed, but Gitlab CE
| (community edition) is open source
| (https://gitlab.com/gitlab-org/gitlab-foss/)
|
| * I didn't follow the Gitea drama too closely, but my
| understanding is that Forgejo was a fork born out of that
| situation
|
| * I've heard the SourceHut guy is a controversial figure,
| so avoiding it because of that isn't unreasonable. I will
| just say that "spite forks" tend not to last very long
|
| EDIT: actually, ignore my pointless negativity. I realize
| now that there really aren't that many open source code
| forges out there, so if you're willing to make one, then go
| for it!
| gavinhoward wrote:
| Thank you for your encouragement. :)
|
| You're very right about "spite forks"; I'm explicitly
| avoiding a fork of SourceHut because of that.
|
| I'm also going to be building the VCS from the ground up
| to solve some problems with Git; otherwise, why would
| anyone switch to my CI?
| mst wrote:
| Maybe validate the idea by prototyping whatever features
| you want atop Fossil first?
|
| Might want to see if you can catch rkeene on libera#tcl
| for a few minutes at some point - IIRC he has his own
| Fossil+CI type setup with custom BuildKit integration to
| produce binary artifacts and enjoys talking about it if
| he has time.
|
| Also the tcl/fossil/etc. crew tend to not care about
| forks/"NIH"/custom versions/etc. because the whole
| hardware background thing means those are just something
| you gotta do sometimes.
|
| (this is -not- me trying to sell you on using fossil in
| your final approach or anything, mind, just musing on
| what might make your experimental phase more tractable
| and mentioning a community I like who might be amenable
| to chatting about it :)
| gavinhoward wrote:
| Your comment brings up good points.
|
| I've thought about prototyping with Fossil; not sure if
| I'll do it because I'm also building a sandboxing build
| system that should slot into CI easily. We'll see.
|
| I haven't dug into CI design much, so I could be wrong,
| but CI seems pretty easy to do on top of a sandboxing
| build system and a VCS with hooks. One thing I know for
| sure is that I'm not going to use YAML. :)
|
| But I do take inspiration from the Fossil guys; for
| example, my VCS will have a bug tracker, wiki, and other
| project management stuff built in. It will also allow
| multiple checkouts and be one self-contained binary. It
| will also use SQLite for storing information about
| commits and other small things.
|
| It will, however, depart from Fossil's format to handle
| large files. And binary files too.
|
| So rest assured that I'm in touch with the Fossil guys.
| :)
| acjohnson55 wrote:
| I love GitHub Actions for basic CI/CD use cases, but I agree with
| the criticisms that testing locally is a major pain point and
| coding as a YAML DSL is a minor pain point.
| digitalsushi wrote:
| I mean to me it's no different than a video game that's "online
| only". And I know how much I love that.
| dventimi wrote:
| > Defining a language to run scripts in parallel with some
| constraints is actually quite easy if you just pass the script to
| an existing interpreter and let the language mainly deal with the
| constraints around order and parallelism (and code reuse)
|
| Isn't this Make?
| rr808 wrote:
| Right I like doing everything in Bazel instead of make now but
| still the same idea to keep it independent of CICD.
| javier2 wrote:
| Bazel? How many hours per year are you allowed to spend on
| CI? :D
| rr808 wrote:
| Lol our company uses it a lot so I use it a bit. I never
| learned make properly so I figure I might as well ignore it
| for now.
| zzzeek wrote:
| Github Actions gives me literally free server time across an
| extremely wide range of OS'es that I dont have to worry about at
| all, including Windows and OSX, which I therefore dont have to
| deal with in any way, buy any license keys, none of that. It's
| nothing short of miraculous for us as it's how Python projects
| can have binary wheel files for dozens of OSes and Python
| versions: https://pypi.org/project/SQLAlchemy/#files . This task
| remained impossible for years (to be clear: because I don't have
| a server farm, or Windows licenses, or however you'd run OSX on a
| headless server, or any kind of resources to fund / manage dozens
| of images and keep them running, or any of that) until GH Actions
| made it possible.
|
| Now is this all part of Microsoft's evil plan ? It probably is!
| But unless someone else wants to give me a free server farm that
| includes Windows / OSX up and running without me paying anything
| / writing containers / storing images / etc. I dont see this
| aspect of Github actions losing any popularity.
| Galanwe wrote:
| > This task remained impossible for years until GH Actions made
| it possible.
|
| Every major distribution used build farms long before GitHub
| (and git) existed...
| mplewis wrote:
| Yes, but I don't have a build farm.
| throw555chip wrote:
| Vendor lock-in hence the evil plan:
|
| https://dev.to/pwd9000/how-github-actions-can-improve-
| cicd-a...
|
| "While GitHub Actions is an open platform, it is still tied
| to the GitHub ecosystem. This can be a concern for teams
| who want to maintain flexibility and avoid vendor lock-in."
|
| https://gsilvapt.me/posts/moving-to-github-actions/
|
| "Actions are a Vendor Lock-in Mechanism"
|
| "But are you really sparing that many lines of code?
| Ultimately, you're going to have to declare what your
| environment needs so other developers can contribute to the
| project anyway, so why not simply write a bash script to
| take care of that for you?"
| ReleaseCandidat wrote:
| For Windows and OS X?
|
| That's still the main reason I use Github (although Gitlab
| has them now in beta(?)
| https://docs.gitlab.com/ee/ci/runners/index.html).
| plugin-baby wrote:
| Local actions can be defined with a dockerfile. I haven't used
| this approach extensively, but it seems like a simple way to use
| actions while also removing significant dependence on the
| platform.
| ttfkam wrote:
| Why not just use act? It was made for this exact set of problems?
|
| https://github.com/nektos/act
| airpoint wrote:
| Because it's incomplete and unstable
| TheCleric wrote:
| I use it when I can, but in my experience it's only useful
| about 50% of the time.
| dboreham wrote:
| Yes Github actions is (are?) a mess lacking conceptual integrity
| and widely misused. But we do the following with success:
|
| 1. Jobs should always (just) execute a script or program. This
| allows running outside the CI system.
|
| 2. To test/debug CI jobs, use act.
|
| 3. To test/debug more complex scenarios spin up a Gitea instance
| (which provides a Github clone wrapping act).
| mdaniel wrote:
| > 2. To test/debug CI jobs, use act.
|
| I can't wait for this meme to die, or for act (or gitea's fork
| thereof) to catch up to the hype train. Then again, I guess
| this fantasy is being promoted by folks who are all "just use
| run: and that's it" because any moderately complex one <https:/
| /github.com/VSCodium/vscodium/blob/1.84.2.23314/.gith...> for
| sure fails with spectacularly illegible error messages under
| both act and gitea's fork
| GiorgioG wrote:
| The over-complication of software deployments is just mind-
| blowing. I hope the end of cheap money reigns in the insane bloat
| that is DevOps today. It doesn't have to be this way, but there
| continues to be too few rational adults in the room.
| BigJono wrote:
| It's not a zero sum game. All the enterprise companies hiring
| 3,000 engineers only need a tiny fraction of their software to
| run their core businesses, and most of it is legacy.
|
| When a tech recession hits and clears out the clown car,
| everyone stops paying for software. You might be working on a
| team of 3 experts instead of 30 morons, but you'll work twice
| as hard for half the pay, with the same dickhead managers that
| treat you as replaceable cog, except this time they'll be
| right.
|
| If I were you I'd stick with the easy money where you get to
| look like a genius for being able to write basic simple code.
| throw555chip wrote:
| Last year George was a football star. George was injured in a
| big game. George learned to code YAML+Python scripts. George is
| now a DevOps rock star.
| asmor wrote:
| The runner is open source, but once you read the code you'd wish
| it wasn't.
|
| https://github.com/actions/runner
|
| For anyone in the audience who didn't know: GitHub Actions is
| based on Visual Studio Team Foundation Server's CI, and later
| Azure DevOps. And nobody, including the current maintainers, seem
| to know exactly how it all works (or how it doesn't).[1] The
| random exit codes is just the cherry on top!
|
| [1]: https://github.com/actions/runner/issues/510
| zamalek wrote:
| I attempted to read that code in order to glean information
| about how it communicates back to GitHub about progress etc.
| The goal was to make a new runner. I ran for the hills, fast.
| capableweb wrote:
| Sounds like most Microsoft projects to be honest. Not a lot
| of them can be highlighted as the pinnacle of software
| engineering, including their GitHub Actions implementation.
| SOLAR_FIELDS wrote:
| It's easy to look at the endgame. Just look at the hurdles
| https://github.com/nektos/act has to work with
| stylepoints wrote:
| I searched for "github actions random exit codes" and didn't
| find anything.
| asmor wrote:
| Personal experience building infrastructure and an autoscaler
| for these things (which performed horribly due to API lag)
| while having weekly meetings with our GitHub rep to get the
| --ephemeral flag in (took about a year longer than promised).
| Sometimes the exit code would be 0 when using `--once`, and
| sometimes it would not be. Sometimes it'd also be 0 if the
| token somehow didn't work and the worker couldn't even
| register no matter how often you restarted it (of course with
| a cryptic Azure IAM error code). Either way, we eventually
| just decided that throwing away the machine if the runner
| exists for any reason was safest.
| chimera77 wrote:
| Looks like GHA was announced[1] around the same time as the
| acquisition by MS. The first commit in that repo is a year
| later when they opensourced it, so we can't see how it evolved
| before then.
|
| For anyone who was at Github at the time, was it always written
| in C# or rewritten/replaced after acquisition? If "GitHub
| Actions is based on Visual Studio Team Foundation Server's CI"
| is the case, then it sounds like the latter.
|
| [1]: https://github.blog/2018-10-16-future-of-software/
| evook wrote:
| I don't recall the exact context anymore but during the
| Microsoft Build this year it was said by an employee at
| Github that the service came with the acquisition which is
| the reason it was and stays C#.
| Kwpolska wrote:
| You can also find many leftovers with the "VSS" acronym,
| like https://github.com/actions/runner/blob/main/src/Sdk/Co
| mmon/C... or https://github.com/actions/runner/blob/main/sr
| c/Sdk/Common/C... - which also mentions TFS (which is yet
| another acronym that used to refer to the Microsoft team-
| development thing).
| asmor wrote:
| This probably answers your question:
|
| https://github.com/actions/runner/blob/a4c57f27477077e57545a.
| ..
| capableweb wrote:
| > And nobody, including the current maintainers, seem to know
| exactly how it all works (or how it doesn't).[1]
|
| I read through this whole issue, and I cannot find exactly what
| you're referring to. What message gives you the impression they
| don't know how it all works? Seems it's mostly people asking
| for updates, and eventually something similar but different got
| implemented.
| asmor wrote:
| The fact that the --once flag was implemented and just didn't
| work. And it didn't work for several reasons. One, the
| control flow in the runner is extremely convoluted and
| exiting out is not trivial, and two, the scheduler doesn't
| handle assigned jobs not being picked up.
| capableweb wrote:
| And reason 0: it was never completely implemented, wasn't
| documented anywhere and wasn't supposed to be used by
| anyone at all.
|
| It's all the linked GitHub issue, begin from the top.
| lucasfcosta wrote:
| This was my first startup idea, and the first one we executed on:
| an open-source CI that runs on your machine exactly like it does
| in the cloud and doesn't use YAML as the main configuration
| format.
|
| Others have tried the same, including Cicada, which was built by
| the same folks as Fig.
|
| Here are the failure modes for this idea:
|
| - CI succeeds most of the time. It's only in the minority of
| cases that running CI locally is useful, as you don't need to
| debug that often.
|
| - The only case where CI fails more often is when building
| pipelines. If you're building pipelines then GitHub actions
| already has bazillions of actions and recipes available. Their
| "whole product" is a huge advantage (shout-out to readers of
| Crossing the Chasm).
|
| - The friction to adopt GHA is much smaller as it's one click
| away on GitHub and all your other stuff is already there.
|
| - Making an MVP of CI is really difficult because it requires
| lots of features. You can't even start if you don't have things
| like permissions, outputs for different workflows, and a
| significant amount of infra for orchestration.
|
| - Even when you have an MVP the switching costs of CI are
| ridiculously high, and it's a critical piece of infrastructure,
| so it's difficult to get people to move over if you're only
| offering marginal gains.
|
| - GitHub is owned by Microsoft, which owns Azure. Their compute
| will be way cheaper 99% of the time. Your product advantage must
| be significant enough to justify higher costs. In any case, their
| margin will be ridiculously higher.
|
| It's funny how many people get to the same conclusion, yet this
| is a really difficult problem to solve.
|
| TL;DR: the author idea is great and many people thought about it
| before. Executing such an idea is extremely difficult though.
| oooyay wrote:
| > CI succeeds most of the time. It's only in the minority of
| cases that running CI locally is useful, as you don't need to
| debug that often.
|
| It's also greatly impacts migration time which would be a
| leading signal for any team considering a transition.
| adobrawy wrote:
| > GitHub is owned by Microsoft, which owns Azure. Their compute
| will be way cheaper 99% of the time.
|
| I hope that regulators in Europe will start to notice and
| respond to these monopolistic practices of IaaS players who
| unfairly compete on the managed services market by providing
| services below cost. Previously, such actions were taken
| against e.g. Microsoft to ensure that Windows licenses are
| offered on equal terms to all cloud providers (without
| preferences for Azure).
| rubenfiszel wrote:
| We have built an open-source generic workflow engine to run
| arbitrary scripts (https://windmill.dev) with a vscode extension
| to build the yaml using a low-code builder and each individual
| script in their dedicated python/ts files so you get your full
| editor assistants https://youtu.be/aSOF6AzyDr8?t=116
|
| One of the area we are expanding next is a github app so you get
| exactly the same UX as github actions but running windmill
| workflows on your windmill workers.
| replwoacause wrote:
| No pwsh support?
| rubenfiszel wrote:
| We do support pwsh :)
| replwoacause wrote:
| Great! Will give it a try.
| throw555chip wrote:
| I had to Google this, apparently pwsh is the incarnation for
| PowerCrap I mean PowerShell.
| replwoacause wrote:
| You had to Google that? It's been widely used since 2018 to
| refer to PowerShell.
| keepamovin wrote:
| I actually really like GitHub actions, I read the article and
| while I get some of the concerns, others I don't understand. In
| any case the author's situation doesn't apply to me, and I wanted
| to share something I really liked about GHA.
|
| So, I recently figured out a way to host a remote browser on them
| by using an Ngrok tunnel. It's really cool to see BrowserBox
| running from inside a GitHub action container. I literally
| couldn't believe it actually worked when I first figured it out!
|
| I was so excited. It started as just this tech prototype in my
| mind (could this be possible? Probably not but I Feel like it
| could be). And to see it actually achieved so cool! :)
|
| It has made CI integration testing SO much easier and more
| repeatable. I _love_ that it can just run it up on Ubuntu and I
| can verify.
|
| Anyhow, I thought this was so cool, and such a useful way for
| people to either just get started with BrowserBox trying it out,
| or even run a quick little VPN-like/proxy browser from another
| region. I've even logged into HN from it on the sly. I liked this
| whoel concept _so_ much, that I even wrote an action that
| integrates with issues to make the process as easy as possible
| for people.
|
| Basically you can just clone or fork the repo:
| https://github.com/BrowserBox/BrowserBox and then open an issue
| and pick the template that is like "Make VPN". The login link
| will get published in the repo. The link is not private (unless
| you make your fork or template private) and there's a bit of
| setup with your ngrok API key (free is OK) but the issue
| conversation automatically guides you through all that.
|
| I thought this was _so cool_ (free server time, actually working
| app), that I even created another version that uses MS Edge under
| the hood instead of Chrome in the original, just to show how easy
| it is: https://github.com/MSEdgeLord/EdgeLord
|
| Just a niggle is that the other services we normally have (secure
| doc viewer, audio, remote devtools) do not work as ngrok only
| maps 1 port. I could use an ngrok config file I think to fix that
| but somehow, easy as that is, I have not gotten around to it!
| Another niggle is I noticed the auto-tab opening used in the GHA
| demo seems a little funky lately, and you may need to manually
| reload or resize them to un-wonkify it. Probably a little
| regression!
|
| Anyway! :)
| adobrawy wrote:
| You might also be interested in
| https://github.com/mxschmitt/action-tmate, which enables SSH to
| workers for debugging. If you have trouble reproducing issues
| on CI, this can be a life saver (or at least save a few hours
| of commit-push-wait rounds).
| keepamovin wrote:
| Cool, thank you! Yes, I haven't so far but I might in future
| need that. Anything that increase the speed of those feedback
| loops! :) I wonder how it works? Looks cool
| PLG88 wrote:
| Super cool. Have you considered using alternative technology
| that allows more mapping than 1 port, e.g., open source
| OpenZiti? This is an example of embedding their SDKs into a
| webhook action to connect to a server in a completely private
| network - https://netfoundry.io/this-is-the-way-invisible-
| jenkins/
| semiquaver wrote:
| > Hosting a git repo is hardly more than providing a file system
| and SSH access. The actual mechanism they use to keep you on
| their platform is the CI-pipelines
|
| Then why was GitHub so popular for the 10+ years it had no built
| in CI system?
| IshKebab wrote:
| Yeah, utter nonsense. The mechanisms they use to keep people on
| GitHub is a) network effects (easy to create issues, PRs etc
| because you already have an account), and b) GitHub is actually
| really good!
| hotnfresh wrote:
| But aside from enterprise SSO, far better permissions
| management than you get with SSH and Linux filesystem
| permissions, a unified open-source project discovery and
| vetting-assistance system, secrets management, integrated CI,
| lfs support, issue tracking, a billion integrations for-free,
| automatic dependency vulnerability detection, et c...
|
| What have the Romans done for us?
| phkx wrote:
| I'd also rather say that it was about hosting the code for free
| in the first place and then the pull requests, including the
| possibility to comment on code in PRs. I use the git cli
| (instead of some UI / IDE extension) for all interactions with
| the repo locally. But as soon as it's about collaboration,
| these platforms come into play.
|
| I just quickly scanned, to find that there is the `git request-
| pull` command, before I wasn't even sure whether pull requests
| are a git built-in feature at all.
|
| Side question: does any code hosting platform allow to comment
| on lines of code outside of pull requests? I've had several
| occasions where I wanted to ask, why something was written the
| way it was.
| throw555chip wrote:
| Before GitHub, even before git, we just emailed or irc'ed a
| patch request to the team. Somehow, "pull request" caught on
| as a term.
| juped wrote:
| a github pull request isn't a pull request; a pull request is
| an email from one of linus torvalds' direct underlings
| (subsystem maintainers) to linus torvalds "requesting" that
| he "pull" (hence the name) some tag. an arbitrary example:
| https://lkml.org/lkml/2017/11/13/229
|
| git request-pull generates these emails.
|
| note that a "pull" is just "merge from a URL", and requires
| some preexisting trust, hence why it's only for the subsystem
| maintainers.
|
| github stole this term for their signature misfeature and
| we've all been suffering since. some of its clones walk back
| this poor naming by saying "merge request" instead, but the
| damage to the name is done.
| jchw wrote:
| A big problem with making an abstraction over CI providers is
| that they are, on the surface, very little more than wrappers
| around steps made up of small scripts, but that abstraction leaks
| like hell. So I think you would wind up needing to go quite high-
| level to make something that is useful here, but even if you _do_
| do this, the variance in runtime environment will make it _very_
| difficult to make sure things are relatively portable, and
| attempts to work around this will lead to problems (e.g. you
| could just wrap everything in Docker, but fetching large images
| on each CI invocation isn 't really a good use of resources.)
|
| Let's say you have a project using CMake with vcpkg. Vcpkg has
| support for the GitHub Actions Cache API, so it can integrate
| very well into GHA and provide good performance for caching
| dependencies so they do not need to be built on each invocation.
| Doesn't seem too unreasonable, you could have a vcpkg+CMake
| plugin of some kind that tries to do the best thing for each CI
| provider. But, what version of vcpkg do you use? what version of
| CMake do you use? Vcpkg is essentially just a rolling-release Git
| repo, there's no specific version to use. But it would save _a
| lot_ of time to make use of the vcpkg installation that already
| exists on GitHub Actions images. And obviously, a tool like CMake
| depends _super_ heavily on the environment around it. A lot of
| people use the Ninja provider for better performance... but if
| you use it on Windows, you need to be careful: CMake with the
| Ninja provider on Windows can not automatically pick up the
| Visual Studio installation, unlike the default Windows provider
| (which is, well, Visual Studio.) So instead it might pick up
| MinGW, which is often unexpected, and does not always work
| correctly. Does Microsoft have an answer for how to deal with
| this? Not really. You might think to use the official setup-
| msbuild step, which has been mistakenly recommended by Microsoft,
| but it only sets up enough environment to do .NET development.
|
| So actually your CI abstraction needs plugins that deal with:
|
| - Every CI provider
|
| - Using every operating system
|
| - For each build step
|
| And, also, it needs to be updated fairly frequently, especially
| if you want to keep up to date on e.g. macOS versions.
|
| Personally, I think this is an unwinnable situation, though I'd
| love to be proven wrong. I think it would take something fairly
| dramatic to actually solve this problem most of the way, though.
| Something that might need a lot of collaboration between
| stakeholders, possibly the CI providers themselves, and probably
| even build/packaging tools at some level.
|
| One thing I thought was interesting was the approach of using
| Nix[1]. I've tried rudimentary implementations of Nix with GitHub
| Actions though, and it does definitely have some issues still.
| The most obvious issue is that it doesn't really support Windows,
| followed by the fact that binaries made using Nix are generally
| not going to be very easy to distribute (unless you're building
| an OCI or AppImage or something like that.) But for just running
| tests, it's enticing at least, and it can do a whole lot of
| caching in theory (for dependencies; you'll still need to handle
| caching of intermediate files for local tests/builds in your
| build system underneath Nix.)
|
| The more I think about this problem, the more I wonder which
| angle is the best one to come at it from. Is this ultimately a
| problem about software SDKs in disguise?
|
| [1]: https://determinate.systems/posts/nix-github-actions
| rollcat wrote:
| I've always been opposing push-to-deploy, unless you actually
| have per-branch, ephemeral test environments, and take the
| required care to set up your data fixtures - which is a major
| technical challenge on any non-trivial project.
|
| The trick is to keep the deployment code simple and stupid, while
| making the _process_ transparent and dependable for the team.
|
| For production environments, rather than thinking in terms of
| pushing a commit, you want to reframe the problem as promoting a
| version from a staging environment - preferably subject to
| approval from QA and/or project owner. ChatOps is great for that,
| and it doesn't have to be painfully complicated, or full of hacky
| Hubot code - you can get huge ROI from a simple script[1] that
| posts a message at the start&end of a deployment, which includes
| a version string (autogenerated from a git tag[2]), environment,
| maybe your login/host name ("$(id -un)@$(hostname)"). CI can
| build the artifacts, then your job is just "./deploy prod v3.14",
| which is also easily delegated.
|
| [1]: def slack_post(data): req =
| urllib.request.Request(
| "https://hooks.slack.com/services/xxx/xxx/xxx",
| headers={"content-type": "application/json"},
| data=json.dumps(data).encode("utf8"),
| method="POST", ) res =
| urllib.request.urlopen(req) assert res.getcode() in
| range(200, 300) return res
|
| [2]: #!/bin/sh set -eu
| slugify() { sed -E -e 's/[^A-Za-z0-9]/-/g' -e
| 's/-+/-/g' -e 's/(^-|-$)//g' } git_tag=$(git
| describe --tags --dirty --always) git_branch=$(git rev-
| parse --abbrev-ref HEAD | slugify) case "${git_branch}"
| in master|HEAD) echo "${git_tag}"; ;; *)
| echo "${git_tag}-${git_branch}"; ;; esac
| juped wrote:
| push to deploy can be _made_ okay by giving it promoting-a-
| version semantics, with the advantage of having a deploy-
| history graph, but the concept as it 's typically described and
| the way it's used in practice push people towards being not
| very okay at all.
|
| your approach is less likely to be misused, which often
| outweighs having a deploy-history graph
| rollcat wrote:
| The chat log is the deploy history ;)
|
| Sometimes stupid really does beat clever. Spend your tech
| debt tokens on the product instead!
| asylteltine wrote:
| Yaml shouldn't be used for anything except configuration files.
| It's just lazy for services to use it as a kind of DSL
|
| These kinds of systems should use an actual programming language
| or their own DSL
| sghill wrote:
| I'd like to see better centralized support generally from CI
| systems. Six months of steady migration and a custom DSL seems
| alarmingly common.
|
| CI Wishlist:
|
| * testable outside of CI
|
| * required steps based on repo characteristics (ex: security
| scans)
|
| * central step and workflow definitions
|
| * controlled rollouts/rollbacks to simplify migrations
| AtlasBarfed wrote:
| Crazy thing about this and is frustratingly obvious, as I become
| a little older in IT, is that this is just a workflow problem.
|
| CI build pipelines aren't even really in the class of workflow
| problems that are even difficult to implement... workflow engines
| start you down a rabbit hole towards different models of
| computation that eventually ends up with a turing machine anyway,
| but I digress.
|
| It's just really kind of ridiculous that the same basic problem
| gets re-implemented literally thousands and thousands of times
| across different platforms and use cases. The fact that the
| current implementation is based on some Frankenstein of former
| visual studio code and other stuff really, isn't that surprising.
|
| Of course! Standardizing workflows leads precisely to the XkCD
| referenced in the article, in standards produced by the workflow
| vendors of of course been horrendous and focused on preserving or
| enhancing market share, and we're done in the era of XML.
|
| The issue of course is that workflows are an "Enterprise" feature
| and solution.
|
| Perhaps someday in the halcyon future someone will actually
| address a useful cross platform workflow model I could conserve
| algorithms, processes, techniques, and maybe even plugins.
| Because right now the workflow engine is balkanized across every
| single CI platform.
| marcosdumay wrote:
| > It's just really kind of ridiculous that the same basic
| problem gets re-implemented literally thousands and thousands
| of times across different platforms and use cases
|
| Why not? It's incredibly easy and quick to implement one, why
| not just make one perfectly fit for your use case every time?
|
| The actual issue here is people insisting on offshoring that
| implementation to GitHub, so that they deal with a brittle,
| badly defined interface that is not aligned with their needs,
| instead of just starting with a simple engine and
| reimplementing whatever they need. It's actually the opposite
| problem that standards were meant to solve.
| baq wrote:
| > It's incredibly easy and quick to implement one, why not
| just make one perfectly fit for your use case every time?
|
| Look around, it's so easy there are corpses of companies and
| projects all around the industry.
| kyrofa wrote:
| I would respectfully suggest that the author is misusing CI. If
| you have trouble running your tests locally, you have a problem.
| If you have trouble deploying from your local code, you have a
| problem. All of those capabilities should exist as simple scripts
| in your project already. Once you have that done, the CI yaml is
| a simple glue layer that defines an order of operations, e.g.:
|
| 1. Run static tests
|
| 2. If those pass, run unit/integration tests
|
| 3. If those pass, deploy
|
| If you find yourself screaming about YAML, you're leaning too
| heavily on it and need to refactor your project's scripts.
|
| Maybe a good question to ask would be "if I had to switch to
| another CI system today, how hard would it be?" If the answer is
| "hard", perhaps you're leaning too heavily on it and need to
| refactor your project's scripts.
| digitalsushi wrote:
| Testing locally, I agree with.
|
| Deployments, I think it'd be fair to consider the requirements.
| At work, our softare can be tested locally but deployments are
| all registered against a central authority, and after a point
| of composing enough access requirements, only then does a role
| (cicd in this case) have enough policy allowance to perform a
| deployment.
|
| The entire transaction is auditable. And I think that with a
| deployment, that's how it should be; allowing that trust down
| to a local environment strikes me that too much permission is
| accured with a single entity.
|
| I guess that we could better define what a deployment is; to
| some nonprod environments I'd agree, but I'd still probably
| insist on the heavy machinery up at the test, perf, qa, areas,
| and then getting into staging and prod, there'd be no wiggle
| room.
| kyrofa wrote:
| Fair critique, totally depends on what kind of software we're
| talking about and where the deployment is happening. In
| general though, how screwed are you if your CI environment
| goes down? Can you not deploy anything? That would be scary.
|
| My point, however, was mostly that the logic necessary to
| deploy should live as part of your codebase, not written out
| in YAML. The privs necessary to deploy are a separate
| discussion.
| msm_ wrote:
| This sounds fine and well, but it's not how Github Actions work
| (or at least, not the encouraged workflow). Let's have a look
| at the snippet from one of the projects I work on:
| - name: Set up Docker Buildx uses: docker/setup-buildx-
| action@v2 - name: Build and push mwdb-core image
| uses: docker/build-push-action@v4 with: file:
| ./deploy/docker/Dockerfile tags: |
| certpl/mwdb:${{ github.sha }} certpl/mwdb:master
| cache-from: |
| type=registry,ref=certpl/mwdb:buildcache outputs:
| type=docker,dest=./mwdb-image - name: Upload mwdb-core
| image uses: actions/upload-artifact@v3 with:
| name: mwdb-image path: mwdb-image
|
| Good luck running this locally. There's no script code to speak
| of, just references to external "actions" and parameters (for
| example, https://github.com/docker/setup-buildx-action).
|
| Some CI platforms are just a simple glue layer (Gitlab CI -
| which I prefer - is one of them), but in most cases Github CI
| is not. Maybe it adds to the author frustration?
| jameshart wrote:
| Building it that way is a choice. It's not mandatory.
|
| You can use gitlab CI with special-purpose docker images for
| all your steps and magic parameters driving everything too
| (Gitlab AutoDevops works that way).
|
| But if you just run your steps in shell scripts in vanilla
| docker images containing your build-time dependencies, you
| should be able to produce something that works the same in
| any CI pipeline, or locally.
|
| The most annoying thing for me is that a lot of CI engines
| make docker-in-docker complicated. I love using compose to
| set up integration test environments, but doing that in CI is
| often a fight.
| kyrofa wrote:
| > Building it that way is a choice. It's not mandatory.
|
| This ^ . In GitHub Actions, I personally try to use pre-
| baked actions as little as possible, for exactly the
| reasons I outlined.
|
| I prefer GitLab CI, but you can make a mess of that just as
| easily. In general, if you approach CI as I suggested, you
| end up with something maintainable regardless of the CI
| engine in use.
| javier2 wrote:
| Yeah, Github Actions has its cool parts, but I still prefer
| Gitlab CI since its just 'dumber', and doing custom things is
| simpler to achieve.
| mdaniel wrote:
| As another nod to GitLab's CI, it has sane autocomplete in
| the editor since the schema is mostly static. I'm not aware
| of any GitHub yaml tooling that downloads the
| descriptors(?) for any actions in use and with then allow
| autocomplete of its uses: or with: blocks
| jameshart wrote:
| 90% of the tricky parts of CI are secret management. As long as
| you write scripts to pick up credentials in a sane way
| (.awsprofile or similar) you should be able to configure your
| CI to provide the credentials just as well as you can locally -
| but in practice, the various different ways that things like
| artifact repositories, integration test databases, and cloud
| deployment tools want to manage auth is the cause of most of
| the complexity in getting your build/test/deploy pipeline
| working on the runner.
| twosdai wrote:
| I would say the secret management part typically isn't the
| most insane or annoying part.
|
| For me it's external system state management. Like making
| sure the integration test db is cleaned up correctly.
| djha-skin wrote:
| This is all fine and good if you are the principal developer of
| the project. However, the author makes it clear that he is
| migrating _other people 's_ CI pipelines. He is a DevOps
| engineer working across several teams.
|
| This is why he makes the important point that discipline is not
| enough. The reason is most teams simply don't care. I find one
| in five teams where everyone on the team cares about the build
| (when I'm lucky!), most teams have one person who cares, and
| some teams have no one that cares.
|
| When I am tasked with the proper care and feeding of the
| pipelines of others, I want tools that can work and help me out
| even when the developers who created the software are Holding
| it Wrong.
|
| With these requirements in mind -- managing and migrating the
| many different CI piplines across an organization -- it would
| be a major breakthrough to have a tool that 1) transpiles to
| the workflows of all the CI tools and 2) allows for local
| testing. So many orgs have different teams using different CI
| stacks, and the local testing problem is always a struggle. I
| would use a tool like that _into the ground_.
|
| So I would qualify your original statement: The author isn't
| misusing CI. Rather, the author is attempting to survive in a
| world where _others_ are misusing it, and where the author is
| tasked with managing all the CI pipelines.
| pyrale wrote:
| > I would respectfully suggest that the author is misusing CI.
| If you have trouble running your tests locally, you have a
| problem.
|
| I would respectfully suggest that you misread the author. The
| issue isn't running tests locally, it's running the CI config
| locally.
|
| I experienced the same problem with gitlab CI years ago, where,
| basically, you can lint the file and not much more. Past that,
| you need to run it through your CI and debug if you get
| slightly different results compared to running a script
| locally.
| capableweb wrote:
| Yeah, it's a horrible experience all around. CircleCI solved
| this problem like a decade ago, enabling SSH builds so you
| can troubleshoot straight up in the build itself, and once
| you've figured it out, just copy-paste the steps to your
| Makefile/CI config.
|
| I don't understand how one could build a CI service so long
| time after CircleCI launched, and still not have that very
| same feature (or something similar).
| buster wrote:
| What? You hopefully do run your tests locally. The premise that
| you have to run them via the GHA configuration is only an
| annoyance. Same as I've never seen people running the Jenkins
| pipelines locally. In the end it's up to you to develop your
| tests "local first".
| digitalsushi wrote:
| After two forced CICD migrations, (shoulda been the first), I
| learned a pattern I think must be very common: decouple
| absolutely everything you can from your CICD. Small
| Jenkinsfiles, no groovy libraries, or Bamboo specs; Keep all of
| it in portable shell script or some other easily movable
| environment, and call it from the tiniest pinning you can in
| your Jenkinsfile.
|
| Then have your team use the scripts locally, and make sure that
| Jenkins is basically just a person on your dev team calling it
| the same exact way. Jenkins will end up testing your
| build/test/deploy solution for free and you can bring bugs it
| catches back to your team so that they are wasting less time on
| those bugs and more on product work.
| baq wrote:
| This is very wise and I recommend this to everyone, but is
| not without problems; notably if you want to parallelize the
| build in a stateless build system with ephemeral runners
| (like GHA) you have to be very careful about caching your
| intermediate artifacts. This can be a lot of complexity which
| doesn't exist on a single build box.
| aaomidi wrote:
| Honestly the fact that you can't run this locally is the main
| problem here
| hnlmorg wrote:
| This is why you should write your business logic in external
| scripts and reference them from your CI/CD. So the YAML is just
| an interface while your pipeline logic is CI/CD agnostic.
| 8organicbits wrote:
| > The fact that you have to push a commit to the repo to test a
| change, and then wait for a runner delays everything
| indefinitely, and makes iterating really painful.
|
| Iterating fast is key. As mentioned `act` helps a lot, getting as
| much as possible to run locally really cuts cycle times.
|
| GitHub Actions feel like bash to me. They are not great
| languages, but they are available so often that knowing them is a
| net productivity boost.
| TheCleric wrote:
| I agree and use act as frequently I can, but it has some pretty
| hard limitations. For example your runners better be using
| Linux as act doesn't support Windows or Mac runners.
| iandanforth wrote:
| Other people must have very different setups to warrant this kind
| of additional overhead and tooling.
|
| In our repo local testing is first. You check out the code and
| can run the tests. The github runners do the same steps the
| readme encourages humans to do when running tests locally.
|
| Also I'm not sure what duplication the author is avoiding with
| their DSL as Github actions can be broken into components and
| reused and parameterized. Of course if you don't like YAML you're
| never going to be happy until you've wrapped it in your own layer
| of tech debt.
| WirelessGigabit wrote:
| This is my approach too.
|
| I don't use any of all-in-one actions.
|
| I write the things that I need to have locally, in bash, or a
| build.rs script or via package.json.
|
| Then I invoke those things in GitHub actions. I split them up
| so they can run concurrently.
| benrutter wrote:
| The times I've seen more complex testing set-ups that use
| actions is normally some combination of: 1. Testing involves
| some kind of complex environment 2. IT infrastructure in some
| corporations makes setting up that environment impossible.
|
| I worked once in a role where I had to write a library aimed
| for working with spark, but getting spark (i.e. java etc)
| installed on my machine involved weeks worth of requests and
| escalations to IT.
|
| A lot of libraries (I'm thinking of adlfs as a good example
| which interacts with azure data lake file systrmd) have to have
| relatively complex testing involving docker by nature of what
| they are.
|
| None if those points make "put in a PR to run tests" a good or
| justifiable workflow. But I understand how people wind up
| falling into that trap.
| sshine wrote:
| We run CI in an image generated by our devcontainer Dockerfile.
| All the commands in the 'run:' steps of GHA can be run locally in
| the devcontainer. Most commands work outside in random Linux
| environments, but a few things will break.
| nimbius wrote:
| patiently waiting for the "github actively considered useless"
| post on medium.com at some point.
|
| actions are a howling trainwreck. Copilot is a dancing bear of
| potential litigation and code exfiltration. neither are
| immediately very dangerous though as Github.com itself, and its
| ecosystems, have had more than sixty outages this year alone. if
| you can get over those, you'd better hope members of your team
| dont live in --or have been actively sanctioned by-- the United
| States because Microsoft will just delete your work and lock you
| out.
|
| Look...when you drop 7.5 billion dollars on a code ecosystem in
| order to capture the worlds developers, its incumbent upon you
| not to squander the braintrust youve just aligned yourself with.
| Microsoft feels like its done nearly everything it could to erase
| whatever goodwill github had as a standalone company in an effort
| to do the same pathetic whitewash its done with every other
| acquisition. Shoehorn its own garbage languages into the product,
| migrate its services to Azures low-performance dumpster fire, and
| lock in whatever users ya got in order to staunch the potential
| bleed to other much more meaningful programs like gitea or
| github.
|
| I give this github.com another 7 years before Microsoft quietly
| packs it up, sells it to a capital management firm, and sweeps it
| under the rug.
| holman wrote:
| I wouldn't deny that Microsoft brings a lot of friction and
| problems to things within GitHub, but you also shouldn't
| underestimate that GitHub itself is doing a lot of this on
| their own merit.
|
| Regardless of the parent company, the GitHub org is huge, with
| a lot of people and execs involved. Just wait until the next
| year or two of reimplementing all these existing features in
| React- that's really going to be fun!
| samcat116 wrote:
| This applies to just about any CI/CD program, so I guess they
| just don't like them in general.
| csnover wrote:
| In addition to the suggestions others have made for locally
| testing workflows, there are also reverse shell actions[0] that
| can be used for troubleshooting CI failures on the GH runners
| themselves.
|
| [0] https://github.com/mxschmitt/action-tmate
| eezing wrote:
| I've been using a Makefile in front end projects for years- use
| it for all dev, build, test, and deploy scripts. Local users and
| GH Actions use the same Make targets. Our GH Actions are very
| lean.
| djha-skin wrote:
| The idea of a DSL over YAML is also a good one. Jenkins got this
| right, using a DSL on top of Groovy. This allows for Jenkins
| build script libraries[1]. I have used them in the past, and they
| were a joy. They allowed for standardizing builds across many
| different small git repos, each with their own terraform or helm
| chart. I miss this feature in GHA.
|
| 1: https://www.jenkins.io/doc/book/pipeline/shared-libraries/
| RichardCA wrote:
| This is a topic of debate in my current job. We use Jenkins
| Multibranch Pipelines and don't want to discard all that work,
| but we also don't want developers doing release management
| based on the Jenkins UI/UX. It seems logical to use GHA with
| local runners to call Jenkins via bash scripts running curl
| commands, except for how "Rube Goldberg" that sounds. But it
| seems like everything is like that these days, and all you get
| to do is pick your poison.
|
| There's also the problem of Jenkins in general, that it's a
| mature product and you never know if the one plugin you depend
| on is going to stop being maintained. Or you know you have
| CVE's but can't upgrade without running two environments, one
| production and one with all the plugins updated. Or you're
| chasing down an issue only to find bugs that have been open
| issues for several years.
|
| https://issues.jenkins.io/browse/JENKINS-52362
| https://issues.jenkins.io/browse/JENKINS-52966
| yesimjack wrote:
| > The fact that you have to push a commit to the repo to test a
| change, and then wait for a runner delays everything
| indefinitely, and makes iterating really painful
|
| This isn't correct. The workflow_dispatch trigger specifically
| exists to allow for manual runs and you can also re-run existing
| failed (and successful) jobs.
| ligurio wrote:
| Cirrus CI had an attempt to make CI-agnostic tool for running
| Dockerized tasks, see [1]. (I'm not affiliated wtih Cirrus CI)
|
| 1. https://cirrus-ci.org/blog/2020/10/07/cirrus-cli-ci-
| agnostic...
| Ayesh wrote:
| This article sounds like an incoherent rant against CI/CD in
| general.
|
| Coming from Jenkins, App Veyor, Circle CI, and GitLab CI, GH
| Actions was such a fresh breath and is a very cohesive solution
| to the cluttered CI space.
|
| Most of the "points" mentioned in the article are merely code
| smells of not using a decent deployment/testing setup.
|
| Having centralized build setups is not new either. Ubuntu PPAs
| and Arch's equivalent of package builds share similar pain
| points, but it's been serving us for decades at this point.
| yeck wrote:
| > the runner is closed source
|
| My team runs our GHA runners on our own infrastructure using
| Actions Runner Controller (ARC).
|
| > The fact that you have to push a commit to the repo to test a
| change, and then wait for a runner delays everything
| indefinitely, and makes iterating really painful.
|
| Sounds like a mistake. Steps in jobs can often be run locally,
| and you could make choices in your jobs to make that more or less
| true. While there are certainly some things that you can only
| realistically test by triggering a CI job, many things can be
| tested locally.
|
| > writing pipelines in YAML is just painful... I invested 3 days
| to write a small DSL that compiles to GHA
|
| Yes, that can definitely be the case. But the thing about YAML is
| that it can easily be made a target for DSLs, as you
| demonstrated. I'd rather GHA use something common like YAML than
| force another DSL on to the world. It is better for users to
| decide if/what abstractions to use in front of YAML.
| jiggawatts wrote:
| Local execution of GitHub actions for testing will become
| possible some time after pigs learn to fly.
|
| Everyone here is asking for it as if it's some minor oversight,
| soon to be rectified.
|
| Unfortunately, this tech stack is a significant revenue source.
| Microsoft charges for pipeline minutes, concurrent runs, etc...
| This is especially true in Azure DevOps which shares much of same
| underlying pipeline software.
|
| Letting anyone run this locally for any reason would let them
| bypass the monetisation.
|
| It's the same reason that ad-supported YouTube is "missing" a
| download offline feature.
|
| It's not an oversight. It's not happening. Stop asking.
|
| The only thing we the dev community can do about this is to
| develop our own open-source CD platform with blackjack and
| hookers.
|
| In other words, we need a Kubernetes of CI/CD.
___________________________________________________________________
(page generated 2023-11-12 23:02 UTC)