[HN Gopher] Write Gitlab CI Pipelines in Python Code
___________________________________________________________________
Write Gitlab CI Pipelines in Python Code
Author : DrSarez
Score : 44 points
Date : 2021-04-29 20:47 UTC (2 hours ago)
(HTM) web link (gitlab.com)
(TXT) w3m dump (gitlab.com)
| kmstout wrote:
| Several weeks ago I poked around Github's CI workflow references.
| It seems that there's a clear path for crafting a workflow that
| simply grabs the text of a each issue (as it's submitted) and
| appends it to the documentation before closing the issue, perhaps
| marking it as "expected behavior". In short, an evening of
| tinkering could automatically turn all of a project's bugs into
| features for the foreseeable future.
|
| This would represent a quantum leap in software quality
| assurance.
| zie wrote:
| This is cool, but I'd rather my CI jobs are just k8s or nomad or
| systemd jobs. Then the code we deploy is the same as the code we
| use to build, and it doesn't matter WHAT you do in build land, go
| nuts and do whatever you want.
|
| CI systems will either grow into general purpose code runners or
| they will wither and die.
| nerdbaggy wrote:
| Gitlab can load its ci file from a remote web server. What I have
| done is generate the yaml programmatically and this gets returned
| from the web server. Less than ideal but it at least allows
| dynamic creation
| StavrosK wrote:
| You can just have a CI job generate a CI config file and have
| GitLab CI load that into a child pipeline. Look up dynamic
| pipelines.
| lyjackal wrote:
| Ya, that's all this python ci lib is doing
| nhoughto wrote:
| Reminds me of the original atlassian bamboo pipeline as code impl
| that had you write the pipeline as Java or Groovy, can't remember
| specifically. They got dunked on for it not being a declarative
| file.
|
| Now we've got a pipeline being defined by a declarative file
| being generated by code.
| takeda wrote:
| To me it reminds me of troposphere[1]. It's similarly using an
| imperative language (also python) to generate a declarative
| file (CloudFormation).
|
| [1] https://github.com/cloudtools/troposphere
| delusional wrote:
| We use bamboo at my job, and it's java that generates a yaml
| file that is then shoved into bamboo. It's a completely
| shitshow. why would i use a turing complete language to
| generate a declarative yaml file containing turing complete
| shell scripts? Why not just write the shell script in the first
| place?
| DrSarez wrote:
| That is interesting. In fact I've googled for it and found a
| blog post with really nastay Java code defining a pipeline. Of
| course, in code you can do nasty things and create insane
| pipelines. Thus I think it is good having both - the
| declarative base and a generator on top. The declarative base
| ensures having an easy way to get into pipeline mechanics.
| However if you plan to write really complex pipelines, the
| pipeline-as-code approach jumbs in. At this point your'e firm
| with the basics and know what your'e doing in code.
|
| Amazon is doing the same with the Cloud Development Kit (CDK).
| With CDK you can code your infrastructure in a number of
| languages (java, csharp, .net, python, typescript), which was
| synthesized into cloud formation to be finally deployed. For
| smaller projects and teams not firm with one of those
| languages, plain CFN may be much better. However after learning
| CDK you won't create any infrastructure without it.
| octopoc wrote:
| What I would really like to see is a CI system that lets me write
| a script in a language of my choice instead of defining a
| pipeline config file. That way I can run the pipeline locally,
| put breakpoints in, etc.
|
| Nuke [1] gets close but there are still a lot of tasks that don't
| have C# bindings, such as publishing build artifacts and
| uploading test results.
|
| While I'm dreaming about my perfect CI, I'd also like the ability
| to download benchmark results from previous commits so that I can
| generate trend graphs and publish them in the build results.
|
| To do this right the CI system would have to have an API using
| REST, GraphQL, gRPC or some such API format that generates
| clients in many languages. That way they don't have to maintain
| bindings in every language.
|
| [1] https://nuke.build/
| gravypod wrote:
| I think the main issue is that most build systems are not
| expressive enough to do complex packaging, code gen, etc that
| most projects need. I've been using Bazel for a few
| projects/teams and it's worked really well. Once your build
| system is as expressive as you need you get a lot of freedom.
| Bazel also let's you execute completely local builds and remote
| builds triggered locally. If you set this up debugging CI
| issues is amazingly simple.
| octopoc wrote:
| Totally agree. I've never used Bazel; it sounds a lot like
| Nuke. I had a coworker whose job was to maintain the CI. His
| commit messages would look like this sometimes:
|
| > Fix CI issue with blah blah blah
|
| > Hmm that didn't work lets try something from stackoverflow
|
| > Build fix
|
| > Build fix please work
|
| > Please
|
| > I hate my life
|
| > I am tired and hungry, I want to go home
|
| > Stupid yaml
|
| The CTO would call him like a week later to make sure he was
| okay. This happened often before we switched to Nuke. It
| rarely happened afterwards. It's awesome being able to debug
| your CI locally.
| gravypod wrote:
| Yep, and if you get everything working smoothly enough
| developers can use the same very polished tooling for their
| own workflow.
___________________________________________________________________
(page generated 2021-04-29 23:00 UTC)