[HN Gopher] Show HN: Layerform - Open-source development environ...
___________________________________________________________________
Show HN: Layerform - Open-source development environments using
Terraform files
Hi HN, we're Lucas and Lucas, the authors of Layerform
(https://github.com/ergomake/layerform). Layerform is an open-
source tool for setting up development environments using plain .tf
files. We allow each engineer to create their own "staging"
environment and reuse infrastructure. Whenever engineers run
layerform spawn, we use plain .tf files to give them their own
"staging" environment that looks just like production. Many teams
have a single (or too few) staging environments, which developers
have to queue to use. This is particularly a problem when a system
is large, because then engineers can't run it on their machines and
cannot easily test their changes in a production-like environment.
Often they end up with a cluttered Slack channel in which engineers
wait for their turn to use staging. Sometimes, they don't even have
that clunky channel and end up merging broken code or shipping bugs
to production. Lucas and I decided to solve this because we
previously suffered with shared staging environments. Layerform
gives each developer their own production-like environment.This
eliminates the bottleneck, increasing the number of deploys
engineers make. Additionally, it reduces the amount of bugs and
rework because developers have a production-like environment to
develop and test against. They can just run "layerform spawn" and
get their own staging. We wrap the MPL-licensed Terraform and
allow engineers to encapsulate each part of their infrastructure
into layers. They can then create multiple instances of a
particular layer to create a development environment.The benefit of
using layers instead of raw Terraform modules is that they're much
easier to write and reuse, meaning multiple development
environments can run on top of the same infrastructure.
Layerform's environments are quick and cheap to spin up because
they share core pieces of infrastructure. Additionally, Layerform
can automatically tag components in each layer, making it easier
for FinOps teams to manage costs and do chargebacks. For example:
with Layerform, a product developer can spin up their own lambdas
and pods for staging while still using a shared Kubernetes cluster
and Kafka instance. That way, development environments are quicker
to spin up and cheaper to maintain. Each developer's layer also
gets a tag, meaning FinOps teams know how much each team's
environments cost. For the sake of transparency, the way we intend
to make money is by providing a managed service with governance,
management, and cost-control features, including turning off
environments automatically on inactivity or after business hours.
The Layerform CLI itself will remain free and open (GPL). You can
download the Layerform CLI right now and use it for free.
Currently, all the state, permissions, and layer definitions stay
in your cloud, under your control. After the whole license change
thing, I think it's also worth mentioning we'll be building on top
of the community's fork and will consider adding support for Pulumi
too. We'd love your feedback on our solution to eliminate "the
staging bottleneck". What do you think?
Author : lucas_vieira
Score : 71 points
Date : 2023-08-15 14:15 UTC (8 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| fwungy wrote:
| This solves so many problems. It's great.
| cloverr20 wrote:
| Developers waiting on the dev/sandbox environment for their turn,
| is such a frequent issue within our team specially when the
| sprint is about to end. This solution really looks cool and will
| be trying this out tomorrow.I am sure our cloud team will be very
| much interested in this.
| lucasfcosta wrote:
| Indeed, we've seen that happening very often. I lost count of
| how many times I've seen developers have a Slack or Discord
| channel to coordinate who's gonna using staging/dev and when.
|
| Please let me know if you need any help setting it up. We'd
| love to help in any way we can.
| cloverr20 wrote:
| One point i am curious about is the way you described you
| intend to make money, by providing a managed service. I am
| just thinking that if this service gets really liked by
| people, won't aws/gcp/azure create managed services out of
| this code and then it will be difficult for you to sell your
| services as most companies like to use the services their
| existing infrastructure is already on. Something similar to
| what happened with elasticsearch.
| DishyDev wrote:
| This solves a really tricky Terraform problem I have at the
| moment about wanting to create Dev environments quickly but not
| wanting to recreate DNS, VNets and other core functions for every
| user. Will definitely be having a play with it.
| lucasfcosta wrote:
| Thanks! As one of the co-founders, I really appreciate that you
| think it could solve your problem.
|
| Looking forward to seeing how you'll use it.
|
| Please don't hesitate to reach out to us if we can help in any
| way.
| sberens wrote:
| Did you generate the diagrams in the README.md with some tool?
| lucasfcosta wrote:
| Yes, we used Excalidraw for creating the diagrams in the README
| and Stable Diffusion to generate the "stickers".
| ryanianian wrote:
| FWIW the transparent backgrounds on the diagrams makes the
| diagrams illegible if Dark Mode is enabled in the github UI.
| [deleted]
| lucasfcosta wrote:
| Thanks for flagging it.
|
| I use the light mode myself so I hadn't noticed it.
|
| I'll go ahead and update the text color for the dark theme
| versions.
| btown wrote:
| This looks really cool - and as a side note, your diagrams on
| https://github.com/ergomake/layerform are super clear!
|
| Agree that "preview apps are hard" is a massive bottleneck -
| especially when specs are evolving and a feature may spend some
| time being QA'd by technical and non-technical team members
| before being ready to land. Not to mention that processes for
| securely populating seed data from recent production data are
| highly domain-specific!
|
| We've built our own system for preview apps on k8s, with a
| complex Github action that tries to encapsulate everything we can
| into a set of helm charts that are installed into a dedicated
| namespace for all the databases and services for that specific
| PR.
|
| Of course, this can only go so far, and doesn't easily allow for
| tracking assets outside of k8s. But given that we can keep many
| of those shared, we've in part stayed away from Terraform due to
| adding yet another learning curve - but also for lack of tooling
| for this shared-some-things-but-not-all per-environment
| customizability - the exact problem you're solving!
|
| I'm excited to see where this system goes - and am excited to see
| what value-add layers you build on the UI and governance/cost-
| control side!
| lucasfcosta wrote:
| Thanks! We really appreciate the in-depth feedback.
|
| Indeed, we've seem many people building similar alternatives
| using in-house solutions on top of Kubernetes. In fact, that's
| the main barrier to adoption we've found so far.
|
| The Terraform point is really interesting. In our earlier
| solution we tried to use compose files because we thought
| they'd be easier to implement. In reality what happened was
| that most people couldn't use it because they had more complex
| infra (or at least _wanted_ to have more complex infra) in
| their previews.
|
| Especially after the whole TF situation we'll probably be
| looking into other ways to configure it anyway.
| dlahoda wrote:
| i did terraform, layers, terragrunt. they are option and
| composable.
|
| not clear what layers are layerform are for?
|
| from other side used nix codespaces/shells/rebuilds terranix.
|
| so how you in comparison to these?
___________________________________________________________________
(page generated 2023-08-15 23:00 UTC)