https://github.com/ergomake/layerform Skip to content Toggle navigation Sign up * Product + Actions Automate any workflow + Packages Host and manage packages + Security Find and fix vulnerabilities + Codespaces Instant dev environments + Copilot Write better code with AI + Code review Manage code changes + Issues Plan and track work + Discussions Collaborate outside of code Explore + All features + Documentation + GitHub Skills + Blog * Solutions For + Enterprise + Teams + Startups + Education By Solution + CI/CD & Automation + DevOps + DevSecOps Resources + Customer Stories + White papers, Ebooks, Webinars + Partners * Open Source + GitHub Sponsors Fund open source developers + The ReadME Project GitHub community articles Repositories + Topics + Trending + Collections * Pricing Search or jump to... Search code, repositories, users, issues, pull requests... Search [ ] Clear Search syntax tips Provide feedback We read every piece of feedback, and take your input very seriously. [ ] [ ] Include my email address so I can be contacted Cancel Submit feedback Saved searches Use saved searches to filter your results more quickly Name [ ] Query [ ] To see all available qualifiers, see our documentation. Cancel Create saved search Sign in Sign up You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. {{ message }} ergomake / layerform Public * Notifications * Fork 2 * Star 185 Layerform helps engineers create reusable environment stacks using plain .tf files. Ideal for multiple "staging" environments. layerform.dev License GPL-3.0 license 185 stars 2 forks Activity Star Notifications * Code * Issues 5 * Pull requests 0 * Discussions * Actions * Projects 0 * Security * Insights More * Code * Issues * Pull requests * Discussions * Actions * Projects * Security * Insights ergomake/layerform This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. main Switch branches/tags [ ] Branches Tags Could not load branches Nothing to show {{ refName }} default View all branches Could not load tags Nothing to show {{ refName }} default View all tags Name already in use A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch? Cancel Create 1 branch 0 tags Code * Local * Codespaces * Clone HTTPS GitHub CLI [https://github.com/e] Use Git or checkout with SVN using the web URL. [gh repo clone ergoma] Work fast with our official CLI. Learn more about the CLI. * Open with GitHub Desktop * Download ZIP Sign In Required Please sign in to use Codespaces. Launching GitHub Desktop If nothing happens, download GitHub Desktop and try again. Launching GitHub Desktop If nothing happens, download GitHub Desktop and try again. Launching Xcode If nothing happens, download Xcode and try again. Launching Visual Studio Code Your codespace will open once ready. There was a problem preparing your codespace, please try again. Latest commit @vieiralucas vieiralucas Merge pull request #21 from ergomake/glob ... 41df022 Aug 15, 2023 Merge pull request #21 from ergomake/glob implement support for glob patterns in layer definitions 41df022 Git stats * 83 commits Files Permalink Failed to load latest commit information. Type Name Latest commit message Commit time .github/workflows add gh workflows August 9, 2023 22:50 assets fix README img for all components August 9, 2023 18:55 cmd/cli add help descriptions August 9, 2023 22:50 examples/beats/terraform implement support for glob patterns in layer definitions August 15, 2023 09:40 internal implement support for glob patterns in layer definitions August 15, 2023 09:40 mocks/internal_ introduce s3 as an option for context August 9, 2023 22:50 .gitignore forget about layer clients for now just reload layers from disk August 9, 2023 22:50 LICENSE Initial commit July 29, 2023 13:29 Makefile forget about layer clients for now just reload layers from disk August 9, 2023 22:50 README.md tag resources with layerform_layer_name and layerform_layer_instance August 15, 2023 07:33 go.mod tag resources with layerform_layer_name and layerform_layer_instance August 15, 2023 07:33 go.sum tag resources with layerform_layer_name and layerform_layer_instance August 15, 2023 07:33 main.go use cobra August 9, 2023 22:50 View code [ ] Layerform Home Page | Documentation | Discord What is Layerform? Why use Layerform Cheaper and quicker development infrastructure It's just like production, every time You own the infrastructure and permissions Encapsulation / Isolation of concerns Cost attribution and cost savings Getting started Layer immutability and layer rebasing How Layerform works Layerform design philosophy Issues & Support Other channels License README.md layerform logo Layerform Layerform helps engineers create reusable environment stacks using plain `.tf` files. Home Page | Documentation | Discord Layerform is released under the GNU GPLv3 license. Discord Chat Follow @GetErgomake After HashiCorp's announcement that they're not using the MPL license anymore, we'll be building on top of a community-supported fork. What is Layerform? Layerform is a Terraform wrapper that helps engineers build reusable infrastructure using plain Terraform files. To enable reuse, Layerform introduces the concept of layers. Each layer contains some infrastructure and can be stacked up on top of another layer. In addition to being much easier to use, Layerform allows teams to reuse core-pieces of infrastructure. That way, development infrastructure is much cheaper and quicker to spin up. With Layerform, Engineers only spawn the infrastructure layers they need. [dev-environments] For those wanting development environments: we don't want to run your text-editor. Layerform is the standard tool for development infrastructure. You can keep using your text-editors, IDEs, and other local development directly on your machine. Why use Layerform [crocodile-spawn] Cheaper and quicker development infrastructure Layerform is much cheaper and quicker than spinning up an entire staging environment for each engineer. When using Layerform, engineers can share the same core-pieces of infrastructure, and only spin up the layers they need on top of it. For example, if you run applications in a Kubernetes cluster, you don't have to create a brand new cluster for each engineer's environment. Instead, you can reuse the same cluster and have multiple engineers spin up their applications on top of it. It's just like production, every time Layerform's environments are just like production because they are spun up from plain .tf files. Whatever you can set up using a .tf file, you can set up in a Layerform layer. That way, your development infrastructure will be just like production, including Lambdas, DynamoDB instances, and whatever else that you need. You own the infrastructure and permissions Layerform runs in your infrastructure. Layerform will store state and spin up resources in your cloud providers. Therefore, you have full control over who gets access to what, and what resources are currently running, and how much they cost. Encapsulation / Isolation of concerns By breaking infrastructure into layers, your organization can define clearer boundaries between teams. Consequently, it will be easier to mirror your organization's structure into your system's structure. [layers-vs-org] Cost attribution and cost savings In addition to saving costs by reusing infrastructure, Layerform allows you to automatically track costs for each layer instance. When applying layers, Layerform will automatically tag the resources it creates with the actual name assigned to the layer instance. If you have production and development base layers, for example, each of those two will contain the tags layerform_layer_name and layerform_layer_instance with their respective names. That way, Layerform can recursively traverse layers' resources to collect cost management information. Consequently, it will be able to tell the cost of your whole production and development layers, as well as an aggregate cost report of everything on top of those layers. Getting started First, install the Layerform CLI. $ go install github.com/ergomake/layerform@latest Then, create the Terraform files you'll use to create each layer of infrastructure. In the example below, we have two layers: eks, which is the "base" layer, and services. layerform/ +- services/ | +- pods.tf | +- outputs.tf | +- inputs.tf +- eks/ | +- eks.tf | +- vpc.tf | +- inputs.tf | +- outputs.tf +- services.tf +- eks.tf Once you have your infrastructure defined as code, you'll create the layer definitions that the CLI will use when spawning instances of each layer. { "layers": [ { "name": "base", "files": ["./layerform/eks.tf", "./layerform/eks/**"] }, { "name": "services", "files": ["./layerform/services.tf", "./layerform/services/**"], "dependencies": ["base"] } ] } Now, configure the place in which the generated layer definitions will be saved. # In config inside ~/.layerform currentContext: remote-context contexts: remote-context: type: s3 bucket: layerform-bucket-example Finally, you should provision S3 with your layer definitions using layerform configure. The Layerform CLI will then take care of creating unique IDs for each layer and sending the Terraform files' contents to the Layerform back-end, which, in this case, is an S3 bucket. After provisioning layer definitions, you can use layerform spawn to create an instance of that particular layer. $ layerform spawn services my-dev-infra Each instance of a layer contains all the pieces of infrastructure defined within that layer's files. [default-base-layer] In this example, running that command will cause Layerform to also create an instance of the underlying eks layer and assign it the ID default. To spawn yet another services layer, just run layerform spawn services another-dev-infra. By default, Layerform will try to use underlying layers whose ID is default as base layers. [multiple-top-layers] As a general rule, underlying layers are always the ones whose ID is default. To specify the desired ID for each underlying layer, you'll have to use the --base parameter. For example: # Creates: # 1. An eks layer with ID "one" # 2. A services layer with ID "two" $ layerform spawn services two --base "eks=one" [one-two-la] Layer immutability and layer rebasing A layer can only mutate itself or the layers above. For example, if you have a base layer and a backend layer, the backend layer's Terraform files will not be able to mutate any infrastructure in a base layer instance. Still, the base layer files can mutate any instances of the layers above it. The way Layerform prevents undesirable mutations is by analyzing each terraform plan and detecting whether any mutation's target belongs to an underlying layer. The reason Layerform prevents a layer from mutating its underlying layer is to avoid breaking sibling pieces of infrastructure. This design allows for platform teams to "rebase" layer instances on top of theirs. For example, assume you have multiple application layers on top of a Kubernetes cluster belonging to a base layer. In that case, if the platform team wants to update the Kubernetes version and needs to patch existing application's manifests, they can do so from their own layer by referencing and patching other Terraform resources. On the other hand, product engineers on the layers above cannot modify the base layer containing the Kubernetes cluster. Otherwise, they could break everyone else's applications. In addition to preventing failures, immutability defines clearer communication interfaces between teams and helps organizations avoid lots of lateral channels. How Layerform works Layerform has three major components. The layerform-provider, the Layerform Back-end, and Layerform CLI. For now, we're only using the JSON file for configurations. The provider will be made publicly available soon. [all-components] The layerform-provider is used by Terraform to provision the Layerform Back-end with all the metadata for each layer, like its name and dependencies, and all the Terraform files associated with that layer. The Layerform Back-end stores the data for each layer definition and stores the state for each instance of each layer so that new layers know which base state to use. There can be multiple types of back-ends. The most common types of back-end are local, for storing data locally, and s3, for storing data on the cloud, in an S3 bucket. Finally, the Layerform CLI talks to the Layerform Back-end to fetch the files for the layer it wants to apply, and the state for the underlying layer. The way the Layerform CLI creates new layers on top of the correct existing layers is by injecting the underlying layer's state when applying each layer. Layerform design philosophy [bear-spawn] Our main goal with Layerform was to make it as easy as possible for engineers to create and share different parts of their infrastructure. That way, we'd empower teams to create their own environments without burdening their organization with unnecessary costs or complex configuration files. When developing Layerform, we also determined it should support virtually any type of infrastructure, including infrastructure for serverless applications. That's why we decided to create a wrapper on top of a comunity fork of Terraform, which supports Kubernetes/Helm, and already has established providers for all major public clouds. Third, we decided Layerform should be simple and intuitive. Engineers shouldn't have to learn new proprietary languages or configuration formats to use Layerform. Whenever possible, we should allow them to reuse their existing configurations. Layerform concepts are the only thing engineers will need to learn about. Everything else should be "just Terraform". Finally, we decided Layerform needs to be open and free. It's for that reason we're using a GPL license for the wrapper and using a community-maintained fork. That's also why you don't necessarily need to pay for anything before you can extract value from Layerform. For the sake of transparency, the way we intend to make money in the future is by providing a managed service with governance, management, and cost-control features. If you wish to bundle the Layerform wrapper itself, the GPL license will take care of ensuring ourselves and the community gets value back. Issues & Support You can find Layerform's users and maintainers in GitHub Discussions. There you can ask how to set up Layerform, ask us about the roadmap, and discuss any other related topics. You can also reach us directly (and more quickly) on our Discord server. Other channels * Issue Tracker * Twitter * LinkedIn * Ergomake Engineering Blog License Layerform is open-source. Licensed under the GNU GPLv3 License. [lgtm-gnu] Layerform is not associated with Terraform. Terraform is a registered trademark of HashiCorp, Inc. About Layerform helps engineers create reusable environment stacks using plain .tf files. Ideal for multiple "staging" environments. layerform.dev Topics devops terraform developer-tools sre dev-environment platform-engineering Resources Readme License GPL-3.0 license Activity Stars 185 stars Watchers 3 watching Forks 2 forks Report repository Releases No releases published Packages 0 No packages published Contributors 2 * @lucasfcosta lucasfcosta Lucas F. da Costa * @vieiralucas vieiralucas Lucas Vieira Languages * Go 98.8% * Makefile 1.2% Footer (c) 2023 GitHub, Inc. Footer navigation * Terms * Privacy * Security * Status * Docs * Contact GitHub * Pricing * API * Training * Blog * About You can't perform that action at this time.