[HN Gopher] Speeding up Azure development by not using Terraform
___________________________________________________________________
Speeding up Azure development by not using Terraform
Author : elisselockhart
Score : 55 points
Date : 2024-03-25 19:35 UTC (3 hours ago)
(HTM) web link (nitric.io)
(TXT) w3m dump (nitric.io)
| wobblyasp wrote:
| It's all fun and games until you want to do something even semi
| non-standard. Nitric even calls this out:
|
| > If you wanted to be able to completely customise your
| infrastructure, using IaC can be practical. However, it is rare
| that an application requires any level of customisation beyond
| the default.
|
| I'm assuming this project is targeting rapid
| prototype/"entrepreneur" market because I can assure you that
| basically every project I've ever worked on at BigTechCorp needed
| some kind of customization beyond the defaults.
| hobs wrote:
| Certainly, but Terraform also sucks at this! I have had all
| sorts of problems with providers that basically lie to your
| face about the guarantees they make, state files always getting
| funky and needing escalation, special things needing to be put
| in special files or "terraform will get mad" (the actual staff
| sre's comments)
|
| It's not a tool I have ever thought was great!
| JohnMakin wrote:
| That's fine but that's not really a fault of terraform
| usually but the provider. Terraform itself is a wrapper
| around the cloud API's the providers use. For your comments
| regarding "state files getting funky" I have also never seen
| this as a problem with terraform itself, but rather people
| doing insanely stupid things with it. "terraform gets mad"
| seems like a constraint of whatever home-grown modules your
| staff SRE is using, not a feature/problem with terraform
| itself.
| aliasxneo wrote:
| Often, I've experienced state file issues as a result of
| some not-so-nice dependencies between AWS services. For
| example, Terraform deletes one thing and then turns around
| and tries to delete another thing, which fails because the
| first thing was deleted (it's been a while, so I don't have
| a detailed example, unfortunately). The result is the state
| file can no longer be reconciled without manual
| intervention.
|
| As you pointed out, though, it's not an issue with
| Terraform, and the above case can be solved by manually
| managing the dependency with lifecycle management. So, I
| agree that most cases are often inflicted by things outside
| of Terraform.
| hobs wrote:
| Well, maybe there's some terraform jedis out there but I
| have seen the same problem at four companies so far, and at
| this point I keep my nose right out of it because its
| always a chore treadmill.
| JohnMakin wrote:
| Much of the consulting work I do surrounds terraform and
| most people are doing it very poorly or incorrectly. I do
| not think this is a fault of terraform. For instance, not
| going all in on IAC and having mixed/poor practice around
| automation and click-ops'ing things for "speed" is a
| great way to have inconsistent terraform states. As far
| as a flat out state corruption, I've only seen that one
| time in ~6 years working with terraform, and it was due
| to the mechanism locking the state file failing during
| simultaneous updates. In today's version I'm not even
| sure how I'd go about corrupting a terraform state file
| sanely. There are a lot more guard rails and QoL stuff
| since the pre-1.0 version days.
| hobs wrote:
| As a database person this pains me to say, but if the
| people keep using the tool wrong you may want to start
| fixing the tool.
| wrnch wrote:
| This tool is going to have the same problems as Terraform
| only worse.
|
| It's wrapping Pulumi and managing the state file for that.
| Yet another abstraction layer on-top of it all.
|
| I'd love to be proven wrong if they can output Pulumi code
| from their application code?
|
| Even then, I'm not sure what the monetization play here is?
| You're standing on the shoulders of Pulumi and you're open-
| source. If you go down the managed route and host the state
| files you'll be in direct competition with Pulumi's cloud
| offering...
| hobs wrote:
| I am not any of those things nor do I know anything about
| their product, just saying that when anyone is crap potting
| for Terraform it just has always sucked in every job I have
| ever worked at.
| icedchai wrote:
| I've seen quite a bit of Terraform code over the past 8
| or 9 years. Most of the problems are related to:
| 1) Rampant copy-and-pasting instead of attempting any
| sort of code reuse 2) Reinventing the wheel poorly:
| instead of bringing in an existing module, they'll
| attempt to build their own half-baked one. 3) Poor
| ops practices like manually editing configuration through
| the console, then complaining about drift when TF
| overwrites their changes. 4) Lack of systems and
| networking fundamentals, like DNS or load balancing. If
| you don't understand it, how can you hope to configure
| something properly?
| tholm wrote:
| Nitric doesn't wrap the state management for Pulumi, we
| actually encourage integrating directly with Pulumi's own
| cloud offerings to manage your infrastructure state.
| https://nitric.io/docs/reference/pulumi
|
| Note this only applies to our OOTB providers which use
| Pulumi as our deployment engine of choice.
|
| When you take away our providers Nitric becomes a
| communication layer between your application and any
| deployment engine that implements the Nitric deployment
| API: https://nitric.io/docs/reference/providers/custom/buil
| ding-c...
|
| This allows collecting a Bill of Materials of cloud
| concepts (e.g. APIs, topics, buckets, queues, policies
| etc.) and providing them however you like with code. This
| could be templating IaC, directly executing IaC, rolling
| your own solution or anything in between.
|
| As for how we make money, currently we're doing consulting
| and dog-fooding our own tooling while working with existing
| teams struggling with more traditional deployment setups to
| make them more productive, their feedback is used to make
| own product even better :).
| jcusch wrote:
| It's a valid point, Nitric isn't a good fit in some cases.
| We've been working on improvements to the underlying provider
| implementation to allow for easier extension for non-standard
| cases. Nitric is a gRPC API under the hood, so alternate
| implementations can be built to satisfy the APIs when
| customization is needed.
| pjmlp wrote:
| Sure, by using their solution instead.
|
| The only real alternative is to use PowerShell or AZ CLI instead.
| k8svet wrote:
| Well, sorta. Azure has Bicep now [1], to compete with Terraform
| and ARM JSON templates. (Separately, there's also the whole
| Dapr thing, what a sight to see that Discord and complete lack
| of interest from anyone outside the company).
|
| [1]: https://learn.microsoft.com/en-us/azure/azure-resource-
| manag...
| nickspag wrote:
| Bicep is fantastic. But it compiles to ARM, so it's still
| limited by ARM's weaknesses and gaps, in addition to the
| underlying general instability of Azure and the
| inconsistencies from the Resource Providers. As well, Bicep
| is declarative- which is elegant in theory, but the stateful
| design of Terraform can cover up some of those underlying
| ARM/Azure gaps in practice. The deployment stacks approach on
| Azure should help further, but there's still a long way to
| go.
| k8svet wrote:
| This just makes it even more asinine that they had to
| invent their own language. I'm sure I'd be just as
| productive with Nix and builtins.toJSON. At least they
| didn't do jsonnet, eh?
| FridgeSeal wrote:
| MS/Azure just seems to have this weird obsession with
| Not-Invented-Here-ing things that didn't need to be nih-
| ed, and their solutions are often...extraordinarily
| Microsoft.
|
| "Use our weird language, to interact with our awful, slow
| API's, nobody else uses it, our support probably doesn't
| know it exists, and it'll be full of weird, MS culture
| specific idioms and eldritch code that compiles down into
| our previous failed experiment"
|
| Like, no? No thank you, can't you just be normal?
| k8svet wrote:
| Do they support ed25519 ssh keys yet? Just hard to take
| 'em seriously.
| pjmlp wrote:
| Sure as the sibling comment replies, depends how much love
| you have to debug the ARM code it generates.
|
| Dapr has some love on the .NET community, and apparently is
| used by Aspire, however I have seen little adoption of it on
| polyglot projects.
|
| When not using terraform, I find the scripting approaches
| easier to follow up.
| throw1234651234 wrote:
| Terraform feels like a useless wrapper on top of AZ CLI and
| resource definition yaml files. In general, Terraform provides
| nothing but a useless wrapper that people need to understand.
|
| Furthermore, most companies don't need K8S either if they are
| using a cloud provider. It only starts being cheaper if you
| forget what you pay the DevOps team (or senior devs who do
| DevOps on top of code) AND have a huge number of resources.
| Otherwise you literally have to be in maybe the top 30
| companies for K8S to actually save you money.
| JohnMakin wrote:
| Sure, you can hide all configuration behind some abstraction, but
| that doesn't really change the fact that the configuration is
| still there. The moment you need to do something even slightly
| different than out-of-the-box, you're hosed. Additionally - IAC
| can work as documentation. "What's running in (environment)?" is
| a question that can usually quickly be answered by looking at the
| IAC itself, whereas this solution, you'd have to go poking around
| in some console and pray to god you're in the right place.
| rk2 wrote:
| The Nitric providers which handle the provisioning piece are
| IaC.
|
| Rather than poking around bespoke IaC on a per-project basis,
| you have the opportunity to inspect modularized and tested IaC
| which in this case is written in Pulumi and maintained by
| either the Nitric team or your team if you need to customize
| and move away from "out-of-box".
|
| I work with the maintaining team, we're happy to discuss more.
| If you're curious then hop into our discord.
| k8svet wrote:
| Why do people do this? I skim the article. I go wait "Nitric
| sounds like Pulumi". A few googles later, I find this :
| https://nitric.io/docs/reference/pulumi
|
| So, it's fast because it uses Pulumi... Does Pulumi no longer
| build on Terraform providers? (EDIT: looking, it seems like many
| Pulumi providers are still terraform wrappers...)
|
| Either way, it feels like this is just a Pulumi feature.
| arcticfox wrote:
| It looks like it's letting you put your application code and
| Pulumi code in one spot. Why...not really sure, outside of a
| very short "Hello World".
| rk2 wrote:
| Not quite,
|
| Nitric infers the necessary cloud runtime by building a spec
| based on the resources you've imported into your application
| code.
|
| At deployment time Pulumi is used to provision the resources
| (because we think it is an awesome tool).
|
| Note that we could have also opted for any other IaC like
| CDKTF, and that you could either modify our providers or
| implement your own providers using your IaC of choice.
| SparkyMcUnicorn wrote:
| Pulumi doesn't rely on Terraform core at all, and Terraform
| providers use a bridge[0].
|
| [0] https://github.com/pulumi/pulumi-terraform-bridge
| lkjaodifywefas wrote:
| No, most of the pulumi providers are wrappers around
| terraform providers.
|
| The exception are _native_ providers which use new stuff like
| AWS Cloud Control API.
| SparkyMcUnicorn wrote:
| I'm well aware of that, and what I said is still true.
|
| You might want to look at the repo I linked, check out the
| source, or hear it directly from the folks at Pulumi if
| you're interested.
| tholm wrote:
| Hopefully this helps give a better idea of how it works under
| the hood:
| https://nitric.io/docs/reference/providers/custom/building-c...
|
| Not just a feature of Pulumi, its just used as the IaC of
| choice for the out of the box deployment providers. This can be
| replaced with anything.
|
| At its core Nitric is really a tool to collect infrastructure
| requirements from an application and those requirements are
| delivered to a server that is built using the Nitric deployment
| APIs.
| junto wrote:
| As a heavy user of Pulumi for C# projects this looks highly
| promising, and the Azure ecosystem has a large number of .NET /
| C# focused developers that are either fighting with bicep or
| terraform (or Pulumi), who would also find this very useful.
|
| Apart from the fact that C# examples seem to be lacking from the
| documentation at the moment, I'd also be interested to know if
| there is, or will be support for managed identities?
|
| Edit. Just found the C# examples. Reading through now.
|
| We are predominantly switching over to managed identities only to
| avoid key rotation hassles, and Entra workload federated
| identities in pipelines to remove those hardcoded identities in
| our pipelines.
|
| I also wonder how you deal with combined function deployments.
| For example we deploy function apps or message consumers that
| process multiple message types as a bounded context, in one
| single app. I assume that Nitric would deploy each function
| endpoint as a separate app container?
|
| Regardless of those niggles, I love the concept of combining the
| code and the infrastructure definitions together.
| jcusch wrote:
| Thanks Junto, for function deployment we use matched files as
| the entrypoint for containers rather than exported functions or
| something similar. This lets you group many message consumers,
| route handlers, etc. together in a single container.
| pepperdetector wrote:
| I'm always hesistant about articles about how a product is good
| written by the company making the product.
|
| However, when it comes to solutions to problems "Separation of
| Concerns" is the best solution. I don't _want_ my application
| code telling azure what resources to make. _I_ want to tell azure
| what resources to make, and I want to be 100% sure that what is
| created is what I want because we 're talking about something
| that _costs my company money to run it_.
|
| While Terraform is a lot of code to get something defined and
| started (and I do think there's room for improvement), the degree
| of what you can define with it and how much you control are the
| key selling points.
| wrnch wrote:
| This is a biased representation of what Nitric can do for you.
|
| The Terraform you've written out looks like a lot, but the Nitric
| code is doing a huge amount of lifting in this example.
|
| Not to mention that Nitric is wrapping a Pulumi provider (granted
| they are using `pulumi-azure-native` unlike the `pulumi-azure`
| provider that's just a Terraform wrapper).
|
| You can do this exact kind of Function Serialization in Pulumi
| anyway: https://www.pulumi.com/docs/concepts/function-
| serialization/....
|
| At the end of the day this article is abstracting a huge amount
| and whilst for startups it might be fine? When something goes
| wrong you'll want to be able to debug the random queue backlog in
| your stack.
|
| For an example of this done beautifully in one language where
| they're managed close the loop with monitoring, I would checkout
| Encore: https://encore.dev/.
| ClassAndBurn wrote:
| Others' concerns are valid; the separation of concerns makes
| infra changes safer and easier to understand. Infra-tooling is
| slow because of the inherent risk of managing stateful services.
|
| Mixing the infra and application logic is the obvious path
| forward, though. Just as most applications don't need more than
| Rails and a single Postgres, most apps don't need customized
| infra. Simplifying the 80% unlocks cycles for more creative work.
| Suppose you are successful enough to have to define custom infra
| at some point. That's good for you. That's a great problem to
| have.
|
| AI is going to standardize architectures going forward. Providing
| a simple set of tools in the same code as the logic makes the
| planner easier for Copilots and reduces the context windows.
| Terraform and application code have implicit dependencies.
| Colocating them lets you define explicit dependencies that are
| more understandable.
| rk2 wrote:
| It is worth noting that the Nitric team (whom I work with)
| aren't suggesting that Nitric is an alternative for every
| Terraform project out there.
|
| However, there are many use-cases _without_ bespoke infra needs
| where it does suit.
|
| Our hope is that people who genuinely are looking for
| assistance with their cloud deployment needs evaluate Nitric
| against their own projects requirements.
| abound wrote:
| I've never used Nitric, but tools of this nature tend to run up
| against the realities of irreducible inherent complexity.
|
| You fundamentally cannot build abstractions that are both easier
| *and* as flexible as the thing you are abstracting away, and so
| tools like this, as other commenters have noted, are great for a
| simple happy path, but that's almost never what you need once
| you're outside the tutorial and building something "real", which
| requires the very flexibility that has been abstracted away.
|
| This is very similar to the argument against (most) ORMs and
| similar tools. For simple, happy path cases, you can avoid
| knowing SQL (or in Nitric's case, infrastructure management,
| platform-specific resources) and use whatever DSL/framework you
| have, but if you want to do something more complicated, you
| either have to hack and kludge your way through with the system,
| or drop down to the lower level and forego the abstraction
| entirely.
|
| As an aside: Azure is a pain to use with Terraform as its API
| endpoints seem to be an order of magnitude slower than the
| equivalent GCP or AWS APIs, so things like `terraform plan` can
| throw you out of flow even on modestly sized projects. Going back
| and forth between Azure and other clouds is night and day.
| smallnix wrote:
| I think it's an a less daunting 'journey' if you don't know
| anything. You start with something friendly and then carve the
| edge cases into the nice scaffolding. E.g. for AWS IaC using
| escape hatches and L1 constructs in CDK over cloudformation.
| BarryMilo wrote:
| I'm not sure if ORM is a great example, as they tend to work
| for the vast majority of use cases. If you're not using an ORM,
| you're either working on something where performance is
| critical, or you're making a mess.
| abound wrote:
| I tend to either use just plain SQL [1] or sqlc [2] [3]. With
| sqlc, you write plain SQL and it generates the (in my case,
| Go) boilerplate bindings for you. I don't find these
| approaches to be a mess, but both linked projects are
| admittedly small. I've used a similar approach on (non-OSS)
| codebases with thousands of DB functions/SQL statements
| though.
|
| I don't disagree with you, ORMs can be a great time-saver for
| your average CRUD app, where the SQL queries tend to be
| straightforward SELECT/UPDATE/DELETEs on primary keys,
| assuming you don't mind the overhead of learning the DSL. But
| as your querying requirements get more complicated, you
| either have to:
|
| - Keep the DB queries simple and do the complicated stuff
| (joining, aggregating, etc) in the business logic, or
|
| - Learn how to express your complicated SQL in the ORM's DSL,
| or
|
| - Drop into the ORM's "raw" mode and write plain SQL
|
| The first option might be fine for non-performance-sensitive
| stuff, but otherwise isn't making good use of the database,
| and the latter two options reveal the SQL behind the
| curtains, at which point the abstraction of the ORM is more
| or less totally broken.
|
| [1] Raw SQL: https://github.com/bcspragu/stronk/blob/main/db/
| sqldb/sqldb....
|
| [2] sqlc: https://github.com/bcspragu/logseq-
| sync/blob/main/db/sqlite/...
|
| [3] https://sqlc.dev/
| WinstonSmith84 wrote:
| Well, to me it is. I've been working on a relatively large
| projects based on Node (Typescript), various Lambdas, some
| doing server stuff only, other serving Client ... It's a
| pretty complex and large application overall and the whole
| architecture was really well designed from the ground up,
| except one thing: typeorm. It was fine at the beginning when
| doing simple things, but it got really tricky after. Then,
| I've had always very complex "migrations" to add sometimes
| just a field. And eventually I've had an issue I logged in
| 2019, and it's not been resolved yet. Well there is no
| solution due to the abstraction layers of typeorm, and I've
| solved (i.e. workarounded) my problem by executing a proper
| SQL query instead ..
|
| I mean there are 2k+ issues open
| https://github.com/typeorm/typeorm/issues/ - yes, not all of
| them are bugs and limitations, but if you filter the bugs,
| that's still more than 1000 ...
| simondotau wrote:
| > You fundamentally cannot build abstractions that are both
| easier _and_ as flexible as the thing you are abstracting away
|
| Easier doesn't have to mean less configuration. It could be
| more sane defaults, better organised abstractions, clearer
| documentation, and/or handier tools.
|
| For the very small environment which I have indirect
| responsibility for, I'd love a simple read-only GUI which could
| present the environment, its sources, and the connections
| between them.
| jcusch wrote:
| I can definitely see your point, but we don't think cloud
| development has hit up agaist the limits of inherent
| complexity. It's still needlessly complex in many cases.
| Projects using Nitric typically write the bulk of the
| application using the abstractions and go direct for anything
| unsupported or specialized.
|
| The point of Nitric isn't to avoid knowing how your
| infrastucture works, it's just to speed up development and
| improve cohesion.
| paulddraper wrote:
| But I love reproducing 1990s network topologies in AWS VPC.
| sevagh wrote:
| After the Terraform relicensing ordeal, do yourself a favor, if
| you're starting greenfield, use your cloud provider's provided
| provisioning tools.
|
| I wish I had CDK/Cloudformation skills, and not Terraform skills.
| Faaak wrote:
| Opentofu is terraform compatible and way better than bicep imho
| thedougd wrote:
| What about outside of AWS? Are folks only using a single
| provider? Our list of providers is long and includes a whole
| ecosystem of stuff from GitHub to Okta and Datadog. Everything
| needed to host and operate an application is in the Terraform
| configuration.
| lijok wrote:
| But Cloudflare, github, hetzner, etc don't provide a
| provisioning tool..
| FridgeSeal wrote:
| Or, don't, because the vast, vast majority of users are almost
| certainly not affected by the licensing change, and your life
| can continue on, literally unchanged.
| kgeist wrote:
| What if something breaks in production and it's not clear what's
| happening because it's hidden somewhere in the application code
| behind layers of magic
| tkiolp4 wrote:
| To be fair, that's happening already.
|
| My stack looks like this:
|
| - framework magic (e.g., spring boot, or ror, or whatever)
|
| - library magic
|
| - terraform magic
|
| - aws magic
|
| - docker magic
|
| - VM magic
|
| - standard magic (OS, networking, decent stuff, etc.)
| jiggawatts wrote:
| I had a laugh at that last bullet point!
|
| I suddenly had a mental image of something like a Discworld
| novel poking fun of corporate America by having a company
| called _Standard Magic Inc_ that commercialises wizardry to
| the chagrin of the University wizards.
| lijok wrote:
| Separation of concerns thrown out the window. Nobody show this to
| Bob Martin.
| tholm wrote:
| One of the co-founders of Nitric here, and I'll bite on this
| one :)
|
| I hear the separation of concerns between IaC and application
| code argument quite a bit, but always see an inherent coupling
| between the two. If changes to your IaC require changes to your
| application code for your application to continue working then
| where is the separation of concerns? If its just deployment and
| runtime, this separation still exists within Nitric:
| https://nitric.io/docs/reference/providers/custom/building-c...
| (this is the same way we build our own deployment providers).
|
| With Nitric the primary goal is to increase cohesion between an
| application and its running infrastructure without increasing
| coupling to that infrastructure. With current IaC application
| and infrastructure cohesion is inherently very low, while
| giving the appearance of low coupling (despite it being quite
| high).
| lijok wrote:
| Application code sets infra requirements, not vice versa. If
| infra shapes your application, it suggests tech limitations
| or significant mistakes.
|
| And I'm sorry, but without some concrete drawings you can't
| just claim current IAC creates low cohesion and strong
| coupling without making people suspect you don't know what
| those terms mean.
|
| What would be very useful is an article covering the points
| of how testing works with Nitric, exactly how it saves you
| time and why the cost/benefit analysis of adopting Nitric
| makes sense.
| tkiolp4 wrote:
| Terraform is probably the most recent tech that I have seen
| playing two different roles in such a short period of time (in
| the last 6 years):
|
| a) guys, look at this! It's called Terraform and you can define
| your infra with HCL and keep it in git. Cool, right? Let's get
| rid of these old Ansible files!
|
| Five years later
|
| b) Guys, we are deprecating Terraform. Please follow this guide
| on how to use CDK/Nitric/etc.
|
| Wow. From hero to zero in such a short time span.
| 0xbadcafebee wrote:
| Most tech goes through the hype cycle in about 5-10 years.
| We're about at the trough of enlightenment with Terraform.
| https://en.wikipedia.org/wiki/Gartner_hype_cycle
| aleksiy123 wrote:
| I'm curious if anyone has experience with writing their own mini
| terraform/pulumi and no persistent state.
|
| What I'm thinking is abstract cloud resource APIs I need into
| unified CRUD interface + has_diff.
|
| Then essentially exposing idempotent functions (ensure_exists,
| ensure_deleted, ensure_updated) based on the unified interface.
|
| No saved state state means slower. But it could be added fairly
| easily.
|
| Something like this for personal use like scripts instead of
| terraform etc..
| jiggawatts wrote:
| Azure Bicep does essentially this, and there are some projects
| in the pipeline to make the language cloud-agnostic and multi-
| vendor.
| oneplane wrote:
| Until one of these advertisements also comes with a provider
| ecosystem and a universal three-way diff every run, it's a bit
| pointless to me. The only thing that does what terraform does is
| OpenTofu, and that's because it's a fork of... terraform.
|
| Maybe if you are small enough to just build a single application
| with some resources in a single vendor's playground all of the
| other IaC variations make sense, but that's hardly the reality.
| Either you're going to be part of something bigger, you're big
| enough to need multiple services, or you're so small that you can
| go straight back to the 90's and clickops+screenshot your way
| through the day.
|
| While there might not be a one-size-fits-all solution, there is
| definitely a cutoff where CDK-style or single-vendor tools just
| can't do what is required.
| knowsuchagency wrote:
| I'm glad someone is doing this. It's a very difficult problem to
| solve, but I see it as similar to JSX in that the business logic
| and the underlying infra are often tightly coupled. There are
| certain kinds of data integration problems where I could see this
| being intuitive and useful.
| jcusch wrote:
| Thanks, that's the way we see it too. We think people mistake
| "separate files" for "separation of concerns", but many
| applications have inherent coupling to their infrastructure.
___________________________________________________________________
(page generated 2024-03-25 23:01 UTC)