[HN Gopher] Show HN: Build for any cloud with the same code
___________________________________________________________________
Show HN: Build for any cloud with the same code
We have been working on Multy, an open-source[1] tool that enables
developers to deploy and switch to any cloud - AWS, Azure and GCP
for now. We realized that, even when using Terraform, writing
infrastructure code is very different for each cloud provider. This
means changing clouds or deploying the same infrastructure in
multiple clouds requires rewriting the same thing multiple times.
And even though most core resources have the same functionality,
developers need to learn a new provider and all its nuances when
choosing a new cloud. This is why we built Multy. Multy is
currently available as a Terraform provider. You can write cloud-
agnostic code and then just choose which cloud you want to deploy
to. Multy will then call the cloud provider APIs on your behalf.
For example, the following Terraform code deploys a virtual network
in AWS and can be easily changed to deploy to Azure or GCP: ```
resource "multy_virtual_network" "vn" { cloud
= "aws" // or azure, or gcp name = "multy_vn"
cidr_block = "10.0.0.0/16" location = "eu_west_1" } ```
Our goal is to expose any configuration that is common across all
clouds, but there's always specific features that are not available
in all of them. For example, if you want a very specific AWS CPU
for your Virtual Machine or use a region that is only available in
GCP. To enable this, we implemented overrides [2] - a way to
configure the underlying infrastructure for cloud-specific
purposes. You can also mix other Terraform code that uses the
cloud-specific providers with Multy. While this makes you somewhat
locked in, having your 80% or 90% of your infrastructure cloud-
agnostic is still very powerful. You can see more complex examples
in our documentation - https://docs.multy.dev/examples/. We're
still in early days and looking for feedback from other developers
on our approach. Let us know what you think! [1]
https://github.com/multycloud/multy [2]
https://docs.multy.dev/overrides
Author : goncalo-r
Score : 42 points
Date : 2022-08-08 12:27 UTC (10 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| actionfromafar wrote:
| It's nice to see a cross platform library for these proprietary
| operating systems. (AWS, Azure and GCP.)
|
| Reminds me of GNU Libc back in the day before Linux when it ran
| on Ultrix, BSD etc.
| ehutch79 wrote:
| Don't most cloud providers have some kind of VM/VPS/whatever,
| that is essentially just a linux instance? It's not hard to just
| script setup and deployment.
|
| Adding layers on top of serverless or whatever just to get back
| to that point seems dumb.
| goncalo-r wrote:
| The cloud environment right now offer so many things that you'd
| be wasting your time recreating everything in plain linux
| instances.
|
| At the end of the day using cloud native managed resources
| offer real advantages from time to market to pricing that we
| believe people should leverage as much as they can.
| Unfortunately it's also how you end up being locked in, so we
| are trying to tackle that.
| YPPH wrote:
| That's kind of trivialising it. It's usually more complicated
| than dropping scripts on instances.
|
| Unless you're doing something very simple, there's a lot of
| wheel-reinvention involved in writing your own scripts when
| there's mature software and services available to manage
| infrastructure at scale.
|
| And what you've said doesn't really address networking, storage
| allocation, and the many other things the cloud provider
| handles, which is not configured on the instance, but rather,
| by vendor-specific APIs.
| joshstrange wrote:
| Has anyone successfully used a tool like this to near-seamlessly
| switch clouds? I'm not trying to be a debbie downer but having
| bought into the Serverless Framework for a project I found that a
| lot of AWS-specific things crawled into the config very quickly
| despite the selling point of "you can switch clouds easily". The
| docs themselves break out GCP/AWS ways to define functions for
| Lambda/Cloud Functions, at that point you are still locking
| yourself in, just with an extra layer to debug. When I get the
| time I'm considering switching to just AWS Cloud Formation
| instead of dealing with a translation layer that translates
| practically nothing. I mean I know there is some syntactic sugar
| that I'll miss or have to recreate but it's not like I could pick
| up my codebase and move to GCP on any reasonable timescale.
|
| Couple all that with the fact that using straight VMs on any
| cloud is one of the more expensive options available to you. As
| in you're better off in most cases using managed services if you
| are looking to save money or make life easier. If all you use is
| EC2/GCP-equivalent then I think in most all cases you'd be better
| off using a different provider (OVH/Hetzner/etc).
| spmurrayzzz wrote:
| It seems like this tool solves the "happy path" insofar as
| folks who use a bunch of managed services that share the same
| API shape between providers. If you just own a bunch of http
| services in load balancers, I bet this works really well. But
| for more exotic deployments (e.g. hybrid cloud & on-prem w/
| DirectConnect/BGP arcana), this is going to fall short.
|
| That of course is the primary issue with any of these multi-
| cloud toolchains, its not feasible for them to understand the
| makeup of every org's cloud infrastructure. The permutations
| are practically infinite. As such, they have to make make
| educated guesses to get a pareto distribution of value. The
| paradox here is that any truly difficult multi-cloud effort is
| difficult solely because of infrastructure complexity, not
| because of API deltas between the providers-- the existing
| abstractions don't improve this at all.
|
| In my experience, you're better off owning your own DSL for
| this sort of thing, but you're going to run into the same issue
| of impedance mismatch when your infra gets more complex. As an
| aside, I always push back on the knee jerk reaction that multi-
| cloud is ever truly necessary (its rarely is, but thats another
| topic altogether).
| count wrote:
| I saw a demo of one years ago that solved this by merely
| overlaying their own SDN/compute/etc. onto VMs in each of the
| supporting underlying clouds.
|
| So you didn't even get the 'least common denominator', you got
| 'tied to one particular startups cloud definition'.
|
| I run away screaming at this stuff, generally.
| goncalo-r wrote:
| I don't believe the goal of Serverless Framework is necessarily
| to tackle vendor lock in, it's more about simplifying and
| somehow unifying the deployment process. Similarly to
| Terraform, it ends up being cloud specific in a lot of the
| details.
|
| I agree that managed services provided by the cloud providers
| allow you to save money and make life easier. Our main goal is
| to enable people to use those that are common across clouds
| without locking themselves in. Of course some features are only
| available in some clouds, and you'll still have to deal with
| data fees if you want to move, but having your configuration
| cloud agnostic saves you a lot of time if you either want to
| move or want to go multi-cloud.
| txutxu wrote:
| Always preferred direct cloudformation, instead of terraform, and
| this was one of the main reasons.
|
| Does it support load balancers? And... block storage? snapshots?
| something like IAM roles and policies?
|
| I can see how to do it with "general" things, like subnets,
| instances... but really curious on how it will manage the real
| and many differences between cloud providers, once you need
| anything else than launch an instance.
|
| Looks promising.
| latchkey wrote:
| I personally think the solution here is to not use terraform.
| You're trying to bolt this onto something that wasn't well
| designed for this in the first place. It is going to create so
| many edge cases that it will be extremely hard for anyone to
| reason about.
| goncalo-r wrote:
| Terraform is quite generic in its nature. At the end of the day
| it's just a state management tool. We decided to start there as
| it's the de facto IaC tool, but we are planning on launching a
| web portal soon and offer other ways to interact with Multy in
| the future such as an SDK and a CLI tool.
| latchkey wrote:
| If TF was generic, your tool wouldn't be needed.
|
| My experience with TF is that it is a terrible SMT because
| they did non-backwards-compatible upgrades to the state
| file... which made it impossible to redeploy and I had to not
| only start over with the state file but also recode a bunch
| of my TF code to support the underlying plugin changes.
|
| Of course this was long ago before it was a final 1.0 version
| so I'm supposed to give some flex there, but it put me off
| badly enough that I'd never consider TF a SMT.
|
| At the end of the day, as long as your tool is on top of TF,
| it will always be a function of the developer to understand
| all of the idiosyncrasies of not only the different cloud
| layers, but also the underlying tool. I think you're trying
| to solve a great problem, but basing it on TF doesn't help.
|
| You asked for feedback... don't use TF.
| NotTameAntelope wrote:
| This just sounds like you don't like TF, and no matter how
| well executed this tool would be, you'd always criticize it
| for using TF.
|
| I wouldn't really call that feedback, just a luddite-esque
| bias on your part.
|
| I'm pattern matching your "feedback" as "New thing scary,
| don't use!"
| latchkey wrote:
| Correct, I don't like TF, based on some real world
| experience using it.
|
| I also think it is fair to say that my complaint is
| valid.
|
| Luddite? The 20k+ servers that I manage disagree with
| you. =)
| NotTameAntelope wrote:
| You even admit you haven't used TF in a long time, and
| when you did use TF it hadn't been 1.0 yet, which means
| you should fully expect breaking changes to take place.
|
| In no way is your complaint valid for a technology that
| hadn't even been released yet, and you should know that.
|
| And a "luddite" is just "a person opposed to new
| technology or ways of working." How many servers you do
| or don't manage is totally irrelevant to that. Terraform
| is new, you dislike it because it's new (my guess),
| therefore you are a luddite.
|
| Did you think I was claiming you belonged to the group of
| 19th century English mill workers who destroyed factory
| machinery to protest those machines taking your job?
| latchkey wrote:
| fool me once, shame on you; fool me twice, shame on me.
| NotTameAntelope wrote:
| Yes shame on you for not realizing a 0.x versioned
| technology would make breaking changes on minor version
| updates.
|
| Given:
|
| 1) Terraform uses Semantic Versioning [0]
|
| 2) Semantic Versioning specifically states that 0.x APIs
| are not stable [1]
|
| Therefore:
|
| Your complaint that a 0.x versioned tool wasn't stable is
| not valid.
|
| Conjecture:
|
| Your complaint exists as an excuse to remain ignorant of
| a new tool or technology simply because it is new. This
| makes you a luddite. [2]
|
| [0] https://www.terraform.io/plugin/sdkv2/best-
| practices/version...
|
| [1] https://semver.org/#spec-item-4
|
| [2] https://www.google.com/search?q=define%3A+luddite
| latchkey wrote:
| Dear anon-fresh-hn-account, it sounds like you enjoy
| terraform. What's your use case?
| NotTameAntelope wrote:
| I don't like or dislike Terraform, you're just not making
| any sense.
| 0xbadcafebee wrote:
| Or at least use CDKTF which allows you to code around
| problems, rather than bash your head against TF's
| awfulness. https://www.terraform.io/cdktf
| latchkey wrote:
| Wow, that's kind of cool. Didn't know about that.
|
| It is me, or is it weird that this has a hard constructor
| dependency on Constructs[1], which is pretty much
| undocumented?
|
| At least in golang, this brings in some other
| dependencies and golang init() reflect code [2], which
| then depends on interacting with javascript!? [3]...
| something smells odd here.
|
| [1] https://github.com/aws/constructs-go
|
| [2] https://github.com/aws/constructs-
| go/blob/main/constructs/co...
|
| [3] https://github.com/aws/jsii/
| 0xbadcafebee wrote:
| _" CDKTF uses Typescript types to generate an assembly
| file that jsii uses to generate the code bindings for the
| other supported languages."_
|
| Yep, sounds like Terraform-level awfulness. Although they
| actually took the design from AWS CDK, so I guess we can
| blame AWS for it.
| solatic wrote:
| I sympathize with what you're trying to do, but I'm not convinced
| by this approach. You're either fully, 100%, completely radically
| anti-vendor-lock-in, or you're fooling yourself.
|
| Either you're doing stuff like:
|
| * You're running your own databases. You use Multy to bring up
| raw virtual machines, you install e.g. PostgreSQL on it, and you
| do all the DBA work yourself. * You're running your own DNS
| infrastructure within the VPC, so that you're not reliant on the
| VPC's provided DNS. * You're running your own network edge. You
| use Multy to bring up a raw virtual machine with a public IP, you
| install e.g. Caddy on it, and you set up the proxy yourself.
|
| Or you end up writing code that implicitly takes advantage of
| stuff like AWS RDS's IAM authentication, or DNS at 10.0.0.2
| pointing to i-0123456789abcdef.us-west-2.compute.internal, or AWS
| Time Sync Service, or using AWS load balancers, with AWS-issued
| TLS certificates, etc.
|
| The simple truth is, there isn't a startup on the planet that
| delivers additional value to customers by mucking around with
| anti-vendor-lock-in measures. Period (and if you really think you
| have a counter-example, I'm VERY curious to hear about it). And
| by the time you've grown to the size where cloud vendor lock-in
| matters (because of cost vs. deploying your own on-premise
| infrastructure), basically, it's too late, and your migration
| project is going to be rather expensive. And Multy doesn't help
| you migrate to on-premise infrastructure, because it only
| abstracts the various cloud providers! So you're in for a massive
| migration project either way.
| 0xbadcafebee wrote:
| I like the idea of it, but there are a hundred things that change
| between different versions of the same provider and resource, and
| lots of quirks, gotchas, limitations, footguns you don't find
| until you run that configuration and then try to modify it. It's
| hard enough making a single cloud provider work reliably.
___________________________________________________________________
(page generated 2022-08-08 23:02 UTC)