[HN Gopher] Launch HN: Gallery (YC S21) - On-demand environments...
___________________________________________________________________
Launch HN: Gallery (YC S21) - On-demand environments on any cloud
provider
Hi HN! We're Jagath and Vignesh, co-founders of Gallery
(usegallery.com). We provision on-demand pre-prod environments on
your own cloud account so you can do manual QA, share feature
previews with team members/clients, and parallelize automated
testing & CI processes, all without bottlenecking on your staging
environment. Engineers typically have a fixed number of cloud
environments, including production, staging, and maybe some
additional pre-production environments. They use these to preview
features, share work with other team members, do manual
QA/automated testing, and more. As the number of engineers grows,
having a fixed number of cloud environments becomes a bottleneck.
Teams end up queuing for access to staging and so on. Creating
additional environments and keeping them in sync, though, is a
major headache, especially when the environments are reasonably
complex. When this problem reaches a boiling point, teams either
have to slow down feature development, test on production, or deal
with building their own on-demand platform internally. That's where
we come in. We enable on-demand spinup of unlimited environments
for quicker development. Engineers use these as environments-per-
feature--in parallel--to share previews with QA/other
engineers/product people, to create demo environments for clients
with features that aren't in production, to run automated testing
on ephemeral environments as part of their CI processes, and more.
Vignesh and I are engineers with experience in infra/research at
Facebook/Apple/Microsoft. We met as roommates at Caltech. We got
into YC with a very different idea (a way for stores to ask
customers questions and offer instant recommendations/promotions
based on their responses) but soon abandoned it (it was more
exciting to us as a technical challenge than to stores as a
practical solution) and found ourselves testing different ideas as
quickly as we could. During this period, right before every launch
without fail, Vignesh and I would collide on staging: features that
worked locally would break on prod infra. We tried to provision our
own individual staging environments, but setting them up and
keeping them in sync sucked up valuable bandwidth. The gold-
standard workflow, in my mind, was the one-click "On-Demand"
environment provisioning I'd had at Facebook, wherein I could click
a button and instantiate a live feature preview. We found solutions
that promised on-demand environment spin-up, but none of them
worked for us; they were either incompatible with our stack (mostly
built around App Engine), didn't interface with our cloud provider,
or required too much overhead and finessing to set up. This was
the seed for our idea. Talking to larger startups, we realized that
the inconveniences we faced were just a few of the major pains
faced by bigger engineering teams with nascent DevOps; they often
had brittle, scattered workflows around managing their environments
and growing queues for features. They needed a solution to flexibly
create, destroy, and replicate environments, through both ad-hoc
and developer triggers like PR's and CI builds. While there are a
number of existing services that aim to simplify DevOps and/or
provide easier workflows around environments, engineering teams we
spoke to were either unable or unwilling to use them due to
incompatibility with their specific setups. We realized that a
better approach would be to abstract away the specifics of each
environment by using infra-as-code as middleware. Infra-as-code
solutions like Terraform, CloudFormation, etc. can represent
infrastructure thoroughly in a standardized fashion. The majority
of people we talked to were maintaining their infrastructure using
Terraform or something similar, and it was easy to build out
pipelines for those who didn't by using Terraform under the hood.
This was the key to what became Gallery. Since every company
manages their environments differently (cloud provider, use of K8s
or infra-as-code, cloud-managed services, security/privacy layers),
creating a product that can work with them all out-of-the-box is a
challenge. Unlike other solutions on the market, Gallery isn't a
Kubernetes orchestrator that replicates containerized environments;
rather, we can represent entire cloud projects, including managed
services (e.g. App Engine, Elastic Beanstalk, etc.), spin them up
in sandboxed cloud projects, and automate their teardown as well.
To create cloud resources on your GCP/AWS accounts, we use
Terraform as a middleware. For select cloud services, we can
understand an existing project, generate the Terraform
corresponding to it, and use it to spin up your resources. We allow
users to link their own Terraform code to allow spin-up of more
complex environments as well. When you trigger the creation of a
new environment, we spin up a worker that applies the Terraform
template and performs any post-deploy actions to seed the newly
generated infrastructure (for example, copying over initialization
data into databases). We store your Terraform state file and manage
tearing down environments when they outlive their TTL, or deletion
is triggered by a specific action (e.g. merging a pull request).
Once we spin up infrastructure, we pull application code from your
app repositories, and allow you to specify build and deployment
commands that target the newly created environment. This lets you
use your current deployment scripts on Gallery almost as-is, for
fast and straightforward onboarding. We have integrations with
Github/Gitlab around the Pull Request/Merge Request flow, letting
you create environments that track a branch whenever a PR is
created. We're a SaaS product, and we have pricing tiers based on
the number of concurrent environments. We've prepared a live demo
account that you can play around with as a read-only team member
here: http://a.getgallery.co/teams/14. I've also prepared a demo
video of how to set up ephemeral environments per-PR in just a
couple of minutes
(https://www.loom.com/share/26165ea69f0d4b7b974019bdf72e5d11). You
can see our docs here: docs.usegallery.com. Thank you so much for
reading! We'd love to hear your thoughts, ideas, and experiences
around the problem we're tackling and the solution we're proposing!
Author : jag729
Score : 36 points
Date : 2021-11-09 17:28 UTC (5 hours ago)
| josh_thurman wrote:
| Hi Jagath and Vignesh,
|
| What you're proposing here sounds like it falls under the
| Continuous Previews umbrella that we've been collaborating with
| others to define and refine via the Continuous Previews
| Manifesto.
|
| Awesome to see others finding the same value proposition in the
| process. We think there's space for several interpretations of CP
| and we see value in the implementation you're proposing.
|
| If you're interested in collaborating please reach out.
| josh_thurman wrote:
| Reference CP Manifesto https://cpmanifesto.org/
| easton wrote:
| I've seen a couple of these, and I think much of the problem is
| that I need to duplicate my ops work (scripts to build
| environments, etc.) somehow for the ephemeral environments to be
| created. You've evaded that somewhat by using Terraform, but what
| if I use CloudFormation to deploy my stuff?
|
| I'm wondering if the better way to solve this problem would be to
| use docker-compose files (or k8s definitions), since we all have
| them checked in already for use with local development. I really
| just need a way to fire the local environment up on a server so I
| can send someone a link to it that will work for a while (and not
| die when I close my laptop). I don't need the full onslaught of
| AWS (or GCP services) when I just want to send someone a link and
| ask them if this CSS looks right (or put the link in a PR), but I
| need it to work better than a picture.
| jag729 wrote:
| Thanks for your feedback! We've weighed these considerations
| quite a bit in choosing the right first step for the product.
|
| The docker-compose approach is a valid angle -- we've seen a
| couple products built around that workflow. A weakness that led
| us to our approach is that they generally lack support for
| projects that involve cloud-specific services. Most of the
| startups we've talked to already have some infra-as-code that
| describes their cloud infrastructure, and we're rolling out
| support for CloudFormation/Pulumi very soon, so customers will
| be able to bring over infrastructure in any major format. As
| far as limiting the breadth of services spun up, we've found
| that removing unnecessary services from the IaC config provides
| reasonably good results. We plan on adding support for a
| Docker-compose approach soon, but we felt that starting with
| IaC would bring us better generalizability out of the gate.
| ramish94 wrote:
| Terraform (as you may well have already found out) is a blessing
| and a curse. It's great for instantiating infrastructure quickly
| and repeatedly, but what it's not great for is managing it going
| forward and streaming changes (it seems to break A LOT when there
| is drift from managed infrastructure vs. real life
| infrastructure).
|
| Do you guys plan to handle managing instances created via
| terraform code via Gallery, or is it strictly a tool to
| instantiate environments and add triggers around it?
|
| Like I'm thinking of cases where you might want to edit your
| existing env created by Gallery with other stuff that other teams
| may have worked on and merged onto prod. If I edit the config,
| there could be a lot breaking changes
| jag729 wrote:
| Absolutely - managing and syncing the state of IaC tools is
| core to the value of Gallery. Infrastructure templates can be
| edited without changing currently-active environments, and this
| is handled gracefully right now. Propagating template updates
| to running environments seamlessly is also on our roadmap.
| ramish94 wrote:
| Thanks for the reply. It certainly will be an interesting
| technical challenge to try and propagate changes efficiently.
| Speaking from experience (I've worked with Terraform a lot
| the past 2 years), it can be a tough nut to crack, so good
| luck!
|
| I would add one more thing (and this is maybe for down the
| line): There's a use case for duplicating envs beyond just
| development environments. There's a lot of value in cases
| where the SaaS product itself requires on-demand environment
| generation. Where customers of it need staging instances so
| they can see what config changes are before they merge to
| production. Obviously in that case, you're treating an actual
| product like a terraform provider, and the API's of that
| product as resources. I would explore that down the line
| because there's a lot of value in a service like that.
|
| FYI, I would love to get involved any way I can to help you
| guys grow this. Let me know how I can help!
| mailto:ramish@ualberta.ca if you're interested :)
| bberenberg wrote:
| So if this requires a demo to use, are you writing the TF code
| manually?
| jag729 wrote:
| We have reverse terraform support for some services (for
| example Cloud Run, multiple firebase services, etc.), where we
| automatically generate the Terraform (if you use services that
| are not supported, we do offer to manually create the terraform
| infrastructure as part of the onboarding process).
|
| The reason we have a read-only demo is because of the friction
| required to set up an account (you have to link your cloud
| account and repositories); we felt that the read-only account
| was the easiest way to take a look at the features without
| needing to go through the account creation process.
| vincentmarle wrote:
| > We realized that a better approach would be to abstract away
| the specifics of each environment by using infra-as-code as
| middleware. Infra-as-code solutions like Terraform,
| CloudFormation, etc. can represent infrastructure thoroughly in a
| standardized fashion. The majority of people we talked to were
| maintaining their infrastructure using Terraform or something
| similar, and it was easy to build out pipelines for those who
| didn't by using Terraform under the hood. This was the key to
| what became Gallery.
|
| The problem you're trying to solve definitely exists, don't get
| me wrong; but the level of effort required to integrate with your
| service sounds more like a high-touch consultant-level engagement
| (definitely sounds enterprise-y and expensive). I'm not sure how
| you can scale this as a self-serve SaaS business.
| jag729 wrote:
| You're right; distribution is one of the biggest challenges for
| a service like this. There's a bit of a tradeoff in my eyes;
| you can either support a few setups with seamless self-service,
| or you can support everyone but on a case-by-case basis. We
| think that the IaC approach bridges the two; our goal at the
| moment isn't to rush to fully hands-free self-service, but
| rather to optimize for the range of configurations we're
| compatible with and how seamlessly we can support clients on
| those stacks.
|
| Using IaC helps a lot with the former, and getting experience
| onboarding different clients has helped with the latter - we're
| now familiar with common hangups (e.g. figuring out the data
| layer), and we've built processes into the product to get
| around those (e.g. immediately populating a test DB from a
| specified staging DB). We've seen progress in how quickly we
| can get people onto the platform; it now generally only takes a
| day, and we're bringing _full_ self-service in sight as that
| duration decreases.
|
| That being said, our broader goal is to hit the larger
| enterprise market, for which (as you mentioned) a higher-touch
| approach is warranted.
| josh_thurman wrote:
| self-removed
| dang wrote:
| I get how tempting it is to hijack someone else's launch
| thread, especially when you've been working in the same space
| for a long time, but you're overdoing it. When I looked in on
| this thread, 3 of 7 comments were yours.
|
| It's maybe fine to post one comment about your competing
| service--maybe. (I'm not even sure about that, to be honest,
| but people do it.) Pushing the envelope beyond that is too
| much.
|
| Btw, this applies to all startup launches, not just YC
| companies, and we moderate other threads exactly the same way
| when we see this dynamic. From my perspective it's about the
| kind of community we want HN to be.
|
| Non-launch threads are different of course. I'm sure your work
| is very good and you're completely welcome to participate on
| HN.
|
| I've detached this comment from
| https://news.ycombinator.com/item?id=29166416.
| josh_thurman wrote:
| Got it. Not my intent to hijack this. I'll delete.
|
| I'm trying to share that much of what is described in the
| post and discussion is currently being formalized - and there
| are different interpretations and technical solutions that
| various teams including what gallery is proposing.
| nrmitchi wrote:
| Sorry, are you part of Gallery, or are you answering questions
| on this Launch from the perspective of a separate, but similar,
| company?
___________________________________________________________________
(page generated 2021-11-09 23:00 UTC)