[HN Gopher] Launch HN: Release (YC W20) - Orchestrate AI Infrast...
___________________________________________________________________
Launch HN: Release (YC W20) - Orchestrate AI Infrastructure and
Applications
Hello Hacker News! We're Erik, Tommy, and David, the founders of
Release (https://release.ai/). We launched on HN in 2020
(https://news.ycombinator.com/item?id=22486031) after leaving
TrueCar, where we managed a 300 person development team. Our
original focus was making staging environments easier with
ephemeral environments, but along the way AI applications started
to emerge as an important and critical component of distributed
applications. As we talked to customers using our original product,
we realized we had built the underlying platform needed to address
the needs of orchestrating AI applications and infrastructure. So
here we are and we're excited to share Release.ai with HN. Here's
a video showcasing the platform and demonstrating how to easily
manage new data and changes using the RAG stack of your choice:
https://www.youtube.com/watch?v=-OdWRxMX1iA If you want to try
release.ai out, we're offering a sandbox account with limited free
GPU cycles so you can play around and get a feel for Release.ai:
https://release.ai. We suggest playing around with some of the RAG
AI templates and adding custom workflows like in the demo video.
The sandbox comes with 5 free compute hours on an Amazon g5.2xlarge
instance (A10 with 24GB VRAM, 8vCPUs and 32GB). You will also get
16 GB and 4vCPUs for cpu workloads such as web servers. You will be
able to run an inference engine plus things like an api server,
etc. After the sandbox expires, you can switch to our free plan,
which requires a credit card and associating an AWS/GCP account
with Release to manage the compute in your cloud account. The free
account provides 100 free managed environment hours a month. If you
never go over, you never pay us anything. If you do, our pricing is
here: https://release.com/pricing. For those that like to read
more, here's the deeper background. It's clear that open source AI
and AI privacy are going to be big. Yes, many developers are going
to choose SaaS offerings like OpenAI to build their AI
applications, but as open source frameworks and models improve,
we're seeing a shift to open source running on cloud. Security and
privacy is a top concern of companies leveraging these SaaS
solutions, which forces them to look at running infrastructure
themselves. That's where we hope to come in: we've built Release.ai
so all your data, models and infrastructure stay in your cloud
account and open source frameworks are first class citizens.
Orchestration - Integrating AI applications into a software
development workflow and orchestrating their lifecycle is a new and
different challenge than traditional web application development.
Release also makes it possible to manage and integrate your web and
AI apps using a single application and methodology. To make
orchestrating AI applications easier, we built a workflow engine
that can create the complex workflows that AI applications require.
For example, you can automate the redeployment of an AI inference
server easily when underlying data changes using webhooks and our
workflow engine. Cost and expertise - Managing and scaling the
hardware required to run AI workloads is hard and can be incredibly
expensive. Release.ai lets you manage GPU compute resources across
multiple clouds with different instance/node groups for various
jobs within a single admin interface. We use K8s under the covers
to pull this off. With over 5 years of building and running K8s
infrastructure our customers have told us this is how it should be
done. Getting started with AI frameworks is time consuming and
requires some pretty in-depth expertise. We built out a library of
AI templates (https://docs.release.com/release.ai/release.ai-
templates) using our Application Template format (which is kind of
a super docker-compose: https://docs.release.com/reference-
documentation/application...) for common open source frameworks to
make it easy to get started developing AI applications. Setting up
and getting these frameworks running is a hassle, so we made it one
click to launch and deploy. We currently have over 20 templates
including temples for RAG applications, fine tuning and useful
tools like Juypter notebooks, Promptfoo, etc. We worked closely
with Docker and Nvidia to support their frameworks: GenAI and
Nvidia NEMO/Nims. We plan to launch community templates soon after
launch. If you have suggestions for more templates we should
support, please let us know in the comments. We're thrilled to
share Release.ai with you and would love to get your feedback. We
hope you'll try it out, and please let us know what you think!
Author : tommy_mcclung
Score : 47 points
Date : 2024-08-07 14:50 UTC (8 hours ago)
| the_pascal wrote:
| How does this compare to managed offerings like Google Gemini and
| AWS Bedrock? Thanks in advance and congratulations on the new
| product!!
| erik_landerholm wrote:
| Release.ai gives you complete control over your infrastructure
| and allows you to create the workflows that are best for your
| particular problem. You can go as deep as you would like using
| the open source AI tools available to you without restrictions.
|
| Release.ai is much cheaper than those options even with our
| pricing on top of your cloud costs.
|
| We are a single pane of glass no matter where your k8s clusters
| are running. You don't need to learn bedrock, sagemaker,
| gemini, etc in order to use gpus in different clouds.
|
| Release also makes it very easy to create complete applications
| with web services and ai services together. No other platform
| allows you to do both easily.
|
| We are a complete platform where you have total control over
| the software you select and how it needs to be integrated in
| your development processes.
|
| Thanks for the kind words!
| sidcool wrote:
| Congrats on launching. Interesting pivot.
| erik_landerholm wrote:
| Thank you! We believe our standard product and Release.ai work
| really well together!
| billclerico wrote:
| congrats!!
| erik_landerholm wrote:
| Thanks!
| mchiang wrote:
| This is cool. I'd like to give it a try. Press a button, and get
| GPU access to build apps on.
| erik_landerholm wrote:
| Thank you! Let us know how you get on with it!
| jakubmazanec wrote:
| Why do you have such generic name? It will make searching so much
| harder.
| tommy_mcclung wrote:
| It's funny you ask this because this question has come up a
| bunch of times. It just maps to our mission and we decided we
| really wanted to be true to it. Our mission is to help
| people/companies release their ideas to the world quickly. So
| it just fit and we decided it was worth the tradeoff for it to
| be a little harder to search for. In the end if we're
| successful it will be because we built something people want
| (i.e. apple.com) vs the name we chose.
| mcsplat2 wrote:
| How do you hook up data to an environment? And what data sources
| do you support (Snowflake/etc?)
| tommy_mcclung wrote:
| We have a way to connect data to an environment through
| workspaces. The docs are here:
| https://docs.release.com/reference-documentation/application...
|
| Right now we support s3 and pulling in full github repos.
| Snowflake is on our roadmap and has been requested by some of
| our customers already.
| BurritoKing wrote:
| This looks awesome, getting started with AI development is
| daunting and I really like how this focuses on integrating with a
| bunch of open source frameworks and then deploying them into your
| own cloud (I always prefer to run the infrastructure, it feels
| weird to rely on something that's a complete black box).
|
| The sandbox environment with free GPU hours is a cool way to try
| things out without a big commitment too. It's nice seeing a
| product that genuinely seems to address the practical challenges
| of AI deployment. Looking forward to seeing how the platform
| develops!
| erik_landerholm wrote:
| Thanks! Hopefully, the sandbox allows people to try out some
| things and see how Release works. Release become most powerful
| when you are deploying into your own infra and mixing and
| matching your web apps and AI apps.
| bluelightning2k wrote:
| Seems pretty vague. Something to do with half self-hosting open
| source LLMs with a proprietary docker-like template thing.
|
| Am I on the right track?
| erik_landerholm wrote:
| Not sure I would describe it that way exactly, :). The
| templates are to get your started. Release itself takes docker
| compose, docker files, helm, terraform, etc and gives you a way
| to organize them and a workflow engine to deploy, update and
| remove every part of the environment. All of the functionality
| works with the AI templates.
|
| The templates are a good place to start and with our concept of
| workspaces, you an easily attach data sources and do
| development with common ai tools and frameworks.
|
| A lot of our customers and others we have talked to don't even
| know where to start with AI, including RAG. Our templates,
| infra management and workflow engine make it easy to spin up
| multiple environments, with varying configs for testing all the
| way to production.
|
| While a lot of companies are getting into the RAG framework or
| Fine Tuning framework business, by releasing open source
| project, it is still very difficult to tie it all together.
|
| Both nvidia and Docker have lamented about this issue to us and
| this is our attempt to make some of these frameworks easier to
| use.
| bradhe wrote:
| Like Together.ai but a bit late to the market I suppose.
| todd3834 wrote:
| This is very cool! I love seeing tooling targeting inference. I
| feel like stable diffusion and LLAMA have to be the primary use
| cases for these types of services. DALL-E is super lacking and
| GPT does actually start to get pretty expensive once you are
| using it in production.
| michaelmior wrote:
| This looks cool, but I'm a little confused about the pricing
| model. It sounds like I'm paying you for every hour my jobs are
| running on my own infrastructure if I'm reading it right. That
| seems like a really odd way to price things if true.
| tommy_mcclung wrote:
| Yeah, I hear you and the confusion. We decided on this as a
| management fee that only gets charged when you're using the
| environment. It was the best tradeoff we could come up with but
| it definitely isn't perfect. Any ideas on how we can make this
| clearer or better?
| richardw wrote:
| It's not a unique model - that's what e.g. Databricks does too.
| You pay for the resources, which means you can deploy them with
| whatever security model you require (good for enterprise, with
| all the firewalls and secrets and security reviews your heart
| desires), but you also pay for the management layer that
| Databricks offers. In Databricks' case you're basically
| doubling the infrastructure cost per hour.
|
| Data and compute stay in your own tenant.
|
| Edit: confirmed - look at that enterprise tier. If you want SSO
| & RBAC you click that button and pay $5k/month minimum.
| Definitely an enterprise play and the pricing model and
| approach to security will make sense to those customers.
| drawnwren wrote:
| I'm pretty much exactly your target market. I run a kubernetes,
| docker, poetry devops hell at an ml startup and was curious how
| your product helped. You got about 2 minutes of my time. I
| scanned your website, I have no idea what you do or whether you
| fix my problem.
|
| Not trying to be negative, but I think there may be a 30 second
| pitch to be made to people like me that isn't made on your site.
| tommy_mcclung wrote:
| Thanks for the feedback and I agree we need to do a better job
| messaging the value on the site. How would this have landed for
| you?
|
| We are an AI orchestration platform that makes deploying open
| source models and frameworks using Kubernetes and docker
| simple. We manage the GPU and k8s resources, provide templates
| to common open source frameworks and an orchestration engine
| for your AI workflows.
| drawnwren wrote:
| I think at the end of the day, I have a bunch of config files
| and commands. Showing a quick translation between the
| spaghetti and your solution would make the most sense. The
| less language I have to parse the better.
|
| i.e. you have a set of kubernetes CRDs probably managed by a
| gitops solution -> we do this for you, you have a bunch of
| different pyproject.toml files that need to be orchestrated
| into a docker container -> this is what that looks like in
| release, tbh our terraform isn't a huge bottleneck because so
| much just ends up going in the kube crds.
|
| maybe have two pitches: 1 for startups getting started and 1
| for startups who already have a lot of this figured out
| (probably your bigger target audience if you have a strong
| pitch)
| tommy_mcclung wrote:
| Awesome feedback, thank you!!
| JoeCortopassi wrote:
| I've noticed that while a bunch of developers have played with
| LLM's for toy projects, few seem to have any actual experience
| taking it to prod in front of real users. I've personally had to
| do so for a few startups, and it's like trying to nail Jell-O to
| a tree. Every random thing you change, from prompts to models,
| yields massively different/unpredictable results.
|
| I think because of this, a bunch of companies/tools have tried to
| hop in this space and promised the world, but often times people
| are best served by just hitting OpenAI/GPT directly, and jiggling
| the results until they get what they want. If you're not
| comfortable doing that, there are even companies that do that for
| you, so you can just focus on the prompt itself.
|
| So that being said, help me understand why I should be adding
| this whole system/process to my workflow, versus just hitting
| OpenAI/Anthropic/Google directly?
| ira23 wrote:
| You're right - hitting OpenAI/Anthropic/Google directly is
| often the quickest way to get started, and for many simple
| applications, it might be all you need. However, Release.ai
| addresses the needs of companies that require more control,
| customization, and scalability in their AI systems.
|
| Release.ai isn't about replacing the big players but about
| giving you options. It's for when you need more than a generic
| API call but don't want to build an entire ML infrastructure
| from scratch. You can build exactly what you need without
| getting a Ph.D. in machine learning or becoming a DevOps
| expert.
| bradhe wrote:
| Super interesting you guys have been working on this since 2020
| if I'm reading the post title correctly? Would love to know the
| iterations you've gone through.
| tommy_mcclung wrote:
| It would be hard to count. Basically every customer we had on
| Release 1.0 was an iteration towards this. If you go to
| https://release.com it's there and we have many happy customers
| using it. There are definitely challenges with that business,
| however, that made us look for easier ways to get people using
| the platform we had built. People looking to do ephemeral
| environments have a lift to move their environment definitions
| into Release and it's a work effort that needs prioritization.
| Release.ai is built on the same platform and because AI
| frameworks are more turn key than bespoke software stacks
| companies have been building on, we believe Release.ai will be
| easier to adopt. In the long run AI applications and
| traditional web applications are going to merge and we think
| we're the platform that will do that in the long run. Long
| story short, hundreds of iterations.
| tommy_mcclung wrote:
| We might not have made it clear in the post how to signup for the
| sandbox. Just head to http://release.ai and click on "Start Free
| Trial".
___________________________________________________________________
(page generated 2024-08-07 23:00 UTC)