[HN Gopher] Show HN: Convect - Instant Serverless Deployment of ...
___________________________________________________________________
Show HN: Convect - Instant Serverless Deployment of ML Models
Author : sameerank
Score : 61 points
Date : 2021-05-27 14:39 UTC (8 hours ago)
(HTM) web link (convect.ml)
(TXT) w3m dump (convect.ml)
| sameerank wrote:
| Hi HN, I'm Sameeran. I'm building Convect (https://convect.ml).
|
| Convect deploys machine learning (ML) models to instantly
| callable, serverless API endpoints. Using Convect, Jupyter
| notebook users can deploy trained models from their notebooks and
| share them with the world in seconds. No web development or
| infrastructure experience is needed. Convect simplifies the
| process by being more opinionated than other model deployment
| workflows.
|
| To give Convect a try, visit https://app.convect.ml. You can also
| try out models without signing into an account on the demo page
| (https://app.convect.ml/#/demo). I would love your feedback.
|
| Some background/context:
|
| Deploying ML models to be used in production entails a different
| set of skills than training models in a sandbox environment and
| can get pretty complicated depending on what you're trying to do.
| For many data scientists, this "sandbox" environment is a Jupyter
| notebook. One common approach to "deploying to production" that I
| saw while previously working as a data scientist at Intuit, is
| turning a model trained with scikit-learn in a notebook into an
| API endpoint.
|
| From my experience, there are a few ways to deploy a model to an
| API endpoint and all of them involve a nontrivial level of effort
| and time. Examples of some of the steps in the process include
| pickling a model and uploading it to cloud storage, Dockerizing a
| model's prediction code and environment, deploying a Flask app,
| or getting set up with an ML framework (e.g. MLFlow) or platform
| (e.g. SageMaker) so you can use the deployment feature in their
| SDK.
|
| While complex workflows make sense for deploying complex models,
| I haven't seen any dead simple deployment solutions for simple
| models, and that's what I am working on building with Convect. In
| this case, simplicity comes at the cost of flexibility, i.e. you
| give up the ability to customize your infrastructure and runtime
| environment in exchange for a simple, one-click workflow. My
| hypothesis is that this tradeoff is worth it in many situations,
| and I'm curious to see what people are enabled to build when this
| aspect of the ML workflow is drastically simplified.
|
| Under the hood, Convect creates two artifacts by serializing 1)
| the model prediction code and 2) all the variables that are in
| scope in the Python session at deployment time. I've made this
| part of the deployment code public here:
| https://github.com/convect-ml/convect/. These artifacts are then
| loaded and executed in an AWS Lambda function upon invocation by
| an API Gateway endpoint.
|
| I've talked with 50+ data scientists at small to medium size
| companies (2-300 employees) and many have identified deployment
| as a pain point in their workflows. I've also spoken with a few
| data scientists who have indicated that this would help save time
| on their after-work/weekend side projects.
|
| I'm sharing this now on Show HN because I'd love for people to
| try out Convect and to hear about how you use it or how I can
| improve it to make it useful. I've also put together a gallery of
| examples for training and deploying models to make it easy to
| quickly get started (https://convect.readme.io/docs) and provided
| example endpoints that you can use query or even build ML-powered
| apps on top of (https://app.convect.ml/#/demo). Find out more at
| https://convect.ml. Thanks for having a look!
| Tycho wrote:
| Looks cool. But what is the ConvectApp exactly? It's blocked on
| my work network.
| sameerank wrote:
| Thanks! app.convect.ml is the ML model admin page.
|
| The workflow looks somewhat like this --
|
| Within a notebook, you can: 1) train a model, 2) deploy a
| model, and 3) get a shareable serverless API endpoint.
|
| Outside the notebook, app.convect.ml provides a UI for
| editing, sharing, or deleting deployed models as well as
| getting the API key for deploying models.
| matheist wrote:
| > _2) all the variables that are in scope in the Python session
| at deployment time._
|
| My eyebrows raised in surprise at this point! Obviously there's
| a tradeoff between convenience ("Oooh it's working! great let
| me deploy _right now_ ") and reproducibility ("Hmm I need to
| re-deploy what my teammate was working on because <reason>, let
| me go and find their notebook... ah here it is, oh weird why do
| I have this undefined variable") and this is hard on the
| convenience side.
|
| Was it surprising to you too that you'd have users who wanted
| this location on the continuum?
| sameerank wrote:
| You're absolutely right that this product is heavy on the
| convenience side of the tradeoff.
|
| > _Was it surprising to you too that you 'd have users who
| wanted this location on the continuum?_
|
| I'm still working on the "have users" piece, but there were
| signals for wanting more simplicity/convenience in
| conversations that I had with data scientists. I also felt as
| a data scientist that existing deployment solutions sometimes
| felt over-engineered for flexibility and reproducibility, at
| the cost of convenience. I'm exploring the other end of the
| continuum with Convect -- how opinionated can we get with ML
| workflows to simplify and still be useful?
| yurisagalov wrote:
| Congrats on the launch, Sameeran! As a non AI/ML engineer, I love
| the demo page with the pre-populated examples. The idea that
| anyone can deploy and share ML models through a REST api seems
| quite cool.
| chetan_v wrote:
| Seems like a cool product, congrats on the launch!
| sameerank wrote:
| Thank you for checking it out!
| jonbaer wrote:
| Congrats on launch. It seems a bit like FastAPI/ColabCode ...
| does this also work w/ Google Colab?
| sameerank wrote:
| Thanks! FastAPI/ColabCode is a fair comparison. And I just
| tried it in Google Colab, and it does work! You can install
| Convect in a notebook cell with "!pip3 install convect"
| spamalot159 wrote:
| Very cool! I have run into the issue of deploying models
| recently. I see that this is using AWS lambda functions. How does
| this hold up with larger models (3-5gb)?
|
| Trying to deploy a model to Azure Functions didn't play nice
| because of my file sizes.
| sameerank wrote:
| Thanks! AWS Lambda supports Docker image sizes up to 10 GB
| (according to their docs), so on the back end, 3-5 GB could
| still work.
|
| Convect, in its current state, is still limited to small models
| (e.g. < 0.5MB). This is because the deployment happens by
| posting to a REST API, which hasn't yet been tested for large
| payloads.
|
| I wrote up some tips for decoupling a large model from the data
| that's causing it to be large
| (https://convect.readme.io/docs/news-topic-model). However, it
| sounds like you're asking about a truly large model. Convect is
| still in the MVP stage right now, but we plan to handle larger
| models. 3-5 GB models will likely be feasible in the future.
| etaioinshrdlu wrote:
| Will you support PyTorch models? Will they run on a GPU? How many
| concurrent requests will you support?
| sameerank wrote:
| At the moment, Convect only deploys scikit-learn models, but I
| am planning to support inference with PyTorch and other ML
| frameworks. Not planning to run on GPUs, as there are other
| products that already help with this, unless I see interest in
| this from my users. Convect would support concurrency according
| to AWS Lambda's quota, which is in the hundreds of thousands.
| ploomber wrote:
| Congrats on launching!
|
| Is there support for including custom code as part of the
| deployment?
|
| Models trained on tabular data (which are the majority of use
| cases in industry) usually require some pre-processing before
| feeding the model (i.e., feature engineering).
| sameerank wrote:
| Great question! Yes, Convect supports deploying pipelines and
| functions in which you can include pre-processing code. Here
| are two tutorials that walk through that:
| https://convect.readme.io/docs/wine-classifier-pipeline and
| https://convect.readme.io/docs/iris-plant-classifier-functio...
___________________________________________________________________
(page generated 2021-05-27 23:01 UTC)