[HN Gopher] We raised $100M for open and collaborative machine l...
___________________________________________________________________
We raised $100M for open and collaborative machine learning
Author : minimaxir
Score : 154 points
Date : 2022-05-09 13:56 UTC (9 hours ago)
(HTM) web link (huggingface.co)
(TXT) w3m dump (huggingface.co)
| kolbe wrote:
| Getting that kind of raise in the current environment is a huge
| accomplishment. Congratulations.
| orangepurple wrote:
| What is this?
|
| Question: _________ is just an attention mechanism
|
| Answer: Huggingface
|
| https://www.microsoft.com/en-us/research/uploads/prod/2020/0...
| orangepurple wrote:
| The brand name only makes me think of Half Life head crabs (they
| hug the face). Maybe that's the joke on us.
| UnpossibleJim wrote:
| Oh, I went straight to Alien.
| spywaregorilla wrote:
| Same. They're literally face huggers
| wodenokoto wrote:
| It comes from the emoji[1], but I always think of Boaty
| McBoatface[2]
|
| [1] https://emojipedia.org/hugging-face/ [2]
| https://en.wikipedia.org/wiki/RRS_Sir_David_Attenborough#Nam...
| ushakov wrote:
| how will you earn that money back to the investors?
| orangepurple wrote:
| Looks like they are selling a chat bot to Monzo and allegedly
| Bing uses them for search?!
| w1nk wrote:
| So this question can only come from a place where you have no
| idea what they do in their field. For every news article or
| arxiv post that you see talking about how this amazing new
| GPT-N model has broken all sorts of language benchmark scores,
| you'll notice that basically nobody can reproduce those
| results. That's mostly due to the barrier of entry with respect
| to hardware for training the models.
|
| Huggingface is releasing APIs and model checkpoints that allow
| any random internet user to execute (almost) SOTA language
| models in production. FYI - that's an amazing leap forward and
| a strong piece of kit for MLEs to have access to.
|
| So let me rephrase your question: Is general access to SOTA
| language models worth 100mm to the software market?
|
| I suspect the answer is a resounding yes.
| jollybean wrote:
| So this answer can only come from a place where you have no
| idea how business works.
|
| There's a gaping difference between 'value creation' and
| 'value capture'.
|
| Some products create incredible value for many parties, but
| don't have an easy way to capture value.
|
| Some products create negative value for the system, but are
| oriented towards capturing a lot of money.
|
| Wikipedia, Web Browsers a lot of Open Source libs. - examples
| of the kinds of things that can be invaluable, but whereupon
| it's difficult to capture value.
| linspace wrote:
| It's a completely different thing to produce something of
| value and to get paid for it. There are plenty of examples.
| Immediately come to mind operating systems or compilers. Or a
| lot of completely underfunded but fundamental open source
| tools that everybody uses but nobody pays.
| karpierz wrote:
| > So let me rephrase your question: Is general access to SOTA
| language models worth 100mm to the software market?
|
| That doesn't answer the original question. The question was:
|
| "How will you extract $100M+ from the software market?"
|
| Your answer was:
|
| "I think that Huggingface will produce $100M worth of value."
|
| Which may or may not be true, but just because something
| produces X amount of value doesn't mean the project will be
| able to extract that value. See any open source project.
| w1nk wrote:
| So if you want someone to answer precisely how they'll
| extract hundreds of millions of dollars from an emerging
| market, I have to imagine this isn't the correct forum to
| expect such answers.
|
| Enabling the general software community access to SOTA
| language models will absolutely unlock an order of
| magnitude more money (than 100mm) over time. At least for
| now their obvious strategy for capturing this value is
| providing these APIs to enable it, and I suspect they'll
| gladly host such versions for the orgs that don't have the
| capacity to fine tune / host their own LLMs.
| prepend wrote:
| > So if you want someone to answer precisely how they'll
| extract hundreds of millions of dollars from an emerging
| market, I have to imagine this isn't the correct forum to
| expect such answers.
|
| I don't think the idea is to get a precise answer. I
| think the idea is to answer what their business model is.
| Like will they sell subscriptions? Or ads? Or patronage?
| Or enterprise support? Or conferences? Or what.
|
| I don't use huggingface, but am a little familiar with
| them and think they are a great group with great
| software. But since it's OSS, I'm not sure how they would
| make such a huge amount as $100M. Not to mention that
| they probably need to make much more than that to have
| happy investors. So there's probably a $1-2B plan for
| making money somewhere and knowing the general idea for
| their business model would be cool.
|
| I'm a bit bitter over what happened with OpenAI, and many
| other great opensource projects that turned into crappy
| companies boxed into making way more than they naturally
| could make (eg, elastic).
| karpierz wrote:
| Let me try to make the point clearer:
|
| 1. Investors expect Huggingface to extract more than
| $100M from the market. Otherwise they'd be called
| 'donors'.
|
| 2. If they openly publish models, then their APIs will be
| undercut by other providers who can take the published
| model and host it for cheaper. It would be cheaper for
| other companies because: they don't need to pay the cost
| of training the model, and they can specialize in simply
| hosting models.
|
| 3. Because of 2), Huggingface would need to avoid
| allowing other companies to host models, including
| internal APIs (because then providers would simply spin
| up to making hosting those internal APIs easy).
|
| 4) Because of 3), their policy of publishing trained
| models openly has to change.
|
| So the question that the original poster was asking is:
| what Huggingface policies will change, given the need to
| make returns on this investment?
|
| The original poster is likely thinking of OpenAI, which
| went down a similar route (starting training open models,
| took in a bunch of money, realized that openly publishing
| them wasn't sustainable, kept the models secret and
| created locked down APIs for accessing them).
|
| > So if you want someone to answer precisely how they'll
| extract hundreds of millions of dollars from an emerging
| market, I have to imagine this isn't the correct forum to
| expect such answers.
|
| This market isn't new; Google, AWS, OpenAI, etc. all have
| APIs they charge for. They also have services to host
| trained models for you. How will Huggingface make money
| without resorting to hiding its models?
| joshcryer wrote:
| Hugging Face is selling CPU cycles. They're also letting
| you upload your own datasets that aren't "limited" like
| others. I'm not quite sure where you think their approach
| of "open models" is wrong, they _still_ sell the CPU
| cycles.
|
| The idea that restricting access to the data is the only
| way to profit is such an archaic way of thinking. Hugging
| Face, if they keep making a good user interface and a
| good front end, will very much be able to fill the niche
| it is designed for: people who can't afford a $10-20k rig
| to run a model but who need to run it for their backend
| project.
|
| Also, it may be due to using HN, but when I think of
| "where can I run a model" or "get a dataset" I think
| Hugging Face. They are leveraging the democratization of
| the data.
| karpierz wrote:
| Thanks for clarifying, I misunderstood what Huggingface's
| product was.
|
| I see the niche. The risks are:
|
| - the mid market is constantly churning; either players
| become too big and you can't meet their requirements or
| they go bankrupt. Customer acquisition becomes a pretty
| big expense.
|
| - selling CPU cycles is a cutthroat business which
| competes pretty directly with AWS, Azure, and Google
| Cloud. Their edge will likely be ease of use, but at some
| scale, the larger providers will be able to undercut them
| hard.
|
| - selling a solution for managing datasets and training
| models using cloud CPUs is a crowded market.
|
| - not sure how trustworthy the company is with private
| datasets. Easier to trust an established vendor.
|
| But it wouldn't be a startup if there weren't risks.
| jollybean wrote:
| "This market isn't new; Google, AWS, OpenAI, etc. all
| have APIs they charge for."
|
| And if they were standalone businesses they'd be losing
| money, it's neither a big nor profitable market.
|
| When the business model for a project is not 'really
| obvious' it's usually a bad sign.
|
| AirBnB, Uber, Stripe etc. - 'how' they make money is
| obvious, it's intrinsic to the product.
| skdotdan wrote:
| I don't think OpenAI is a valid comparison. Huggingface's
| mission, unlike in the case of OpenAI, is not training
| models, but being the standard service for sharing them.
| The vast majority of models and datasets available at the
| Huggingface Hub have been provided by third-party
| companies or researchers. They aim to be the Github of ML
| models and data, not an AI-building startup.
| k8si wrote:
| Yep - "Huggingface Enterprise" just like there's "Github
| Enterprise" seems like a straightforward way to make
| money, at least to me? Does Microsoft make good money
| from Github Enterprise?
| k8si wrote:
| It's expensive to hire NLP labor right now, and has been
| for awhile. Seems like one strategy could be: HF provides a
| cheaper & more scalable alternative to having to hire an
| in-house NLP team. Basically NLP becomes synonymous with
| HF.
|
| And they amortize the cost of hiring their own NLP
| engineers by developing a few models/model-based services
| that lots of businesses would be willing to pay for. E.g.
| 'foundation models' for different verticals like healthcare
| etc. Then it'll also be a lot easier to either fully
| automate or at least scale up work that's specific to each
| paying customer (because fine-tuning should go much more
| quickly, just essentially be a hyperparameter tuning cycle
| in as many cases as they can get away with).
| Areibman wrote:
| Since everyone seems to be avoiding direct answers, I have a
| few ideas:
|
| 1. Open source the free, lower quality models (I.e. fewer
| epochs trained) but sell perpetual licenses to higher quality
| ones.
|
| 2. API access to high quality pre-trained models. Similar to
| OpenAI, but code would be open source.
|
| 3. Licensed access to a Databricks/Collab-esque style
| development environment. Jupyter is great, but once they
| establish a big enough community and enough killer features,
| they could adopt paying users.
|
| 4. ML Ops infrastructure as a service for enterprise
|
| 5. Consulting
| option wrote:
| 4 alone is obviously a 1B+ business in the next 5 years
| prepend wrote:
| I think that anyone will have a hard time competing with
| AWS and Azure in this space (although it will be a $1B+
| business).
| joshcryer wrote:
| One of the draws to use Hugging Face is _because_ the open
| source community is giving those trained models away. What
| they are doing is permitting people to _use_ those models
| affordably (something like a million words would be $10 if I
| recall correctly, you need to spend $10k minimum to run GPT-
| Neo-x in your bedroom just to print out one prompt).
|
| I'm not saying these are bad ideas but they need to more
| focus on their friendly front end and clean API access. There
| will be a DALL-E 2-Neo-x in a few months. People are going to
| want to run it without being limited by OpenAI's terrible
| interface.
| m_ke wrote:
| Wish them luck but having gone through something similar with
| another deep learning company that raised a ton of money, things
| will probably get a lot harder for them after the C round when
| investors start looking at revenue and not growth, and with a
| large valuation that limits the pool of potential acquirers.
| hackernewds wrote:
| Interesting you mentioned revenue and not net income / profits.
| How are revenue and growth detached?
| m_ke wrote:
| A lot of companies use other metrics to show growth, like
| github stars and package downloads
| lumost wrote:
| I recently searched through the ML companies listed on
| https://topstartups.io/.
|
| One repeating pattern that surprised me is that there are few
| successful ML startups, the ones that are there don't seem to
| be remotely close to self-sustaining - or even product market
| fit in many cases.
|
| Why is ML such a struggle? are we overstating the impact
| relative to "old fashioned" data collection and analytics? Is
| the tech to expensive for companies to adopt in terms of man
| hours?
| joshcryer wrote:
| A lot of people come up with an idea and then have no
| product. And investors are just happy to throw money at them.
| It's that whole idea that if you want to start an AI company,
| hire a bunch of people in India to do your work, then once
| the funding round is done, actually build the system.
|
| Hugging Face actually has a product and I think they'll be
| fine. I actually think $100 million is undervalued, because
| when I think "open models" or "training datasets" I think
| "Hugging Face."
|
| If you really want free money as an AI startup just say
| you're going to solve safe / friendly AI. People throw money
| at that without even showing anything. Hugging Face actually
| recently put out a job for someone who can work in "bias
| mitigation":
| https://nitter.net/mmitchell_ai/status/1520483233132990464
| mistrial9 wrote:
| great - somewhere in your game plan, please recognize that not
| everyone is going to use opaque, trained-elsewhere binary blobs
| as base models; nor does everyone require CNN/DeepLearning to do
| useful, real research or invention. Things like python skLearn,
| an internally built supervised model or a few hundred of them;
| analysis runs that are done without logging into a cloud with
| AUTH linked to some records behind a closed glass door
| somewhere..
|
| If you are FAIR, then you let people try things without being
| attached to you. agree?
| k8si wrote:
| You can turn off the part where it logs into the cloud and
| insert your own logging backend fairly easily.
|
| Also if your research/biz needs are satisfied by sklearn, then
| why not just use sklearn? But for a lot of NLP systems, BERT is
| actually really really useful. And if you don't want to use a
| pretrained BERT, you can easily initialize their BERT
| implementation randomly and train it on your own data.
| adamsmith143 wrote:
| >great - somewhere in your game plan, please recognize that not
| everyone is going to use opaque, trained-elsewhere binary blobs
| as base models; nor does everyone require CNN/DeepLearning to
| do useful, real research or invention.
|
| Shocking that a company focused on Deep Learning, NLP and CV
| might focus their efforts on people who do Deep Learning, NLP
| and CV??
| minimaxir wrote:
| Hugging Face is one of the few companies that recognizes that
| creating good OSS (e.g. the transformers Python package) and
| supporting that with managed services is better than focusing on
| managed services primarily and having open source as a crippled
| afterthought designed to drive people to said managed services.
|
| An example is HF's push toward ONNX export support for their
| major AI models in the transformers package, which allows faster
| model inference and could theoretically compete with their
| managed inference service.
| bjourne wrote:
| Congrats. I'm so jealous. Personally, I found Huggingface's
| models clunky to work with and for me it was easier to build and
| train my own. I really didn't like how they tried to make a
| "unified api" and treat PyTorch, TensorFlow 2, etc as backends.
| Though I realize that for those who are not adept in ml, or don't
| have the need for much customization, using Huggingface's models
| makes a lot of sense. No doubt they'll make themselves rich and
| their investors even richer.
| throwaway83242 wrote:
| We tried huggingface.co with great hope recently. Unfortunately,
| though their system was well orchestrated, we could not make
| progress on baby steps.
|
| We uploaded our model to github and then downloaded to hugging
| face. Why the package installed correctly, it failed because
| underlying Glibc headers were compiled with a version that is
| different from Hugging face's. So, while the platform works for
| some situations, it still has a long way to go.
| axg11 wrote:
| Congrats to the Huggingface team! They closed this round at a
| great time. I except there will be much stronger pressure over
| the next 18-24 months for them to show revenue growth.
| fxtentacle wrote:
| I don't get how they can earn those $100M back without becoming
| significantly less open.
|
| The gist of huggingface is that they host pre-trained open source
| models for free. Anyone can download them and then use them
| offline without HF. And their paid offerings aren't even close to
| being cost competitive with buying a few 3090 workstations.
|
| Also, HF doesn't really have any technical moat. Other people
| research and train those models, they just provide the hosting.
| In my opinion, their biggest value is the community. But how do
| you monetize a group of motivated volunteers?
|
| What stops GitHub from offering free LFS hosting for AI models,
| thereby copying the foundation of HFs community?
|
| And lastly, is there really much of a market in making SOTA AI
| beginner-friendly? You still need to buy/rent that A100 GPU
| server. Who's going to be greedy on salary for people operating a
| $20k/month machinery?
| skdotdan wrote:
| The moat is network effects, not the hosting service. GitHub is
| indeed a potential competitor but, again, you are just thinking
| about the hosting, while Huggingface provides an API compatible
| with all the uploaded models and datasets. GitHub would have to
| create their own API and then hope that users would switch to
| their service.
| lvl102 wrote:
| Huggingface + WandB are some of my favorite tools doing ML.
___________________________________________________________________
(page generated 2022-05-09 23:02 UTC)