[HN Gopher] The industry structure of LLM makers
___________________________________________________________________
The industry structure of LLM makers
Author : paulpauper
Score : 72 points
Date : 2024-11-26 18:35 UTC (4 hours ago)
(HTM) web link (calpaterson.com)
(TXT) w3m dump (calpaterson.com)
| roca wrote:
| Interesting article but gets at least one thing wrong. Not all
| models are trained on Nvidia chips.
| https://blog.google/technology/ai/google-gemini-ai/
| phillipcarter wrote:
| This article says something that seems very false to me once you
| step outside of the developer sphere:
|
| > Most LLM users seem willing to change from Chat-GPT to Claude,
| for example.
|
| Talk to people who aren't engineers and it's all ChatGPT. Many
| don't even know about the concept of an LLM or a provider, just
| literally "ChatGPT". The South Park episode where they parody
| this stuff? They call it ChatGPT. The stuff students use every
| year to help with homework? ChatGPT. The website that "chat.com"
| redirects to? ChatGPT. And cai has cornered to market on
| horny/lonely male teens.
|
| The moat here is the broader consciousness that a very very large
| population of people have adopted. Articles like this take
| something technical -- the cost of switching over to an LLM,
| which is cheap -- as an assumption that it will happen, without
| taking into account just how difficult it is to change social
| forces.
|
| This doesn't mean ChatGPT will forever be what people use. Maybe
| it will fail spectacularly in a year. But it's OpenAI's game to
| lose here, not the other way around.
| bee_rider wrote:
| If people don't know what the LLM behind the chat service is,
| then it seems likely (or plausible at least) that one could
| easily replace the chat bot used by these services with one
| backed by a different LLM, right?
| WorkerBee28474 wrote:
| Just like ChatGPT changes out models silently. Even if it's
| mentioned to the user, they don't care.
| moffkalast wrote:
| People just want a solution to their problem. Does a Google
| user care what iteration of their index engine they're
| using? No, they just want a picture of a god dang hot dog I
| tell ya hwat.
| gnfargbl wrote:
| I agree. The author makes the argument that airlines have a
| terrible business partly because consumers don't have any brand
| loyalty and Coca-Cola has a wonderful business partly because
| consumers have brand loyalty. What distinguishes those cases?
| Why should we consider LLMs to be more like one business or the
| other?
| kyoji wrote:
| Brand loyalty might matter when the cost of a good is
| relatively low and the availability high. I can basically
| choose between coke or Pepsi anywhere, and they cost about
| the same, so why not go with my favorite?
|
| For airlines availability with a preferred carrier is not
| guaranteed, and prices can vary wildly. Do I have so much
| brand loyalty that I will pay perhaps 2x the cost? Like most
| people, I wouldn't.
|
| In terms of availability and cost, LLM providers are much
| closer to Coke than to an airline.
| ljlolel wrote:
| An article last year said that LLMs quickly become like
| brands of bottled water
| scarface_74 wrote:
| Yes you will pay 2x the cost for your preferred airline
| when it's not your money and you are getting reimbursed by
| your company.
| scarface_74 wrote:
| The major airlines very much have brand loyalty via loyalty
| rewards programs, lounges, and cobranded credit cards.
|
| If you are business traveler gaining status by flying a
| preferred airline and using other people's money, you aren't
| going to go to the cheapest airline.
|
| Most of the profit from the Big three airlines come from
| business travel and credit cards
| ajmurmann wrote:
| This! I'd argue that the only reason loyalty might not
| always matter is because I am frequently not given a real
| choice because a given route likely has a very limited
| number of airlines offering flights and those might be
| dramatically different in number of stops, price and times.
| Air travel is one area where I frequently wonder how many
| benefits of it being a free market on paper we are actually
| getting. There is limited choice and direct competition
| seems limited
| scarface_74 wrote:
| One of my semi-frequent routes is between MCO (current
| home) and ABY - a small airport in Southwest GA where my
| parents live.
|
| There are only two commercial flights a day, both on
| Delta and both to ATL. A round trip ticket is $540 for
| two 1 hour segments (MCO - ATL - ABY).
|
| A round trip ticket from MCO (Orlando) to LAX (Los
| Angeles) is about the same price
|
| Of course I know the trick for former - book through a
| partner AirFrance for 17K miles
| throwaway314155 wrote:
| > What distinguishes those cases?
|
| It's in the article. Making coke is relatively easy compared
| to running an airline.
|
| > Why should we consider LLMs to be more like one business or
| the other?
|
| Also in the article. LLM's are analogous to airlines.
| gnfargbl wrote:
| You are doing the thing of asking if I read the article
| without actually directly asking if I read the article.
| Please don't do that, at least without carefully reading
| the comment that you're replying to.
|
| My specific point was that the article doesn't appear to
| support the assertions that it makes about brand loyalty.
| throwaway314155 wrote:
| I'm simply following the HN guidelines on the subject
| which prohibit directly asking if people have read the
| article.
|
| It's a pretty bad guideline in my opinion but my opinion
| isn't worth shit here.
|
| I'll re-read your comment when I have more time. Sorry if
| I missed the point.
| gnfargbl wrote:
| Most people who bother to comment on HN have an
| interesting opinion, and I value yours.
|
| The point of that guideline is to ensure that the
| conversation is substantive. Repeating points from the
| article with an assertion that those points are indeed in
| the article doesn't really add to the conversation and
| it's something that I do find frustrating on HN, which is
| why I mentioned it. I agree that it isn't a great
| guideline.
| moffkalast wrote:
| OAI has certainly positioned themselves culturally the same way
| as Google did for search engines. Google this, Tweet that, ask
| ChatGPT.
|
| We know now how much actual competition[0] Google had after the
| dust had settled, in all practical terms - zero. Even after all
| the SEO spam and enshittification they haven't lost any notable
| market lead.
|
| Time will tell if ChatGPT ends up that way but unless OAI
| implodes (which isn't all that unlikely) they're on the way
| there.
|
| [0] https://gs.statcounter.com/search-engine-market-share
| quonn wrote:
| But Google came years late. I used multiple search engines
| before Google finally emerged as a winner. altavista, excite,
| hotbot and others; there was a huge hype around the Lycos IPO
| and then alltheweb was a thing for a time and then Google
| won.
|
| So being first does not necessarily mean winning.
|
| And Twitter had strong network effects.
| bhouston wrote:
| I expect it to sort of be like AWS, Azure and Google Cloud.
|
| Many people started with AWS as it was first, and it leads to
| quite a bit of momentum in terms of market share long past when
| there was significant differentiators. It is just that there
| are switching costs and most people have already learnt AWS's
| APIs.
| ljlolel wrote:
| Good comparison
| JohnMakin wrote:
| What are the significant differentiators? I have worked much
| of a decade in the cloud infrastructure space, and from the
| POV of a business owner, AWS is such a stupidly superior
| product that I could not even imagine considering the
| alternatives. Google offers mostly AWS products but
| "googleized," and their support is practically nonexistent.
| Microsoft support isn't as bad, but their products are
| unreliable at best (from my view) and what differentiators
| they do have, which to me is better support for MS products
| in general don't really matter to me or my business at all.
|
| These are the big 3 so the only ones I mentioned. I know
| alibaba/yandex/digitalocean/etc exist but lack as much
| experience with them so only commented on the big 3.
| xnx wrote:
| > The moat here is the broader consciousness that a very very
| large population of people have adopted.
|
| That's not nothing, but switching costs are very low, and an
| alternative could arise faster than the switch from Friendster
| to Myspace or Myspace to Facebook.
| theolivenbaum wrote:
| Specially because there are no network effects and no lock
| in.
| ljlolel wrote:
| Only lock in could be if they become smart enough to truly
| know you and small preferences as a person that would be
| hard to repeat all the nuances to the next chatbot
| otherme123 wrote:
| Me and my coworkers pass around opinions about what LLM does
| what task better. The only conclusion is that they are 100%
| interchangeable, some prefer ChatGPT over Claude, and that
| just means that when ChatGPT credits get exhausted, they
| switch tab to Claude, Gemini or whatever their second option
| is. If ChatGPT started charging money or closed, they won't
| care at all.
| kibwen wrote:
| The general public doesn't care to understand the difference
| between "LLM" and "ChatGPT" any more than they care to
| understand the difference between "web browser" and "Chrome".
| Most people will happily use whatever you put in front of them,
| and if the product is bad, they'll generally grumble and shrug
| their shoulders in learned helplessness rather than do the
| research necessary to switch to a better alternative.
| Discerning consumers are a rounding error.
|
| Which is to say, the platform holders will determine who wins
| and loses. ChatGPT will win if they pay sufficient fealty to
| Microsoft, Google, and Apple.
| scarface_74 wrote:
| ChatGPT is not just an LLM.
|
| It is:
|
| - Dall-e for image generation
|
| - a real time Python interpreter that can run Python code to
| answer relevant questions
|
| - can search the web to retrieve and validate information
|
| - has the infrastructure to handle processing at scale
| PittleyDunkin wrote:
| Well that's just a branding failure.
| scarface_74 wrote:
| How is it a branding failure that ChstGPT - the product -
| augments the weaknesses of LLM by adding more capabilities?
| PittleyDunkin wrote:
| Well for one thing it calls itself "chat" when it offers
| so much more.
| Workaccount2 wrote:
| This is like Bitcoin.
|
| It's objectively a very bad crypto. It's the prototype and
| everything it does there is a coin that does it 100x better
| now.
|
| But man, Bitcoin, that name has serious influence and staying
| power. It's a testament to the power of branding and being the
| first mover.
| ajmurmann wrote:
| Bitcoin inherently relies on buy-in for its value. It's a
| shared fiction that becomes real because we share it. In that
| regard it's similar to countries. I literally cannot switch
| from Bitcoin to another coin and get the same value unless we
| collectively do it. It's a inherent property of its usage as
| a currency. I can switch from ChatGPT to Claude though
| without anyone else doing so and I get the same value. In
| fact, if Claude is superior I might actually get more value
| than if everyone switched because I now have a leg up on
| everyone else.
| indigoabstract wrote:
| > It's a shared fiction that becomes real because we share
| it.
|
| It's called the network effect, I believe.
| HarHarVeryFunny wrote:
| Having masses of people using ChatGPT and not paying for it
| doesn't make for a successful business. The people who are
| willing to pay are more likely to be aware of the alternatives
| and choose the one best suited for their use.
|
| For many school kids I think it's all just "AI", not "ChatGPT".
| tokioyoyo wrote:
| We said the same about Google, Uber, DoorDash, Facebook,
| TikTok, <insert any other unprofitable business that
| eventually became profitable>. Sure, most of them are making
| money through ads, but for that you need some audience.
| There's absolutely survivorship bias here, but eventually it
| might just pan out.
| kaptainscarlet wrote:
| That's true. Some business models succeed in the most
| unexpected of ways. They can pivot and change the recipe
| until it works.
| PittleyDunkin wrote:
| I'm one of those people--I use a variety of models but I call
| them all "chatgpt" (ironically, not including OpenAI's
| product). For the most part the model used doesn't really
| impact usage or quality that much, at least for my use-cases.
| It helps that I tend to keep my expectations very low. I think
| it's going to become a generic term for "llm chat bot" pretty
| rapidly, if it's not already metastasized.
| lkrubner wrote:
| There is no money to be made from individual users. All of the
| money comes from companies building something on top of the
| LLMs, and those of us building startups on top of LLMs are very
| much aware of the differences between the LLMs. And, to the
| point made in the article, it is trivially easy for us to
| switch from one LLM to another, so the LLMs don't have much of
| a moat and therefore they cannot charge much money.
| HarHarVeryFunny wrote:
| Probably true in the long run, but at the moment OpenAI is
| making about 90% of their revenue from ChatGPT subscriptions.
| dehrmann wrote:
| Google users are theoretically willing to become Bing users,
| though I'll admit that ChatGPT is the consumer leader mostly
| because of brand recognition and being the first mover.
| onlyrealcuzzo wrote:
| > This doesn't mean ChatGPT will forever be what people use.
| Maybe it will fail spectacularly in a year. But it's OpenAI's
| game to lose here, not the other way around.
|
| The AVERAGE person still does not even know what ChatGPT is.
|
| At most, 1 in 10 people have ever used ChatGPT.
|
| This is like saying Social Networking is MySpace's to lose. Not
| really. Most people hadn't heard of Social Media or MySpace
| when MySpace was already huge and - by far - the biggest
| player.
|
| It is likely easier for Facebook, Apple, Microsoft, or Google
| to introduce >50% of the population to an LLM than for ChatGPT
| to get from ~2.5% to >50%.
|
| ChatGPT monthly users is about 1 in 40 people, by the way.
|
| Does that mean ChatGPT is doomed to fail. No.
|
| ChatGPT could easily be the winner.
|
| But declaring the race over unless ChatGPT blows both its legs
| off seems very premature.
| littlestymaar wrote:
| That's true, but that doesn't mean much as long as these
| particular users are free users that don't bring any money to
| the company (and cost _a lot_ compared to similar users in
| other technology companies).
|
| The real business is enterprise API endpoint billed by the
| millions of tokens, and in that particular domain OpenAI has
| literally zero market lock-in (and they probably depends more
| on Microsoft sales power than on their own brand value).
|
| Unless OpenAI can show that they are able to make money from
| the mass of casual users, they are in a tough spot.
| JohnMakin wrote:
| It seems like the points you're making are in support of the
| statement you are quoting - if most people don't know the
| concept of an LLM or a provider, why would that make it
| difficult for them to switch? Seems like ChatGPT's only
| competitive advantage here if I am understanding what you wrote
| correctly is name recognition. If ChatGPT's "game" to lose here
| is just staying relevant in the public consciousness, it would
| appear to me that the main point of this article, that building
| LLM's is not going to be a great business, is largely correct.
| I would expect a company such as OpenAI with such fantastic
| claims they make to have some kind of technical advantage over
| their competitors.
| wkat4242 wrote:
| Yeah but early stronghold brands don't always keep the market.
| Before word there was WordPerfect. Before Excel there was lotus
| 123. Everyone swore by both but they have been dead for
| decades.
|
| It's funny because ChatGPT is such a bad mainstream branding.
| Technical name, hard to pronounce, nobody even knows what GPT
| stands for. They really got overwhelmed by their own success
| otherwise they would have done more on the branding side to
| appeal to mainstream users. But their first mover advantage
| won't last forever.
| tdesilva wrote:
| I'd say it's more like kleenex. Lots of people ask you to 'pass
| them a kleenex' when their nose is runny, but they just mean
| tissue. They don't actually care what the brand is. Similarly
| for LLMs most people may not care (or maybe they will, and it
| will be more like Google search), especially if they just use
| it via some other app that calls LLM provider APIs. My anecdata
| so far says early adopters try multiple LLM providers and use
| the best one for their use-case. No clue on what non-tech folks
| think though.
| otherme123 wrote:
| Exactly. One of my coworkers prefers Gemini to overcome the
| blank page hurdle, and he happily describe it as "the ChatGPT
| from Google". What does that mean for ChatGPT as a business?
| Nothing. Google would like people to use Gemini, but at least
| they retain this user and can target him with better ads,
| their real business. ChatGPT is just a layman synonym for
| LLM.
| crabmusket wrote:
| It amuses me that ChatGPT actually seems like a generic
| term already. You Chat with a Generative Pre-trained
| Transformer. Does what it says on the tin!
| joe_the_user wrote:
| Your argument is essentially "no one will buy generic tissue
| when everyone calls it Kleenex". That's only powerful when
| ChatGPT is free. When there's price pain, we can see people
| adopting alternatives.
|
| Branding is a moat but it's not a deep moat. Branding
| ironically works best (most profitablye) for incidental things
| that people exhibit to others - designer clothing is the most
| obvious - and this is because then brands have a social aspect
| (there's also branding a real signal of real superior quality -
| I'd buy a good brand of drill 'cause I have a rational reason
| to expect better quality but maintaining the quality of a
| branded product is more costly and hence less profitable than
| maintain the pure image of something like Coke and LLMs turn
| out not to really differentiate on quality). Whether they call
| LLMs "ChatGPTs" or not, people use LLMs for a result - they'll
| use a different LLM that gives equivalent result _if_ they 're
| motivated to do so. No one else is going to what brand of
| "ChatGPT" someone "drives", etc.
| YetAnotherNick wrote:
| > Branding is a moat but it's not a deep moat.
|
| Let's do an opposite question. What's Google's moat? What's
| Apple's moat? All I hear from everyone is "X is not a moat",
| which while true doesn't mean company couldn't be ahead of
| the competition forever.
| yifanl wrote:
| Google's moat with this current wave of AI is pretty
| obvious: They own the compute resources inhouse.
|
| Apple isn't immediately seeking to compete in this field,
| presumably because they don't see a deep enough moat.
| dingnuts wrote:
| Google and Apple's moat in the mobile world is the monopoly
| Qualcomm has on modems and those two players being the only
| ones who can afford them, but nobody wants to talk about
| that.
| joe_the_user wrote:
| Google's moat for search on the user side is quality, habit
| and integration but Google search is free and compared
| other "FANG" companies, Google is actually fairly
| vulnerable imo.
|
| Apple's moat is people's hardware investment, their
| interface, their brand in a way that is socially
| significant as well as implying a real quality difference.
| Apple's overall moat is much larger than Google's.
|
| Edit: and the specific non-moaty part of LLMs is that their
| answers are generic - LLMs don't have "personalities"
| because they are a trained average of all publicly
| available human language. If a given LLM had restrictions,
| it wouldn't be as useful.
| YetAnotherNick wrote:
| > quality, habit and integration
|
| > hardware investment, their interface, their brand
|
| Exactly, you gave all the possible moats for LLMs. Not
| saying OpenAI has it right now, I am disagreeing with the
| premise that LLM provider can never have moat.
| phillipcarter wrote:
| Like I said, this is OpenAI's game to lose, not someone
| else's.
| mbesto wrote:
| > Most LLM users seem willing to change from Chat-GPT to
| Claude, for example
|
| There is some nuance to this. If you're building an application
| that embeds an LLM, then your "user" might be a prompt
| engineer, not necessarily the user using the application. It
| just so happens you can use the embedded magic using the prompt
| yourself.
|
| Example: https://asana.com/product/ai
|
| Not a single mention of Chat-GPT or Claude, but if you google
| you'll see they use Claude under the hood. So I would argue the
| branding is actually "AI" not ChatGPT.
|
| It's a bit like Crypto and Bitcoin. Not all Crypto is bitcoin,
| but all bitcoin uses crypto to power it. People recognize both
| the branding of Crypto and Bitcoin.
| 627467 wrote:
| There's an interesting parallel between the subjective nature
| of LLMs (being blackboxes of nondeterministic output) and
| brands. The whole point of investing in brands is to create a
| moat. And maybe LLM are converging but because they are hard to
| predict there will always be factor that people's psyche will
| favour
| WorkerBee28474 wrote:
| > Really though, LLM makers have only one true supplier: NVIDIA
|
| The argument relies on the axiom that NVIDIA will have a
| persistent hardware advantage. Maybe they will, but even if they
| were always 2 years ahead of the competition, if NVIDIA-trained
| LLMs would 'good enough' in 2025, then non-NVIDIA-trained LLMs
| would be 'good enough' in 2027.
| ljlolel wrote:
| No because that would compete with a 2025 model not state of
| the art.
| CharlieDigital wrote:
| > NVIDIA will have a persistent hardware advantage
|
| Is it a hardware advantage?
|
| I think it probably has more to do with CUDA. The reason Python
| is the undisputed champion in AI and ML isn't because Python is
| a better, more performant programming language so much as
| because ecosystem of software in the AI/ML space is extremely
| concentrated, dense, and rich in the Python ecosystem compared
| to Java, Go, or C#.
|
| Likewise, it seems like NVDAs advantage isn't even necessarily
| the hardware but the suite of tools and software that are built
| up to take advantage of that hardware.
| YetAnotherNick wrote:
| It's definitely not CUDA advantage. If you can get
| Pytorch/flash attention/triton well supported in any
| hardware, a huge chunk of client don't care if it means cost
| saving. Case in point Google's TPU had extensive usage
| outside Google when they were cheaper for the same
| performance. Now that isn't the case.
| dehrmann wrote:
| It's TSMC more than NVIDIA.
| tomjohnneill wrote:
| And it's ASML more than TSMC.
|
| Or maybe there's a highly profitable role for all the
| different parts of the value chain.
| sixhobbits wrote:
| I've seen the airplane/railway comparison a lot and I'm not sure
| I buy it.
|
| Nvidia is currently important to training new LLMs but it's not
| that important to running inference on existing ones.
|
| I think email might be a better comparison? If LLMs really do
| become something that everyone uses without thinking about (and
| at least anecdotally it's the first tech trend I've seen that all
| my non tech friends are using) then yes sure you can easily
| change provider but in reality most people are just going to use
| whoever wins, just like most people use Gmail.
|
| So investors are putting in huge amounts of money to have part of
| the next Gmail, and many of them will lose but there will
| probably be some dominant player and sure you can change but once
| you've used one for a few years and it is as good as or equal to
| another, and it approximates to free, then you'll probably stick
| with it, compatible api and interface or not.
| ljlolel wrote:
| No because Gmail has strong network effects (people email your
| @gmail address)
| rrr_oh_man wrote:
| I'm not sure you're portraying network effects correctly.
| WorkerBee28474 wrote:
| The article calls out trademarks as part of the reason why soft
| drinks are successful. I think the impact of laws and regulations
| cannot be ignored for the future of LLMs either. Legal/regulatory
| moats make companies more profitable than they otherwise would
| be.
|
| For example, in the future if you ask a model "are trans women
| real women" it will have to say "yes" in one country and "no" in
| another. In one country the LLM will have to talk about the "5000
| years of Chinese history" and in another will have to say that's
| just an invention to feed their superiority complex.
| MeetingsBrowser wrote:
| It doesn't technically detract from the overall point, but almost
| everything about airlines is wrong.
|
| It's incredibly difficult to start a new airline. Small and
| medium sized airlines are almost non existent. There is currently
| a pilot shortage. Most people stick with the same airline for the
| rewards.
| rrr_oh_man wrote:
| _> Most people stick with the same airline for the rewards._
|
| Are you sure?
| ashishb wrote:
| I looked at the numbers at various startups and felt the same
|
| https://ashishb.net/all/llms-great-for-business-but-bad-busi...
| highfrequency wrote:
| Agree that the foundation LLM business model is challenging, but
| I'm not very convinced by these particular arguments.
|
| Yes, Nvidia GPUs are currently expensive. But they will soon be
| under tremendous competitive pressure from AMD, and more
| importantly Moore's law is relentless (both in terms of model
| size capacity and performance per dollar). The price evolution of
| miniaturized transistors is basically the opposite of the
| airplane example.
|
| Second, barriers to entry will keep increasing. Frontier models
| require stacking many new research and engineering insights. Of
| course the extent of secrets is currently limited because they
| only stopped publishing breakthroughs a couple years ago.
| Obviously that's going to look very different 5 years from now.
|
| On the other hand, competition between the leading frontier model
| companies is increasingly fierce (Google and Facebook have been
| slow to ramp up but theoretically should pose a big threat, and I
| am suddenly seeing Gemini topping leaderboards in the last few
| weeks), the moat is indeed questionable and the price of talent
| is very high. So it's by no means an easy place to build a
| profitable business. But it's at least possible for one or
| multiple of these firms to achieve process complexity that is
| extremely hard to replicate, and in the asymptote I really don't
| think GPU costs will be a material threat to the business model.
| thefaux wrote:
| He lost me at Pepsi is the same as Coke. GTFO.
| indigoabstract wrote:
| I was following along nicely until I hit this line:
|
| > This is despite the fact that they are identical in both taste
| and colour.
|
| The only way I could ever mistake Coca-Cola for Pepsi is if I
| were to completely lose my sense of taste. I'm quite surprised to
| encounter someone who considers them identical or
| interchangeable.
|
| He's right, though, that I could change the supplier for
| airlines, browsers (or ChatGPT) and my world wouldn't be all that
| different. It would taste the same.
| elzbardico wrote:
| One thing the author is forgetting:
|
| Regulatory capture.
|
| The incumbents can lobby for overly restrictive legislation based
| on copyright or "safety" concerns that make absurdly expensive
| for new entrants to enter the market.
|
| Think about big tobacco.
| rossdavidh wrote:
| "If LLM makers seem cursed to an airline-style business destiny,
| how come they are able to raise so much money? ...What do they
| know that I don't? It is a mystery - but let's consider the
| options..."
|
| Not mentioned: a lot of the "money" that they raised, was not
| actually cash but credits for cloud compute. If you've already
| bought too much cloud capacity, giving away some of it and
| claiming it as an investment in AI, looks like a good idea.
___________________________________________________________________
(page generated 2024-11-26 23:00 UTC)