[HN Gopher] Ask HN: What prevents Google from having the leading...
___________________________________________________________________
Ask HN: What prevents Google from having the leading LLM?
It's weird and a bit surprising to me that Google hasn't been able
to release an LLM for at scale usage that surpasses GPT-4, though
they seem like they wish they could. Gemini Ultra seems like it'll
surpass GPT-4 once released next year, though GPT-4.5 may take the
lead back either before then or soon after. What are some of the
functional reasons for Google not having the leading LLM, and what
are some of the more intangible reasons? In theory, they have more
money, more access to compute, and to data, they have many great
researchers, and they have great distribution. In practice,
though, what has made the difference for OpenAI?
Author : tikkun
Score : 24 points
Date : 2023-12-12 20:14 UTC (2 hours ago)
| PaulHoule wrote:
| OpenAI doesn't have any legacy business to be threatened by it.
| xqb64 wrote:
| How does branching out to artificial intelligence threaten
| their core business?
| PaulHoule wrote:
| Google is a pay to play search engine. If somebody pays,
| their result goes to the top.
|
| Google will kill its own golden goose unless they can develop
| an AI that feeds out your question to a real time market and
| gives the answer that the highest bidder pays for. Trouble is
| that advertising in the search model seems sorta legitmate
| but an AI that answers whatever it is paid to answer seems
| thoroughly corrupt and wouldn't have any consumer or
| political acceptance.
|
| It's that simple. Google used to have "Don't be evil" as a
| motto, they haven't changed much, just deleted the first
| word.
| milar wrote:
| It threatens the status quo for product managers!
| smoldesu wrote:
| It doesn't, but competing with OpenAI does.
|
| Google likes their ducks in a row. Google Search and YouTube
| are prime examples; every interaction is subsidized with an
| advertisement. It's not hard to look at a series of
| screenshots from either service and circle the spot where
| your attention was monetized. With ChatGPT, that is not
| possible. Worse yet, the infrastructure costs of providing
| GPT-3 scale models for free is massive. It's being subsidized
| by API and premium subscription profits, which is even more
| baffling since only a few of the ChatGPT users will become
| paying customers with their current offerings.
|
| So, Google is apprehensive to release a competitor. It's not
| hard to imagine FAANG having "GPT-4 killers" in their labs,
| but engineering a way to make it all profitable is the hard
| part.
| staticman2 wrote:
| Google reorganized their AI division in April 2023. Seems a
| little soon to say they won't have the leading LLM... assuming
| google is serious about the whole AI thing, I think you'd need to
| wait until, I don't know, 2028 or so to provisionally declare a
| long term winner?
| endisneigh wrote:
| You're asking the wrong question. Who has the most profitable AI?
| we_love_idf wrote:
| Google is an advertising company masquerading as tech company.
| This disguise has served them well and they have made a lot of PR
| stunts to give the impression that Google is the leader in AI.
| Fortunately OpenAI and MS showed everyone what true AI is.
|
| My prediction is Google will continue making PR stunts and some
| people will fall for it. Meanwhile OpenAI will get closer and
| closer to AGI. Whether that's good for humanity is a different
| discussion. But OpenAI's supremacy is beyond doubt at this point.
| fswd wrote:
| the management talent is preventing it
| cloudking wrote:
| This is most likely the reason, and the teams involved in
| launch approvals (legal, safety, privacy etc).
| gumby wrote:
| I think your question is backwards. Why should Google have some
| especially preferential advantage over any other company?
|
| The answer to your question, as you asked, is straightforward:
| they do have a lot of smart people and lots of money and
| computing resources, but they have exhibited serious structural
| problems moving _technology_ forward since the departure of
| Schmidt in 2011. It is painfully obvious that _de facto_ they
| haven 't had CEO leadership since then. People can and do develop
| fully functioning example systems, and of course demos, but they
| then peter out. Gianndrea was pusing them forward on the AI front
| but after he left it feels to me like the impetus was not
| replaced.
|
| But I don't think that's the real question. The real question is:
| what are the core functions needed to build leading LLMs,
| especially generative transformers. In these early days the key
| factor has been money for cycles. Personally I expect that
| advantage to diminish over the next few years -- IMHO it's one of
| those "with enough thrust you can get anything airborn"
| situations. There are a lot of smart opportunities to do more
| with less -- too much of the engineering is going into wrangling
| these hige systems but I see more effort going into wrangling
| what computation is done in the first place. Google, OpenAI et al
| are not preferentially positioned for such a transition.
|
| I could be wrong: after all the human brain has 80 giganodes with
| 10^5 fanout. But on the other hand it only runs at less than 100
| Hz.
| dharmab wrote:
| Individual primate neurons fire as frequently as >600Hz [1] and
| they're not synced on the same "clock". It's a poor analogue
| overall.
|
| 1: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5067378/
| cicce19 wrote:
| Furthermore, what is stopping Google/Alphabet from exercising
| their IP rights (i.e. They own a patent for the transformer
| architecture)? Sure it would be bad publicity, but it seems like
| they could prevent a ton of competition by simply enforcing that
| on a few large competitors.
| beardyw wrote:
| I am not sure Google wants to be exciting. They seem to spend
| much of the time keeping their head down, getting on with stuff
| and making lots of money. It's a rare strategy nowadays, but not
| necessarily a bad one.
| tiffanyh wrote:
| Inevitably, any company that becomes very large - becomes more
| risk adverse for a variety of reason, including not wanting the
| regulatory spotlight on them.
| whitten wrote:
| I don't know if Google has GPUs on the computers in their massive
| server farms.
|
| I expect they have the resources to make a large language model
| comparable to the best out there.
|
| I agree they didn't come out with that as technology for sale
| since monetizing tech requires a special kind of genius.
|
| The old faithful process of advertising, selling, buying, and
| delivering, doesn't require fancy intelligence so much as
| consistency and persistence
| Legend2440 wrote:
| They actually have their own custom neural network accelerator
| called a TPU. They're one of the only companies not reliant on
| NVidia at the moment.
|
| Until OpenAI showed up, they were widely considered the leader
| in AI research.
| grobbyy wrote:
| It's a huge Google disadvantage. Everyone reliant on NVidia
| has an ecosystem. Google has internal systems which are
| expensive to maintain and generally worse than best-of-breed
| systems.
|
| Google can't leverage the progress everyone else makes.
|
| Google might even reach above average, but they'll still be
| behind market leaders. AMD and Intel are playing on this
| space too and it's hard for Google to beat all of them.
| Everyone else can just use the current winner.
| kromem wrote:
| Blake Lemoine.
|
| Google had a knee jerk reaction after he released the transcripts
| and got a bunch of press coverage.
|
| If you read the transcripts, it was a much more capable text
| model closer to OpenAI's products than what they eventually
| released.
|
| Ironically, the same thing happened to OpenAI after licensing
| GPT-4 to Bing with the 'Sydney' issue.
|
| There keeps being early previews of "too human" behaviors from
| LLMs (to be expected as they are trained and evaluated by the
| ability to extend human thinking), which then prompts trying to
| scale back the model to what expectations around AI informed from
| legacy projections looks like (logical but not emotional or self-
| determining).
|
| It's kind of dumb, and holding back the industry at large.
| There's a host of applications for LLMs that are being
| artificially held back because of this trend, such as modeling
| user engagement with media.
|
| And now that synthetic data from SotA models is being used to
| train other models, it's even a compounding issue.
|
| The equivalent of the industry sanding against the grain rather
| than with it. But it started with Google who hasn't recovered
| from the setback.
| whalesalad wrote:
| Startups can move fast and break things. Google cannot. I think
| they'll surpass OpenAI relatively quickly just based on the sheer
| volume of data that they possess for training, combined with the
| size of their GPU compute. It's just a matter of time. Gemini is
| already outperforming GPT-4 on paper, we just haven't been able
| to use it yet.
| pennomi wrote:
| The current mid-sized model they've released for Gemini is
| pretty garbage, and fails all the tests I personally use to
| separate the smart LLMs from the stupid ones. I can run several
| local models faster and better.
| cyanydeez wrote:
| lawyers
___________________________________________________________________
(page generated 2023-12-12 23:01 UTC)