[HN Gopher] Big tech and the pursuit of AI dominance
___________________________________________________________________
Big tech and the pursuit of AI dominance
Author : MoSattler
Score : 44 points
Date : 2023-03-27 20:31 UTC (2 hours ago)
(HTM) web link (www.economist.com)
(TXT) w3m dump (www.economist.com)
| Forestessential wrote:
| It is like reading the news or a study paper but you know it is
| correct because a robot is telling you. This is the closest thing
| to a prophet people will have. Go downvote me.
| fdgsdfogijq wrote:
| Big Tech hasnt dominated anything. They got trounced by a 400
| person startup. Alexa, Google Search, FB research all spend
| untold money on ML systems. Neither is releasing game changing
| products anywhere close to OpenAI.
| tomComb wrote:
| It is true that big companies have brands and existing revenue
| stream to protect (Google), so maybe only a little company with
| nothing to lose could do what OpenAI did.
|
| And maybe MS found the perfect arrangement giving them an
| ability to adjust how closely to associate themself with the
| tech according to need.
|
| But is this all necessarily a good thing? OpenAI gave MS access
| to GPT and MS clearly wants to use GPT to destroy Google, so
| Google now has respond aggressively, and now we have exactly
| the AI arms race that many predicted would lead to the worst
| outcomes.
| pydry wrote:
| In that case they might have picked the worst time to push
| through layoffs.
| lwhi wrote:
| I think you forget what money and connections bring.
|
| Anything that can be provided by a micro or small sized company
| can be provided by billion dollar corporations too.
|
| Maybe size brings a drawback, in the same way a mega ship finds
| it difficult to turn around without a day's notice -- but these
| companies can and will.
|
| Microsoft once betted all chips on the internet being fad; look
| at them now.
| Karrot_Kream wrote:
| The Transformer architecture that powers GPT models is based on
| a now-famous Google Brain paper, Attention is All You Need.
| You're right that OpenAI has productized this research much
| faster than Big Tech, but the research itself came from
| established R&D in Academia and Big Tech.
| pclmulqdq wrote:
| Well, Google had to figure out how to cancel itself somehow.
| That's the only Google project, aside from search, that has
| had staying power.
| yreg wrote:
| And Stable Diffusion is based on Google's Imagen paper.
| nr2x wrote:
| Yes, but only one of the authors is still at google.
| taneq wrote:
| So if you work on a kickass product at FlorpCorp and then
| leave, FlorpCorp didn't develop that product?
| xigency wrote:
| I disagree. All of the big players have been involved in some
| way, creating their own systems (which are quite competitive)
| as well as laying much of the foundation for existing models
| with research they published.
| galaxytachyon wrote:
| OpenAI was backed by probably the biggest tech company around,
| Microsoft. Without the computing afforded by Azure, they would
| have problems training and running the LLM.
| Forestessential wrote:
| I cant even discuss this without going powerlevel 9000+
| danbrooks wrote:
| I wouldn't say big tech was "trounced" by OpenAI. Advances in
| LLMs are driven by the research community. OpenAI is a small,
| well-resourced organization with the sole purpose of quickly
| deploying and hyping state of the art AI models. Big tech
| naturally has a slower product development cycle - and will
| make full use of LLMs in due time.
| rcme wrote:
| > I wouldn't say big tech was "trounced" by OpenAI.
|
| I mean not a single "big tech" company is offering any LLM
| products not powered by Open AI. It's been almost 3 years
| since GPT-3 was released. I'm not sure they can catch up at
| this point. And I think the GPs point was a good one: big
| tech threw billions and billions at personal assistants.
| Nothing innovative was delivered.
| nico wrote:
| Dominance is such a charged word.
|
| We have been corrupted by words like that.
|
| Why should we care about dominance?
|
| We should care about collaboration and people's well being.
|
| Barring a catastrophe, AI is not going to stop.
|
| At the current rate, by next month we will have some previously-
| never-seen model/algorithm/chip/tech that will run locally on our
| phones and computers. At that point is game over for the big
| server-model companies.
|
| After that... well, pretty much all companies will be at stake.
|
| There's a much bigger revolution happening right now than just
| the tech industry.
|
| We will be forced to ask ourselves what it means to be human.
|
| No company is going to dominate AI, AI is going to dominate AI.
| atleastoptimal wrote:
| AI will dominate but only on huge distributed compute clusters.
| Local will be able to run models at reasonable speeds only when
| they're 4 years old.
| xyzzy123 wrote:
| All indications are that you can scale LLMs up & down pretty
| smoothly, I bet there will be lots of workflows in future
| using AI on pc-or mobile scale hardware which can call out to
| cloud AI for "deep thinking" or to double check stuff.
|
| It seems likely there will be AIs specialised in taxes or
| medicine, people/systems will consult multiple AIs to smooth
| out errors and look for consensus etc. In short, I agree with
| OP that we will probably end up with something that looks
| more like an ecosystem than a hegemony.
| yreg wrote:
| Doesn't the recent development (Stable Diffusion, LLaMA) tell
| us that these things actually run surprisingly well locally
| and that the models can be compressed much further?
|
| And aren't we finding out that the amount of parameters seems
| to matter less than we used to believe?
| thatjoeoverthr wrote:
| Four years will come and go.
| samspenc wrote:
| > The $11bn that Microsoft has reportedly put into Openai would,
| at the startup's latest rumoured valuation of $29bn, give the
| software giant a stake of 38%.
|
| That's not quite accurate, Microsoft first invested $1 billion in
| OpenAI in 2019 (four years ago) and another $10 billion a few
| months ago when it was valued at $29B.
|
| So it's latest $10B investment gives it a 35% ownership, but it's
| unclear how much the $1B 2019 investment gave Microsoft. Some
| news reports peg Microsoft's current ownership at 49% of OpenAI,
| which means Microsoft's 2019 investment gave it 14% of the
| company (which would further indicate that OpenAI was valued at
| $7-8 billion at that time).
| gsatic wrote:
| They will not dominate anything cuz of the number of useless
| teams within all fighting each other about who should own it. Big
| tech is going the way of IBM and good riddance.
| cardosof wrote:
| This. Some folks will spend their lives trying to find faster
| horses and build lighter carriages, and all is good and well
| until someone shows up with a car.
| HPMOR wrote:
| "in which Microsoft is implanting machine intelligence"
|
| I thought it was extremely adept change of referring to it as
| machine intelligence instead of artificial. There's nothing
| artificial about this intelligence, merely different. I think
| this will probably be increasingly more correct moving forward
| with additional advances.
| fnordpiglet wrote:
| As I'm sure someone will point out LLMs aren't intelligent.
|
| But then again, I don't think humans are either.
| acchow wrote:
| Probably half of Americans today would fail the Turing Test.
| Yoric wrote:
| Probably all of Americans today would fail one half of
| Americans on the Turing Test. Not all Americans the same
| half, of course.
| galaxytachyon wrote:
| Probably half of Americans and some government officials
| will fail the US citizenship test so not much to expect
| from the general populace...
| peteradio wrote:
| No you are wrong this is good and getting gooder.
| christkv wrote:
| I'm sure we will get model foundry businesses where we can buy a
| model and a runtime to run the models ourselves very soon.
| 13years wrote:
| It will be interesting to see how long anyone can dominate. AI
| might actually result in actually the opposite. Meaning that the
| era of years and decades of dominance is not sustainable any
| longer.
|
| Why would this be? Because AI allows you to copy your competitor
| in ways that was never possible before. It essentially becomes a
| skill and technology replicator. For example, look how Alpaca was
| able to somewhat replicate the multi million dollar models with
| $600 dollar cost.
| acchow wrote:
| The data that you train your model on is certainly a huge
| moat...
| kmeisthax wrote:
| Yes, but you don't own that data, it's scraped off the
| Internet. Either anyone can scrape data and train an AI with
| it, or nobody can do that: there is no special privilege or
| advantage for OpenAI.
| weixiyen wrote:
| what about RLHF at scale?
| sacnoradhq wrote:
| The other moat is tech to capture feedback with actionable
| metadata from users.
| 13years wrote:
| It is a moat to be the first. The question is can you defend
| that moat.
| hiddencost wrote:
| I'm excited for home assistant's voice efforts these years.
| Packaging speech and an assistant into open source hobby
| electronics could be incredibly powerful.
|
| https://www.home-assistant.io/blog/2022/12/20/year-of-voice/
| 13years wrote:
| Excellent. It will be interesting if this trend continues.
| Devices that you actually own instead of they owning you.
| amelius wrote:
| Yeah you can "own" your hardware until Nvidia and Intel
| start copying Apple's walled garden business practices.
| crop_rotation wrote:
| But Alpaca was possible only after Llama, and Llama is not
| gonna happen with 600 or 60000 USD.
| bogwog wrote:
| I don't have the link, but someone estimated that training a
| Llama clone from scratch would cost around ~$80k. That's
| pocket change for most corporations and some individuals.
| crop_rotation wrote:
| The prospect of not having your own LLM would be frightening to
| any big tech CEO. LLMs might be a massive productivity boost very
| soon (IMHO GPT4 is already there), and if you don't have one, but
| your competitor does, you will either have to use their product
| or perish.
| gumby wrote:
| I already have (before GPT 3 or 4) a plug in for Zoom which
| summarizes zoom calls, makes transcripts, and does all sorts of
| other things I don't care about (e.g. tries to gauge engagement
| and sentiment). It's pretty handy.
|
| Of course like everything these days it's a cloud silo: the
| transcripts, videos etc are in their cloud. There's no way to
| have them simply posted to slack channels so they can be found
| later in searches etc.
|
| The economic models and thinking of 2023 are seemingly still
| stuck in the PC era rather than being focused on the needs of the
| customer.
| mostlysimilar wrote:
| What do you mean by the "PC" era? At least with PCs as the
| dominant computing platform we had files and programs separate
| from each other. Smartphones began the trend of app-bound data.
| MoSattler wrote:
| https://archive.ph/ot8EH
___________________________________________________________________
(page generated 2023-03-27 23:01 UTC)