[HN Gopher] What does a leaked Google memo reveal about the futu...
___________________________________________________________________
What does a leaked Google memo reveal about the future of AI?
Author : Brajeshwar
Score : 85 points
Date : 2023-05-14 08:56 UTC (14 hours ago)
(HTM) web link (www.economist.com)
(TXT) w3m dump (www.economist.com)
| [deleted]
| gumby wrote:
| Was it leaked or "leaked"?
|
| Google has a good, two part strategy available.
|
| First is to neutralize the threat: support the open source
| efforts covertly and/or overtly, as those efforts undercut the
| revenue of OpenAI and other potential competitors for the base
| technology. This is what they did with Android: at a minimum all
| they had to do was ensure that Apple did not become a monopoly,
| which they accomplished (the secondary effect was to cut the
| underbrush of any other competitors). A name for this is to
| "commoditize your competition".
|
| The second half is to use their internal experience with LLMs and
| their other ecosystem to use supercharge their strengths. Side
| inventions can either be expanded in the hope of a (desperately
| needed, unrelated to LLMs) new revenue stream or given away to
| starve potential new entrants.
|
| The big guys have good experience in weaponizing open source. But
| google has become flabby and seemingly lost any culture of
| execution so I don't know if they can pull it off. But the path
| is there.
| [deleted]
| sitkack wrote:
| The doc is too short and not puffy to be a normal google doc.
| There is a high likelihood this was placed to proactively shake
| off any regulation or scruitiny while they get the deck chairs
| in order.
|
| They might _not_ appear to have a moat, but they definitely
| have an advantage that they can keep for 5+ years. They also
| have a "copy of the internet" that no one else does.
| akiselev wrote:
| What are the chances that their copy is more useful than
| CommonCrawl? It's been the target of SEO for decades - are
| they even capable of separating the good content from the
| crap anymore?
|
| Google may already be terminal
| chaxor wrote:
| Their data may not be that much better; however Deepmind is
| still one of the best institutions that study AI
| fundamentals. OpenAI works hard and focused towards
| products, but the academic research is far better from
| Deepmind. So since Google hasn't been trying to put any AI
| into products really at all for the past 10 years, they're
| going to have to push a little bit to make the public
| facing interfaces and put polish on things, but the
| underlying models they have made are far better than OpenAI
| (just not the ones made publicly accessible to anyone).
| visarga wrote:
| They have no model on par with GPT-4.
| sitkack wrote:
| Data pollution isn't the problem it is made out to be. You
| can even use a model trained on garbage data to relabel the
| original set and retrain.
|
| Google of all companies is sitting on exaflops of TPU
| capacity, they can retrain models larger than anyone else,
| faster than anyone else.
| visarga wrote:
| I agree, but it would probably require to use LLMs to
| filter or augment the raw text. They can afford it.
| UncleMeat wrote:
| > The doc is too short and not puffy to be a normal google
| doc.
|
| Loads of internal position papers read exactly like this. _I
| 've_ written docs that write like this.
| [deleted]
| mdp2021 wrote:
| > _AI burst into the public consciousness with the launch in late
| 2022_
|
| Because 70 years of named work based on centuries of interest and
| efforts can remain unseen if one really tries. /R
|
| > _But easier access to AI also means bad actors will be able to
| fine-tune systems for nefarious purposes, such as generating
| disinformation. It means Western attempts to prevent hostile
| regimes from gaining access to powerful AI technology will fail.
| And it makes AI harder to regulate, because the genie is out of
| the bottle_
|
| And the fact that after centuries of steep increase of technical
| possibilities in spite of low spiritual growth, with clear
| consequences every time met with bewilderment - you know, e.g.,
| WWI and the surprise (historically noted) post-facto after the
| heightened possible devastation -, should not suggest that what
| are to be tackled are the root issues? The <<genie is out of the
| bottle>> at least after the industrial revolution.
| sitkack wrote:
| You are just starting a flame war.
| scrollaway wrote:
| > _Because 70 years of named work based on centuries of
| interest and efforts can remain unseen if one really tries._
|
| You're really trying to be uncharitable eh?
|
| We've been talking about time travel for millennia. But I can
| guarantee you that if tomorrow we figure out a way for most
| people to actually reliably perform it and play with it, it
| will be defined as "bursting into the public consciousness".
| SoftTalker wrote:
| And it would be, no normal person (i.e. the public) thinks
| about time travel in any regular, meaningful way.
| AnonymousPlanet wrote:
| I must have overlooked all those conferences on time travel
| that are on the same level as, e.g., AAAI.
|
| But yes, public perception is a different thing. Most people
| are totally baffled when I tell them that the first spam
| email was sent in 1978, because for a lot of people the
| internet (and with it email) popped into existence somewhere
| around 2004.
| medoc wrote:
| Not to nitpick but this would rather be around 199x when
| AOL joined the Internet.
| AnonymousPlanet wrote:
| Oh, it's a different time for any group of people you
| ask. For some it was AOL, for others the media coverage
| that picked up around 1995/1996. And some were late to
| the game and only noticed when every one around them got
| DSL in the early 2000's. I kind of picked 2004 at random.
| It represents a (sadly rather sizeable) group of people
| in rural areas who neither got DSL nor internet over
| cable before that time and therefore just didn't pay
| attention.
| edgyquant wrote:
| But we had a presidential candidate run on fears of AI in
| 2020, so that poster has a real point.
| mdp2021 wrote:
| > _time travel_
|
| Really not the same. Time travel is hardly achievable and has
| a crisp goal. Artificial Intelligence, on the contrary, has
| been producing results since the start, and there are (there
| have been) entire businesses based on it.
|
| > _uncharitable_
|
| I was being <<uncharitable>> in front of the depiction of
| "masses discovering with surprise things that would have been
| noted by exploitation of the information available (and never
| had been as available as it is today)", also because on
| practical terms I find this unawareness of the world socially
| dangerous.
| siliconc0w wrote:
| On the low end, google can sprinkle in well integrated cost-
| optimized 'pretty good' LLMs into all their products which
| already have billions of DAU. It's hard for OpenAI to compete
| here, their models have to be significantly better to encourage
| people to copy and paste into ChatGPT and yet still cost
| effective. The high end is more interesting in that I'm not sure
| Google has a response to Gpt4. Specialized Palm2 models are
| allegedly SOTA in certain verticals but they're not generally
| available so it's hard to say. It's possible Google could offer a
| beefy model to Google One subscribers or something but they seem
| to be focused on the low end for now at least in the B2C arena
| with specialized models for B2B. I honestly don't think OSS is a
| big factor business-wise but it's nice to see and certainly
| enables more participation in the ecosystem.
| chaxor wrote:
| This idea that ChatGPT is better than the models that Google
| has is false; but I understand the perception. Deepmind has far
| better models than OpenAI, and the models OpenAI have made are
| based on the research done at Deepmind. For example, ChatGPT
| was not the first model trained by RLHF - that was Sparrow by
| Deepmind. The difference is that Google has been very
| disinterested in making products out of these systems, and
| didn't spend any effort to make them publicly accessible.
|
| Deepmind is still the leader in AI, and they have much better
| models - it's just going to take a while for people who don't
| study ML to realize this because Google hasn't released any of
| their good models for public use yet (and yeah, I'm not
| counting Bard, etc because the real model was probably put
| together by one guy in an afternoon at Google, and the rest of
| the time working was marketing).
| visarga wrote:
| > I honestly don't think OSS is a big factor business-wise
|
| Lots of companies are evaluating fine-tuning strategies for
| <60B LLMs. Such models can be efficient, private, cheaper and
| adapted to the use case. They may not be generalists like
| GPT-4, but you need them for scaling up usage on high volume
| calls on the same task if you don't want to have to pay a
| fortune to OpenAI, or for processing confidential information
| such as in medical, B2B and other cases. Not to mention the
| fear that OpenAI might snoop on their private data and use it
| in their corpus.
| janalsncm wrote:
| I don't know. If you define "AI" as having a great chatbot then
| probably no company has a moat anymore. But there are a lot more
| types of tasks which have nothing to do with that. Robotics, for
| one. Self-driving cars is another example. "AI" in the abstract
| is a meaninglessly vague concept but for all intents and purposes
| Google still has advantages over others in important areas.
|
| Their perpetual disadvantage of course is comically poor product
| vision, which puts any of their actual advantages in jeopardy.
| CharlesW wrote:
| > _But there are a lot more types of tasks which have nothing
| to do with that._
|
| I've found the Gartner AI Hype Cycle to be a useful tool for
| making this point with people who are under the impression that
| "AI" is mostly generative AI, and LLMs specifically:
| https://www.gartner.com/en/articles/what-s-new-in-artificial...
|
| In particular, the chart conveys this really quickly.
| Animats wrote:
| Wasn't this note from someone at Google discussed on HN last
| week, before The Economist picked it up?
|
| It is a huge issue for Google. Not because they're behind, but
| because chat engines don't seem to map well to an ad-supported
| model. Unless they talk like the Youtubers who start with "But
| first, let me tell you about our sponsor, FBX Crypto".
|
| They do map well to a surveillance model. That's already getting
| regulator attention. Look what happened to TikTok, which is
| really good at focusing on a user's interests. Chat engines focus
| like that. They have to, to work well.
| flutas wrote:
| Yep.
|
| Related:
|
| Google "We have no moat, and neither does OpenAI" -
| https://news.ycombinator.com/item?id=35813322 (May 2023)
| warrenmiller wrote:
| https://archive.is/pK4nv
| UncleMeat wrote:
| All the presentation of this has been weird.
|
| "Leaked Google memo" is true in a sense, but the doc was written
| by one person who isn't the primary leading force within relevant
| orgs at Google. The headline is more accurately described as
| "what does the opinion of one Googler reveal..."
| FooBarBizBazz wrote:
| "Leaks" can be press releases. "Oops, you weren't supposed to
| see that" can make the message seem more credible. Is this
| Google preemptively fending off accusations of monopoly in an
| important space?
|
| On the other hand, if Google is perceived as weak, then its
| share price will be negatively affected. So maybe not.
|
| Maybe it's just what some guy wrote.
|
| But then how did the leak happen? _Somebody_ chose to publicize
| this particular bit of text. Why?
| UncleMeat wrote:
| > "Leaks" can be press releases.
|
| It'd be weird not to integrate the comments first if this was
| being done by PR.
|
| > But then how did the leak happen?
|
| Somebody thought the doc was interesting and shared it with a
| news outlet.
|
| I've read 100+ docs that read like this. Some sort of
| position paper designed to spark discussion, much of which
| happens in the comments alongside the doc. All of them are
| just the opinion of a person, not the company.
| mrtranscendence wrote:
| Well, how do you tell the difference between a truly leaked
| internal document that reflects the opinion of one person,
| and a "leaked" document that just so happens to put forth a
| position that's beneficial to Google given its current
| standing in the LLM space? I think some skepticism is
| healthy here, as long as we understand that no one knows
| for sure.
| UncleMeat wrote:
| In this case, the trick is to look at the document
| internally and see the comment history.
| startupsfail wrote:
| Does it matter what is the origin of that opinion? As long as
| it is a good insight into the situation and accurately reflects
| the state of affairs, it shouldn't matter, right?
|
| And as to not being the leader at Google, this hardly matters.
| Being politically adept and leadership skills don't necessarily
| correlate with having better feel and insights into the future
| technology.
|
| And not being the leader reduces filters, doesn't have a pull
| of a contract to protect specific people you lead and your own
| project, etc.
|
| It'd be better to hear that from a founder, for certain, but a
| honest opinion of a well seasoned engineer often enough is a
| better reflection of the reality, than an official statement by
| a "leading force".
| oh_sigh wrote:
| Was any previously unknown info about Google leaked in the
| memo, that only an insider could know? If not, then why does
| it even matter that a googler wrote it? If "Anonymous" wrote
| the doc and uploaded it to pastebin, would you lend it the
| same credence you are to this doc?
| MrScruff wrote:
| It matters because the memo as being portrayed as 'the
| opinion of Google' by websites looking for clicks.
|
| The opinion is as relevant or not as any opinion expressed on
| Hacker News, the fact that the person who expressed it works
| for Google is irrelevant.
| startupsfail wrote:
| Well, there a honest opinion of an unbiased engineer,
| ideally a researcher who was trained to filter his own
| biases, from the inside of the organization , speaking
| honesty is likely the best approximation that you can get
| for "an opinion of Google".
|
| It is important that the engineer is an insider, as from
| outside of Google access to its internal knowledge is
| limited.
| skybrian wrote:
| Googlers are no more unbiased than someone who gets a job
| somewhere else. Getting a job at Google might affect your
| opinions somewhat when you see what people are doing, but
| there is no special training in being unbiased.
|
| Also, the document we're discussing was not written as a
| serious research paper. It's a more like one of Steve
| Yegge's rants, written to be provocative.
|
| When something Yegge wrote while at Google leaked, people
| knew who he was, so it was interpreted as "here's what
| Yegge thinks." It's the same in this case, but it's
| someone else who isn't famous.
| mattlondon wrote:
| > but there is no special training in being unbiased
|
| Citation needed! ;-)
| skybrian wrote:
| Well, I didn't get any training like that when I worked
| there, but it was a long time ago and I don't know what
| they're teaching Nooglers these days.
|
| I suppose you might be referring to diversity training,
| but it's not the same thing.
| ninth_ant wrote:
| It does matter, because Google is an important actor in this
| "industry".
|
| Even if it doesn't represent a monolithic "googles thoughts
| on AI" it is interesting and newsworthy to consider how at
| least some folks who work at Google think about the recent
| developments.
|
| If Google goes on a AI-everything spree and forces it
| awkwardly into all their products a la Google+ -- this may
| indicate how and where AI will be useful to people (or not).
| UncleMeat wrote:
| Google has nearly 200,000 employees. Almost every possible
| opinion is held by somebody within the company.
|
| Imagine this was written by Blake Lemoine, who believed that
| one of the earlier models was sentient. Would it be
| reasonable to present this as a company-wide belief?
| fauigerzigerk wrote:
| _> Does it matter what is the origin of that opinion? As long
| as it is a good insight into the situation and accurately
| reflects the state of affairs, it shouldn't matter, right?_
|
| It does matter a great deal whether a particular opinion is
| held by someone with the power to act on it or by someone who
| can only try to influence those who can act on it.
| dehrmann wrote:
| At the same time, I'm not sure I'd give any more weight to
| something written by "leading forces within relevant orgs" when
| these are the same people who missed ChatGPT and are playing
| catch-up.
| reillyduncN wrote:
| [dead]
___________________________________________________________________
(page generated 2023-05-14 23:01 UTC)