https://www.economist.com/leaders/2023/05/11/what-does-a-leaked-google-memo-reveal-about-the-future-of-ai

 Skip to content

  * Menu
  * Weekly edition
  * The world in brief
  * Search

Log in

  * Featured

      + War in Ukraine
      + Recession watch
      + US politics
      + Climate change
      + Coronavirus
      + A-Z of economics
      + Big Mac index
      + 1843 magazine
  * Sections

      + The world this week
      + Leaders
      + Letters
      + By Invitation
      + Briefing
      + United States
      + The Americas
      + Asia
      + China
      + Middle East & Africa
      + Europe
      + Britain
      + International
      + Business
      + Finance & economics
      + Science & technology
      + Culture
      + Economic & financial indicators
      + Graphic detail
      + The Economist explains
      + Obituary
      + Special reports
      + Technology Quarterly
      + The World Ahead
      + Essay
      + Schools brief
      + The Economist reads
      + Summer reads
      + Christmas Specials
  * More

      + Newsletters
      + Podcasts
      + Films
      + Subscriber events
      + The Economist app
      + Online courses

  * Opinion

      + Leaders
      + Letters to the editor
      + By Invitation
      + Current topics

          o War in Ukraine
          o Climate change
          o Coronavirus
          o The Biden presidency
          o Recession watch
          o The Economist explains
  * World

      + The world this week
      + China
      + United States
      + Europe
      + Britain
      + Middle East & Africa
      + Asia
      + The Americas
      + International
  * In depth

      + Science & technology
      + Graphic detail
      + Special reports
      + Technology Quarterly
      + The World Ahead
      + Briefing
      + Essay
      + Schools brief
  * Business & economics

      + Finance & economics
      + Business
      + Big Mac index
      + A-Z of economics
      + Economic & financial indicators
  * Culture & society

      + 1843 magazine
      + Culture
      + Obituary
      + The Economist reads
      + Summer reads
      + Christmas Specials
  * More

      + Podcasts
      + Newsletters
      + Films
      + The Economist app
      + Subscriber events
      + Online courses

  * My Economist
  * Saved stories
  * Log out

  * Saved stories
  * Account
  * Log out

Search
[                    ]
Leaders | A stochastic parrot in every pot

What does a leaked Google memo reveal about the future of AI?

Open-source AI is booming. That makes it less likely that a handful
of firms will control the technology

[20230513_LDD002]
May 11th 2023
Share

THEY HAVE changed the world by writing software. But techy types are
also known for composing lengthy memos in prose, the most famous of
which have marked turning points in computing. Think of Bill Gates's
"Internet tidal wave" memo of 1995, which reoriented Microsoft
towards the web; or Jeff Bezos's "API mandate" memo of 2002, which
opened up Amazon's digital infrastructure, paving the way for modern
cloud computing. Now techies are abuzz about another memo, this time
leaked from within Google, titled "We have no moat". Its unknown
author details the astonishing progress being made in artificial
intelligence (AI)--and challenges some long-held assumptions about the
balance of power in this fast-moving industry.

Listen to this story. Enjoy more audio and podcasts on iOS or Android
.

Your browser does not support the <audio> element.

AI burst into the public consciousness with the launch in late 2022
of ChatGPT, a chatbot powered by a "large language model" (LLM) made
by OpenAI, a startup closely linked to Microsoft. Its success
prompted Google and other tech firms to release their own LLM-powered
chatbots. Such systems can generate text and hold realistic
conversations because they have been trained using trillions of words
taken from the internet. Training a large LLM takes months and costs
tens of millions of dollars. This led to concerns that AI would be
dominated by a few deep-pocketed firms.

But that assumption is wrong, says the Google memo. It notes that
researchers in the open-source community, using free, online
resources, are now achieving results comparable to the biggest
proprietary models. It turns out that LLMs can be "fine-tuned" using
a technique called low-rank adaptation, or LoRa. This allows an
existing LLM to be optimised for a particular task far more quickly
and cheaply than training an LLM from scratch.

Activity in open-source AI exploded in March, when LLaMa, a model
created by Meta, Facebook's parent, was leaked online. Although it is
smaller than the largest LLMs (its smallest version has 7bn
parameters, compared with 540bn for Google's PaLM) it was quickly
fine-tuned to produce results comparable to the original version of
ChatGPT on some tasks. As open-source researchers built on each
other's work with LLaMa, "a tremendous outpouring of innovation
followed," the memo's author writes.

This could have seismic implications for the industry's future. "The
barrier to entry for training and experimentation has dropped from
the total output of a major research organisation to one person, an
evening, and a beefy laptop," the Google memo claims. An LLM can now
be fine-tuned for $100 in a few hours. With its fast-moving,
collaborative and low-cost model, "open-source has some significant
advantages that we cannot replicate." Hence the memo's title: this
may mean Google has no defensive "moat" against open-source
competitors. Nor, for that matter, does OpenAI.

Not everyone agrees with this thesis. It is true that the internet
runs on open-source software. But people use paid-for, proprietary
software, from Adobe Photoshop to Microsoft Windows, as well. AI may
find a similar balance. Moreover, benchmarking AI systems is
notoriously hard. Yet even if the memo is partly right, the
implication is that access to AI technology will be far more
democratised than seemed possible even a year ago. Powerful LLMs can
be run on a laptop; anyone who wants to can now fine-tune their own 
AI.

This has both positive and negative implications. On the plus side,
it makes monopolistic control of AI by a handful of companies far
less likely. It will make access to AI much cheaper, accelerate
innovation across the field and make it easier for researchers to
analyse the behaviour of AI systems (their access to proprietary
models was limited), boosting transparency and safety. But easier
access to AI also means bad actors will be able to fine-tune systems
for nefarious purposes, such as generating disinformation. It means
Western attempts to prevent hostile regimes from gaining access to
powerful AI technology will fail. And it makes AI harder to regulate,
because the genie is out of the bottle.

Whether Google and its ilk really have lost their moat in AI will
soon become apparent. But as with those previous memos, this feels
like another turning point for computing. #

This article appeared in the Leaders section of the print edition
under the headline "A stochastic parrot in every pot?"

Leaders May 13th 2023

  * Is Chinese power about to peak?
  * Small, sensible steps could help ease America's border woes
  * The rehabilitation of Syria's dictator raises awkward questions
    for the West
  * Joe Biden is more responsible for high inflation than for
    abundant jobs
  * What does a leaked Google memo reveal about the future of AI?

Peak China?

From the May 13th 2023 edition

Discover stories from this section and more in the list of contents

Explore the edition
Share
Reuse this content

More from Leaders

[20230513_LDP002]

Small, sensible steps could help ease America's border woes

The art of the practical in dealing with migrants, drugs and gangs

[20230513_LDP502]

The rehabilitation of Syria's dictator raises awkward questions for
the West

Clearer principles about how and when to ease sanctions are needed

---------------------------------------------------------------------
[20230513_LDD001]

Is Chinese power about to peak?

The country's historic ascent is levelling off. That need not make it
more dangerous

---------------------------------------------------------------------

  * Subscribe
  * Group subscriptions
  * Reuse our content
  * The Trust Project
  * Help and contact us

Keep updated

  *  
  *  
  *  
  *  
  *  
  *  

Published since September 1843 to take part in "a severe contest
between intelligence, which presses forward, and an unworthy, timid
ignorance obstructing our progress."

The Economist

  * About
  * Advertise
  * Press centre

The Economist Group

  * The Economist Group
  * Economist Intelligence
  * Economist Impact
  * Economist Events
  * Working Here
  * Economist Education Courses
  * Which MBA?
  * Executive Jobs
  * Executive Education Navigator

  * Terms of Use
  * Privacy
  * Cookie Policy
  * Manage Cookies
  * Accessibility
  * Modern Slavery Statement
  * Sitemap
  * California: Do Not Sell My Personal Information

Copyright (c) The Economist Newspaper Limited 2023. All rights
reserved.