https://arxiv.org/abs/2108.07258

close this message

Donate to arXiv

Please join the Simons Foundation and our generous member
organizations in supporting arXiv during our giving campaign
September 23-27. 100% of your contribution will fund improvements and
new initiatives to benefit arXiv's global scientific community.

DONATE

[secure site, no need to create account]

Skip to main content
Cornell University
We gratefully acknowledge support from
the Simons Foundation and member institutions.
 
arXiv.org > cs > arXiv:2108.07258
[                    ]

Help | Advanced Search

[All fields        ]
Search
arXiv
Cornell University Logo
[                    ] GO
quick links

  * Login
  * Help Pages
  * About

Computer Science > Machine Learning

arXiv:2108.07258 (cs)
[Submitted on 16 Aug 2021 (v1), last revised 18 Aug 2021 (this
version, v2)]

Title:On the Opportunities and Risks of Foundation Models

Authors:Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ Altman,
Simran Arora, Sydney von Arx, Michael S. Bernstein, Jeannette Bohg,
Antoine Bosselut, Emma Brunskill, Erik Brynjolfsson, Shyamal Buch,
Dallas Card, Rodrigo Castellon, Niladri Chatterji, Annie Chen,
Kathleen Creel, Jared Quincy Davis, Dora Demszky, Chris Donahue,
Moussa Doumbouya, Esin Durmus, Stefano Ermon, John Etchemendy, Kawin
Ethayarajh, Li Fei-Fei, Chelsea Finn, Trevor Gale, Lauren Gillespie,
Karan Goel, Noah Goodman, Shelby Grossman, Neel Guha, Tatsunori
Hashimoto, Peter Henderson, John Hewitt, Daniel E. Ho, Jenny Hong,
Kyle Hsu, Jing Huang, Thomas Icard, Saahil Jain, Dan Jurafsky,
Pratyusha Kalluri, Siddharth Karamcheti, Geoff Keeling, Fereshte
Khani, Omar Khattab, Pang Wei Kohd, Mark Krass, Ranjay Krishna,
Rohith Kuditipudi, Ananya Kumar, Faisal Ladhak, Mina Lee, Tony Lee,
Jure Leskovec, Isabelle Levent, Xiang Lisa Li, Xuechen Li, Tengyu Ma,
Ali Malik, Christopher D. Manning, Suvir Mirchandani, Eric Mitchell,
Zanele Munyikwa, Suraj Nair, Avanika Narayan, Deepak Narayanan, Ben
Newman, Allen Nie, Juan Carlos Niebles, Hamed Nilforoshan, Julian
Nyarko, Giray Ogut, Laurel Orr, Isabel Papadimitriou, Joon Sung Park,
Chris Piech, Eva Portelance, Christopher Potts, Aditi Raghunathan,
Rob Reich, Hongyu Ren, Frieda Rong, Yusuf Roohani, Camilo Ruiz, Jack
Ryan, Christopher Re, Dorsa Sadigh, Shiori Sagawa, Keshav Santhanam,
Andy Shih, Krishnan Srinivasan, Alex Tamkin, Rohan Taori, Armin W.
Thomas, Florian Tramer, Rose E. Wang, William Wang
, Bohan Wu, Jiajun Wu, Yuhuai Wu, Sang Michael Xie, Michihiro
Yasunaga, Jiaxuan You, Matei Zaharia, Michael Zhang, Tianyi Zhang,
Xikun Zhang, Yuhui Zhang, Lucia Zheng, Kaitlyn Zhou, Percy Liang
et al. (14 additional authors not shown)  You must enable JavaScript
to view entire author list.
Download PDF

    Abstract: AI is undergoing a paradigm shift with the rise of
    models (e.g., BERT, DALL-E, GPT-3) that are trained on broad data
    at scale and are adaptable to a wide range of downstream tasks.
    We call these models foundation models to underscore their
    critically central yet incomplete character. This report provides
    a thorough account of the opportunities and risks of foundation
    models, ranging from their capabilities (e.g., language, vision,
    robotics, reasoning, human interaction) and technical principles
    (e.g., model architectures, training procedures, data, systems,
    security, evaluation, theory) to their applications (e.g., law,
    healthcare, education) and societal impact (e.g., inequity,
    misuse, economic and environmental impact, legal and ethical
    considerations). Though foundation models are based on standard
    deep learning and transfer learning, their scale results in new
    emergent capabilities,and their effectiveness across so many
    tasks incentivizes homogenization. Homogenization provides
    powerful leverage but demands caution, as the defects of the
    foundation model are inherited by all the adapted models
    downstream. Despite the impending widespread deployment of
    foundation models, we currently lack a clear understanding of how
    they work, when they fail, and what they are even capable of due
    to their emergent properties. To tackle these questions, we
    believe much of the critical research on foundation models will
    require deep interdisciplinary collaboration commensurate with
    their fundamentally sociotechnical nature.

          Authored by the Center for Research on Foundation Models
Comments: (CRFM) at the Stanford Institute for Human-Centered
          Artificial Intelligence (HAI)
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI);
          Computers and Society (cs.CY)
Cite as:  arXiv:2108.07258 [cs.LG]
          (or arXiv:2108.07258v2 [cs.LG] for this version)

Submission history

From: Rishi Bommasani [view email]
[v1] Mon, 16 Aug 2021 17:50:08 UTC (16,173 KB)
[v2] Wed, 18 Aug 2021 17:07:22 UTC (16,193 KB)
Full-text links:

Download:

  * PDF
  * Other formats

[by-4]
Current browse context:
cs.LG
< prev   |   next >
new | recent | 2108
Change to browse by:
cs
cs.AI
cs.CY

References & Citations

  * NASA ADS
  * Google Scholar
  * Semantic Scholar

a export bibtex citation Loading...

Bibtex formatted citation

x
[loading...          ]
Data provided by:

Bookmark

BibSonomy logo Mendeley logo Reddit logo ScienceWISE logo
(*) Bibliographic Tools

Bibliographic and Citation Tools

[ ] Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
[ ] Litmaps Toggle
Litmaps (What is Litmaps?)
( ) Code & Data

Code and Data Associated with this Article

[ ] arXiv Links to Code Toggle
arXiv Links to Code & Data (What is Links to Code & Data?)
( ) Related Papers

Recommenders and Search Tools

[ ] Connected Papers Toggle
Connected Papers (What is Connected Papers?)
[ ] Core recommender toggle
CORE Recommender (What is CORE?)
( ) About arXivLabs

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and
share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have
embraced and accepted our values of openness, community, excellence,
and user data privacy. arXiv is committed to these values and only
works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community?
Learn more about arXivLabs and how to get involved.

Which authors of this paper are endorsers? | Disable MathJax (What is
MathJax?)

  * About
  * Help

  * Click here to contact arXiv Contact
  * Click here to subscribe Subscribe

  * Copyright
  * Privacy Policy

  * Web Accessibility Assistance
  * arXiv Operational Status
    Get status notifications via email or slack