https://arxiv.org/abs/2111.14232

close this message
arXiv smileybones icon

Global Survey

In just 3 minutes, help us better understand how you perceive arXiv.
Take the survey

TAKE SURVEY
Skip to main content
Cornell University
We gratefully acknowledge support from
the Simons Foundation and member institutions.
 
arXiv.org > q-bio > arXiv:2111.14232
[                    ]

Help | Advanced Search

[All fields        ]
Search
arXiv
Cornell University Logo
[                    ] GO
quick links

  * Login
  * Help Pages
  * About

Quantitative Biology > Neurons and Cognition

arXiv:2111.14232 (q-bio)
[Submitted on 28 Nov 2021]

Title:Long-range and hierarchical language predictions in brains and
algorithms

Authors:Charlotte Caucheteux, Alexandre Gramfort, Jean-Remi King
Download PDF

    Abstract: Deep learning has recently made remarkable progress in
    natural language processing. Yet, the resulting algorithms remain
    far from competing with the language abilities of the human
    brain. Predictive coding theory offers a potential explanation to
    this discrepancy: while deep language algorithms are optimized to
    predict adjacent words, the human brain would be tuned to make
    long-range and hierarchical predictions. To test this hypothesis,
    we analyze the fMRI brain signals of 304 subjects each listening
    to 70min of short stories. After confirming that the activations
    of deep language algorithms linearly map onto those of the brain,
    we show that enhancing these models with long-range forecast
    representations improves their brain-mapping. The results further
    reveal a hierarchy of predictions in the brain, whereby the
    fronto-parietal cortices forecast more abstract and more distant
    representations than the temporal cortices. Overall, this study
    strengthens predictive coding theory and suggests a critical role
    of long-range and hierarchical predictions in natural language
    processing.

          Neurons and Cognition (q-bio.NC); Artificial Intelligence
Subjects: (cs.AI); Computation and Language (cs.CL); Machine Learning
          (cs.LG); Neural and Evolutionary Computing (cs.NE)
Cite as:  arXiv:2111.14232 [q-bio.NC]
          (or arXiv:2111.14232v1 [q-bio.NC] for this version)

Submission history

From: Charlotte Caucheteux [view email]
[v1] Sun, 28 Nov 2021 20:26:07 UTC (17,352 KB)
Full-text links:

Download:

  * PDF
  * Other formats

[by-4]
Current browse context:
q-bio.NC
< prev   |   next >
new | recent | 2111
Change to browse by:
cs
cs.AI
cs.CL
cs.LG
cs.NE
q-bio

References & Citations

  * NASA ADS
  * Google Scholar
  * Semantic Scholar

a export bibtex citation Loading...

Bibtex formatted citation

x
[loading...          ]
Data provided by:

Bookmark

BibSonomy logo Mendeley logo Reddit logo ScienceWISE logo
(*) Bibliographic Tools

Bibliographic and Citation Tools

[ ] Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
[ ] Litmaps Toggle
Litmaps (What is Litmaps?)
[ ] scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
( ) Code & Data

Code and Data Associated with this Article

[ ] arXiv Links to Code Toggle
arXiv Links to Code & Data (What is Links to Code & Data?)
( ) Related Papers

Recommenders and Search Tools

[ ] Connected Papers Toggle
Connected Papers (What is Connected Papers?)
[ ] Core recommender toggle
CORE Recommender (What is CORE?)
( ) About arXivLabs

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and
share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have
embraced and accepted our values of openness, community, excellence,
and user data privacy. arXiv is committed to these values and only
works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community?
Learn more about arXivLabs and how to get involved.

Which authors of this paper are endorsers? | Disable MathJax (What is
MathJax?)

  * About
  * Help

  * Click here to contact arXiv Contact
  * Click here to subscribe Subscribe

  * Copyright
  * Privacy Policy

  * Web Accessibility Assistance
  * arXiv Operational Status
    Get status notifications via email or slack