https://arxiv.org/abs/2509.22906

Skip to main content
Cornell University
We gratefully acknowledge support from the Simons Foundation, member
institutions, and all contributors. Donate
 
arxiv logo > cs > arXiv:2509.22906
[                    ]

Help | Advanced Search

[All fields        ]
Search
arXiv logo
Cornell University Logo
[                    ] GO
quick links

  * Login
  * Help Pages
  * About

Computer Science > Computation and Language

arXiv:2509.22906 (cs)
[Submitted on 26 Sep 2025]

Title:Extract-0: A Specialized Language Model for Document
Information Extraction

Authors:Henrique Godoy
View a PDF of the paper titled Extract-0: A Specialized Language
Model for Document Information Extraction, by Henrique Godoy
View PDF HTML (experimental)

    Abstract:This paper presents Extract-0, a 7-billion parameter
    language model specifically optimized for document information
    extraction that achieves performance exceeding models with
    parameter counts several orders of magnitude larger. Through a
    novel combination of synthetic data generation, supervised
    fine-tuning with Low-Rank Adaptation (LoRA), and reinforcement
    learning via Group Relative Policy Optimization (GRPO), Extract-0
    achieves a mean reward of 0.573 on a benchmark of 1,000 diverse
    document extraction tasks, outperforming GPT-4.1 (0.457), o3
    (0.464), and GPT-4.1-2025 (0.459). The training methodology
    employs a memory-preserving synthetic data generation pipeline
    that produces 280,128 training examples from diverse document
    sources, followed by parameterefficient fine-tuning that modifies
    only 0.53% of model weights (40.4M out of 7.66B parameters). The
    reinforcement learning phase introduces a novel semantic
    similarity-based reward function that handles the inherent
    ambiguity in information extraction tasks. This research
    demonstrates that task-specific optimization can yield models
    that surpass general-purpose systems while requiring
    substantially fewer computational resource.

Subjects: Computation and Language (cs.CL); Artificial Intelligence
          (cs.AI)
Cite as:  arXiv:2509.22906 [cs.CL]
          (or arXiv:2509.22906v1 [cs.CL] for this version)
          https://doi.org/10.48550/arXiv.2509.22906
          Focus to learn more
          arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Henrique Godoy [view email]
[v1] Fri, 26 Sep 2025 20:34:43 UTC (1,102 KB)
Full-text links:

Access Paper:

    View a PDF of the paper titled Extract-0: A Specialized Language
    Model for Document Information Extraction, by Henrique Godoy
  * View PDF
  * HTML (experimental)
  * TeX Source
  * Other Formats

view license
Current browse context:
cs.CL
< prev   |   next >
new | recent | 2025-09
Change to browse by:
cs
cs.AI

References & Citations

  * NASA ADS
  * Google Scholar
  * Semantic Scholar

export BibTeX citation Loading...

BibTeX formatted citation

x
[loading...          ]
Data provided by:

Bookmark

BibSonomy logo Reddit logo
(*) Bibliographic Tools

Bibliographic and Citation Tools

[ ] Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
[ ] Connected Papers Toggle
Connected Papers (What is Connected Papers?)
[ ] Litmaps Toggle
Litmaps (What is Litmaps?)
[ ] scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
( ) Code, Data, Media

Code, Data and Media Associated with this Article

[ ] alphaXiv Toggle
alphaXiv (What is alphaXiv?)
[ ] Links to Code Toggle
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
[ ] DagsHub Toggle
DagsHub (What is DagsHub?)
[ ] GotitPub Toggle
Gotit.pub (What is GotitPub?)
[ ] Huggingface Toggle
Hugging Face (What is Huggingface?)
[ ] Links to Code Toggle
Papers with Code (What is Papers with Code?)
[ ] ScienceCast Toggle
ScienceCast (What is ScienceCast?)
( ) Demos

Demos

[ ] Replicate Toggle
Replicate (What is Replicate?)
[ ] Spaces Toggle
Hugging Face Spaces (What is Spaces?)
[ ] Spaces Toggle
TXYZ.AI (What is TXYZ.AI?)
( ) Related Papers

Recommenders and Search Tools

[ ] Link to Influence Flower
Influence Flower (What are Influence Flowers?)
[ ] Core recommender toggle
CORE Recommender (What is CORE?)

  * Author
  * Venue
  * Institution
  * Topic

( ) About arXivLabs

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and
share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have
embraced and accepted our values of openness, community, excellence,
and user data privacy. arXiv is committed to these values and only
works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community?
Learn more about arXivLabs.

Which authors of this paper are endorsers? | Disable MathJax (What is
MathJax?)

  * About
  * Help

  * Click here to contact arXiv Contact
  * Click here to subscribe Subscribe

  * Copyright
  * Privacy Policy

  * Web Accessibility Assistance
  * arXiv Operational Status
    Get status notifications via email or slack