https://arxiv.org/abs/2402.16671

Skip to main content
Cornell University
We gratefully acknowledge support from the Simons Foundation, member
institutions, and all contributors. Donate
 
arxiv logo > cs > arXiv:2402.16671
[                    ]

Help | Advanced Search

[All fields        ]
Search
arXiv logo
Cornell University Logo
[                    ] GO
quick links

  * Login
  * Help Pages
  * About

Computer Science > Computation and Language

arXiv:2402.16671 (cs)
[Submitted on 26 Feb 2024 (v1), last revised 24 Apr 2024 (this
version, v6)]

Title:StructLM: Towards Building Generalist Models for Structured
Knowledge Grounding

Authors:Alex Zhuang, Ge Zhang, Tianyu Zheng, Xinrun Du, Junjie Wang,
Weiming Ren, Stephen W. Huang, Jie Fu, Xiang Yue, Wenhu Chen
View a PDF of the paper titled StructLM: Towards Building Generalist
Models for Structured Knowledge Grounding, by Alex Zhuang and 9 other
authors
View PDF HTML (experimental)

    Abstract:Structured data sources, such as tables, graphs, and
    databases, are ubiquitous knowledge sources. Despite the
    demonstrated capabilities of large language models (LLMs) on
    plain text, their proficiency in interpreting and utilizing
    structured data remains limited. Our investigation reveals a
    notable deficiency in LLMs' ability to process structured data,
    e.g., ChatGPT lags behind state-of-the-art (SoTA) model by an
    average of 35%. To augment the Structured Knowledge Grounding
    (SKG) capabilities in LLMs, we have developed a comprehensive
    instruction tuning dataset comprising 1.1 million examples.
    Utilizing this dataset, we train a series of models, referred to
    as StructLM, based on the Mistral and the CodeLlama model family,
    ranging from 7B to 34B parameters. Our StructLM series surpasses
    task-specific models on 16 out of 18 evaluated datasets and
    establishes new SoTA performance on 8 SKG tasks. Furthermore,
    StructLM demonstrates strong generalization across 6 novel
    held-out SKG tasks, outperforming TableLlama by an average of 35\
    % and Flan-UL2 20B by an average of 10\%. Contrary to
    expectations, we observe that scaling model size offers marginal
    benefits, with StructLM-34B showing only slight improvements over
    StructLM-7B. This suggests that structured knowledge grounding is
    still a challenging task and requires more innovative design to
    push to a new level.

Comments: Technical Report
Subjects: Computation and Language (cs.CL)
Cite as:  arXiv:2402.16671 [cs.CL]
          (or arXiv:2402.16671v6 [cs.CL] for this version)
          https://doi.org/10.48550/arXiv.2402.16671
          Focus to learn more
          arXiv-issued DOI via DataCite

Submission history

From: Wenhu Chen [view email]
[v1] Mon, 26 Feb 2024 15:47:01 UTC (554 KB)
[v2] Wed, 28 Feb 2024 14:49:03 UTC (555 KB)
[v3] Sun, 31 Mar 2024 20:14:20 UTC (1,141 KB)
[v4] Sun, 21 Apr 2024 01:06:24 UTC (1,043 KB)
[v5] Tue, 23 Apr 2024 17:29:25 UTC (1,042 KB)
[v6] Wed, 24 Apr 2024 21:13:24 UTC (1,059 KB)
Full-text links:

Access Paper:

    View a PDF of the paper titled StructLM: Towards Building
    Generalist Models for Structured Knowledge Grounding, by Alex
    Zhuang and 9 other authors
  * View PDF
  * HTML (experimental)
  * TeX Source
  * Other Formats

view license
Current browse context:
cs.CL
< prev   |   next >
new | recent | 2402
Change to browse by:
cs

References & Citations

  * NASA ADS
  * Google Scholar
  * Semantic Scholar

a export BibTeX citation Loading...

BibTeX formatted citation

x
[loading...          ]
Data provided by:

Bookmark

BibSonomy logo Reddit logo
(*) Bibliographic Tools

Bibliographic and Citation Tools

[ ] Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
[ ] Litmaps Toggle
Litmaps (What is Litmaps?)
[ ] scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
( ) Code, Data, Media

Code, Data and Media Associated with this Article

[ ] Links to Code Toggle
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
[ ] DagsHub Toggle
DagsHub (What is DagsHub?)
[ ] GotitPub Toggle
Gotit.pub (What is GotitPub?)
[ ] Links to Code Toggle
Papers with Code (What is Papers with Code?)
[ ] ScienceCast Toggle
ScienceCast (What is ScienceCast?)
( ) Demos

Demos

[ ] Replicate Toggle
Replicate (What is Replicate?)
[ ] Spaces Toggle
Hugging Face Spaces (What is Spaces?)
[ ] Spaces Toggle
TXYZ.AI (What is TXYZ.AI?)
( ) Related Papers

Recommenders and Search Tools

[ ] Link to Influence Flower
Influence Flower (What are Influence Flowers?)
[ ] Connected Papers Toggle
Connected Papers (What is Connected Papers?)
[ ] Core recommender toggle
CORE Recommender (What is CORE?)

  * Author
  * Venue
  * Institution
  * Topic

( ) About arXivLabs

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and
share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have
embraced and accepted our values of openness, community, excellence,
and user data privacy. arXiv is committed to these values and only
works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community?
Learn more about arXivLabs.

Which authors of this paper are endorsers? | Disable MathJax (What is
MathJax?)

  * About
  * Help

  * Click here to contact arXiv Contact
  * Click here to subscribe Subscribe

  * Copyright
  * Privacy Policy

  * Web Accessibility Assistance
  * arXiv Operational Status
    Get status notifications via email or slack