https://www.preprints.org/manuscript/202308.0148/v1

[sciprints_]  
   

  * Submit preprint
  * 
  * How it Works
  * Instructions for Authors
  * Subject Areas
  * Advisory Board
  * screening preprints
  * About
  * Statistics
  * 
  * Log in/Register

  * preprints  
  * Instructions for Authors
  * Statistics
  * About
  * FAQ
  * [                    ]
    Submit Log in/Register

[                    ]

Share this article with

 
 
 
 
x

Create alert

Email: [                    ]
[                    ]
CaptchaLoad new image
---------------------------------------------------------------------
Submit
x
preprints.org > computer science and mathematics > artificial
intelligence and machine learning > doi: 10.20944/
preprints202308.0148.v1
Preprint Article Version 1 Preserved in Portico This version is not
peer-reviewed

GPT-4 Can't Reason

[unknown-us]Konstantine Arkoudas
^*
Version 1 : Received: 27 July 2023 / Approved: 2 August 2023 /
Online: 2 August 2023 (04:10:57 CEST)
Version 2 : Received: 3 August 2023 / Approved: 4 August 2023 /
Online: 7 August 2023 (07:13:00 CEST)

How to cite: Arkoudas, K. GPT-4 Can't Reason. Preprints 2023,
2023080148. https://doi.org/10.20944/preprints202308.0148.v1 
Arkoudas, K. GPT-4 Can't Reason. Preprints 2023, 2023080148. https://
doi.org/10.20944/preprints202308.0148.v1 Copy

Cite

Export citation file: Bib TeX | RIS
MDPI and ACS Style

Arkoudas, K. GPT-4 Can't Reason. Preprints 2023, 2023080148. https://
doi.org/10.20944/preprints202308.0148.v1

APA Style

Arkoudas, K. (2023). GPT-4 Can't Reason. Preprints. https://doi.org/
10.20944/preprints202308.0148.v1

Chicago/Turabian Style

Arkoudas, K. 2023 "GPT-4 Can't Reason" Preprints. https://doi.org/
10.20944/preprints202308.0148.v1

Abstract

GPT-4 was released in March 2023 to wide acclaim, marking a very
substantial improvement across the board over GPT-3.5 (OpenAI's
previously best model, which had powered the initial release of
ChatGPT). Despite the genuinely impressive improvement, however,
there are good reasons to be highly skeptical of GPT-4's ability to
reason. This position paper discusses the nature of reasoning;
criticizes the current formulation of reasoning problems in the NLP
community and the way in which the reasoning performance of LLMs is
currently evaluated; introduces a collection of 21 diverse reasoning
problems; and performs a detailed qualitative analysis of GPT-4's
performance on these problems. Based on the results of this analysis,
the paper argues that, despite the occasional flashes of analytical
brilliance, GPT-4 at present is utterly incapable of reasoning.

Keywords

GPT-4; LLM; AI; reasoning; inference

Subject

Computer Science and Mathematics, Artificial Intelligence and Machine
Learning

Copyright: This is an open access article distributed under the
Creative Commons Attribution License which permits unrestricted use,
distribution, and reproduction in any medium, provided the original
work is properly cited.

Download PDF Download Supplementary

Comments (0)

Not displayed online.

Importance: How significant is the paper to the field?
( ) Outstanding/highlight paper
( ) Significant contribution
( ) Incremental contribution
( ) No contribution
( ) I am not qualified to judge
Soundness of evidence/arguments presented:
( ) Conclusions well supported
( ) Most conclusions supported (minor revision needed)
( ) Incomplete evidence (major revision needed)
( ) Hypothesis, unsupported conclusions, or proof-of-principle
( ) I am not qualified to judge
Please leave your comment to this article below.
1
2
S
O

Mathematical equations can be typed in either LaTeX formats \\[ ... \
\] or $$ ... $$, or MathML format <math> ... </math>. Try the LaTeX
or MathML example.

Type equation: [                    ] Preview:

Copy equation into message below
Close menu

Please click a symbol to insert it into the message box below:

+
++
%0




-
-
|
-
|
(tm)
...
-
--
!
C/
PS
$?
Y=
|
SS
(c)
a
<<
!

(r)
-
deg
+-
2
3
'
u
P
*
1
o
1/4
1/2
3/4
?
A
A
A
A
A
A
AE
C
E
E
E
E
I
I
I
I
D
N
O
O
O
O
O
x
O
U
U
U
U
Y
Th
ss
a
a
a
a
a
a
ae
c
e
e
e
e
i
i
i
i
d
n
o
o
o
o
o
/
o
u
u
u
u
y
th
y
A
a
B
b
G
g
D
d
E
e
Z
z
E
e
Th
th
I
i
K
k
L
l
M
m
N
n
Ks
x
O
o
P
p
R
r
S
s
T
t
U
u
Ph
ph
Kh
kh
Ps
ps
O
o
*
*

Close menu

Please enter the link here:

[http://             ]
Please enter a valid URL

Optionally, you can enter text that should appear as linked text:

[                    ]
Copy link into message below
Close menu

Please enter or paste the URL to the image here (please only use
links to jpg/jpeg, png and gif images):

[http://             ]
Please enter a valid URL
Copy image into message below
Close menu

Type author name or keywords to filter the list of references in this
group (you can add a new citation under Bibliography):
[                    ]

---------------------------------------------------------------------

No existing citations in Discussion Group

Wikify editor is a simple editor for wiki-style mark-up. It was
written by MDPI for Sciforum in 2014. The rendering of the mark-up is
based on Wiky.php with some tweaks. Rendering of mathematical
equations is done with MathJax. Please send us a message for support
or for reporting bugs.

Close menu
[                    ]
Please declare any conflicts of interest you may have with this
paper, financial or otherwise.
(*) I do not have a conflict of interest
( ) I would like to declare a conflict of interest
[                    ]
[*] Display my name on the website Send my name and email address to
the authors
Captcha
[_captcha_captcha]Renew[                    ]
[ ] I accept the following conditions:

Comments must follow the standards of professional discourse and
should focus on the scientific content of the article. Insulting or
offensive language, personal attacks and off-topic remarks will not
be permitted. Comments must be written in English. Preprints reserves
the right to remove comments without notice. Readers who post
comments are obliged to declare any competing interests, financial or
otherwise.

Cancel Post 
We encourage comments and feedback from a broad range of readers. See
criteria for comments and our Diversity statement.

Leave a public comment
Send a private comment to the author(s)
* All users must log in before leaving a comment

what's this?

Add a record of this review to Publons to track and showcase your
reviewing expertise across the world's journals.

x
Views 0
Downloads 0
Comments 0
Metrics 0
---------------------------------------------------------------------
Get PDF Supplementary Files Cite Share 0
---------------------------------------------------------------------
Bookmark BibSonomy Mendeley Reddit Delicious
---------------------------------------------------------------------
x
Alerts
Notify me about updates to this article or when a peer-reviewed
version is published.

  * [author_ser]
  * [encycloped]

[sciprints_]
Preprints.org is a free preprint server subsidized by MDPI in Basel,
Switzerland.
 
 
 
Contact us RSS

MDPI Initiatives

  * SciProfiles
  * Sciforum
  * Encyclopedia
  * MDPI Books
  * Scilit
  * Proceedings
  * JAMS

Important links

  * How it Works
  * Advisory Board
  * FAQ
  * Friendly Journals
  * Instructions for Authors
  * About

Subscribe

Choose the area that interest you and we will send you notifications
of new preprints at your preferred frequency.

Subscribe

Subscribe

[                    ]
[                    ]
Captcha 
---------------------------------------------------------------------
Submit
x
(c) 2023 MDPI (Basel, Switzerland) unless otherwise stated
Disclaimer
Privacy Policy
Terms of Use
We use cookies on our website to ensure you get the best experience.
Read more about our cookies here.
OK
[sciprints_]
RSS feed for
Preprints: The Multidisciplinary Preprint Platform
Paste the URL in your RSS reader:
[paste]https://www.preprints.org/rss
Please note that by using ScienceDirect RSS feeds, you agree to our
Terms of Use and Privacy Policy.
x