[HN Gopher] Google Co-Scientist AI fed previous paper with the a...
___________________________________________________________________
Google Co-Scientist AI fed previous paper with the answer in it
Author : pcfwik
Score : 86 points
Date : 2025-02-24 17:52 UTC (5 hours ago)
(HTM) web link (pivot-to-ai.com)
(TXT) w3m dump (pivot-to-ai.com)
| dmitrygr wrote:
| not surprising. why would you expect a simple next token
| predictor to think?
| triceratops wrote:
| Technically everyone is a next token predictor. The only
| difference is how good the prediction is.
| olyjohn wrote:
| Technically, everyone is just an ugly giant bag of mostly
| water.
| abenga wrote:
| Not true, we build novel knowledge all the time. You only
| have to look at *gestures at everything *.
| triceratops wrote:
| Right, sometimes we invent new tokens. It's unclear whether
| large models can do that yet.
| Frieren wrote:
| > everyone is a next token predictor
|
| As true as "everyone is a calculator". There are superficial
| similarities, but that we achieve a similar result does not
| mean that we follow the same reasoning (or lack of).
| triceratops wrote:
| > does not mean that we follow the same reasoning
|
| Finally some logic. Token prediction alone - the behavior -
| isn't proof of a lack of intelligence.
|
| Better reasoning gives better token predictions. And,
| sometimes, the ability to invent new tokens entirely.
| aezart wrote:
| In humans, language is based on the state of the world, and
| anticipates future states of the world. In an LLM, language
| is just language.
|
| If an LLM says "my favorite flavor of ice cream is cookie
| dough", it's just a statistically plausible thing to say.
|
| If _I_ say "my favorite flavor if ice cream is cookie
| dough", it's because I'm trying to deceive someone, because
| for some reason I don't want them to know my real favorite
| flavor is cake batter.
| fennecbutt wrote:
| I mean the logic behind it is sound. Just infantile compared
| to what we do. But yes I believe that all we do boils down to
| "if x is true and also y is true, therefore under these
| circumstances z must be false, I'll give it a go and find
| out". Plus a little qualified randomness, look at the large
| percentage of human discoveries being made accidentally
| (albeit by qualified professionals making best guesses).
| umanwizard wrote:
| Please rigorously define what the term "next token predictor"
| means.
| dmitrygr wrote:
| A device or system that, given a set of tokens, predicts the
| next token purely based on the previous tokens and its own
| initial state. It is incapable of coming up with a new token
| type or changing its own workings
| euroderf wrote:
| If the AI found a needle in a haystack, that isn't bad, is it?
| Lastminutepanic wrote:
| Do you think it's possible that the person who literally wrote
| (multiple) papers with the answer AI gave, may phrase their
| question to the AI with similar, or even the exact phrasing
| they used in previous papers on the same subject ? If I load a
| bunch of lorem ipsum into AI scientist, labeled as scientific
| papers, then ask the AI "lorem ipsum" do you think it'll spit
| out some gibberish from those specific papers?
| kedean wrote:
| The point isn't that it's bad, the point is that the AI did not
| discover the solution to the problem independently, as news
| headlines had implied.
| hirenj wrote:
| Colour me unsurprised - even not knowing data leakage had
| occurred, the hypothesis was underwhelming, as I mentioned in a
| comment on an earlier discussion. I sometimes despair for the
| state of thinking in science these days given how quickly people
| fawn over entirely pedestrian thinking and work.
|
| https://news.ycombinator.com/item?id=43105759
| mandevil wrote:
| I mean, to be fair, "knowing all of the relevant literature" is a
| great first step for solving a problem! And a LLM can probably be
| better about not forgetting than humans are (though I'm guessing
| they do much worse on the hallucination front than humans do-
| humans tend to know when they are playing a hunch). But "This
| paper from a lower-tier journal two years ago suggests you should
| look at X" is a very valuable thing to have!
| rideontime wrote:
| Instant subscribe when I saw David Gerard's name. He cut through
| the BS in crypto and we need more people like him focusing on AI
| fraud like this.
___________________________________________________________________
(page generated 2025-02-24 23:01 UTC)