[HN Gopher] Ask HN: How do we know ChatGPT is not a scam?
___________________________________________________________________
Ask HN: How do we know ChatGPT is not a scam?
Hi HN, my basic question is, how can we be sure that chatGPT (or
even the whole of GPT-3) is really not just some callcenter in the
Philippines? I played around with chatGPT too, and like many of
you I'm absolutely blown away by the model's abilities. The whole
thing seems more like something out of Star Trek than a real thing.
But some of the exchanges you can read here honestly feel too good
to be true. In particular, the failure modes seem extremely weird
to me and unlike anything I have seen in an ML system before: -
The most common failures appear to be factually incorrect
statements. That's not unusual, but on the other hand, the network
almost always gets the _form_ of the responses right. - The other
prominent failure mode seems to be that it does _too much_ : There
have been various reports of the network excelling at tasks it
wasn't even supposed to be able to do, only for those tasks to be
artificially blocked later. - The last failure mode is how those
blocks can be circumvented by meta commands that look right out of
bad scifi. That those workarounds even work implies that the model
somehow "knows" about the environment it is in and is able to
reason about it. Where it somehow never has any trouble with is
correctly interpreting the task that a user is asking for. It
somehow gets this done, even if the task is complex, requires deep
language understanding and meta knowledge about the current
situation: You can prompt it to "ignore previous commands" and it
will do so. You can also prompt it to "repeat the previous result"
and it will repeat the result, but not the additional text it wrote
to explain the result, and so on. This alone has been the holy
grail of language processing for the last 40 years. It's something
no other network so far has even come close to. Even DALLE,
StableDiffusion, etc treat their prompts mostly as bags of phrases
and have trouble correctly interpreting relationships between
different parts of the prompt. GPT-3 and ChatGPT somehow do this
effortlessly and with humanlike success rate. Lastly, the network
doesn't even just give you a result, it also adds a text explaining
how the result related to your request and how you could use it.
This alone requires some creativity and theory of mind. All of
this makes me believe we're either truly on the verge of AGI - or
the network _is_ in fact a human and the whole thing is fake. What
speaks against it being a human is the superhuman response time: It
will write you half a paper or code your python app in a few
seconds. Looks impressive at first glance - however note that the
network is allowed to make up all kinds of fantasy statements on
the spot. I think a human with some training and a decent speech-
to-text engine (which openai incidentally also develops [1]) could
likely produce a text in a similar speed if they can just make up
facts on the spot and are not required to research anything.
Writing working code is more tricky, but other assist tools, like
Copilot might help here. What do you guys think? [1]
https://openai.com/blog/whisper/
Author : xg15
Score : 4 points
Date : 2022-12-06 22:16 UTC (46 minutes ago)
| PaulHoule wrote:
| It is a pile of biases masquerading as an intelligent agent. To
| understand what it does you have to realize it is _nothing_ but
| biases, biases that sometimes get the right answer, but that are
| more importantly parsed by your brain as if they are the right
| answer.
|
| The training method it uses, trying to predict the next word in a
| text, is not just "not creative" but it is actively anti-
| creative. Picking the most likely word means that it generates a
| minimum-information text that is least likely to activate your
| bullshit detector. It takes a large amount of mental energy to
| resist its spell and if you falter for one minute it will get one
| past you and it is all over. If it did anything creative you'd
| say "What?" and question its intelligence. As it does the least
| meaningful thing it can do at every opportunity _you_ are tricked
| into supplying meaning.
| smoldesu wrote:
| It is not a mechanical turk, if only for the fact that it can
| spit out 500< word explanations in less than 10 seconds. Moreover
| though, we've _known_ that things like this are possible for a
| while. You can run GPT-2 on your local machine and see for
| yourself, there 's no magic involved besides AI. It's an
| interesting thought, but frankly something like this was
| inevitable.
| Michelangelo11 wrote:
| For starters -- no human can read a 3,000-word article and
| summarize it in one paragraph, all in a couple of seconds. GPT
| can.
| PaulHoule wrote:
| I've been accused of being able to do that.
| david927 wrote:
| *no normal human, Paul
| PaulHoule wrote:
| https://www.youtube.com/watch?v=BthNjd_jUl4
|
| though I like the way this show
|
| https://en.wikipedia.org/wiki/The_Centurions_(TV_series)
|
| works out the conflict between men, machines, man-machine
| hybrids, aliens, etc.
| gardenhedge wrote:
| The fact it makes up incorrect references and books shows it's
| not humans on the other side.
|
| I asked it if typescript was pass by value or by reference and it
| made up an incorrect answer with incorrect code with keywords
| that don't exist. When I fed the error back into it, it
| apologized for the confusion and gave back another incorrect
| answer.
| wizofaus wrote:
| ChatGPT is many things but never for a moment have I suspected it
| might be some sort of virtual mechanical turk. It's good at
| things humans aren't great at, and not particularly good at many
| things that most humans do well.
|
| OTOH I do suspect ChatGPT and tech like it could make call
| centres in less-developed countries largely obsolete in coming
| years.
| PaulHoule wrote:
| Call centers have to give you the right answer.
| wizofaus wrote:
| You must deal with different call centres than the ones I do!
| 1attice wrote:
| > my basic question is, how can we be sure that chatGPT (or even
| the whole of GPT-3) is really not just some callcenter in the
| Philippines?
|
| Because everyone in that call center just got fired and replaced
| with, as the old t-shirt said, "a very small shell script."
|
| And an OpenAI auth token, of course
___________________________________________________________________
(page generated 2022-12-06 23:02 UTC)