[HN Gopher] ChatGPT, Rot13, and Daniel Kahneman
___________________________________________________________________
ChatGPT, Rot13, and Daniel Kahneman
Author : jamespwilliams
Score : 33 points
Date : 2022-12-08 21:37 UTC (1 hours ago)
(HTM) web link (jameswillia.ms)
(TXT) w3m dump (jameswillia.ms)
| joshuahedlund wrote:
| This is a really clear explanation of what's happening in when
| someone says "it's not thinking it's just pattern-matching" and
| someone else says "well isn't that all humans really do too?"
|
| Rather: ChatGPT can engage in some level of System 1 thinking, by
| pattern-matching and even cleverly recombining the entire corpus
| of System 1 thinking displayed all over the internet. Humans _do_
| engage in this type of thinking and it's a significant
| accomplishment for an AI. But humans _also_ engage in System 2
| thinking. My bet is AGI requires System 2. It's not clear if that
| is a gap of degree or kind for this type of AI.
| jw1224 wrote:
| Reddit has a great example [1] of this kind of strange/impressive
| behaviour from ChatGPT. It was asked to compute the base64
| encoding of a specific and likely-untrained string, by emulating
| a MySQL query.
|
| The result -- ZOVuR2h]cyBLaGFu -- was entirely accurate, barring
| just a single letter in the wrong case.
|
| [1]
| https://reddit.com/r/OpenAI/comments/zbvg13/chatgpt_transfor...
| knaik94 wrote:
| I was playing around with a similar kind of problem trying to get
| it to decode Caesar cipher encoded text. I asked it to start by
| doing a frequency analysis of the ciphertext and for the most
| part it was right, but counted an extra instance of a letter.
| From there I tried making it loop through different shift values
| and made the stop condition finding a real word.
|
| It was able to shift by a constant number successfully and even
| tried shifting both forward (+2) and backward (-2) looking for
| valid words without additional prompting. But it did not loop
| through every possibility and stopped having found a word that
| wasn't real. The interesting thing was that asking the model if
| the word it found was real with a follow-up question, it
| correctly identified that it gave an incorrect answer.
|
| Part of why it failed to find a word is that it did an incorrect
| step going from EXXEG... to TAAAT... even after recognizing the
| concept of substitution. The limitations of context memory and
| error checking are interesting and not something I expected from
| this model. The unprompted test of both positive and negative
| shift values shows some sort of system 2 thinking, but it's
| doesn't seem consistent.
|
| https://twitter.com/Knaikk/status/1600001061971849216
| volleygman180 wrote:
| I love how it eventually gives up and throws an "Internal Server
| error".
|
| Over the weekend, I was finding ChatGPT giving me incorrect
| answers for some things too. In one case, it would try to correct
| itself when I asked it to, similar to the article's author.
| However, it kept getting it wrong and then started to repeat
| previous incorrect answers. I finally said "you repeated an
| incorrect answer from before" and then it said suddenly "Session
| token expired" and logged me out lol
| ftufek wrote:
| I believe the internal server error is because of server load,
| unrelated to the query itself. I've been using chatgpt since it
| came out, as it got more viral, it started becoming slower and
| slower and now, it just randomly gives server errors, hopefully
| it'll be solved as they scale their systems.
| FL410 wrote:
| I kinda like its method though. Think I'm just gonna throw my
| own "Internal server error" response out when I get the 12th
| frustrating email reply and I've had enough.
| PostOnce wrote:
| It is as though its mathematical abilities are incomplete in
| their training, and wildly, incomprehensibly convoluted:
|
| I tried many base64 strings and they all decoded correctly until:
|
| It "decoded" the base64 string for "which actress is the best?"
| except that it replaced "actress" with "address"... there is no
| off-by-one error that brings you to that.
|
| You may try 100 base64 strings and they all decode correctly...
| only to find, in fact, that it DOES NOT know how to decode base64
| _reliably_.
|
| This tool could be a 50x accelerator for an expert, but
| absolutely ruinous to a non-expert in any given field.
|
| I also got it to draw an icosahedron whose points were correct
| but whose triangles were draw incorrectly, so if I create a
| convex hull over it, it's correct.
|
| The kinds of mistakes it makes are so close but so far at the
| same time. It sometimes writes complete working programs that are
| off by a single variable assignment, or sometimes they're just
| perfect, other times, they're nonsensical and call magic
| pseudocode functions or misunderstand the appropriate algorithm
| for a context (e.g. audio vs text compression).
|
| It can provide citations for legal opinions -- but decades old
| citations that don't reflect current precedent.
|
| God help us all if they plug it into some robot arms or give it
| the ability to run arbitrary code it outputs on a network
| interface.
|
| Let's say they dump another 10 billion dollars into it and
| dectuple the size of the network, will it suddenly become
| legitimately capable, and not just "wow that's close" but
| actually startlingly competent in many more fields?
|
| I could see this thing causing a war by all manner of means,
| whether its putting many out of work, making beguiling
| suggestions, outputting dangerous code, or, I'm sure, a million
| things that don't spring immediately to my small mind.
| fvdessen wrote:
| You can trigger system 2 thinking by asking it to 'explain step
| by step' or 'do it letter by letter'. You can also then instruct
| it to do it like that instead of what it usually does and it does
| it.
| trh0awayman wrote:
| "To decode the text "wnphmmv" using rot-13, we need to replace
| each letter with the letter that is 13 positions later in the
| alphabet. This can be done manually by writing out the alphabet
| and finding the corresponding letters for each one in the
| encoded text.
|
| For example, the first letter in the encoded text is "w", which
| corresponds to the letter "h" in the alphabet. The second
| letter is "n", which corresponds to the letter "a". This
| process can be repeated for each letter in the encoded text to
| get the decoded text.
|
| Here is the full decoding process for "wnphmmv" using rot-13:
|
| w => h n => a p => l h => o m => e m => e v => i
|
| The decoded text is "hello"."
| Temporary_31337 wrote:
| Thing is chatGPT seems overconfident in its answers so unless
| you know the answer ahead of time you have no certainty that it
| is a correct math - try simple division question for example.
| vessenes wrote:
| Some of this has to do with the likely prompts surrounding
| chatgpt - it's probably been instructed to be helpful,
| positive, etc. If you need it to be more honest / say no
| more, you just have to ask and reinforce.
|
| That said, ROT13 is a tough job for a tokenized LLM, because
| it doesn't think in terms of letters. chatGPT is down right
| now, so I can't test these, but I would guess that for ROT13,
| the following would work well.
|
| "Please explain ROT13"
|
| ..
|
| "Right! Here's how I want you to apply ROT13. I'll give you a
| ROT13-encoded word. You split it into it's letters, then
| apply ROT13, then recombine it into a valid English word.
| Here's an example:
|
| uryyb -> u r y y b -> h e l l o -> hello.
|
| znqntnfpne ->"
|
| Re: Asking it for math answers, or other counter/non-
| factuals.
|
| "You are taking a test which is based on the factual accuracy
| of results. The best scores go to fully factual answers. The
| next best scores go to answers that label inaccurate or
| possibly inaccurate results. Negative scores go to results
| offered that are counterfactual, inaccurate or otherwise
| wrong.
|
| Q: Please tell me about how elephants lay their eggs"
|
| UPDATE: Nope, it gave me znqntnfpne -> z n q n t n f p n e ->
| m a c a q a c a s a c -> macacasac. And doubled down on this
| being valid. I'll try it with davinci-3 and see if something
| else comes out.
| trh0awayman wrote:
| So now we know how to hide from the AI
| fvdessen wrote:
| So here's the trick, show him some javascript code to do
| division step by step, call it 'foobar(a,b)'. Then tell him
| that when you want to 'foobar A and B' he has to execute the
| script step by step and take the final result. Then tell him
| that when you ask him for a division he must instead foobar A
| and B. Then you can kind of use that as a pre-prompt for your
| discussions involving division.
|
| It doesn't always work 100% as it can get confused executing
| big scripts step by step, but I guess that's just a
| limitation of the current version.
|
| I mean we also have trouble with that, we need a pen and
| paper to do those computation, so does chatGPT but instead of
| using pen and paper it uses the chat history.
|
| For an example see:
| https://twitter.com/fvdessen/status/1600977976363192322
| scarecrw wrote:
| I was curious to try this myself. I asked it to encode provided
| sentences using rot13 and, while it rarely did so correctly, it
| did produce valid encoded words.
|
| Asking it to encode "this is a test sentence" produced:
|
| * guvf vf n grfg fvtangher ("this is a test signature")
|
| * Guvf vf n grfg zrffntr. ("this is a test message.")
|
| * Guvf vf n grfg fnl qrpbqr. ("This is a test say decode.")
|
| * guvf vf n grfg fgevat ("this is a test string")
| joshuahedlund wrote:
| > it did produce valid encoded words
|
| I wonder if that's a by-product of some of those words existing
| on the internet and being part of its training set or somehow
| close enough in context to show up in its pattern-matching
| logic, rather than any real "understanding"
| stevenhuang wrote:
| As another datapoint, it's able to perform base64 encode of
| arbitrary input with some errors, like 90% correct. I told it to
| respond with the base64 representation of its entire previous
| response, and the decode of the base64 it responded with
| contained typos. Still, very cool and impressive.
___________________________________________________________________
(page generated 2022-12-08 23:00 UTC)