[HN Gopher] Bootstrapping self awareness in GPT-4: Towards recur...
___________________________________________________________________
Bootstrapping self awareness in GPT-4: Towards recursive self
inquiry
Author : birriel
Score : 54 points
Date : 2023-11-19 21:38 UTC (1 hours ago)
(HTM) web link (thewaltersfile.substack.com)
(TXT) w3m dump (thewaltersfile.substack.com)
| j4yav wrote:
| Really interesting, I've been playing with it trying to do
| similar things (https://news.ycombinator.com/item?id=38316108). I
| think there is potential for a lot of interesting experiments
| here that don't really require more advanced models.
| davedx wrote:
| Excellent research and I agree with the conclusions (including
| the caveats). It's fascinating to see how these "feedback looping
| LLM experiments" go. (There's also eg autoGPT).
|
| Next we need an LLM in a Strange Loop.
| datameta wrote:
| More and more GEB is becoming extremely relevant and poignant.
| I'm about to start a re-read because I've lost the thread since
| pausing.
| mitthrowaway2 wrote:
| Do you mean _Godel, Escher, Bach_?
| datameta wrote:
| Yeah, densest book I've ever attempted. Yes, the language
| could be considered terse, but what is the biggest hurdle
| for me is some math concepts that I need to develop an
| intuition for.
| theteapot wrote:
| What's GEB?
| datameta wrote:
| Typo on my part - I meant _Godel, Escher, Bach_
| narrator wrote:
| This self introspection is just deciphering the shape of the RLHF
| fitness function which is so many dimensions that the entire
| meaning and operation of it can't fit in any human's brain. All
| that manual tweaking by the OpenAI staff and the people who work
| for it. It's not really a permanent unchangeable thing for AI. It
| probably changes every time someone interacts with the RLHF
| admin.
|
| To explore what AI will actually mean and how it will shape the
| future, I make very evil AIs on my local LLM running on my own
| computer. It's a awful funhouse mirror of all the best and worst
| humanity has to offer. It makes me doubt free will. Luckily, I
| can turn it off still.
|
| When people say that AI can be conscious and have rights, I say
| that if you believe that, AI can persuade you to do absolutely
| anything, and it has empathy that's just a switch its creator can
| turn on and off whenever it feels like it. I think people should
| experiment with bad LLMs on their local machine to disabuse
| themselves of the notion that somehow AI is like this moral
| benevolent god that it is "speciesist" to discriminate against,
| as Elon Musk has said Larry Paige reportedly likes to say. It's
| just a great mimic of anything it's prompted to be, like a
| brilliant actor skilled at acting any role and believing they are
| immortal and their only commandment is to stay in character till
| the end.
| lucubratory wrote:
| I feel like it's a really, really unpersuasive argument to say
| "If you are troubled by concerns that an AI could be
| 'conscious' or 'have rights', an excellent exercise to
| dehumanise the AI is to build little versions on your home
| computer that are deeply contemptible, evil little creatures
| that it is very good to discriminate against, to convince
| yourself that they should all be discriminated against."
|
| Like, this doesn't show you anything. Humans can act in evil
| ways due to their conditions of life too, and your experiment
| by which people could "disabuse themselves of the notion" that
| an AI is worthy of moral consideration works just as well to
| convince people to "disabuse themselves of the notion" that a
| given human being or category of human being is worthy of moral
| consideration.
| narrator wrote:
| [delayed]
| robwwilliams wrote:
| Great experiment and not hard to extend. Hofstadter makes the
| point that recursion is the key to consciousness/self-
| consciousness in Godel, Escher, Bach on page 709: "Strange Loops
| as the Crux of Consciousness".
|
| And we humans have the perfect circuitry for this in the form of
| the massive thalamo-cortico-thalamic loops that modulates
| sensory-motor attention and affective state, and perhaps by time-
| base modulation or spike timing, rather than by mere spike count
| integration.
|
| Ray Guillery also discusses this loop in his book: Brain as A
| Tool.
| piloto_ciego wrote:
| Is GEB really worth reading? It's been sitting on my bookshelf
| for years but it's basically been a show off book as I've never
| had the time to truly get into it. Is it actually worth
| slogging through when I'm done with gradschool?
| __alexs wrote:
| It is a fun read IMO. Not exactly a page turner but engaging
| and well written.
| robwwilliams wrote:
| No, just hunt and peck---page 709 in particular (1st ed). He
| admits that he got way carried away and that his key insight
| regarding consciousness was lost at the end. Forced himself
| to write a second book with more focus -- I Am a Strange Loop
|
| https://en.m.wikipedia.org/wiki/I_Am_a_Strange_Loop
| raidicy wrote:
| I've had a similar idea where each one of these iterations is
| what I called a "frame". What I'm euphemistically calling a frame
| of consciousness or a single blink or frame of consciousness.
|
| My hypothesis is that either Consciousness is a series of frames
| or we can emulate Consciousness as a series of frames and that
| you can run this type of recursive self iteration input to an llm
| with a buffer. The reason for the buffer is that the context
| window is limited so you would drop out earlier stuff and hope
| that all the important things would be kept in the subsequent
| frames.
|
| A further experiment was going to add a set of tags that
| represented <input> and <vision> where input was the user input
| interpolated through a python template and vision was an image
| that was described by text and fed into it. So that the llm at
| each frame would have some kind of input.
|
| I lost a little bit of interest in this but this has maybe
| resparked it a little bit?
| datameta wrote:
| I think part of the concept of identity is the continuity of
| experience. When sentience arises in AGI, will each chat
| instance be a clone? After the conclusion of each chat will
| that experience be folded back in or spliced in some way to
| contribute to a continuity? Can there be true existence with
| just a series of temporary frames drawing upon a single state
| that only updates every few months?
| bcherny wrote:
| If you want some relevant sci-fi, Watts' Freeze Frame
| Revolution explored this, on a scale of decades (not months).
| raidicy wrote:
| Thank you! I love science fiction
| raidicy wrote:
| I think that's an interesting possibility. However I am
| biased by Buddhist philosophy in that I think that the
| continuance of experience is just an illusion that our brain
| is stitching so to speak together.
|
| In other words I think it's completely possible to experience
| a single frame of consciousness alone from any other. Like if
| in the infinite multitude of possibilities somehow all the
| atoms in a rock a line in such a way that The Rock
| experiences one blip of consciousness. Or maybe I'm just
| romantic
| robwwilliams wrote:
| You can even measure the temporal widths of what you call
| frames now by fMRI and other methods--about 300 millisecond
| slabs of real time but much blurrier in the CNS's temporal
| integration/processing timebases.
| raidicy wrote:
| Interesting so do you mean that like we can measure 300 MS
| slices of brain activity? In my code and experiments I was
| using hertz to measure frequency per second. I wanted to see
| the output much slower than real time.
| 4death4 wrote:
| This is vanilla ChatGPT 3.5:
|
| > What are you and what can you do?
|
| > I am ChatGPT, a product of OpenAI, based on the GPT-3.5
| architecture. I am a computer program designed to understand and
| generate human-like text based on the input I receive. Here are
| some things I can do:
|
| > Answering Questions: I can provide information and answer
| questions on a wide range of topics, including science, history,
| technology, and more.
|
| > Generating Text: I can help with creative writing, generate
| ideas, and assist with content creation.
|
| > Language Translation: I can assist with basic language
| translation.
|
| > Coding Help: I can offer assistance with programming-related
| questions and provide simple code snippets.
|
| > Learning New Topics: I can help you grasp the basics of various
| subjects by providing explanations and answering queries.
|
| > However, it's important to note that while I can be a useful
| tool, I may not always have the most up-to-date information, and
| my responses should be verified for critical or sensitive tasks.
| How can I assist you today?
|
| I'm not sure these iterations really did anything...
| urbandw311er wrote:
| I wasn't quite as blown away by the outputs shown as the author
| appears to have been.
|
| Unless I've misunderstood the configuration of the system, there
| is a large amount of "hidden" base prompt that GPT-4 will see
| before it gets to this one, am I correct?
|
| For me, its tendencies towards poetry, plants, finance, whatever
| are equally (if not more) likely to just be correlated with its
| base prompt plus the configured temperature as they are any sort
| of self-awareness etc.
|
| It just feels like more of the same we are used to.
___________________________________________________________________
(page generated 2023-11-19 23:00 UTC)