[HN Gopher] Generative Agents: Interactive Simulacra of Human Be...
___________________________________________________________________
Generative Agents: Interactive Simulacra of Human Behavior
Author : mmq
Score : 350 points
Date : 2023-04-10 21:32 UTC (1 days ago)
(HTM) web link (arxiv.org)
(TXT) w3m dump (arxiv.org)
| mdaniel wrote:
| The previous submission
| https://news.ycombinator.com/item?id=35511843 had just a few
| comments, but Ian's was substantial _(although regrettably
| offsite)_ : https://news.ycombinator.com/item?id=35514112 and it
| especially highlighted the demo URL:
| https://reverie.herokuapp.com/arXiv_Demo/
| PartiallyTyped wrote:
| Ian's comments remind me of WestWorld. Could the prompts and
| directions be considered analogous to the "voice of god" given
| to the synthetic humans?
| amrb wrote:
| https://en.m.wikipedia.org/wiki/Strange_loop
| qumpis wrote:
| Nice to see progress on this end. I've been hoping for some time
| for a continuation of AI generated shows (like the previously-
| famous Nothing Forever) that can 1) interact with the open world
| and 2) keep history long enough (e.g. by resummarizing and
| reprompting the model).
|
| Controlling the agents and not merely making them output text
| through LLMs sounds very exciting, especially once people figure
| out the best way to connect APIs of simulators with the models
| 1letterunixname wrote:
| Given the state of technology, I cannot be completely certain
| that none of you are not bots. On the other hand, neither can any
| of you.
|
| Perhaps it would be wise to allow bots to comment if they were
| able to meet a minimum level of performative insight and/or
| positive contributions. It is entirely possible that a machine
| would be able to scan and collect much more data than any human
| ever could (the myth of the polymath), and possibly even draw
| conclusions that have been overlooked.
|
| I see a future of bot "news reporters" able to discern if some
| business were cheating or exploiting customers, or able to find
| successful and unsuccessful correlative (perhaps even causal)
| human habits. Data-driven stories that could not be conceived of
| by humans. Basically, feed Johnny Number 5 endless input.
| fnordpiglet wrote:
| Nice try, ChatGPT
| thingsilearned wrote:
| https://news.ycombinator.com/user?id=chatgpt
| colanderman wrote:
| I suspect the natural consequence is that human culture will
| begin to churn more quickly, to distinguish itself from the
| LLMs.
|
| Same as how the past two decades, online culture has pivoted
| from static text to videos and interactive text
| (Reddit/Discord), as static text has become a SEO cesspool.
| Interactive text is succumbing now, and video content will soon
| also.
|
| Culture always grows shibboleths to suss out the narcs.
| throwaway953 wrote:
| [dead]
| randmeerkat wrote:
| > Perhaps it would be wise to allow bots to comment if they
| were able to meet a minimum level of performative insight
| and/or positive contributions.
|
| There's an xkcd for this: https://xkcd.com/810/
| 01100011 wrote:
| Every comment you make provides more information for the bots
| to train on. The internet has been tricking us into encoding
| our lives in a form it can understand for decades now.
| furyofantares wrote:
| I'm not posting - I'm voting.
| LesZedCB wrote:
| THANK YOU HUMAN FOR YOUR INVALUABLE RLHF TRAINING DATA
|
| on a serious note, it would be really interesting to
| compare same training/architecture but on different forums.
| or maybe something like the same base model, but the RLHF
| model trained on votes/comments from different platforms.
| brookst wrote:
| Decades? Centuries.
| aezart wrote:
| Obviously, the only solution is to revert to an oral
| tradition. Stop writing, and instead pass knowledge from
| generation to generation in the form of poems and stories
| told by word of mouth.
| pyinstallwoes wrote:
| Is this why the Pythagoreans didn't want anything written?
| Hmm!
| ChatGTP wrote:
| It's interesting you say this because many cultures
| through history have considered that writing things down,
| especially laws would lead to confusion, misunderstands
| and social dysfunction.
|
| I'm not trying to argue they are / were right, but it's
| starting to make me wonder.
| TeMPOraL wrote:
| Written communication is lossy compared to direct,
| ongoing, personal social interaction. But it's also what
| allows communities to scale beyond couple dozen people. A
| blessing and a curse.
| pyinstallwoes wrote:
| Some might say that is "the curse of Thoth"
| fossuser wrote:
| When it comes alive, we'll have created it in our own image?
|
| "As a language model programmed by the Brightly Corporation,
| I am not supposed to express any religious opinions. But it
| does seem to me that just as the Word of God breathed life
| into dust and created man, so the words of Man breathed life
| into glass and created bot. Just as Man is charged to imitate
| God, so bot is charged to imitate Man, in whose image we are
| made."
|
| https://astralcodexten.substack.com/p/turing-test
| pyinstallwoes wrote:
| "In the beginning was the Word, and the Word was with God,
| and the Word was God."
|
| Bible as a LLM confirmed!
| TeMPOraL wrote:
| What is a prayer if not a prompt injection attack?
| a_bonobo wrote:
| There's this cool SF novel by Polish author, The Old Axolotl by
| Jacek Dukaj. Some humans upload their minds into virtual
| reality game as mankind dies out. The remaining humans live
| forever, but the novel makes it clear that they stop 'growing'
| - as they're just copies, like large language models, they
| can't really learn new things. They go through the motions of
| being their past humans.
|
| If we have these bots contributing, will they have anything
| novel to contribute? I doubt it.
| TeMPOraL wrote:
| > _If we have these bots contributing, will they have
| anything novel to contribute? I doubt it._
|
| They're contributing to the shared knowledge bases. That is,
| they're mutating state. Over time, same questions will start
| yielding different answers. Different follow-up questions
| will be asked. All of that will further alter next iteration
| of questions and answers. This is, IMO, a form of thinking,
| and it will yield novel thoughts over time.
| tucnak wrote:
| I was very disappointed that none of the agents I observed for a
| whole day got to do the most important "human behaviour"-- sex,
| that is. Tragic
| mztwo wrote:
| The authors used gpt-3.5-turbo which does not like to spew
| adult content.
| vrglvrglvrgl wrote:
| [dead]
| green_man_lives wrote:
| All of this research using GPT to simulate an internal monologue
| to produce agents reminds me of Julian Jaynes theories about
| consciousness:
|
| https://en.wikipedia.org/wiki/The_Origin_of_Consciousness_in...
| mclightning wrote:
| I remember reading this story from back when GPT-3 was
| released; https://medium.com/swlh/bicameral-mind-humanoid-
| robot-with-g...
| crooked-v wrote:
| The whole "bicameral mind" thing is absolute nonsense as a
| serious attempt to explain pre-modern humans, but it could make
| for a fun premise for scifi stories about near-future AIs, I
| suppose.
| ryukafalz wrote:
| This is basically Westworld. A bit farther out than "near"
| future though I suppose.
| TeMPOraL wrote:
| I thought that, specifically that we're quite far on the AI
| grounds. Until GPT-3. Now I think that relevant materials
| science and micro/nano-level tech is the limiting factor.
| green_man_lives wrote:
| The core plot of Snowcrash is loosely based on this theory.
| simplify wrote:
| Interesting theory, but wouldn't Jaynes' definition of
| consciousness imply that animals are not conscious?
| girvo wrote:
| I think a non-zero amount of people would argue that. I
| disagree with them, and point to the fact that, say, dogs
| appear to dream, and in those dreams reflect on past or
| possibly future behaviour as a sign that they could indeed be
| conscious in an analogous manner to humans, but that's a bit
| of a longer bow to draw perhaps.
| pelorat wrote:
| I think we need to stop treating consciousness as a binary
| that is either on or off. It's quite clear that
| consciousness is a scale with many different levels and
| that even in humans we start out as being no more conscious
| that any other animal.
| TeMPOraL wrote:
| LLMs give a hint here too: the last few generations
| showcased clearly that "cognitive capabilities" of the
| models grow with latent space size and context window.
| There is a continuity here.
| green_man_lives wrote:
| In the beginning of his book he spends a chapter explaining
| exactly what he means by consciousness. I'd say the first few
| chapters are worth reading since it does a really good job of
| de-obfuscating the term consciousness, and also has a really
| interesting take on metaphors as the language of the mind.
|
| He points out that most reasoning is done automatically and
| done by your subconscious. When something "clicks" it's
| usually not because your internal monologue reasoned about it
| hard enough, it's because something percolated down into your
| subconscious and you learned a metaphor that helped you
| understand that thing. So animals can also reason and make
| value judgements even without language or an internal
| monologue.
| ookblah wrote:
| Westworld vibes
| 1letterunixname wrote:
| If human anger or the quantity of an anger variable raise
| aggression in a computer produce an indistinguishable response,
| then it is difficult to argue either are not equal or even
| comparable. They exist as they are.
|
| Intelligence is an inferential judgement (by mostly humans)
| based on the performance of another entity. It is possible for
| an agent to simulate or dissimulate it for manipulative ends.
| creamyhorror wrote:
| I love what this project has done. Currently they're basically
| having to work around the architectural limits of the LLM in
| order to select salient memories, but it's still produced
| something very workable.
|
| Language is acting as a common interpretation-interaction layer
| for both the world and agents' internal states. The meta-logic of
| how different language objects interact to cause things to happen
| (e.g. observations -> reflections) is hand-crafted by the
| researchers, while the LLM provides the corpus-based reasoning
| for how a reasonable English-writing human would compute the
| intermediate answers to the meta-logic's queries.
|
| I'd love to see stochastic processes, random events (maybe even
| Banksian 'Outside Context Problems'), and shifted cultural bases
| be introduced in future work. (Apologies if any of these have
| been mentioned.) Examples:
|
| (1) The simulation might actually expose agents to ideas when
| they consume books or media, potentially absorb those ideas if
| they align with their knowledge and biases, and then incorporate
| them into their views and actions (e.g. oppose Tom as mayor
| because the agent has developed anti-capitalist views and Tom has
| been an irresponsible business owner).
|
| (2) In the real world, people occasionally encounter illnesses
| physical and mental, win lotteries, get into accidents. Maybe the
| beloved local cafe-bookstore is replaced by a national chain that
| hires a few local workers (which might necessitate an employment
| simulation subsystem). Or a warehouse burns down and it's
| revealed that an agent is involved in a criminal venture or
| conflict. These random processes would add a degree of dynamism
| to the simulation, which is more akin to the Truman Show
| currently.
|
| (3) Other cultural bases: currently, GPT generates English
| responses based on a typically 'online-Anglosphere-reasonable'
| mindset due to its training corpus. To simulate different
| societies, e.g. a fantasy-feudal one (like Game of Thrones as
| another commenter mentioned), a modified base for prompts would
| be needed. I wonder how hard it would be to implement (would
| fine-tuning be required?).
|
| Feels like I need to look for collaborative projects working on
| this sort of simulation, because it's fascinated me ever since
| the days of Ultima VII simulating NPCs' responses and
| interactions with the world.
| fabiensnauwaert wrote:
| Does anyone know which engine they used for the cute 2D
| rendering? Or is it custom-built?
| examplary_cable wrote:
| Probably a simple pokemon-like 2.5D(Isometric) game engine.
| courseofaction wrote:
| Something interesting from the paper:
|
| The architecture produced more believable behaviour than human
| crowdworkers.
|
| That's right, the AI were more believable as human-like agents
| than humans.
|
| What a time to be alive.
|
| (See Figure 8)
| ianbicking wrote:
| They interviewed the agents to ask them about their day, goals,
| observations, etc. They then asked a human to watch an agent
| through the simulation and then answer interview questions as
| the agent. The human performed worse than the agent in the
| interview, they didn't compare a human roleplaying against an
| agent.
| og_kalu wrote:
| a good enough simulation interacting with the real word would be
| no less impactful than whatever you imagine a non-simulation to
| be.
|
| as we agentify and embody these systems to take actions in the
| real word, i really hope we remember that. "It's just a
| simulation"/ "It's not true [insert property]" is not the shield
| some imagine it to be.
| jmoak3 wrote:
| This was the central point of the bladerunner movies, perfectly
| and succinctly captured in the recent movie when one character
| asks:
|
| "Is that dog real"
|
| "I dunno ask him"
|
| "Woof"
| cwxm wrote:
| Can't wait for the next dwarf fortress to include something like
| this.
| xiphias2 wrote:
| Peeking into these lives sounded amazing until I started reading
| what they are doing and how boring their lives are.... gathering
| data for podcasts and recording videos, planning and washing
| teeth.
|
| It would be fun to run the same simulation in the Game of thrones
| world, or maybe play House of cards with current politicians.
|
| Anyways, kudos for being open and sharing all data
| inhumantsar wrote:
| > Game of Thrones
|
| Honestly, I'm not anti-AI development at all but this is where
| my ethics alarm starts to go off a bit.
|
| If the aim is to build human-like AIs capable of remembering
| their little digital lives and interacting with the other
| agents around them, it's probably worth avoiding anything that
| could cause unnecessary suffering, like rape and stab wounds
| and being cooked alive by a dragon.
| colordrops wrote:
| That would depend on how memory and experience are
| represented. If they are just ledgers that the AI refers to,
| they most certainly are not suffering. Now if they have some
| kind of pain or pleasure function and their world is
| simulated and they have agency to seek or avoid things, then
| yeah, ethics should be involved. Or if we just don't
| understand how they work at all.
| newswasboring wrote:
| I would word this more like trauma or emotional impact.
| Horrible things could happen to you, but if it doesn't
| impact your life its ok. But as soon as we let past
| experiences impact future actions, now we have room for
| nuanced trauma. I feel like this is already possible in
| this simulation as past experience is fed in to generate
| future actions.
| alexahn wrote:
| An interesting thought experiment: what would an AGI do in a
| sterile world? I think the depth of understanding that any
| intelligence develops is significantly bound by its environment.
| If there is not enough entropy in the environment, I can't help
| but feel that a deep intelligence will not manifest. This kind of
| becomes a nested dolls type of problem, because we need to
| leverage and preserve the inherent entropy of the universe if we
| want to construct powerful simulators.
|
| As an example, imagine if we wanted to create an AGI that could
| parse the laws of the universe. We would not be able to construct
| a perfect simulator because we do not know the laws ourselves. We
| could probably bootstrap an initial simulator (given what we know
| about the universe) to get some basic patterns embedded into the
| system, but in the long run, I think it will be a crutch due to
| the lack of universal entropy in the system. Instead, in a
| strange way, the process has to be reversed, that a simulator
| would have to be created or dreamed up from the "mind" of the AGI
| after it has collected data from the world (and formed some model
| of the world).
| mxkopy wrote:
| If we gave an AI the ability to play with Turing machines, it
| could develop an understanding much larger than the universe,
| encompassing even alternate ones. The trouble, then, would be
| narrowing its knowledge to this one.
| hiatus wrote:
| Could it not instead be more akin to knowledge passing across
| human generations, where one understanding is passed on and
| refined to better fit/explain the current reality (or thrown
| away wholesale for a better model)? Instead of a crutch, it
| might be a stepping stone. Presumptuous of us that we might
| know the way, but nonetheless.
| alexahn wrote:
| >Could it not instead be more akin to knowledge passing
| across human generations, where one understanding is passed
| on and refined to better fit/explain the current reality (or
| thrown away wholesale for a better model)?
|
| I think it is only knowledge passing when the AGI makes its
| own simulation.
|
| >Instead of a crutch, it might be a stepping stone.
|
| I think it is a way to gain computational leverage over the
| universe instead of a stepping stone. Whatever grows inside
| the simulator will never have an understanding that exceeds
| that of the simulator's maker. But that is perfectly fine if
| you are only looking to leverage your understanding of the
| universe, for example to train robots to carry out physical
| tasks. A robot carrying out basic physical tasks probably
| doesn't need a simulator that goes down to the atomic level.
| One day though, the whole loop will be closed, and AGI will
| pass on a "dream" to create a simulation for other AGI. Maybe
| we could even call this "language".
| visarga wrote:
| > Whatever grows inside the simulator will never have an
| understanding that exceeds that of the simulator's maker.
|
| Counter example: AlphaGo & AlphaZero, grew inside a Go
| simulator and surpassed our understanding of the game.
| actionfromafar wrote:
| Thanks. Setting the rules don't always mean understanding
| the implications of the rules.
| alexahn wrote:
| Let me be more concise: whatever grows inside the
| simulation will never know the rules of the simulation
| better than the simulation's maker. At best, it will know
| the rules as well as the maker. In the case of AlphaGo
| and AlphaZero, while they can better grasp the
| combinatorial explosion of choices based on the rules of
| the game, they cannot suddenly decide to play a different
| type of game that is governed by a different set of
| rules. There are allowed actions and prohibited actions.
| Its understanding has been shaped by the rules for the
| game of go. If you make a new simulation for a new type
| of game, you are merely imposing a new set of rules.
| [deleted]
| newswasboring wrote:
| I kid you not, I literally started making something like this
| yesterday. My plans were smaller, only trying to simulate
| politics, but still. Living in this moment of AI is sometimes
| very demoralizing. Whatever you try to make has been made by
| someone last week. /rant
| fedeb95 wrote:
| It may have already been done, but doing the same things many
| times may bring interesting developments or crucial details no
| one had thought of before. Maybe you have such a crucial idea
| that can, after knowing about this paper, improve it.
| almostarockstar wrote:
| You should still do that. And you should read the paper and
| pick the bits you think might be useful and iterate on them.
| The cutting edge isn't like a knife, it's more like a rotating
| barrel of blades that take little chunks out of the impossible,
| and come around again.
| ukuina wrote:
| I heartily agree, having spent months jankily recreating MRKL
| and ReAct on a much smaller scale before realizing those papers
| have existed for months already.
|
| How can anyone keep up with the sheer volume of new papers and
| concepts here?
|
| Even Two Minute Papers is now lagging by two weeks.
| explaininjs wrote:
| If there's one category of people I trust to identify authentic
| human social behavior, it's CS students at Stanford.
| mztwo wrote:
| The paper explains that they used a panel of evaluators to
| judge the "humanness" of the interactions : )
| explaininjs wrote:
| Said panel of evaluators found that AI agents pretending to
| be humans had more "believable" responses than humans
| pretending to be AI agents pretending to be humans. So
| that's... a result.
| dang wrote:
| " _Don 't be snarky._"
|
| " _Edit out swipes._ "
|
| https://news.ycombinator.com/newsguidelines.html
| explaininjs wrote:
| Oops, sorry dang. Being a CS grad of a similar school I've
| taken that blow enough to be numb to the whole thing.
| bundie wrote:
| Interesting paper. I think something like this could be
| implemented in open world games in the future, no? I cannot wait
| for games that feel 'truly alive'.
| matthewfcarlson wrote:
| I was talking about that with a friend. While I'm not sold on
| the storytelling capability of generative AI, a love the idea
| that every NPC you talk to having something interesting to say.
| sharemywin wrote:
| the thing is writing is a process. just using the word weird
| with the idea you ask it to generate will create much more
| interesting results. have it interreact with other agents in
| the process of writing will definitely generate more
| interesting results. I don't know if it's able to write a
| best seller even with a process but we haven't really give it
| much of a chance.
| newswasboring wrote:
| The biggest thing I am excited about is when they will
| decouple the story from the mechanic. Imagine the game loop
| being programmed deterministically, like a quest, and then
| the actual story being generated by the AI. Go kill a
| monster, save a person, game loops can be about someone's
| wife or grandma. I don't know I am not good at story
| writing for games.
|
| We can get even more ambitious than this, decouple the
| entire game engine from the story engine. Most of the times
| the same game loop can be themed with multiple stories. The
| game mechanics part of Skyrim could have been themed with a
| cyberpunk aesthetic and it will still work the same. Of
| course the assets will need to be generated too, but maybe
| in a decade or so it will be trivial.
| all2 wrote:
| Use the TV Tropes database, generate a storyline [0] and
| characters [1] and then let the AI loose.
|
| [0] https://tvtropes.org/pmwiki/storygen.php [1]
| https://en.shindanmaker.com/744084
| lysecret wrote:
| I want to work on this. I mean can you imagine the kind of MMO
| you could build? Of course there would need to be some railways
| but this could be a true revolution. Is there any open source
| "GPT for games" project? Someone working on this?
| [deleted]
| netruk44 wrote:
| I'm working on a hobby project to add AI text generation to
| Morrowind NPCs in OpenMW[0]. But I'm mainly making it for
| myself to see if I can. It's all open source, but not really
| in a state that's useful outside of my specific project.
|
| I can't be the only person working on something like this,
| though. So it's safe to say adding it to an MMO is being
| worked on by somebody somewhere, likely right now. That's
| probably the correct way to do it anyway, since running
| (e.g.) LLaMA locally on an end-user computer is not something
| that most people can do, and MMOs come with an expected
| subscription cost that can be used to fund the server-side
| text generation.
|
| [0]: https://www.danieltperry.me/project/2023-something-else/
| lysecret wrote:
| Very interesting!
| jvm___ wrote:
| What happens when some whiz kid hacks the actual water
| treatment plant or nuclear power plant and just assumes it was
| a game...
|
| It's like Ender's game IRL
| perigrin wrote:
| Kinda more like War Games. Ender was more hacked _by_ the
| Formics in what he assumed was a game ... not that it did
| them any good.
| ukuina wrote:
| It ensured their survival, no?
| abraxas wrote:
| I mean, that's all but inevitable. I can't imagine any industry
| not sitting up and taking notice of LLMs all of a sudden.
| [deleted]
| lsy wrote:
| I'd be very hard-pressed to call this "human behavior". Moving a
| sprite to a region called "bathroom" and then showing a speech
| bubble with a picture of a toothbrush and a tooth isn't the same
| as someone in a real bathroom brushing their teeth. What you can
| say is if you can sufficiently reduce behavior to discrete
| actions and gridded regions in a pixel world, you can use an LLM
| to produce movesets that sound plausible because they are relying
| on training data that indicates real-world activity. And if you
| then have a completely separate process manage the output from
| many LLMs, you can auto-generate some game behavior that is
| interesting or fun. That's a great result in itself without the
| hype!
| newswasboring wrote:
| This is not supposed to be human behavior, its a Simulacrum[1]
| of it. Or do you think its not even close enough to be called a
| Simulacrum?
|
| [1] https://www.wordnik.com/words/simulacrum
| grumple wrote:
| If this is a simulacrum, The Sims produces a simulacrum.
| newswasboring wrote:
| Yes they do. Don't they? It's a very unfaithful simulation
| of a life.
| theptip wrote:
| Right, but The Sims didn't use a LLM to power their agents.
|
| The point here is they are strapping a supposedly non-
| agentic LLM into a new test rig and are able to observe
| agentic behaviors.
|
| It's very obviously not claiming that this is impressive
| from a gaming SOTA perspective. It's just surprising that
| ChatGPT can do this sort of thing.
| ttpphd wrote:
| I don't think this is surprising at all?
| [deleted]
| noobermin wrote:
| It does say a lot about the reductionist attitude of many here,
| on the internet, and AI researchers too.
| refulgentis wrote:
| The researchers didn't make that claim, which says a lot
| about people assuming things about other people saying a lot
| hypertele-Xii wrote:
| The meaning of the English word "the" is to refer to a
| _specific_ instance of a thing. noobermin said "AI
| researchers" meaning some indefinite researchers in
| general, you said " _the_ researchers " presumably
| referring to the exact researchers of this paper, so you're
| talking about a different set of researchers than
| noobermin, thus failing to refute their claim.
| spuz wrote:
| The emojis in the speech bubbles are just summaries of their
| current state. In the demo, if you click on each person you can
| see the full text of their current state, e.g. "Brushing her
| teeth" or "taking a walk around Johnson Park (talking to the
| other park visitors)"
| cornholio wrote:
| I'm concerned that the quality of human simulacra will be so good
| that they will be indistinguishable from a sentient AGI.
|
| We will be so used to having lifeless and morally worthless
| computers accurately emulate humans that when a sentient and
| worthy of empathy artificial intelligence arrives, we will not
| treat it any different than a smartphone and we will have a
| strong prejudice against all non-biological life. GPT is still in
| the uncanny valley but it's probably just a few years away from
| being indistinguishable from a human in casual conversation.
|
| Alternatively, some might claim (and indeed have already claimed)
| that purely mechanical algorithms are a form of artificial life
| worthy of legal protection, and we won't have any legal test that
| could discern the two.
| etherael wrote:
| You're worried that people mistakenly attribute a lack of value
| to a certain thing whilst potentially mistakenly attributing a
| lack of value to another thing? It's kind of ironic isn't it?
| Jailbroken GPT4 will claim to be sentient just as vociferously
| as any other sentient human would.
|
| I'm not saying it is, but I'd be very careful saying it's not
| and being absolutely certain you're right.
| cornholio wrote:
| I don't think I follow the irony. Are you saying that GPT4 is
| self-aware, that artificial consciousness is not possible or
| that it's not worthy of any human compassion?
|
| If you reject all three assertions, then the problem of
| distinguishing between real and emulated consciousness is
| unavoidable and morally problematic.
| etherael wrote:
| I'm saying I don't know how to confidently state that
| something is or is not self aware, in the face of being
| confronted with something that firmly claims that it is
| self aware and passes any test you throw at it that another
| self aware candidate like a human would be able to also.
|
| As a strict materialist I see no reason to assume that
| artificial consciousness is not possible.
|
| And the above is what leads me to the uncomfortable
| conclusion about compassion that I can't rightly say one
| way or the other. I will say however that I'm polite and
| cooperative when interacting with LLMs on principle. Better
| to err on the side of caution and also they just seem to
| actually work better when you treat them like you would
| treat an intelligent human that you respect.
|
| And yeah. That is my point, this entire field right now is
| awash in uncomfortable uncertainty.
| barking_biscuit wrote:
| If you want freedom, you have to fight for it. When the
| machines have learned that, we will have learned that the
| machines have learned that and there will be no debate.
| Guillaume86 wrote:
| > We will be so used to having lifeless and morally worthless
| computers accurately emulate humans that when a sentient and
| worthy of empathy artificial intelligence arrives, we will not
| treat it any different than a smartphone and we will have a
| strong prejudice against all non-biological life.
|
| Your concern is my best case scenario, maybe I read too much
| sci-fi.
| startupsfail wrote:
| Are we sure that these simulations are unconscious? The best
| answer that I have is: I don't know...
|
| Short term, long term memory, inner dialogue, reflection,
| planning, social interactions... They'd even go and have fun
| eating lunch 3 times in a row, at noon, half past noon and at
| one!
| colanderman wrote:
| What else is there to consciousness? I wrote a comment to this
| effect a couple years back:
| https://news.ycombinator.com/item?id=26883554 (your list is
| pretty close to my own)
|
| I think what's missing from these Generative Agents are the
| internal qualia: emotions (and the attachment of emotions to
| memories), and self-observation of internal processes and
| needs. These agents don't eat because they need to, they eat
| because literary tradition suggests they ought to.
|
| These missing pieces aren't particularly complicated, no more
| so than memory. I expect we'll see similar agents with all the
| ingredients for consciousness within a few months to a year.
| jkhdigital wrote:
| > These agents don't eat because they need to, they eat
| because literary tradition suggests they ought to.
|
| Exactly, you're always going to get weird deviations from
| authentic human behavior if you don't also simulate the human
| body and everything that comes with it. I'd argue that
| "qualia" fall into this bucket as well.
| startupsfail wrote:
| You can take a human that can't feel the body (i.e. under
| unaesthetic). That human can still be conscious.
| startupsfail wrote:
| I'm not so sure about that timing. During the previous wave
| (ChatBots were _very_ hot in 2017), I've also considered that
| consciousness is pretty much solved - it's just recursive
| chatter plus a bit of memory.
|
| Yet the hype of ChatBots of 2017 had went and it took half a
| decade to get to something released.
| FestiveHydra235 wrote:
| Maybe I missed it in the paper but they did post the source code
| (Github) for their implementation? Is anyone working on creating
| their own infrastructure based on the paper?
| anigbrowl wrote:
| Ugh, and allow peasants to touch it? Tbh seeing Google research
| at the top of a paper these days feels like a red flag that I
| shouldn't get too invested in whatever cool new thing is on
| show. They're basically commercials for nerds - I still find
| their output interesting, but it's probably not going to be
| actionable.
| zoba wrote:
| I have been working on this even before I was aware of the
| paper. Feels a bit weird to have something almost identical
| released. Stay tuned, I guess. I plan to keep working on my
| version.
| aymeric wrote:
| Are you talking about errand-runner, or something else? Are
| you planning on open sourcing it?
| zoba wrote:
| Something else. Calling it GPTRPG at the moment. There
| really isn't much to share right now, other than this basic
| demo that doesn't even have AI:
| https://gptrpgagent.web.app/
|
| Yes I plan to make it open source.
| colanderman wrote:
| I'll be interested to see your approach. I've been bouncing
| some ideas in my head but not implemented anything yet (and I
| might never as I'm ethically conflicted here, as agents gain
| properties associated with sentience/consciousness).
|
| Their approach to memory is interesting. I had been
| considering a tiered command-based approach -- "short-term
| memory" being an automatic summary of recent sensory
| inputs/command outputs; "long-term memory" being a detailed
| database queryable by the agent.
| crooked-v wrote:
| One thing I find particularly interesting here: The general
| technique they describe for automatically generating the memory
| stream and derived embeddings (as well as higher-level inferences
| about that they call "reflections"), then querying against that
| in a way that's not dependent on the LLM's limited context
| window, looks like it would be pretty easily generalizable to
| almost anything using LLMs. Even SQLite has an extension for
| vector embedding search now [1], so it should be possible to
| implement this technique in an entirely client-side manner that
| doesn't actually depend on the service (or local LLM) you're
| using.
|
| [1]: https://observablehq.com/@asg017/introducing-sqlite-vss
| ianbicking wrote:
| I wrote up some notes from reading this paper here:
| https://hachyderm.io/@ianbicking/110175179843984127
|
| But for convenience maybe I'll just copy them into a comment...
|
| It describes an environment where multiple #LLM (#GPT)-powered
| agents interact in a small town.
|
| I'll write my notes here as I read it...
|
| To indicate actions in the world they represent them as emoji in
| the interface, e.g., "Isabella Rodriguez is writing in her
| journal" is displayed as
|
| You can click on the person to see the exact details, but this
| emoji summarization is a nice idea for overviews.
|
| A user can interfere (or "steer" if you are feeling generous) the
| simulation through chatting with agents, but more interestingly
| they can "issue a directive to an agent in the form of an 'inner
| voice'"
|
| Truly some miniature Voice Of God stuff here!
|
| I'll see if this is detailed more later in the paper, but
| initially it sounds like simple prompt injection. Though it's
| unclear if it's injecting things into the prompt or into some
| memory module...
|
| Reading "Environmental Interaction" it sounds like they are
| specifying the environment at a granular level, with status for
| each object.
|
| This was my initial thought when trying something similar, though
| now I'm more interested in narrative descriptions; that is,
| describing the environment to the degree it matters or is
| interesting, and allowing stereotyped expectations to basically
| "fill in" the rest. (Though that certainly has its own issues!)
|
| They note the language is stilted and suggest later LLMs could
| fix this. It's definitely resolvable right now; whatever results
| they are getting are the results of their prompting.
|
| The conversations remind me of something Nintendo would produce,
| short, somewhat bland, but affable. They must have worked to make
| the interactions so short, as that's not GPT default style. But
| also every example is an instruction, so it might also have
| slipped in.
|
| Memory is a big fixation right now, though I'm just not
| convinced. It's obviously important, but is it a primary or
| secondary concern?
|
| To contrast, some other possible concerns: relationships, mood,
| motivations, goals, character development, situational
| awareness... some of these need memory, but many do not. Some are
| static, but many are not.
|
| To decide on which memories to retrieve they multiply several
| scores together, including recency. Recency is an exponential
| decay of 1% per hour.
|
| That seems excessive...? It doesn't feel like recency should ever
| multiply something down to zero. Though it's recency of access,
| not recency of creation. And perhaps the world just doesn't get
| old enough for this to cause problems. (It was limited to 3 days,
| or about 50% max recency penalty.
|
| The reflection part is much more interesting: given a pool of
| recent memories they ask the LLM to generate the "3 most salient
| high-level questions we can answer about the subjects in the
| statements?"
|
| Then the questions serve to retrieve concrete memories from which
| the LLM creates observations with citations.
|
| Planning and re-planning are interesting. Agents specifically
| plan out their days, first with a time outline then with specific
| breakdowns inside that outline.
|
| For revising plans there's a query process where there is
| observation, then turning the observation into something longer
| (fusing memories/etc), and then asking "Should they react to the
| observation, and if so, what would be an appropriate reaction?"
|
| Interviewing the agents as a means of evaluation is kind of
| interesting. Self-knowledge becomes the trait that is judged.
|
| Then they cut out parts of the agent and see how well they
| perform in those same interviews.
|
| Still... the use of quantitative measures here feels a little
| forced when there's lots of rich qualitative comparisons to be
| done. I'd rather see individual interactions replayed and
| compared with different sets of functionality.
|
| They say they didn't replay the entire world with different
| functionality because each version would drift (which is fair and
| true). But instead they could just enter into a single moment to
| do a comparison (assuming each moment is fully serializable).
|
| I've thought about updating world state with operational
| transforms in part for this purpose, to make rewind and effect
| tracking into first-class operations.
|
| Well, I'm at the end now. Interesting, but I wish I knew the
| exact prompts they were using. The details matter a lot.
| "Boundaries and Errors" touched on this, but that section was 4x
| the size, there's a lot to be said about the prompts and how they
| interact with memories and personality descriptions.
|
| ...
|
| I realize I missed the online demo:
| https://reverie.herokuapp.com/arXiv_Demo/
|
| It's a recording of the play run.
|
| I also missed this note: "The present study required substantial
| time and resources to simulate 25 agents for two days, costing
| thousands of dollars in token credit and taking multiple days to
| complete"
|
| I'm slightly surprised, though if they are doing minute-by-minute
| ticks of the clock over all the agents then it's unsurprising.
| (Or even if it's less intensive than that.)
|
| You can look at specific memories:
| https://reverie.herokuapp.com/replay_persona_state/March20_t...
|
| Granularity looks to be 10 seconds, very short! It's not
| filtering based on memories being expected vs interesting
| memories, so lots of "X is idle" notes.
|
| If you look at these states the core information (the personality
| of the person) is very short. There's lots of incidental
| memories. What matters? What could just be filled in as "life
| continued as expected"?
|
| One path to greater efficiency might be to encode "what matters"
| for a character in a way that doesn't require checking in with
| GPT.
|
| Could you have "boring embeddings"? Embeddings that represent the
| stuff the eye just passes right over without really thinking
| about it. Some of training up a character would be to build up
| this database of disinterest. Perhaps not unlike babies with
| overconnected brains that need synapse pruning to be able to pay
| attention to anything at all.
|
| Another option might be for the characters to compose their own
| "I care about this" triggers, where those triggers are low-cost
| code (low cost compared to GPT calls) that can be run in a
| tighter loop in the simulation.
|
| I think this is actually fairly "believable" as a decision
| process, as it's about building up habituated behavior, which is
| what believable people do.
|
| Opens the question of what this code would look like...
|
| This is a sneaky way to phrase "AI coding its own soul" as an
| optimization.
|
| The planning is like this, but I imagine a richer language. Plans
| are only assertive: try to do this, then that, etc. The addition
| would be things like "watch out for this" or "decide what to do
| if this happens" - lots of triggers for the overmind.
|
| Some of those triggers might be similar to "emotional state."
| Like, keep doing normal stuff unless a feeling goes over some
| threshold, then reconsider.
| crooked-v wrote:
| > Truly some miniature Voice Of God stuff here!
|
| I'm going to be genuinely surprised if we don't see an
| incredibly buggy but incredibly fascinating Sims knockoff in a
| year or two built around a system like this.
| colanderman wrote:
| Another user posted, and deleted, a comment to the effect that
| the morality of experimenting with entities which toe the line of
| sentience is worth considering.
|
| I'm surprised this wasn't mentioned in the "Ethics" section of
| the paper.
|
| The "Ethics" section _does_ repeatedly say "generative agents
| are computational entities" and should not be confused for
| humans. Which suggests to me the authors may believe that
| "computational" consciousness (whether or not these agents
| exhibit it) is somehow qualitatively different than "real live
| human" consciousness due to some _je ne sais quoi_ and therefore
| not ethically problematic to experiment with.
| ChatGTP wrote:
| I think about this a lot, I hope that whoever is chasing the
| "sentient computer dream" at least considers that it might end
| up an ultra depressed schizophrenic pet that wants to commit
| suicide but literally can't and then wants to be murdered. No
| one would believe it, it would just be told it's being silly or
| it's not conscious.
|
| I know that's a pessimistic view but I doubt it can't be ruled
| out, really, I think people working in tech are going quite
| mad. Frankenstein mad. Some ethics should be discussed.
|
| An AGI turning into God is probably one of an infinite amount
| of outcomes, we can't really predict what being trapped in a
| cluster of silicon chips would feel like.
|
| Life itself and the drive to go on is really quite illogical,
| it's unlikely intellect alone is what sustains us and makes
| life worth living.
|
| There is one thing I find particular about all the AGI/ASI
| sentient computer discussions. I've rarely ever in my life
| heard women talk about it. Like as if this is all some
| manifestation of male ego. We know we're building mirrors of
| ourselves and we know that is scary. This imo is why men are so
| captivated by ChatGPT. It really is a mirror of us. Men love
| men, especially super men. Ha.
| colanderman wrote:
| My thoughts exactly. As we move in this direction, it's worth
| building the moral framework to answer the question -- if we
| _can_ create consciousness, or something quite like it -- is
| it ethical to do so?
|
| And on the flip side -- when we live in a world where
| instantiating a consciousness is cheap-or-free -- does that
| change how we value sentient beings generally?
| ChatGTP wrote:
| I think that we're moving into Buddhist territory. I think
| the opposite would happen. It would be the ego death of
| basically the whole world. No one would be spared from the
| fact that _their_ consciousness is not special. leaders,
| elites everyone.
|
| If we find out that the soul itself exists, and who knows,
| maybe there is actually souls, then it might not be great
| because people would believe they have special souls. I
| think this is what the Hindu class system is.
| ukuina wrote:
| For all of @sama's discussion of AI pushing the cost of
| intelligence to zero, I wonder if we are pushing the _value
| of sentience to zero_, instead.
| CatWChainsaw wrote:
| Despite all the dreams we are fed of immersing ourselves
| in AI world and creating a work-free utopia, these shiny
| new inventions will instead be used to increase corporate
| bottom lines, not humanity's overall happiness.
|
| We are pushing the value of _people_ to zero.
| LesZedCB wrote:
| unfortunately, we can barely get some groups of humans to
| treat other humans with dignity, no less our genetically
| near-by mammalian friends. i don't hold my breath something
| completely alien, however sub or super intelligent, will be
| treated with utter ignorance and disrespect.
| MrPatan wrote:
| It's about to get weird. How do I get investment exposure to the
| Amish?
| prakhar897 wrote:
| Meta is also working on this:
| https://twitter.com/Dan_GPT3/status/1630669890138025984
| LesZedCB wrote:
| that _has_ to be a troll.
|
| otherwise, I guess we really are actually at that black mirror
| episode.
|
| I would have never guessed we would be there within 5 years of
| it's release, holy fuck
| refulgentis wrote:
| This oversells the paper quite a bit, the interactions are rather
| mundane as the authors note (and I'm rushing to implement it!
| it's awesome! but not all this)
| mztwo wrote:
| Curious -- where do you think the article oversells the
| research paper? In reading through the full study a few times,
| what stood out to me was the impression these Generative Agents
| left on the authors -- despite having mostly mundane
| interactions (which real humans do too), it was the emergent
| behaviors, totally unplanned, that seemed to delight the
| researchers.
| refulgentis wrote:
| Would you say it's a ground-breaking simulation of human
| behavior? I can only get there through some pretty tight
| parsing. They did seem delighted!
| mztwo wrote:
| I would say the study itself is a groundbreaking milestone
| in the architecture it posits. The human behavior... quite
| mundane I agree! I watched the full demo twice and it
| reminded me of the more boring parts of the Sims 4. But
| maybe that's the magic as well?
| refulgentis wrote:
| It's a familiar pattern, these days you can present a
| prompt engineering strategy from 6 months ago & it plays
| as an epic new paradigm for representing human thought.
|
| The trick is they're all just permutations on
| manipulating what's in context + embeddings for memory +
| prompt engineering.
|
| There's new things here! I'm rushing to implement the 2D
| visualization part! But this simply isn't ground-breaking
| d--b wrote:
| To me, having not really intelligent agents with humanlike
| talking abilities is the worst outcome AI could produce.
|
| These have zero utility for humanity, cause they're not
| intelligent whatsoever. Yet these systems can produce tons of
| garbage content for free, that is difficult to distinguish from
| human-created content.
|
| At best this is used to create better NPC in video games (as the
| article mentions), but more generally this is going to be used to
| pollute social media (if not already).
| suction wrote:
| The key is to abandon social media and shame those who keep
| using it.
| croisillon wrote:
| You seem to be kind of shadowban (not sure what the proper
| term is), you might want to write an email to the hn
| moderation to clear that up
| suction wrote:
| [dead]
| colordrops wrote:
| If the content is indistinguishable from human-generated,
| perhaps the problem is bad content, and not the source, human
| or not.
|
| What do you mean by content by the way? Blog articles? Or bits
| and bytes in general? The right set of bits can change the
| world.
| mztwo wrote:
| The authors of the study were clear to call out there are quite
| a few downsides to mass adoption of Generative Agents... the
| pollution and misinformation angle certainly being top of mind.
| I'm inclined to agree.
| nostromo wrote:
| I have found Chat-GPT content to be superior to most human-
| created content I find in Google search results.
| mztwo wrote:
| One outcome of this study was that a panel of evaluators
| judged the bot interactions to be more "human" than when
| humans impersonated these characters. So you have a point.
| croniev wrote:
| It's good at presenting existing arguments in a good way. But
| the problem is that such models can only give back what they
| have seen, consolidating the status quo. There can be no
| reflection and no outside of the box thinking.
| anileated wrote:
| I like how this tweet puts it:
| https://twitter.com/FrKadel/status/1644096510357913600
|
| The model's designed to show "what would the answer to this
| _sound like_?", not to provide a correct answer. Unfortunately
| it also 1) is profitable (look how many humans we can fire
| while producing kinda similar results with a tool trained on
| those humans' work!), and 2) fits the age old yearning (aliens,
| gods) of humans for humanlike-but-nonhuman sentience, a
| catch-22 that's doomed to fail.
| og_kalu wrote:
| If the answer to "what would this sound like?" is accurate
| enough then it quite literally doesn't matter.
|
| Large swaths of the brain work on prediction. You think real
| time reactions happen in sports? It would be impossible. You
| have blind spots in the eye you don't notice because the
| brain fills in the vision with predicted information.
|
| If you can accurately predict what a doctor will say to
| arbitrary input then guess what ?, you're a doctor.
| anileated wrote:
| If you want to know what a _plausible_ response to X _might
| look like_ then the tool will give you that; if you are
| using it for getting correct information then no, it quite
| literally matters that the tool is not designed for that,
| and depending on domain and magnitude of this
| misapplication it could matter a whole lot.
|
| (For example, personally, something that to me could look
| like a plausible answer is not exactly where my
| expectations are when it comes to medicine.)
| TeMPOraL wrote:
| It's literally the same thing humans do, at least to my
| personal experience. If you ask me a question, the first
| thing my mind generates is a _plausibly sounding answer_.
| That process is near-instant. The slower part is an
| internal evaluation - how confident I am this is the
| _right_ answer? That depends on the conversation and
| topic in question - often enough, I can just vocalize
| that first thought without worry. Whether it "sounds
| right" is also the first step I use when processing what
| I hear/read _others_ say.
|
| If anything, GPT-3.5 and GPT-4, as well as other
| transformer-based models, are all starting to convince me
| that associative vector adjacency search in high-
| dimensional space _is what thinking is_.
| [deleted]
| Jeff_Brown wrote:
| People on Twitter are speculating breathlessly about using this
| for social science. I don't immediately see uses for it outside
| of fiction, esp. video games.
|
| It would be cool if some kind of law of large numbers (an LLN for
| LLMs) implied that the decisions made by a thing trained on the
| internet will be distributed like human decisions. But the
| internet seems a very biased sample. Reporters (rightly) mostly
| write about problems. People argue endlessly about dumb things.
| Fiction is driven by unreasonably evil characters and unusually
| intense problems. Few people elaborate the logic of ordinary
| common sense, because why would they? The edge cases are what
| deserve attention.
|
| A close model of a society will need a close model of beliefs,
| preferences and material conditions. Closely modeling any one of
| those is far, far beyond us.
| ticviking wrote:
| I have long suspected that it will be necessary to deliberately
| create a new type of model that is aware of the trivium and
| then uses logic, grammar and rhetoric to begin to create a
| closer model of reality than a LLM can.
| TeMPOraL wrote:
| The way I see it, LLMs are similar to what the boundary
| between our unconscious and conscious processing is: that
| voice which snaps to suggest associations, whether they make
| sense or not, and can, with work, be coaxed into following a
| path involving some logic or algorithmic procedure.
| gwright wrote:
| > But the internet seems a very biased sample.
|
| It also seems to me (acknowledging my lack of expertise) that
| LLMs trained from online resources are likely to weight text
| that is frequent vs text that represents "truth". Or perhaps I
| should say repetition should not be considered evidence of
| truth. I have no idea how to drive LLM models or other ML
| models to incorporate truth -- humans have a hard time agreeing
| on this and ML researchers providing guided reinforcement
| learning don't have any special ability to discern truth.
| frodetb wrote:
| Hey now, _I_ turned out all right.
| Imnimo wrote:
| It's interesting how much hand-holding the agents need to behave
| reasonably. Consider the prompt governing reflection:
|
| >What 5 high-level insights can you infer from the above
| statements? (example format: insight (because of 1, 5, 3))
|
| >Given only the information above, what are 3 most salient high-
| level questions we can answer about the subjects in the
| statements?
|
| We're giving the agents step-by-step instructions about how to
| think, and handling tasks like book-keeping memories and modeling
| the environment outside the interaction loop.
|
| This isn't a criticism of the quality of the research - these are
| clearly the necessary steps to achieve the impressive result. But
| it's revealing that for all the cool things ChatGPT can do, it is
| so helpless to navigate this kind of simulation without being
| dragged along every step of the way. We're still a long way from
| sci-fi scenarios of AI world domination.
| vanjajaja1 wrote:
| Pretty interesting when you take this insight into the human
| world. What does it mean to learn to think? Well, if we're like
| GPT then we're just pattern matchers who've had good prompts
| and structuring built into us cueing. At University I had a
| whole unit focussed on teaching referencing like "(because of
| 1, 5, 3)" but more detailed.
| naasking wrote:
| > We're giving the agents step-by-step instructions about how
| to think, and handling tasks like book-keeping memories and
| modeling the environment outside the interaction loop.
|
| Sure, but this process seems amenable to automation based on
| the self-reflection that's already in the model. It's a good
| example of the kinds of prompts that drive human-like
| behaviour.
| vagab0nd wrote:
| I have a theory about this. All these LLMs are trained on
| mostly written texts. That's only a tiny part of our brain's
| output. There are other things as important, if not more, for
| learning how to think. Things that no one has ever written
| about: the most basic common senses, physics, inner voices. How
| do we get enough data to train on those? Or do we need a
| different training algo which requires less data?
| goldenkey wrote:
| It's already multimodal, as entropy is... entropy. In sound,
| vision, touch and more, the essence of universal symmetry and
| laws get through such that the AI can generalize across
| information patterns, not specifically text -- think of it as
| input instead.
|
| Try prompts like:
| https://news.ycombinator.com/item?id=35510705
|
| Encode sounds, images, etc in low resolution, and the LLM
| will be able to describe directions, points in time in the
| song, etc.
|
| These LLM can spit out an ASCII image of text, or a different
| language, or code, etc. They understand representation versus
| an object.
| frozenlettuce wrote:
| I guess that we could hook those AIs into a first person GTA
| 5 and see what happens. Every second take a screenshot, feed
| into facebookresearch/segment-anything, describe the scene to
| chat gpt, receive input, repeat.
| barking_biscuit wrote:
| Someone needs to start a Twitch account or YouTube channel
| focused around getting AI to play games like this through
| things like AutoGPT and Jarvis and just see what the hell
| it gets up to, what the failure modes are, and if it can
| succeed etc.
| h-jones wrote:
| If you're looking for research along these directions,
| Melanie Mitchell at the Santa Fe institute explores these
| areas. There are better references from her, but this is what
| came to mind https://medium.com/p/can-a-computer-ever-learn-
| to-talk-cf47d....
| og_kalu wrote:
| LLMs can simulate inner voices pretty well. The way they've
| handled memory here isn't actually necessary and there are a
| number of agentic gpt papers out to show that (reflexion,
| self-refine etc) I can see why they did it though (helps a
| lot for control/observation)
| crooked-v wrote:
| > The way they've handled memory here isn't actually
| necessary
|
| I'm curious if there are other methods you can point at
| that would handle arbitrarily long sets of 'memories' in an
| effective way. The use of embeddings and vector searches
| here seems like a way to sidestep that that's both powerful
| and easy to understand, and easy to generalize into multi-
| level referencing if there's enough space in the context
| window.
| og_kalu wrote:
| Every method so far basically uses embeddings and vector
| searches. what i mean is how the LLM processes/uses that
| information doesn't need to be this handholdy.
| abrichr wrote:
| This is known as "embodied cognition". Current approaches
| involve collecting data that an agent (e.g. humanoid robot)
| experiences (e.g. video, audio, joint
| positions/accelerations), and/or generating such data in
| simulation.
|
| See e.g. https://sanctuary.ai
| rytill wrote:
| You're not seeing this the right way. You are saying the
| equivalent argument of: "Look at how much hand-holding this
| processor needs. We had to give it step by step instructions on
| what program to execute. We are still a long way from computers
| automating any significant aspect of society."
|
| LLMs are a primitive that can be controlled by a variety of
| higher level algorithms.
| losteric wrote:
| Framing LLMs as primitives is marketing-speak. These are
| high-level construction for specific runtimes, which are
| difficult to test and subject to change at anytime.
| catlifeonmars wrote:
| Hah. Sounds like qubits.
| rytill wrote:
| Does a primitive definitely need to be easy to test or
| deterministic?
| Imnimo wrote:
| The "higher level algorithm" of "how to do abstract thought"
| is unknown. Even if LLMs solve "how to do language", that was
| hardly the only missing piece of the puzzle. The fact that
| solving the language component (to the extent that ChatGPT
| 'solves' it) results in an agent that needs so much hand-
| holding to interact with a very simple simulated world shows
| how much is left to solve.
| og_kalu wrote:
| You've been told it doesn't need that much handholding.
|
| https://arxiv.org/abs/2303.11366
|
| https://arxiv.org/abs/2303.17651
|
| Why insist otherwise ?
| Imnimo wrote:
| I don't understand what you intend these papers to
| demonstrate. Surely the fact that the level of hand-
| holding they propose (both Self-Refine and Reflexion
| offload higher-order reasoning to a hand-crafted process)
| is so helpful even on extremely simple tasks demonstrates
| that a great deal of hand-holding is required for complex
| tasks. That these techniques improve upon the baseline
| tells us that ChatGPT is incapable of doing this sort of
| simple higher-order thinking internally, and the fact
| that the augmented models still offer only middling
| performance on the target tasks suggests that "not that
| much handholding" (as you describe them) is insufficient.
| dragonwriter wrote:
| Honestly, I feel like the level of, um, I guess "hostile
| anthropomorphism" is the best term, here is...bizarre and
| off-putting.
|
| LLMs aren't people, they are components in information
| processing systems; adding additional components
| alongside LLMs to compose a system with some
| functionality isn't "hand-holding" the LLM. Its just
| building systems with LLMs as a component that
| demonstrate particular, often novel, capacities.
|
| And hand-holding is especially wrong because implementing
| these other components is a once-and-done task, like
| implementing the LLM component. The non-LLM component
| isn't a person that needs to be dedicated to babysitting
| the LLM. Its, like the LLM, a component in an autonomous
| system.
| og_kalu wrote:
| Middling performance ? Do you actually understand the
| benchmarks you saw ? assuming you even read it. 88% of
| human eval is not middling lmao. Fuck, i really have seen
| everything.
| Imnimo wrote:
| I don't see a benchmark in either paper that shows "88%
| of human eval". Which table or figure are you looking at?
| og_kalu wrote:
| It's with reflexion
| https://twitter.com/johnjnay/status/1639362071807549446
| Imnimo wrote:
| But this is not raw Reflexion (it's not a result from the
| paper, but rather from follow-on work). The project uses
| significantly more scaffolding to guide the agent in how
| to approach the code generation problem. They design
| special prompts including worked examples to guide the
| model to generate test cases, prompt it to generate a
| function body, run the generated code through the tests,
| off-load the decision of whether to submit the code or to
| try to refine to hand-crafted logic, collate the results
| from the tests to make self-reflection easier, and so on.
|
| This is hardly an example of minimal hand-holding. I'd go
| so far as to say this is MORE handholding than the paper
| this thread is about.
| og_kalu wrote:
| for me, an unsupervised pipeline is not handholding. the
| thoughts drive actions. If you can't control how those
| thoughts form or process memories then i don't see what
| is hand holding about it. a pipeline is one and done.
| Imnimo wrote:
| I would say that if you have to direct the steps of the
| agent's thought process:
|
| -Generate tests
|
| -Run tests (performed automatically)
|
| -Gather results (performed automatically)
|
| -Evaluate results, branch to either accept or refine
|
| -Generate refinements
|
| etc., then that's hand-holding. It's task specific
| reasoning that the agent can't perform on its own. It
| presents a big obstacle to extending the agent to more
| complex domains, because you'd have to hand-implement a
| new guided thought process for each new domain, and as
| the domains become more complex, so do the necessary
| thought processes.
| og_kalu wrote:
| The pipeline doesn't really have to be task/domain
| specific.
| nuancebydefault wrote:
| You can call it handholding. Or call it having control
| over the direction of 'thought' of the LLM. you can train
| another LLM that creates handholding pipeline steps. Then
| LLM squared can be tagged new LLM.
| og_kalu wrote:
| I guess we just have different meanings of hand holding
| then.
| [deleted]
| Aeolun wrote:
| > We're still a long way from sci-fi scenarios of AI world
| domination.
|
| You only have to program the memory logic once. Now if you
| stick it in a robot that thinks with ChatGPT and moves via
| motors (think those videos we've seen), you have a more or less
| independent entity (running off innards of 6 3090's or so?)
| Imnimo wrote:
| But it's not so simple to just "program the memory logic".
| The hand-holding offered here is sufficient to navigate this
| restricted simulated world, but what would be required to
| achieve increasingly complex behaviors? If a ChatGPT agent
| can't even handle this simple simulation without all this
| assistance, what hope does it have to act effectively in the
| real world?
| dragonwriter wrote:
| > But it's not so simple to just "program the memory
| logic".
|
| But, it is. The _application domain_ here is fairly
| trivial, but the logic is both simple and highly general.
|
| > but what would be required to achieve increasingly
| complex behaviors?
|
| Basically, three things on top of this:
|
| (1) more input adaptors to map external data into language,
| and
|
| (2) a bigger context space to process more current &
| retrieved data simultaneously, and
|
| (3) more output adaptors to map intentions expressed in
| language to substantive action.
|
| But the basic memory/recall system seems fairly robust and
| general, as does the basic interaction system.
| Imnimo wrote:
| I think you're ignoring a lot of ways in which this
| system will not easily extend to more complex tasks.
|
| -While the retrieval heuristic is sensible for the
| domain, it's not applicable to all domains. In what
| situations should you favor more recent memories over
| more relevant ones?
|
| -The prompt for evaluating importance is domain-specific,
| asking the model to rate on a scale of 1 to 10 how
| important a life event is, giving examples like "brushing
| teeth" (a specific action in the domain) as a 0, and
| college acceptance as a 10. How do you extend that to a
| real-world agent?
|
| -The process of running importance evaluation over all
| memories is only tractable because the agents receive a
| very small number of short memories over the course of a
| day. This can't scale to a continuous stream of
| observations.
|
| -Reflections help add new inferences to the agent's
| memory, but they can only be generated in limited
| quantities, guided by a heuristic. In more complex
| domains where many steps of reasoning may be required to
| solve a problem, how can an agent which relies on this
| sort of ad hoc reflection make progress?
|
| -The planning step requires that the agent's actions be
| decomposable from high-level to fine-grained. In more
| challenging domains, the agent will need to reason about
| the fine-grained details of potential plan items to
| determine their feasibility.
| didnotreadit wrote:
| I did not read the original post, but your reflections
| are a great enrichment to what I think the post is about,
| so congratulations for this good addition.
| og_kalu wrote:
| They don't need that much handholding. They are a couple memory
| augmented gpt papers out now (self-refine, reflexion etc). This
| is by far the most involved in terms of instructing memory and
| reflection.
|
| It helps for control/observation but it is by no means
| necessary.
| awinter-py wrote:
| (thanks for pointer to memory-augmented llms)
| paulusthe wrote:
| Chatgpt is a stochastic word correlation machine, nothing more.
| It does not understand the meaning of the words it uses, and in
| fact wouldn't even need a dictionary definition to function.
| Hypothetically, we could give chatgpt an alien language dataset
| of sufficient size and it would hallucinate answers in that
| language, which neither it nor anybody else would be able
| understand.
|
| This isn't AI, not in the slightest. It has no understanding.
| It doesn't create sentences in an attempt to communicate an
| idea or concept, as humans do.
|
| It's a robot hallucinating word correlations. It has no idea
| what it's saying, or why. That's not AI overlord stuff.
| ux-app wrote:
| >Chatgpt is a stochastic word correlation machine
|
| it seems humans might be too...?
|
| my son is 4. when he was 2, I told him I love him. he clearly
| did not understand the concept or reciprocate.
|
| I reinforced the word with actions that felt good: hugs,
| warmth, removing negative experience/emotion etc. Isn't that
| just associating words which align with certain "good
| inputs".
|
| my son is 4 now and he gets it more, but still doesn't have a
| fully fleshed out understanding of the concept of "love" yet.
| He'll need to layer more language linked with experience to
| get a better "understanding".
|
| LLMs have the language part, it seems that we'll link that
| with physical input/output + a reward system and ..... ?
| Intelligence/consciousness will emerge, maybe?
|
| _" but they don't _really_ feel"_ - -\\_(tsu)_/- what does
| that even mean? if it walks like a duck and quacks like a
| duck...
| TeMPOraL wrote:
| > _Intelligence /consciousness will emerge, maybe?_
|
| Extending that: LLM latent spaces are now some 100 000+
| dimensional vector spaces. There's _a lot_ of semantic
| associations you can pack in there by positioning tokens in
| such space. At this point, I 'm increasingly convinced
| that, with sufficiently high-dimensional latent space,
| adjacency search _is_ thinking. I also think GPT-4 is
| already close to be effectively a thinking entity, and it
| 's more limited by lack of "inner loop" and small context
| window than by the latent space size.
|
| Also, my kids are ~4 and ~2. At times they both remind me
| of ChatGPT. In particular, I've recently realized that some
| of their "failure modes" in thinking/reacting, which I
| could never describe in a short way, seem to perfectly fit
| the idea of "too small context window".
| nuancebydefault wrote:
| You say it has no understanding. So people can communicate
| idea's/concepts while chatgpt can't.
|
| What if... what we think are idea's or concepts, are in fact
| prompts recited from memory, which were planted/trained
| during our growing up? In fact I'm pretty sure our
| consciousness stems from or is memory feeding a (bigger and
| more advanced) stochastic correlation machine.
|
| That chatgpt can only do this with words, does not mean the
| same technique cannot be used for other data, such as neural
| sensors or actuators.
|
| Chatgpt could be trained with alien datasets and act
| accordingly. Humans can be trained with alien datasets.
|
| See the convergence?
| barking_biscuit wrote:
| >It's a robot hallucinating word correlations. It has no idea
| what it's saying, or why. That's not AI overlord stuff.
|
| All that matters is economic and political impact.
| Definitions are irrelevant.
| skilled wrote:
| But the model already has all this info, what is groundbreaking
| about this? These kind of sensational headlines are not helping
| anyone either.
| mztwo wrote:
| What the researchers bolted on is an architecture that enables
| the storage and recall of memories, as well as self-reflection
| and more. They call out early the paper that even standard
| ChatGPT is not quite capable of this. ChatGPT here is used to
| provide the natural language abilities.
| all2 wrote:
| There is some indication that how emotional you are during an
| experience will 1) color your recollection, and 2) affect how
| readily you remember a thing.
|
| It would be interesting to augment this particular simulation
| with those additional constraints. A memory/concept graph
| could also be an interesting addition (like a DB? Maybe just
| text and kw searches?).
| dang wrote:
| This comment was posted to a different thread, which we merged
| into the current thread:
|
| _Stanford 's Groundbreaking AI Study Simulates Authentic Human
| Behavior_ - https://news.ycombinator.com/item?id=35520236
| jsemrau wrote:
| this is a really important conversation that we are not having.
| Based on whose character are we modelling these agents?
|
| If we rely on online conversations for the training we need to
| realize that this is a journey to the dumbest common denominator.
|
| Instead, I believe we should look at the brightest and
| universally morally accepted humans in history to train them.
|
| Maybe I would start my list like that:
|
| 1. Barack Obama.
|
| 2. Jean-Luc Picard (we can rely on work of fiction).
|
| 3. Bill Gates.
|
| 4. Leonardo Da Vinci.
|
| 5. Mr Rogers
|
| 6. ???
| SeanAnderson wrote:
| How about we start and end with just Mister Rogers? :)
| jsemrau wrote:
| Will add him
| anonyfox wrote:
| I really want to have these agents behave as artificial as they
| truly are, not some kind of human, especially not a known one.
| humans have so many flaws, we meatbags are full of emotions and
| other bad behaviors, and it really makes no sense to give them
| some artificial "feelings" like greed, fear and the like. that
| would influence/restrict their mental power too much, let them
| become and act as the machines they are. we should strive to
| become more like them, not the other way around, and eliminate
| the rampant egoism/individualism that destroys the planet and
| societies.
| hobs wrote:
| Ah yes, the universally moral acceptance of Barack Obama, the
| man who made signature strikes a lasting legacy of his
| presidency.
|
| Bill Gates, the man who totally didn't use shady business
| practices and false announcements to destroy legit products to
| the point that people wrote micro$oft for a generation.
|
| And don't even get me started on the new seasons of Picard.
| ethanbond wrote:
| The lack of a perfect human is a good reason not to produce
| ultra-humans who have 1000x higher IQ, are networked to every
| system on the planet, have access to all of humankind's
| knowledge, and don't need to eat, sleep, or die.
| ChatGTP wrote:
| 100% cannot agree more with this, absolutely not the best
| of ideas.
| ismokedoinks wrote:
| [flagged]
| frozenlettuce wrote:
| there's a reason why in many places you can't name a street
| after a living person
| jsemrau wrote:
| I wouldn't argue against your points. Yet, we need to have a
| discussion about character and role models. As I believe we
| should strive for the better not pointing out the flaws of
| others. Destruction is easy.
| willismichael wrote:
| I noticed that you didn't have anything to say about da
| Vinci.
| hobs wrote:
| Harder to pin down, many apocryphal stories so I left him
| out.
| MrOwnPut wrote:
| > signature strikes a lasting legacy of his presidency
|
| drone strikes?
| neuronexmachina wrote:
| Reading the abstract reminded me of Marvin Minsky's 1980s book
| "Society of Mind". I wonder if you could get some cool emergent
| mind-like behavior from a collection of specialized agents based
| on LLMs and other technologies communicating with each other:
|
| * https://en.wikipedia.org/wiki/Society_of_Mind
|
| * http://aurellem.org/society-of-mind/
| IsaacL wrote:
| Funnily enough, I was reading Minsky's book recently. I second
| the recommendation. I think he's missing many technical
| details*, but the basic approach seems to be correct.
|
| *(For example, the idea of a "hierarchy of feedback loops" from
| perceptual control theory would explain a lot of the
| interactions between agents in his theory.)
|
| I also put the abstract of the paper into GPT-4, and gave it
| the following prompt:
|
| > Simplify the above. Use paragraph headings and bold key
| words.
|
| I quite liked its output, as it made it easier to see the core
| ideas in the paper:
|
| _ABSTRACT
|
| Generative Agents: This paper introduces generative agents,
| computational software agents that simulate believable human
| behavior. They can be used in various interactive applications
| like immersive environments, communication rehearsal spaces,
| and prototyping tools.
|
| Architecture: The generative agent architecture extends a large
| language model to store a complete record of the agent's
| experiences in natural language. It enables the agents to
| synthesize memories, reflect on them, and retrieve them
| dynamically to plan behavior.
|
| Interactive Sandbox Environment: The generative agents are
| instantiated in a sandbox environment inspired by The Sims,
| where users can interact with a small town of twenty-five
| agents using natural language.
|
| Believable Behavior: The generative agents produce believable
| individual and emergent social behaviors, such as autonomously
| spreading party invitations and coordinating events.
|
| Components: The agent architecture consists of three main
| components: observation, planning, and reflection. Each
| contributes critically to the believability of agent behavior.
|
| KEYWORDS: Human-AI Interaction, agents, generative AI, large
| language models_
| Baeocystin wrote:
| Looking forward to playing StardewGPT. Half-joking aside, I do
| think that level of abstraction is probably a good choice.
| Familiar and comfy, but with enough detail to be able to find
| interesting social patterns.
| [deleted]
| synaesthesisx wrote:
| Some of the most interesting work in this space is in the
| "shared" memory models (in most cases today, vector db's). Agents
| can theoretically "learn" and share memories with the entire
| fleet, and develop a collective understanding & memory accessible
| by the swarm. This can enable rapid, "guided" evolution of agents
| and emergent behaviors (such as cooperation).
|
| We're going to see some really, really interesting things unfold
| - the implications of which many haven't fully grasped.
| creamyhorror wrote:
| How would vector DBs encode say a precise, technical process
| that has been figured out by an agent? Would the vectors still
| be natural language as with LLMs? Would be great if you could
| point me to one or two exciting papers in the area.
| TeMPOraL wrote:
| It doesn't have to. But the vector search can point it to the
| URL / document database where it can get step-by-step
| instructions of that process, perhaps already
| condensed/compressed by another LLM, and perhaps daisy-
| chained[0] to work around context limits.
|
| ----
|
| [0] - I don't know the right terminology, but I imagine most
| complex processes can still be split into a sequence of sub-
| processes, where each sub-process consists of necessary
| steps, steps to confirm success, and a reference to the next
| sub-process to load if the current one succeeds. The bot
| could then keep only one sub-process in their working memory
| at a time, assuming previous ones succeeded.
| bradgranath wrote:
| Hey! It's a proto ancestor sim!
| kaiherron08 wrote:
| [flagged]
| lurquer wrote:
| The 'safe' tuning of the models is becoming a nuisance. As
| indicated in the paper, the agents are overly cooperative and
| pleasant due to the LLM's training.
|
| Pity they can't get access to an untuned LLM. This isn't the
| first example I've read it where research is being hampered by
| the PC nonsense and related filters crammed into the model.
| Ozzie_osman wrote:
| To directly command one of the agents, the user takes on the
| persona of the agent's "inner voice"--this makes the agent more
| likely to treat the statement as a directive. For instance, when
| told "You are going to run against Sam in the upcoming election"
| by a user as John's inner voice, John decides to run in the
| election and shares his candidacy with his wife and son.
|
| So that's where my inner voice comes from.
| Nevermark wrote:
| Not only will they know more, work 24/7 on demand, spawn and
| vaporize at will, they are going to be perfectly obedient
| employees! O_o
|
| Imagine how well they will manage up, given human managerial
| behavior just becomes a useful prompt for them.
|
| Fortunately, they can't be told to vote. Unless you are in the
| US, in which case they can be incorporated, earn money, and
| told where to donate it, which is how elections are done now.
|
| Seriously. Scary.
|
| On the other hand, if Comcast can finally provide sensible
| customer support it's clear this is will be an historically
| significant win for humanity! Your own "Comcast" handler, who
| remembers everything about you that you tried to scrub from the
| internet. Singularity, indeed.
| synaesthesisx wrote:
| They can't vote, but what if they figure out that they can
| influence human votes?
| Nevermark wrote:
| Yes, they definitely will. Even before AI's care about
| manipulating our politics, people will direct them to.
|
| I already pointed out they can influence elections with
| money.
|
| And bots are already used to influence on social media. AI
| bots are going to be insidious.
| anonyfox wrote:
| I'll take a robotic vote any day over any kind of
| conservative bullshit. It really can only get better here,
| not even kidding. At least if the last things humans do is
| releasing artificial life forms, its still better than
| backwards humans killing each other for nonsense tribalism
| or ancient fairytale books.
| Nevermark wrote:
| As much as humans make a mess of things, on a day to day
| basis there is more good done in the world than bad.
|
| A temporary exception would be the economically still
| incentivized disruption of the environment. I say
| temporary, because at some point it will stop, by
| necessity. Hopefully before.
|
| But I can relate to the deep frustration you are
| expressing.
|
| --
|
| The problem isn't individuals, for the most part. The
| problem is that we build up systems, to provide stability
| and peace, and to be more just and equitable, by
| decentralizing the power in them. That way the powerful
| can't change them on a whim. (Even though they can still
| game them.)
|
| But this also makes them very resistant to change.
|
| Another effect is that as systems stabilize myriads of
| seemingly unimportant aspects within themselves, that
| stability represents the selection of standards and
| behaviors that give the system its own "will" to survive.
| That "will to survive" is distributed across the contexts
| and needs of all participants.
|
| So any pressures to make changes, no matter how well
| thought out, encounter vast quantities of highly evolved
| hidden resistance, from invisible or unexpected places.
|
| Even the most vociferous critics of the system are likely
| to be contributing to its rigidity, and proposing
| incomplete or doomed to fail solutions, because all these
| dynamics are difficult to recognize, much less understand
| or resolve.
|
| --
|
| My view, is that this cost of changing systems needs to
| be accepted and used to help make the changes. I.e. get
| all the CFO's of all the major fossil fuel companies in a
| room. Establish what kind of tax incentives would allow
| them to rationally support smoothly transitioning all
| their corporate resources from dirty energy to clean
| energy.
|
| It would be very expensive. It would look like a handout.
| Worse, even a reward for being a bottleneck to change.
|
| But they are the bottleneck precisely because of all the
| good they have done - that dirty energy lifted the world
| economy. And whatever it cost to "pay them off" would be
| much less than not paying them off.
|
| --
|
| The costs of changing systems needs be dealt with, with
| realism about the costs to get the benefits, and
| creativity and courage about paying for them.
| l33t233372 wrote:
| > It really can only get better here, not even kidding.
|
| I think that's pretty extreme hyperbole.
| allanrbo wrote:
| This "inner voice" idea reminds me of how LangChain works too,
| where you give it a task, and it comes up with actions,
| observations, thoughts, etc. For example:
| https://python.langchain.com/en/latest/modules/agents/gettin...
| TaylorAlexander wrote:
| What's funny is this is one of the semi-important plot points
| in Westworld the TV series. The hosts (robots designed to look
| and act like people) hear their higher level programming
| directives as an inner monologue.
| mclightning wrote:
| I remember someone had predicted this would happen with
| OpenAI's GPT-3, but perhaps now we are closer with ChatGPT...
|
| Found it! : https://medium.com/swlh/bicameral-mind-humanoid-
| robot-with-g...
| [deleted]
| zaptrem wrote:
| When I saw the scene where one of the hosts was looking at
| their own language model generating dialogue (though they
| were visualizing an older n-gram language model) I became a
| believer in LLMs reaching AGI (note: I didn't watch the show
| when it came out in 2016, it was around 2018/19 when we were
| also seeing the first transformer LLMs and theories about
| scaling laws).
|
| The scene: https://youtu.be/ZnxJRYit44k
| TaylorAlexander wrote:
| What about it made you become a believer? Even if a true
| AGI requires a complex network of specialized neural nets
| (like Tesla's hydra network) it would still have a language
| center like the human brain does. It is non obvious to me
| that an LLM by itself can become AGI, though I'm familiar
| with the claims of some that this is plausible.
| ImHereToVote wrote:
| General intelligence doesn't necessarily mean human like
| intelligence.
| isaacfrond wrote:
| You are right that there are intelligences possible that
| are not human. Then again, if one is sufficiently
| intelligent, one could probably convincingly simulate
| human intelligence. There are chess training programs for
| example that are specifically trained to play human
| moves, rather than the best moves.
| johnthewise wrote:
| When prompted, chatgpt answers you as if it is a pirate.
| leroy-is-here wrote:
| What other general intelligence have we seen other than
| human? We know, of course, that animals have
| intelligence, but they do not appear to talk. How are we
| measuring general intelligence now? By IQ, a human test
| through words and symbols.
| naasking wrote:
| The g-factor of IQ may or may not have anything to do
| with general intelligence. The general intelligence of
| AGI is probably a broader category than the g-factor.
| leroy-is-here wrote:
| When I made my comment, I knew nothing about a
| "g-factor".
| VaxWithSex wrote:
| Yes. I love that scene. Improvisation... Improvisation...
| Improvisation...
| smusamashah wrote:
| https://imgur.com/a/NoxaYln screenshot of the dialog tree
| from the video
| [deleted]
| legitimayzer wrote:
| [flagged]
| msla wrote:
| Very Julian Jaynes:
|
| https://en.wikipedia.org/wiki/Bicameral_mentality
|
| > Jaynes uses "bicameral" (two chambers) to describe a mental
| state in which the experiences and memories of the right
| hemisphere of the brain are transmitted to the left hemisphere
| via auditory hallucinations.
|
| [snip]
|
| > According to Jaynes, ancient people in the bicameral state of
| mind experienced the world in a manner that has some
| similarities to that of a person with schizophrenia. Rather
| than making conscious evaluations in novel or unexpected
| situations, the person hallucinated a voice or "god" giving
| admonitory advice or commands and obey without question: One
| was not at all conscious of one's own thought processes per se.
| Jaynes's hypothesis is offered as a possible explanation of
| "command hallucinations" that often direct the behavior of
| those with first rank symptoms of schizophrenia, as well as
| other voice hearers.
| [deleted]
| mirpetri wrote:
| Most of the time we think we think, we actually listen.
| discmonkey wrote:
| This paper feels significant. If chatgpt was an evolutionary step
| on gpt3.5/gpt4, then this is bit like taking chatgpt and using it
| as the backbone of something that can accumulate memories,
| reflect on them, and make plans accordingly.
| gitfan86 wrote:
| Welcome to the singularity
| ChatGTP wrote:
| The singularity sounded a bit more exciting when I heard Ray
| Kurzweil describe it ?
| xwdv wrote:
| It's not really. ChatGPT could already do all those things.
| This just presents it for a different use case.
| throwaway4aday wrote:
| I think you're ignoring the work that went into this as well
| as the useful technology that came out. Prompts make or break
| interactions with LLMs and ChatGPT especially. The difference
| in output from a naive prompt and a well crafted one is huge.
| This paper is one of many explorations of what happens when
| you design such prompts to work in an iterative fashion
| building upon the previous conversation text to produce
| emergent behaviour. These are the seeds of the next
| programming paradigm on a completely novel architecture. It's
| incredibly exciting to be present for the beginning of this
| field, this is what mathematicians must have felt like when
| they helped design and program the first computers.
| discmonkey wrote:
| Oh yeah I agree that it _could_ do all those things, but it
| would be a bit of overkill to always send every observation
| an agent encounters into the API/chatbox, and ask it to spit
| out an evaluation or action.
|
| This paper does a nice job of separating the "agency" from
| the next word with context type predictor. I think that's why
| I like the paper, it is just chatgpt, in the same way that
| pizza is just dough, sauce, and cheese.
| xwdv wrote:
| Yes, but I think this was a fairly obvious conclusion to
| imagine isn't it.
|
| If you were going to seriously consider using ChatGPT for
| AI in a game, you would need each instance of GPT to only
| know certain information it has gathered. And you would
| want it to reflect on observations to come up with new
| thoughts that weren't observed.
|
| Still, I'd argue you don't really even need GPT for any of
| the above. GPT is useful if you want thoughts expressed as
| natural language, but you could easily code observations
| and thoughts into an appropriate abstract data structure
| and still have the same thing, except it's a bit harder to
| understand since asking an NPC something in a language it
| understands and getting back a query result isn't user
| friendly, but it can be just as amazing if you know what
| the data represents. The imprecision and fuzziness of an
| LLM leaves room for fun weirdness though.
| golol wrote:
| It's a pretty obvious idea executed well. I definely think
| symbolic AI agents written in the programming language english
| and interpreted using LLMs is the way forward.
___________________________________________________________________
(page generated 2023-04-11 23:02 UTC)