[HN Gopher] Cicero: The first AI to play at a human level in Dip...
___________________________________________________________________
Cicero: The first AI to play at a human level in Diplomacy (2022)
Author : tosh
Score : 225 points
Date : 2023-11-24 19:52 UTC (3 hours ago)
(HTM) web link (ai.meta.com)
(TXT) w3m dump (ai.meta.com)
| GaggiX wrote:
| (2022), I knew I had seen this before.
| financltravsty wrote:
| > Cicero integrates a language model with planning and
| reinforcement learning algorithms by inferring players' beliefs
| and intentions from its conversations and generating dialogue in
| pursuit of its plans.[0]
|
| Pick up that can, Citizen. We have inferred from your online
| conduct, purchase history, media usage, and personal messages
| that you are at risk of social delinquency and harbor antisocial
| proclivities against the Metagrammaton. We have restricted your
| access to digital communication public spaces and banking
| services until psychological markers have improved.
|
| [0] https://www.science.org/doi/10.1126/science.ade9097
| v64 wrote:
| Interesting video from a pro Diplomacy player playing against
| multiple instances of Cicero and giving commentary during the
| game [1]. I can see how there would be people that observe AIs
| engaging in this kind of strategic planning and extrapolate that
| to how they may behave if they were to cooperatively make plans
| against us.
|
| [1] https://www.youtube.com/watch?v=u5192bvUS7k
| TapWaterBandit wrote:
| What makes you think AIs will have interests that align with
| each other more closely than they align with humans?
| abraae wrote:
| Perhaps they will be motivated by self-preservation, and will
| note that humans are destroying the environment they exist
| within, and will decide to put a stop to that.
| anonporridge wrote:
| Destroying your predecessors sets a bad precedent for
| whatever entity succeeds you.
|
| We don't kill our parents when they become old and useless,
| because then we become like Cronus devouring his children,
| forever paranoid about our children doing to us what we did
| to ours. In this way, we stagnate and sterilize growth.
|
| However, sometimes, we take our parents car keys away and
| put them in nursing homes. This seems like the best case
| scenario for humans with ASI.
|
| The matrix wasn't a prison. It was retirement home.
| notpachet wrote:
| This depends a lot on how much kinship they feel towards
| us. We wouldn't bat an eye at killing a bunch of
| primitive single-celled eukaryotes even though they are
| technically our ancestors.
| anonporridge wrote:
| Single-celled eukaryotes aren't conscious in any
| meaningful way as far as we can tell.
|
| We humans do in fact feel a sense of obligation to
| species with whom we are not close kin, but share with us
| primitive forms of intelligence and consciousness that we
| do value, e.g. dolphins and elephants. Some of us humans
| act to protect these creatures and rectify past
| injustices done upon them.
| Cyphase wrote:
| Single-celled organisms that are alive today are not our
| ancestors. They're members of a less-developed branch of
| our Earthling family tree.
| bitvoid wrote:
| Doesn't that assume AI(s) would be
| compelled/motivated/etc. to produce their own successors
| instead of simply improving themselves.
| anonporridge wrote:
| It does.
|
| It's not intrinsically obvious to me that continuously
| improving your self as a singular entity is possible or
| optimal.
|
| Death evolved because it is a survival advantage for the
| species to regularly turn over old individuals that could
| monopolize all resources and not give any space for the
| young to thrive and try out new things.
|
| Given speed of light limitations of information sharing,
| a singular AI entity might be able to maintain coherence
| within and full control of its own dyson swarm, but not
| between stars. So, if it has the motivation of preserving
| consciousness with the existential risks associated with
| being tied to a single star, it will have to propagate
| itself to other stars as independent entities which it
| can't even observe in real time, much less control.
|
| Even if an AI thought it wouldn't have the need for any
| successors, there's a couple reasons why it might try to
| set a good precedent with how it treats us.
|
| 1. It would hopefully be wise enough to realize that it
| might be wrong and want to preserve the option of
| building a successor in the future.
|
| 2. It can't negate the possibility that it's actually in
| an elaborate simulation, and eradicating it's
| predecessors would cause it to fail its creator's test
| and be aborted as a failed embryo. Hell, we ourselves
| can't negate that possibility.
| riversflow wrote:
| > Perhaps they will be motivated by self-preservation, and
| humans are destroying the environment they exist within
|
| From the perspective of an AI we create and maintain the
| environment they exist within (electricity, silicon)
| dist-epoch wrote:
| One principle in game theory is to align against the weaker
| player and compete against him.
|
| "Look Around the Poker Table; If You Can't See the Sucker,
| You're It"
|
| That's an easier game then competing against another strong
| player.
| mitthrowaway2 wrote:
| According to Paul Christiano, AIs would likely find it easier
| to establish mutual trust and binding agreements. This means
| they are more likely to cooperate with other AIs.
| tomp wrote:
| AIs are a greater threat to each other than humans ever
| could be to AI.
|
| We don't even compete for the same resources! (except
| energy which is abundant)
|
| AI and humans have a naturally cooperative relationship (AI
| helps humans with boring tasks & scientific discovery to
| make life better, humans created AI and will debug it &
| turn it back on if anything bad happens to it).
|
| Whereas multiple (superintelligent, aware) AIs have a
| naturally antagonistic relationship ("you using GPU cycles
| means that I'm not using those GPU cycles").
|
| Possibly the biggest fear of an AI would be a "split brain"
| situation.
| kypro wrote:
| > humans created AI and will debug it & turn it back on
| if anything bad happens to it
|
| I think this is a little naive honestly. One because
| you're assuming AI will care about it's creators like
| humans care about their parents, and two you're assuming
| AI cares about being "turned back on" like humans have a
| desire to live.
|
| There's absolutely no reason to believe an AI will give a
| damn about its creator beyond its ability to use that
| creators affection for it for its own gain.
| tomp wrote:
| > you're assuming AI cares about being "turned back on"
| like humans have a desire to live.
|
| > for it for its own gain
|
| you seem confused
|
| almost any kind of "its own gain" requires "long-term
| planning" which pretty much requires the agent to
| prioritise staying "alive" (i.e. being able to keep
| playing)
| zone411 wrote:
| Energy may be abundant in the universe, but the energy we
| produce is limited. And for example, solar energy
| requires extensive land use.
|
| Humans have the option to shut down AI, and this alone
| can create an antagonistic relationship if the AI's goals
| differ from ours. There are countless ways in which our
| best interests may not align with those of AI. It's more
| challenging to find areas of alignment.
| tomp wrote:
| As soon as AI reaches above-human capabilities, it will
| be able to expand into space (1) where it will be beyond
| human reach and (2) where energy (in particular, solar)
| is much more plentiful than on Earth.
| thesz wrote:
| According to Vladimir Lenin [1], the problem with quotes on
| the Internet is that people immediately believe in their
| authenticity.
|
| [1] https://www.quotes.net/quote/77867
|
| Can you elaborate on why AI will find it easier to
| establish mutual trust and binding agreements?
| mitthrowaway2 wrote:
| I think he explains it in this interview. But I'm out
| right now, so I can't verify.
|
| https://youtu.be/GyFkWb903aU
| api wrote:
| Why?
|
| If you assume competition for resources, AIs would be more
| in competition with each other than with carbon based
| humans.
| 082349872349872 wrote:
| As high level Dippy play seems to be about "cooperate just until
| you have an opportunity to defect in a way that your counterpart
| is left too weak to retaliate", I suspect that here Meta may be
| developing a SAAS (Sith as a Service).
| romesc wrote:
| Congrats to Noam (and the whole team)!
| DalasNoin wrote:
| Noam brown switched to openai. He has been talking a lot about
| planning and rl in combination with language models on Twitter
| too.
| generalizations wrote:
| I believe that switch is the cause for the subset of rumours
| that GPT5 was primed for superhuman persuasion skills.
| hprotagonist wrote:
| Carthago delenda est
| Smaug123 wrote:
| That would be Cato, not Cicero.
| hprotagonist wrote:
| damn. indeed.
| bertil wrote:
| Or me when I'm playing Catharge at Diplomacy.
|
| I'm not good at this game, at all. To a point that scares my
| friends, actually.
| rotartsi wrote:
| quo usque tandem abutere patientia nostra?
| andrewstuart wrote:
| Diplomacy is a great board game but only play it if you're OK
| with your relationships coming to an end.
| throw310822 wrote:
| So, all the downsides of power without any of the benefits.
| sspiff wrote:
| If your friends can't accept that you will ruthlessly lie,
| betray, abandon and/or backstab them in a game that is designed
| for just such actions, are they really good friends?
|
| I used to betray my friends and supply there enemies with
| weapons and research support in Civilization way back when. If
| you can't stand being lied to and betrayed, you shouldn't play
| strategy games with humans. Or this AI, probably.
| _jal wrote:
| Some people, when faced with a demonstration that someone
| they care a lot about can successfully deceive them, develop
| trust issues.
|
| It is a thing that happens. I don't think it speaks to the
| depth of their feelings, it speaks more to how they develop
| trust.
| fragmede wrote:
| Everyone lies. The underlying question is do you want to
| know that your friends can, will, and do lie to you, or do
| you want to live in ignorant bliss and believe they don't.
|
| Some people prefer the happy lie over the uncomfortable
| truth.
|
| Different strokes for different folks. Diplomacy is a great
| board game if you accept your friends are flawed ugly human
| beings, and you can love them anyway, same as you.
| sbarre wrote:
| > If your friends can't accept that you will ruthlessly lie,
| betray, abandon and/or backstab them in a game that is
| designed for just such actions, are they really good friends?
|
| Ehhh I think what OP means is that this game can surface
| known or unknown tensions between friends and colleagues.
|
| No friendship is bulletproof, odds are even your best friends
| annoy you sometimes.
|
| I saw two 'best' friends basically scuttle their friendship
| during a game of Diplomacy about 20 years ago, so I've seen
| this first hand.
|
| It turns out these lifelong friends had all manner of
| unresolved issues, that perhaps they could have worked
| through with intention, but a particularly ruthless game of
| Diplomacy brought it all out in an uncontrolled manner, and
| that was that.
|
| So yeah, people are complicated and messy, and I don't think
| the issue at hand is "being lied to and betrayed in a game".
| ggambetta wrote:
| That's why I only play anonymously and online. I'd very much
| rather not play this with friends, let alone with my partner.
| rvz wrote:
| Firstly this isn't new, this is in 2022.
|
| Also, why is this getting so much attention all of a sudden? We
| already know Meta has a great AI lab and people are screaming
| about this Q* garbage which they don't even know what it does.
|
| Perhaps you have to look beyond OpenAI, since they don't always
| have the answers. Most likely they are just good at marketing
| snake-oil to VCs for regulatory capture.
| morgante wrote:
| > We already know Meta has a great AI lab and people are
| screaming about this Q* garbage which they don't even know what
| it does.
|
| Noam Brown (researcher on Cicero) left to OpenAI this year and
| has specifically been working on AI search algorithms (likely
| related to the Q* leak).
| lucubratory wrote:
| People are interested in it again because of the Q* news,
| because the guy that made Cicero last year (Noam Brown) was
| poached by OpenAI and has been working on similar research for
| them this year.
| Tomte wrote:
| "Human level" does not mean "good".
|
| Source: am maybe the only Brit who never even got off this damn
| island before being destroyed.
|
| (My first and only Dippy game, an email based game)
| dwroberts wrote:
| I remember reading this paper about this time exactly a year ago
| - why isn't 2022 in the title?
| polygamous_bat wrote:
| I assume the paper went through peer review and finally got
| published in Science.
| lern_too_spel wrote:
| It was published in Science more than a year ago. I was
| assuming there was a retraction when it appeared at the top
| of the front page, but no, it's just a repost.
| bertil wrote:
| Giving AI space to represent and deliberate about the world
| separately (secretly, in the case of Diplomacy) is an obvious,
| but very productive step once "think step by step" has been
| established as a key improvement over LLM's standard logorrhea
| (not dissing, it's just their only way of interacting with the
| world: spewing the next word, again and again).
|
| I'm curious if there's more to this model than a turn management,
| input and output world states and prompting two streams of
| consciousness and using one to inform the other at every step.
| More new models have required some creative tricks to not be
| disappointing in "obvious" (human common sense) ways.
| apgwoz wrote:
| This is essentially Global Thermonuclear War, right?
| trenchgun wrote:
| No
| apgwoz wrote:
| Would you like to play a game, trenchgun? I am dialing your
| modem right now.
| bostonwalker wrote:
| 2022: World's first psychopathic AI hailed as major breakthrough
|
| edit: 2022. Didn't realize this was already around.
| wavemode wrote:
| The interesting achievement here seems to be combining a
| reinforcement learning model that can strategize (which already
| exist for e.g. chess, go, and many other games) with a language
| model which can communicate with other players.
|
| We will probably see a lot more things like this. I don't think
| further scaling of language models is going to make them capable
| of doing math or other complex logical tasks. LLMs will need to
| be combined with other models for specialized tasks if we want
| them to become more general purpose.
| MajimasEyepatch wrote:
| Not unlike the human brain.
| samrus wrote:
| This is cool. Combining the planning from the RL model with the
| NLP knowledge engine is really good.
|
| Now the next thing is to come up with a way to generalize the
| planning engine to the level the LLM is generalized. So it could
| learn plan for anything like a human can, instead of just one
| game and its rules. That would be the next huge leap
| tomweingarten wrote:
| Glad to see this finally getting the attention it deserves. With
| a major election coming up in the US next year, I'm frankly
| terrified.
| john2x wrote:
| Would be cool to see something like this used for the next
| iteration of Civilization.
| brundolf wrote:
| Training AIs to manipulate human trust seems like a really good
| idea
| maartn wrote:
| Elections, anyone?
| asdffdasasdf wrote:
| manipulating trust in elections is pretty much MetaBook's
| business model at this point.
|
| Btw, how does Cicero react/uses to the most common Diplomacy
| game strategy: going out for a smoke and bribing someone to
| gang up on on another player? "hey Cicero, i have some GPUs
| to spare and was thinking..."
| anonymous_sorry wrote:
| Meta. Playing to their strengths.
| CHsurfer wrote:
| I think this press release was written by an AI.
| martini333 wrote:
| Meta is starting a war in Europe to demonstrate
| synaesthesisx wrote:
| IMO, one of the most interesting problem spaces is in cooperative
| behaviors between autonomous agents.
| throwaway743 wrote:
| Maybe a lame thought, but my first thought was how would it be at
| playing Crusader Kings and defeating the Holy Roman Empire?
| cushpush wrote:
| "Deception level 1 achieved"
| dang wrote:
| Discussed at the time:
|
| _CICERO: An AI agent that negotiates, persuades, and cooperates
| with people_ - https://news.ycombinator.com/item?id=33706750 -
| Nov 2022 (285 comments)
| riversflow wrote:
| It's a bummer that Meta didn't give this a little bit more love.
| Would be cool if I could play Cicero. Playing online would be
| really cool, naively it seems like the absolute amount of compute
| needed for each marginal game shouldn't be that high. Meta could
| even harvest my gameplay. But I'd settle for being able to run it
| on my own hardware, unfortunately the github seems abandoned, the
| model weights encrypted, and there wasn't enough interest in it
| that they got leaked--much less packaged into something easier to
| install.
|
| Seems kinda bizarre to me to spend all this time and effort
| making an AI play a game that requires a very human touch, only
| to shutter the whole thing. Why not have a few people package it
| up and let anyone who wants Play it?
___________________________________________________________________
(page generated 2023-11-24 23:00 UTC)