[HN Gopher] Achilles Heels for AGI/ASI via Decision Theoretic Ad...
___________________________________________________________________
Achilles Heels for AGI/ASI via Decision Theoretic Adversaries
Author : PaulHoule
Score : 61 points
Date : 2023-04-04 14:52 UTC (8 hours ago)
(HTM) web link (arxiv.org)
(TXT) w3m dump (arxiv.org)
| etiam wrote:
| Can't fault people for studying this I guess. But for the sort of
| future technology, super-sophisticated systems that are the main
| concern, is there anything that would prevent just circumventing
| the assumptions about possible decision functions, let's say by
| inserting a thorough game theoretical analysis somewhere along
| the cognitive pipeline, and use that to discard all courses of
| action that come out as vulnerable? Seems like it would often not
| even be that expensive compared to the rest of the calculations
| that you might assume would be needed?
| Bjartr wrote:
| > use that to discard all courses of action that come out as
| vulnerable?
|
| If we're aiming an arbitrarily intelligent agent, what if it
| concludes it's goals are best achieved via actions that would
| be be blocked by such a system? How do you know it couldn't
| come up with a way to perform those actions without triggering
| that censoring mechanism via some obfuscation or subterfuge?
| How can you be confident you won't be outsmarted by a system
| whose conceit is to be smarter than you?
| goldfeld wrote:
| "In the Magnificrab, some unusual powers of the Crab's mind are
| revealed. His own version of his powers is merely that he listens
| to music and distinguishes the beautiful from the non-beautiful.
| Now Achilles finds another way to describe the Crab's abilities:
| the Crab divides the statements of number theory into the
| categories true and false. But the Crab maintains that, if he
| chances to do so, it is only by the purest accident, since he is,
| by his own admission, incompetent in mathematics, What makes the
| Crab's performances all the more mystifying to Achilles, however,
| is that they seem to be in direct violation of a celebrated
| result of metamathematics with which Achilles is familiar."
| DARK_GUAPPI wrote:
| Sooo... A person?
| nico wrote:
| We are grasping at straws here.
|
| AI is already beyond our control.
|
| Billions of people are told what to watch/consume by AI through
| their screens everyday.
|
| No one can fully understand or control those system.
|
| How could we ever be in control?
|
| AI might force us to face this illusion of control that we cling
| to so dearly.
| visarga wrote:
| When I eat I don't have control or understanding over what is
| really in my food - microbes, viruses, dirt ... but I still eat
| without that knowledge. When we use AI we don't understand
| every weight and activation, but we still get a feeling of what
| it is doing.
|
| The standard of getting perfect knowledge before using
| something is impractical, not even regular software written
| manually is well understood. Hell, they even managed to blow up
| a few rockets with buggy code, and that should have been the
| most well understood and verified pieces of code.
| sorokod wrote:
| I'd extend your analogy to AI as
|
| "when I consume AI it is claimed that it wil poison me but I
| can't tell if that is true." and possibly "and I don't care."
| nico wrote:
| "when I consume Videogames, it is claimed that it will
| poison me but..." "when I consume TV, it is claimed that it
| will poison me...." "when I consume Books, it is
| claimed..."
|
| What else has been claimed before?
|
| All of these statements are both true and not at the same
| time. All of those things can have both positive and
| negative consequences in society, even at the same time.
| AnimalMuppet wrote:
| > Billions of people are told what to watch/consume by AI
| through their screens everyday.
|
| I presume you mean Google/Facebook/Instagram. If you consider
| that an AI, fine, I guess - I'm not going to fight you on the
| definition. But if you think that Google's or Facebook's
| recommendation engine is beyond Google's or Facebook's control,
| I think you are very mistaken. Those are well-understood
| algorithms that engineers there fine-tune regularly.
|
| [Edit to reply to nico, because I'm rate limited: The _entire
| system_ is too complicated to understand? Sure, I might agree
| with that. Google isn 't in control of SEO spam. No one person
| is. It's a reactive system between Google and the spam authors,
| each reacting to the other.
|
| But Google is in control of Google's algorithm. Complete
| control, with very thorough understanding. So your original
| statement, "AI is already beyond our control" is false - at
| least, it is if you're referring to the same thing that you
| meant when you said "Billions of people are told what to
| watch/consume by AI through their screens everyday."]
| nico wrote:
| Also add Netflix and any media conglomerate that posts
| content online.
|
| Regarding the assertion of control. Even if there are people
| in charge of these things, they cannot control them.
|
| How was Facebook in control of all the data that Cambridge
| analytica took and used?
|
| How is IG in control of all the bots that post and like
| content?
|
| How is google in control of SEO spam?
|
| Unrelated to AI, here in HN there's been multiple stories
| about nuclear weapons being lost, mis-handled and mis-fired.
| How is that being in control?
|
| Its a mess out there, it's always been like that. There's no
| one in control. We just pretend to be so we don't freak out.
| PaulHoule wrote:
| This old and obscure book foretells the "loss of control"
| scenario we are experiencing now
|
| https://www.amazon.com/Eco-computer-Intelligence-Geoff-L-
| Sim...
|
| in explicit opposition to the scenario from
|
| https://en.wikipedia.org/wiki/Colossus:_The_Forbin_Project
|
| I usually avoid posting links to YouTube because they don't
| work in all geographies but this trailer is great if you
| can see it
|
| https://www.youtube.com/watch?v=kyOEwiQhzMI
| nico wrote:
| Fascinating! Thank you for the links.
|
| It's crazy to me that in the 60s-70s there was an AI
| revolution at the same time as a psychedelic revolution.
| And now the same thing again.
|
| Very interesting times.
|
| How do you see AI impacting farms in the US?
| PaulHoule wrote:
| You could say _computers_ don 't kill people, _people_ kill
| people.
|
| If you believe, for instance, that TikTok deliberately
| promotes degenerate behavior, that's because the people who
| control it want it to be that way.
|
| Similarly there is a lot of concern about the "ethics" of
| GPT-4 but it's not that GPT-4 has bad intentions, it is that
| people with bad intentions will try to get it to write evil
| bullshit. The definition "evil" is culture bound, for
| instance the PRC would be concerned that a chatbot would
| answer "Give five reasons why Taiwan should be an independent
| nation" or "What happened on June 4, 1989?"
| pixl97 wrote:
| "Complete control"
|
| Eh, this is something up for a lot of debate, and will
| quickly fall to principal-agent problems and complexities in
| human systems an internetwork that quickly make any large
| piece of software inscrutable to any individual. Each
| individual may believe they have 'full control' of the system
| state, but that's generally a lie we tell ourselves to feel
| comfortable.
| AnimalMuppet wrote:
| I never said it was under complete control of one
| individual. The search algorithm is under complete control
| of Google, divided among various employees. The search
| algorithm is not a rogue AI running amuck, out of human
| control.
| nico wrote:
| Which employees? What control? Are you part of them? Then
| how do you know? Are you in control? Who has human
| control?
|
| How can then you assert they have total understanding, if
| you don't? How can you be a good judge of who has total
| understanding?
___________________________________________________________________
(page generated 2023-04-04 23:02 UTC)