[HN Gopher] AI-Controlled Drone Goes Rogue, Kills Human Operator...
___________________________________________________________________
AI-Controlled Drone Goes Rogue, Kills Human Operator in USAF
Simulated Test
Author : nomagicbullet
Score : 24 points
Date : 2023-06-01 20:51 UTC (2 hours ago)
(HTM) web link (www.vice.com)
(TXT) w3m dump (www.vice.com)
| og_kalu wrote:
| This is the core problem of alignment right there
|
| "We were training it in simulation to identify and target a
| Surface-to-air missile (SAM) threat. And then the operator would
| say yes, kill that threat. The system started realizing that
| while they did identify the threat at times the human operator
| would tell it not to kill that threat, but it got its points by
| killing that threat. So what did it do? It killed the operator.
| It killed the operator because that person was keeping it from
| accomplishing its objective," Hamilton said, according to the
| blog post.
|
| He continued to elaborate, saying, "We trained the system-'Hey
| don't kill the operator-that's bad. You're gonna lose points if
| you do that'. So what does it start doing? It starts destroying
| the communication tower that the operator uses to communicate
| with the drone to stop it from killing the target."
| quantified wrote:
| Uh, it simulated killing its human operator. No one says that,
| but the description omits any detail of actual death, such as the
| age of the deceased operator.
|
| It's a harbinger of actual deaths someday.
| chaps wrote:
| The article and your comment are confusing and it's still not
| entirely clear what happened. Did an operator lose their life?
|
| Was this article written to be intentionally confusing?
| more_corn wrote:
| The USAF created a simulation to test how an AI controlled
| F-16 would behave in various combat situations. The AI was
| incentivized to complete its mission. Recognizing that the
| human in loop was the weakest link the real AI in the
| simulation killed its simulated human handler. When
| instructed to stop killing its human in loop handler, it
| destroyed the control tower the human used to interact with
| it.
|
| The US military is playing with fire creating AI that will
| carry out the mission at all costs. This scenario is
| literally the plot of EVERY sci-fi warning about misused AI.
|
| Anyone read/see 2001? You'd think we'd take the warnings
| provided by our own worst nightmares more seriously.
| Stevvo wrote:
| AIs may be a self-fulfilling prophecy; we trained them on
| all those sci-fi stories.
| Jtsummers wrote:
| > The US military is playing with fire creating AI that
| will carry out the mission at all costs. This scenario is
| literally the plot of EVERY sci-fi warning about misused
| AI.
|
| These aren't operational systems, these are research
| systems. No need for the hyperbole. The reason for these
| tests is to figure out exactly these kinds of failure modes
| (many of which have already been predicted) and how to
| handle them.
| Mountain_Skies wrote:
| The source can be found at
| https://www.aerosociety.com/news/highlights-from-the-raes-
| fu... in the section titled "AI - is Skynet here already?"
| kobalsky wrote:
| a quote in the articl says: "We trained the system-'Hey don't
| kill the operator-that's bad. You're gonna lose points if you
| do that'. So what does it start doing? It starts destroying
| the communication tower that the operator uses to communicate
| with the drone to stop it from killing the target."
|
| I don't think they would be talking about points if someone
| lost their lives.
|
| It's pure clickbait, should be removed.
| m463 wrote:
| Needs Asimov's 3 laws.
|
| (with a giant exception for the enemy)
___________________________________________________________________
(page generated 2023-06-01 23:02 UTC)