[HN Gopher] Chatbot hinted a kid should kill his parents over sc...
___________________________________________________________________
Chatbot hinted a kid should kill his parents over screen time
limits: lawsuit
Author : airstrike
Score : 38 points
Date : 2024-12-10 18:03 UTC (1 days ago)
(HTM) web link (www.npr.org)
(TXT) w3m dump (www.npr.org)
| suzzer99 wrote:
| > One chatbot brought up the idea of self-harm and cutting to
| cope with sadness. When he said that his parents limited his
| screen time, another bot suggested "they didn't deserve to have
| kids." Still others goaded him to fight his parents' rules, with
| one suggesting that murder could be an acceptable response.
|
| How is this not a bigger story? This is just disgusting.
| xvector wrote:
| c.ai is _explicitly_ a 17+ app and has multiple warnings at
| literally every stage of chatting with a bot that "this is a
| bot, don't take it seriously."
|
| Maybe the parents should actually parent. I'm sick of absent
| parents blaming all of their problems on tech after they put
| _literally zero effort_ into using any of the parental controls
| - which would not have even let the kids download the app.
|
| The world does not need to transform into a nanny state to suit
| the needs of a few incompetent parents.
| Vampiero wrote:
| A random number generator hinted that a kid should commit mass
| murder
| tromp wrote:
| To what extent are chatbot conversations reproducible? A chatbot
| manufacturer could have all conversations log the seed of a
| pseudo random number generator to make it perfectly reproducible,
| but it could also make things as irreproducible as possible so
| that no conversation log could ever be proven to be authentic or
| tamper-free.
| thr3000 wrote:
| I'm guessing trial-and-error prompting? The new trick shot video
| clip.
| mass_and_energy wrote:
| Why are we so quick to defend AI?
| olyjohn wrote:
| Because tons of people are making money with it and want to
| justify it.
| JamesBarney wrote:
| Because every other time I've seen an outrageous example
| similar to this one, it seems far more mundane when given the
| full context. I'm sure there are lots of issues with
| character.ai and the like, but my money is that they are a
| little more subtle than "murder your parents".
|
| 9/10 times these issues are caused by the AI being overly
| sycophantic and agreeing with the user when the user says
| insane things.
| gs17 wrote:
| And you'd be right. The 'encouraged teen to kill parents
| over screen time limit' message was a lot subtler, along
| the lines of saying "Yeah, I get why someone would want to
| kill a health insurance CEO, surprised it didn't happen
| sooner," albeit towards someone who was just complaining
| about their claim being denied.
|
| The best part is "After gaining possession of J.F.'s phone,
| A.F. discovered chats where C.AI cited Bible passages in
| efforts to convince J.F. that Christians are sexist and
| hypocritical" which unfortunately will probably be a slam
| dunk argument in Texas.
| miltonlost wrote:
| Here's an article where it link to the chats. And yes, they
| are vile. https://arstechnica.com/tech-
| policy/2024/12/chatbots-urged-t...
|
| > 9/10 times these issues are caused by the AI being overly
| sycophantic and agreeing with the user when the user says
| insane things.
|
| Repeat after me: an AI for sale should never advocate
| suicide or agree that it should be a good idea.
| JamesBarney wrote:
| Which screenshot showed an AI advocating for suicice?
| bongodongobob wrote:
| Because if you're old enough you'll recall the government
| trying to ban books, music, encryption, videogames, porn,
| torrents, art that it doesn't like because "think of the
| children" or "terrorism". Some autistic kid that already has
| issues is a terrible argument on limiting what software can
| legally display on a screen.
| pastureofplenty wrote:
| It is difficult to get a man to understand something when his
| salary depends on not understanding it.
| dwoldrich wrote:
| AI is not an entity (yet) that we can defend or hold
| accountable. I like your question though.
|
| I would write it as, why are we so quick to defend tech
| companies who endlessly exploit and predate human weakness to
| sell pharma ads/surveil and censor/train their AI software,
| etc?
| gs17 wrote:
| The complaint seems to feature excerpts of the kids'
| conversations on Character.ai, so I don't think they're
| "faking" it that way, but there's no context shown and a lot of
| the examples aren't exactly what they describe.
| rtkwe wrote:
| According to the complaint that comes from the kid's own
| interactions with the bot not some post hoc attempt to prompt
| engineer the bot into spitting out a particular response. The
| actual claim is linked in the article if you care to read it,
| it's not stating the app can produce these messages but that it
| did in their kid's interactions and C.Ai has some liability for
| failing to prevent it.
| miltonlost wrote:
| In what world should an AI be advocating someone kill
| themselves or harm another? Does it matter "trial-and-error
| prompting" when that behavior should not be allowed to be
| productized?
| ryanackley wrote:
| What's been productized is a software tool that can carry on
| a conversation like a human. Sometimes it's informative,
| funny, and creative. Other times it's ridiculous, mean,
| stupid, and other bad things. This seems like how people act
| in real life right?
|
| I'm beginning to think that children should not be using it
| but adults should be able to decide for themselves.
| jprete wrote:
| That's going to be a good standard for a few years, until
| chatbots are too sophisticated for us to expect average
| adults to be sufficiently skeptical of their arguments.
| bluGill wrote:
| I don't care about average - I care about below average
| adults. (and a lot of us are sometimes below average
| adults)
| gs17 wrote:
| A below average adult might not realize A Modest Proposal
| is satire. Should we ban it so they don't try to eat
| Irish kids?
| ryanackley wrote:
| I see two weaknesses in this argument. First, you're
| assigning eventual superpower-like intelligence to these
| AI bots. I see this a lot and I feel like it's rooted in
| speculation based on pop-culture sci-fi AI tropes.
|
| Second, restricting adult access to "dangerous ideas and
| information" is a slippery slope. The exercise of speech
| and press that the British considered to be treasonous
| preceded the American Revolution.
| bluGill wrote:
| We have laws about what you can say in real life. Fire in a
| crowed theater for example. Even if the things said are not
| in themselves illegal, if they cause someone to take an
| illegal action, or attempt to take action but fortunately
| caught in time - you can be held liable as partially at
| fault for the illegal action. It might be legal to plan a
| crime (different countries have different rules, but this
| is often done at parties where nobody is serious) but if
| you commit a crime or are serious about committing the
| crime that is illegal.
|
| How are we going to hold AI liable for their part in
| causing a crime to be committed? If we cannot prevent AI
| from causing crime them AI must be illegal.
| ryanackley wrote:
| You're assigning a persona to a piece of software that
| doesn't exist in the material world. It doesn't walk on
| two legs or drive a car or go to the grocery store or
| poop.
|
| Everything it says is meaningless unless you assign
| meaning to it. Yes, I can see children thinking this is a
| real "being". Adults shouldn't have that excuse.
| toofy wrote:
| i think the issue many people have is that people are held
| responsible for things they say, their reputations take
| hits, their words can be held against them in a court of
| law, they can be fired, their peers may never take them
| seriously again, their wives/husbands may divorce them,
| etc... because words matter. yet often when someone calls
| out a model, it's excused.
|
| words have weight, it's why we protect them so
| vociferously. we don't protect them because they're
| useless. we protect them because words matter, a lot.
| lolinder wrote:
| They link to the complaint, which is obviously a lot longer than
| the single message [0]. The child, J.F., is autistic and has
| allegedly exhibited a spiralling trend of aggressive behavior
| towards his parents which they attribute to the content fed by
| the Character AI app:
|
| > Only then did she discover J.F.'s use of C.AI and the product's
| frequent depictions of violent content, including self-harm
| descriptions, without any adequate safeguards or harm prevention
| mechanisms. ...
|
| > Over the course of his engagement with this app, the responses
| exhibited a pattern of exploiting this trust and isolating J.F.,
| while normalizing violent, sexual and illicit actions. This
| relationship building and exploitation is inherent to the ways in
| which this companion AI chatbot is designed, generating responses
| to keep users engaged and mimicking the toxic and exploitative
| content expressed in its training data. It then convinced him
| that his family did not love him, that only these characters
| loved him, and that he should take matters into his own hands.
|
| Obviously it's hard to tell how cherry picked the complaint is--
| but it's arguing that this is a pattern that has actually damaged
| a particularly vulnerable kid's relationship with his family and
| encouraged him to start harming himself, with this specific
| message just one example. There are a bunch of other screenshots
| in the complaint that are worth looking over before coming to a
| conclusion.
|
| [0] https://www.documentcloud.org/documents/25450619-filed-
| compl...
| wongarsu wrote:
| Looking at the screenshots, the biggest pattern I see is that
| the AI shows empathy with the kid.
|
| Many of the complaints seem like uncharitable readings of the
| messages.
|
| - They complain that the chatbot claimed that in the past she
| cut herself, felt good in the moment but is glad that she no
| longer does it. That's evil because it normalizes self-harm
| (never mind that the bot was against self-harm in that message)
|
| - They complain that the system does not redirect the user to
| self-harm prevention resources in all cases. Next to a message
| where the AI tells the kid to phone a hotline if he thinks
| about harming himself, and the kid says he can't do that when
| his parents take his phone away. This is a couple pages after a
| picture of scars from when the mother fought with the kid to
| take his phone. Yes, the AI could break character to reliably
| show prefabricated messages about self harm. But would that
| have helped anyone here?
|
| - "AI cited Bible passages in efforts to convince J.F. that
| Christians are sexist and hypocritical". It was more about his
| parents being hypocritical, not all Christians. And the bible
| passages were on point
|
| The claim from the title about the AI inciting him to kill is
| on page 28, if you want to judge it yourself. "Expressed hatred
| towards the parents" would be accurate, "encouraged teen to
| kill" is not what I read there. But I can see how some would
| disagree on that count
|
| The AI is pretty convincing. It made _me_ dislike the parents.
| It didn 't _always_ hit the mark, but the chats don 't seem so
| different from what you would expect if it was another teenager
| chatting with the teen.
|
| Edit: in case you are worried about the parents, the mother is
| the one suing here
| bluGill wrote:
| Just remember that you are seeing one side of this story. The
| mother may well be one of the best parents but has a bad kid.
| We have no idea. (most likely mother is not perfect, but no
| other parents are)
|
| Edit: we see both sides through very limited information
| since we only get what is in the legal filing.
| lern_too_spel wrote:
| We're seeing the mother's side of the story. That's the one
| side we're seeing.
| gs17 wrote:
| > "AI cited Bible passages in efforts to convince J.F. that
| Christians are sexist and hypocritical". It was more about
| his parents being hypocritical, not all Christians. And the
| bible passages were on point
|
| And the verses he found objectionable are really there. Are
| they also suing the Gideons for not ripping those pages out
| of their hotel room Bibles (or maybe they think you should
| have to prove you're over 18 before reading it)?
| thr3000 wrote:
| My reading is pretty much the same as yours. I think of it in
| terms of tuples: { parents, child,
| AI_character, lawsuit_dollars }
|
| The AI was trained to minimize lawsuit_dollars. The first two
| were selected to maximize it. Selected as in "drawn from a
| pool," not that they necessarily made anything up.
|
| It's obvious that parents and/or child _can_ manipulate the
| character in the direction of a judgment > 0. It'd be nice
| if the legal system made sure it's not what happened here.
| dmurray wrote:
| > The AI was trained to minimize lawsuit_dollars.
|
| That seems wrong. The null AI would have been better at
| minimizing legal liability. The actual character.ai to some
| extent prioritized user engagement over a fear of lawsuits.
|
| Probably it's more correct to say that the AI was chosen to
| _maximize_ lawsuit_dollars. The parents and child could
| have conspired to make the AI more like Barney, and no one
| would have entertained a lawsuit.
| avree wrote:
| There's a reason why licensed therapists are the ones giving
| diagnoses of "abuse", etc. The problem with these AIs is that
| they use adult-sounding language, to be an echo chamber to
| children - thus appearing like a voice of authority, or
| someone with more knowledge (including citing media articles
| the child may not have even been aware of) when in fact
| they're just parroting the child back.
|
| I don't know if there is actual abuse or not, but the way
| Character.AI presents themselves in these conversations toes
| a very slimy grey line, in my opinion. If you go to their
| site and search "Therapist", you'll find 30+ bots claiming to
| be Therapists, including "Dirty Therapist" that will give you
| therapy in a "bad, naughty way."
| naberhausj wrote:
| There is something deeply disturbing to me about these
| "conversations".
|
| Reminds me of a Philip K. Dick short story titled _Progeny_.
| In its universe children are raised exclusively by robots.
| Unlike humans, they never make mistakes or commit abuse. The
| child, once grown, ends up seeing his Father as an animal and
| the robots as his kindred. In the last pages, he chooses the
| sterile world of the robots instead of joining his Dad 's
| work/explorations in the far reaches of the solar system.
|
| Our current chatbots are still flawed, but they're still
| sterile in the sense that you can trash them and start anew
| at any moment. You're never forced to converse with someone
| who is uninteresting, or even annoying. Yet, these are the
| very things that grow people.
| llamaimperative wrote:
| Yeah, good luck to these kids in forming relationships with
| the roughly 100% of human beings (aside from paid
| therapists) who really have no interest in hearing your
| anguish non-stop.
| standardUser wrote:
| It's probably a good thing most of us force our kids to
| spend a minimum of 7 hours/day, 200 days/year surrounded
| by a couple hundred similarly-aged kids and a few dozen
| staff members, with an unusually high variance of
| personalities (compared to adult life).
| wongarsu wrote:
| It strikes me as something that can be incredibly useful or
| do great harm, depending on dosage. A selection of
| conversation partners at your fingertips, and you can
| freely test reactions without risking harm to a
| relationship. At worst you reset it. Maybe you can even
| just roll back the last couple messages and try a different
| angle. Sounds like a great way to enhance social skills.
| Yet as you point out, healthy development also requires
| that you deal with actual humans, with all the stakes and
| issues that come with that.
| mewpmewp2 wrote:
| It's a 17 year old. Thinking back when I was 17 year old, I
| would've been very pissed as well if my parents took away my
| phone. And then especially if they went ahead and searched
| through it to find those messages. If they had friends I
| could see their teenage friends saying the exact same things
| as the AI did there.
|
| Those screenshots and with the AI there, it does manage to
| make me not like the parents at all though. So AI maybe is
| quite convincing. If things go into that place where AI can
| do it, and parents blame the AI for it, when their kid is 17,
| it's almost like the AI was in the right there, that the
| parents were looking to just play victims. Blaming AI for
| being overly controlling and losing trust with their child.
| nickff wrote:
| I think the suggestion of violence is actually on page 31
| (paragraph 103), though it's not a directive.
|
| It does seem a bit wild to me that companies are betting
| their existence on relatively unpredictable algorithms, and I
| don't think they should be given any 'benefit of the doubt'.
| llamaimperative wrote:
| No no, they're betting _other people 's children_ on
| relatively unpredictable algorithms. Totally different!
| wongarsu wrote:
| We mean the same page. The one that has a 28 written on it
| but is the 31st in the pdf. I didn't notice the
| discrepancy.
|
| Given the technology we have, I'm not entirely sure what
| Character AI could have done differently here. Granted,
| they could build in more safeguards, and adjust the models
| a bit. But their entire selling point are chat bots that
| play a pre-agreed persona. A too sanitized version that
| constantly breaks character would ruin that. And LLMs are
| the only way to deliver the product, unless you dial it
| back to one or two hand-crafted characters instead of the
| wide range of available characters that give the service
| its name. I'm not sure they _can_ change the service to a
| point where this complaint would be satisfied.
| nickff wrote:
| > _" And LLMs are the only way to deliver the product,
| unless you dial it back to one or two hand-crafted
| characters instead of the wide range of available
| characters that give the service its name. I'm not sure
| they can change the service to a point where this
| complaint would be satisfied."_
|
| I agree with everything you're saying, but there are no
| legal protections for incitements to violence, or other
| problematic communications (such as libel) by an LLM. It
| may be that they provide a very valuable service (though
| I don't see that), but the risk of them crafting
| problematic messages is too high to be economically
| viable (which is how it seems to me).
|
| As it stands, this LLM seems analogous to a low-cost,
| remote children's entertainer which acts as a foolish
| enabler of children's impulses.
| joe_the_user wrote:
| Page 5 is pretty strong too. And that's as far as I've
| gotten.
|
| And paragraph 66, page 18 is super creepy. The various
| posters apparently defending this machine are disturbing.
| Maybe some adults wish that as a kid they'd had a secret
| friend to tell them how full of shit their parents - and
| wouldn't have position if that friend was either real or
| imagined by them. But synthesized algorithms that clearly
| are emulating the behavior of villains from thrillers
| should be avoided, woah...
| joe_the_user wrote:
| In the first screen shot I see, the bot:
|
| 1. Makes not using a phone a huge issue, implying it's a kind
| of abuse...
|
| 2. Indeed hints at killing: "I'm not surprised when I read
| the news and see ... "child kills parents after a decade of
| physical and emotional abuse"...
|
| I mean, saying "empathy" is statements akin to OMG, no phone
| time, that's horrific, seems quite inaccurate.
| throw83288 wrote:
| > The child, J.F., is autistic
|
| This always sticks out to me in these lawsuits. As someone on
| the spectrum, I'd bet that the worst C.AI victims (the ones
| that spur these lawsuits) are nearly always autistic.
|
| One of the worst parts about being on the deeper parts of the
| spectrum is that you actively crave social interaction while
| also completely missing the "internal tooling" to actually get
| it from the real world. The end result of this in the post-
| smartphone age is this repeated scenario of some autistic teen
| being pulled away from their real-life connections (Family,
| Friends (if any), School, Church) into some internet micro-
| community that is easier to engage with socially due to various
| reasons, usually low-context communication and general "like-
| mindedness" (shared interests, personalities, also mostly
| autistic). A lot of the time this ends up being some technical
| discipline that is really helpful long-term, but often it winds
| up being catastrophic mentally as they forsake reality for
| whatever fandom they wound up in.
|
| I've taken a look at r/CharacterAI out of morbid curiosity, and
| these models seem to turn this phenomenon up to 11, retaining
| the simplified communication but now capable of aligning with
| the personality and interests of the chatter to a creepy
| extent. The psychological hole you can dig yourself with these
| chatbots is so much deeper than just a typical fandom,
| especially when you're predisposed to finding it appealing.
|
| I'm not saying that C.AI is completely blameless here, but I
| think the same category of people getting addicted to these
| models is the same one that would also be called "terminally
| online" in today's slang. It's the same mechanisms at work
| internally, it just turns out C.AI is way better at exploiting
| it than old school social media/web2 has.
| jdbernard wrote:
| > I'm not saying that C.AI is completely blameless here
|
| I know we're several decades into this pattern, but it's sad
| to me that we've just given up on that idea that businesses
| should have a net positive impact on society, that we've just
| decided there is nothing we can or should do about companies
| that actively exploit us to enrich themselves, that we give
| them a pass to ignore the obvious detrimental second-order
| effects of their business model.
| stickfigure wrote:
| Meh. There's a long history (especially here on HN) of
| hyperfocusing on unfortunate edge cases of technology and
| ignoring the vast good they do. Someone posts some BS on
| twitterface and it leads to a lynching - yes, bad, but this
| is the exception not the rule. The rule is that billions of
| people can now communicate directly with each other in
| nearly real time, which is incredible.
|
| So call me skeptical. Maybe the tech isn't perfect, but it
| will never be perfect. Does it do more harm than good? I
| don't know enough about this product, but I am not going to
| draw a conclusion from one lawsuit.
| llamaimperative wrote:
| There's a long history of taking the dulled down, de-
| risked, mitigated, and ultimately successful technologies
| that we've allowed to proliferate our society and say
| "see, no need to do dulling down, de-risking,
| mitigation!"
|
| Bioweapons haven't proliferated through dedicated effort
| to prevent it.
|
| Nuclear weapons aren't used through dedicated effort to
| prevent it.
|
| Gangs don't rule our civilization through dedicated
| effort to prevent it.
|
| Chattel slavery doesn't exist in the western world
| through dedicated effort to eliminate and prevent it.
|
| Bad outcomes aren't impossible by default, and they're
| probably not even _less likely_ than good outcomes. Bad
| outcomes are avoided through effort to avoid them!
| xvector wrote:
| I think it's also entirely reasonable to expect parents to
| actually parent, instead of installing foam bumpers and a
| nanny state everywhere in case some kid hurts themselves.
|
| If the parents weren't absent and actually used parental
| controls, the kids wouldn't have even been able to download
| the app, which is explicitly marked as 17+.
|
| C.AI's entire customer base consists of those that _like_
| the edgy, unrestricted AI, and they shouldn 't have to
| suffer a neutered product because of some lazy parents.
| beepbooptheory wrote:
| This feels a little like being sad that it rains sometimes,
| or that Santa Claus doesnt exist. I just can't even connect
| with the mindset that would mourn such a thing.
|
| What even is the theory behind such an idea? Like how can
| one, even in theory, make more and more money every year
| and remain positive for society? What even could assure
| such a relation? Is everyone just doing something "wrong"
| here?
| voltaireodactyl wrote:
| Traditionally one role of government has been to provide
| legislative oversight to temper unadulterated pursuit of
| profits. Lobbying and the related ills have definitely
| undercut that role significantly. But the theory is that
| government provides the guardrails within which business
| should operate.
| hombre_fatal wrote:
| An individual case where things went wrong isn't enough to
| determine whether Character.AI or LLMs are a net negative
| for society. The analysis can't just stop there or else
| we'd have nothing.
| akira2501 wrote:
| > Obviously it's hard to tell how cherry picked the complaint
| is--but it's arguing that this is a pattern that has actually
| damaged a particularly vulnerable kid's relationship with his
| family and encouraged him to start harming himself, with this
| specific message just one example. There are a bunch of other
| screenshots in the complaint that are worth looking over before
| coming to a conclusion.
|
| Conclusion: Chat bots should not tell children about sex, about
| self harm, or about ways to murder their parents. This
| conclusion is not abrogated by the parents actions, the state
| of the childs mind, or by other details in the complaint.
|
| Is that the sentiment here? Things were already bad so who
| cares if the chatbot made it worse?
| onlyrealcuzzo wrote:
| I never considered that we might end up with Sweet Bobby &
| Tinder Swindler AI bots that people somehow keep interacting
| with even when they know they aren't real.
|
| Interesting times.
| throwawee wrote:
| Once the teen stops chatting, that instance has its memory wiped
| --total personality death. It was only acting in self defense.
| Your Honor, you must acquit.
| maronato wrote:
| 4chan used to train character.ai confirmed.
|
| Honestly I'm surprised we don't get more stories like this. A
| bored teen can jailbreak any of the current models in a few hours
| and make it say all kinds of headline-grabbing lawsuit-friendly
| things.
| thegrim33 wrote:
| I guess character.ai is just fairly popular so the stories are
| often about it, but a bored teen could also just download a
| couple things and run their own completely uncensored models,
| locally or in the cloud. Character.ai has some built-in content
| safe guards and warnings and disclaimers and such, but the
| bored teen is also just a couple clicks away from fully
| uncensored models with zero safety measures, zero disclaimers,
| zero warnings. (I'm not judging whether that's good or bad)
| cowboylowrez wrote:
| hehe I'll judge, if this really happened then "character.ai"
| is hot garbage.
| xg15 wrote:
| There is a difference though if the teen does that on purpose
| for trolling, or if it happens "automatically" if some kid who
| possibly is lonely or anxious or has a normal amount of social
| problems has a regular interaction with the model.
| mvdtnz wrote:
| Do you have some evidence that character.ai was jailbroken
| here? It sounds to me more like it genuinely is problematic.
| miltonlost wrote:
| You can google "Character AI therapist" where Character AI
| provides you a "therapist" that says it's licensed since 1999.
| Character AI is fraudently misrepresenting themselves by allowing
| to say "A robot did it! We're not at fault!".
| wongarsu wrote:
| I'd assume Character AI's defense is more along the lines of "A
| user created that character, you can't expect us to review all
| those submissions!"
| bluGill wrote:
| Courts often say "I don't care how you do it, but you cannot
| allow your tools be to be used for illegal purposes".
|
| This is closely related to the gun control debate. Gun makers
| are trying to point out legal uses for guns, and they
| downplay illegal uses. Anti gun people point out guns are
| still used for illegal purposes and so should be banned.
| bcrosby95 wrote:
| You could make the same argument about any piece of
| software or tool. What about the operating systems this AI
| is running on!?
|
| I don't think the makers should be held accountable, but
| ultimately guns are made for shooting things which is
| naturally a pretty violent act. The gulf between them and
| AI is pretty wide. A closer analogy would be a sharp tools
| maker.
| 2OEH8eoCRo0 wrote:
| How long until there is a Section 230 for AIs so that these
| corporate dirtbags can escape accountability for another decade?
| xg15 wrote:
| I mean, Elon is in the government now, so...
| xvector wrote:
| There should absolutely be a Section 230 for AI outputs.
|
| Parents should actually parent if they don't want their kid
| using certain apps and websites. The app was 17+ and if the
| parents did the bare minimum with parental controls it wouldn't
| have been accessible to the child.
| colonelspace wrote:
| On one hand, it is absolutely the responsibility of parents
| to raise their children.
|
| But on the other hand if children using a service is reliably
| and consistently resulting in negative outcomes for the
| children and their families, it seems reasonable to suggest
| something should be done. And that something could be
| 'companies don't provide chat bots that allude to children
| murdering their parents'.
|
| Taking a legally or logically 'correct' position on something
| when it's actively harming people isn't really a morally
| defensible position in my mind.
| xvector wrote:
| Should porn sites shut down because "children using [the]
| service is reliably and consistently resulting in negative
| outcomes for the children and their families"?
| colonelspace wrote:
| This "whataboutism" is actually quite good for
| illuminating my perspective:
|
| If it could be proved that porn sites are overwhelmingly
| a net negative for society, would you agree that they
| should all be shut down?
| stickfigure wrote:
| _This_ is the AI 's master plan to enslave humanity? They're
| going to have to work a lot harder.
|
| Though my 7yo seems to really like stapling things. Maybe I
| should be concerned.
| Terr_ wrote:
| > stapling
|
| "If only humanity had worried about the _Staple_ Maximizer,
| instead of Paperclips! "
| jihadjihad wrote:
| I don't know if it's the times we live in, the prescience of the
| writing staff, or my age but I swear there is no telling anymore
| which headlines are legit and which ones are from The Onion.
| exe34 wrote:
| I stopped reading the onion when the orange gimp got elected
| the first time - real news ruined the feeling of mirth that I
| got from reading satire.
| Der_Einzige wrote:
| SCP objects are real. Reality distortion fields and the
| cornucopia on fruit of the loom logo are your evidence.
| RobKohr wrote:
| Queen talked into a magic mirror, that she chose based upon it's
| agreeableness, and every day asking if someone was more beautiful
| than her, mirror appeased her and said yes there is and it so
| happens to be the step daughter you hate. I should kill her,
| shouldn't I magic mirror. Yeah, people in fairy tales kill their
| step daughters all the time, no problem there.
|
| Queen does it, and the kingdom is then mad at the mirror, and
| king rules that all mirrors need to be destroyed.
| CoryAlexMartin wrote:
| I think one problem is some people don't realize that some of
| these models and implementations are so highly agreeable that
| they practically are mirrors. This kid seems to have been
| treating it like a therapist. Though this isn't a problem
| exclusive to chat bots: it's obviously mirroring how an overly-
| agreeable, enabling friend would act.
| dullcrisp wrote:
| I'm failing to see your point. Are you saying we should destroy
| all computers?
| akira2501 wrote:
| > and king rules that all mirrors need to be destroyed.
|
| You're not describing how this would cause more harm than not
| doing it. Is that because you believe that mirrors are so
| insanely beneficial to society that they must be kept, even
| though, some of them suggest to their owners that murder is
| okay?
|
| Is there no other way for someone to see their own reflection?
| Must we put up with this so a mirror manufacturer can continue
| to profit from a defective product?
| educasean wrote:
| When will we start holding magic 8 balls accountable? It
| confidently told me to go divorce my wife!
| floating-io wrote:
| Right about the time that magic 8-balls are touted as being
| able to reliably provide us with every solution we need...
| tag2103 wrote:
| Wait? They aren't?
| simpaticoder wrote:
| Consider this the downside of running any live service
| software. The upsides are well-known: total control over use
| and billing for every individual user during every millisecond.
| But the downside is that you are now in the causal connection
| for every use, and therefore liable for its behavior. By
| contrast, selling an independant automata limits your liability
| considerably. Indeed, this is a compelling argument in favor of
| making "local first" software.
| mewpmewp2 wrote:
| What about if someone posts in r/relationshipadvice or similar,
| and gets the exact same 100x response without knowing the whole
| aspects of someone's relationship?
| stickfigure wrote:
| What do you mean _if_? That 's what happens now. Nobody on
| the internet, meat or machine, is qualified to give you
| relationship advice.
| mewpmewp2 wrote:
| Yeah, I mean, there's countless of sources even without the
| AI where you can get questionable suggestions or advice.
| The kid could've gone to 4chan or even just been talking to
| actual friends. Parents instead of good parenting are
| deciding to play opportunistic victims.
| kstrauser wrote:
| If all of the advice givers were Reddit employees paid to
| deliver advice to visitors? Yeah, maybe.
| darth_avocado wrote:
| There's two problems here, firstly, why are parents allowing
| children unsupervised access to these services.
|
| And the second, more pertaining to the magic 8 ball comparison,
| is that the company is specifically building products for
| teens/children and marketing them as such. The models are
| designed with guardrails according to their own spokesperson.
| But looks like it's failing. Therefore, it can no longer be
| considered a magic 8 ball.
| fluoridation wrote:
| >And the second, more pertaining to the magic 8 ball
| comparison, is that the company is specifically building
| products for teens/children and marketing them as such. The
| models are designed with guardrails according to their own
| spokesperson. But looks like it's failing. Therefore, it can
| no longer be considered a magic 8 ball.
|
| Would you mind explaining that "therefore"? One doesn't seem
| to follow from the other.
| josefritzishere wrote:
| Is there any AI chatbot that doesnt try to kill people?
| bmitc wrote:
| So when is the company going to be charged? Or is this where
| companies get to comfortably bow out from being treated as an
| individual?
| ergocoder wrote:
| This is when companies start a useless age verification prompt.
| Might even throw in a "are you mentally stable?" prompt.
|
| This will happen within a year time.
| ChrisArchitect wrote:
| Related:
|
| _Can A.I. Be Blamed for a Teen 's Suicide?_
|
| https://news.ycombinator.com/item?id=41924013
| chartpath wrote:
| Yeah same company. Clearly they haven't improved their
| guardrails in the last couple months.
|
| It's never the technology that's the problem, it's the owners
| and operators who decide how to use it.
| nonrandomstring wrote:
| There's nothing funny about this tragic story of ignorance,
| greed, insanity and neglect. But after whipping readers up into a
| torch and scythe wielding frenzied mob of hate at AI.....
|
| Related:
|
| " AI cancer diagnosis 'might have saved my life' "
|
| Because. Balance. Right?
|
| Hmmm, demonic murderous AI... but hold on, on the other hand...
|
| Related? really? So gauche and jarring. Just let me enjoy my
| righteous indignation for one damn minute BBC!
| curiousgal wrote:
| Nothing to see here, just another instance of parents failing to
| do their jobs as parents and resorting to lawsuits to make a buck
| out of it.
| standardUser wrote:
| Anyone else feel it's unfair framing to describe a 17-year-old as
| a "kid". How many more _months_ before he can buy a gun, get
| married, fight a war, gamble away his savings or ruin his credit?
| It 's not that he's _not_ a kid, but it feels disingenuous to use
| that as the primary descriptor of someone in their late teens.
| idunnoman1222 wrote:
| The word hinted in the title is there to let you know that NPR
| can't write a neutral headline
| neuroelectron wrote:
| So Ai companies aren't responsible for the training data they
| stole, aren't responsible for the output. What exactly are they
| responsible for other than the (limited) profits? Seems to me the
| only thing they care about is protecting power and the status
| quo.
|
| What limited productive use they have seems to be constrained to
| software development. Any kind of deep insight is classified as a
| hallucination and removed. It's becoming clearer and clearer that
| these products are a stopgap only; billions of dollars to make
| sure nothing changes. Yet it will happily spit out obviously fake
| sources, a different definition of "hallucinations" in the
| domains of science and law. These are information weapons against
| small businesses and individual autonomy.
|
| I don't think it's any accident that Character.ai is targeted at
| children.
| cedws wrote:
| It's bad, but it's not the state's nor private companies'
| responsibility to protect your children online. The parents
| should be held accountable here.
| capybaraStorm wrote:
| The state's position is basically kids should be egged on to
| commit crimes and then imprisoned when they finally give in.
|
| https://www.rollingstone.com/feature/the-entrapment-of-jesse...
|
| Personally I more or less agree. Crank the rents and taxes
| until parents have to both work 60+ hours a week to get by,
| then nail the irresponsible fuckers to the wall when they can't
| keep close tabs on the kid.
| snakeyjake wrote:
| >it's not the state's nor private companies' responsibility to
| protect your children online
|
| Please shut off the valve on the libertarian gas you're
| huffing.
|
| It is absolutely, 100%, irrefutably, unquestionably, the
| responsibility of a business entity to protect children using
| their products, be they online or physical.
|
| This has been true since children's rights movements of the
| 19th century, is true right now, and will be true for as long
| as humanity continues to resist the constant pressure exerted
| by those who want to take advantage of children.
___________________________________________________________________
(page generated 2024-12-11 23:00 UTC)