[HN Gopher] What OpenAI did when ChatGPT users lost touch with r...
___________________________________________________________________
What OpenAI did when ChatGPT users lost touch with reality
Author : nonprofiteer
Score : 51 points
Date : 2025-11-24 05:58 UTC (17 hours ago)
(HTM) web link (www.nytimes.com)
(TXT) w3m dump (www.nytimes.com)
| cc62cf4a4f20 wrote:
| https://archive.is/v4dPa
| ArcHound wrote:
| One of the more disturbing things I read this year was the my
| boyfriend is AI subreddit.
|
| I genuinely can't fathom what is going on there. Seems so wrong,
| yet no one there seems to care.
|
| I worry about the damage caused by these things on distressed
| people. What can be done?
| 2OEH8eoCRo0 wrote:
| That subreddit is disturbing
| j-pb wrote:
| After having spoken with one of the people there I'm a lot less
| concerned to be honest.
|
| They described it as something akin to an emotional vibrator,
| that they didn't attribute any sentience to, and that didn't
| trigger their PTSD that they normally experienced when dating
| men.
|
| If AI can provide emotional support and an outlet for survivors
| who would otherwise not be able to have that kind of emotional
| need fulfilled, then I don't see any issue.
| ArcHound wrote:
| phew, that's a healthy start.
|
| I am still slightly worried about accepting emotional support
| from a bot. I don't know if that slope is slippery enough to
| end in some permanent damage to my relationships and I am
| honestly not willing to try it at all even.
|
| That being said, I am fairly healthy in this regard. I can't
| imagine how it would go for other people with serious
| problems.
| j-pb wrote:
| I completely agree that it is certainly something to be
| mindful of. It's just that found the people from there were
| a lot less delusional than the people from e.g.
| r/artificialsentience, which always believed that AI Moses
| was giving them some kind of tech revelation though magical
| alchemical AI symbols.
| netsharc wrote:
| A friend broke up with her partner. She said she was using
| ChatGPT as a therapist. She showed me a screenshot, ChatGPT
| wrote "Oh [name], I can feel how raw the pain is!".
|
| WTF, no you don't bot, you're a hunk of metal!
| darepublic wrote:
| I got a similar synthetic heartfelt response about losing
| some locally saved files without backup
| avensec wrote:
| It may not be a concern now, but it comes down to their level
| of maintaining critical thinking. The risk of epistemic
| drift, when you have a system that is designed (or
| reinforced) to empathize with you, can create long-term
| effects not noticed in any single interaction.
|
| Related: "Delusions by design? How everyday AIs might be
| fuelling psychosis (and what can be done about it)" (
| https://doi.org/10.31234/osf.io/cmy7n_v5 )
| j-pb wrote:
| I don't disagree that AI psychosis is real, I've met people
| who believed that they were going to publish at Neurips due
| to the nonsense ChatGPT told them, that believed that the
| UI mockup that claude gave then were actually producing
| insights into it's inner workings instead of just being
| blinking SVGs, and I even encountered someone participating
| at a startup event with an Idea that I'm 100% is AI slop.
|
| My point was just that the interaction I had from
| r/myboyfriendisai wans't one of those delusional ones. For
| that I would take r/artificialsentience as a much better
| example. That place is absolutely nuts.
| ArcHound wrote:
| Dear god, there's more! I'll need a drink for this one.
|
| However, I suspect I have better resistance to schizo
| posts than emotionally weird posts.
| notpachet wrote:
| Wouldn't there necessarily be correlative effects in
| professional settings a la programming?
| butlike wrote:
| Acceptance of vibe coding prompt-response answers from
| chatbots without understanding the underlying mechanisms
| comes to mind as akin to accepting the advice of a
| chatbot therapist without critically thinking about the
| response.
| DuperPower wrote:
| typical anglosaxon thinking of putting short term happyness
| above all even if It implies self delusion. You guys never
| see the Risk of self delusion until something happens
| averageRoyalty wrote:
| You're being weird and racist, please stop.
| jmcgough wrote:
| Most people who develop AI psychosis have a period of healthy
| use beforehand. It becomes very dangerous when a person
| decreases their time with their real friends to spend more
| time with the chatbot, as you have no one to keep you in
| check with what reality is and it can create a feedback loop.
| cactusplant7374 wrote:
| NYT did a story on that as well and interviewed a few people.
| Maybe the scary part is that it isn't who you think it would be
| and it also shows how attractive an alternative reality is to
| many people. What does that say about our society.
| belval wrote:
| > I worry about the damage caused by these things on distressed
| people. What can be done?
|
| Why? We are gregarious animals, we need social connections.
| ChatGPT has guardrails that keep this mostly safe and helps
| with the loneliness epidemic.
|
| It's not like people doing this are likely thriving socially in
| the first place, better with ChatGPT than on some forum a la
| 4chan that will radicalize them.
|
| I feel like this will be one of the "breaks" between
| generations where millennial and GenZ will be purist and call
| human-to-human real connections while anything with "AI" is
| inherently fake and unhealthy whereas Alpha and Beta will treat
| it as a normal part of their lives.
| MengerSponge wrote:
| load-bearing "mostly"
|
| https://www.cnn.com/2025/11/06/us/openai-chatgpt-suicide-
| law...
| fullshark wrote:
| The tech industry's capacity to rationalize anything,
| including psychosis, as long as it can make money off it is
| truly incredible. Even the temporarily embarrassed founders
| that populate this message board do it openly.
| rustystump wrote:
| Social media aka digital smoking. Facebook lying about
| measurable effects. No gen divide same game different
| flavor. Greed is good as they say. /s
| venturecruelty wrote:
| We need a Truth and Reconciliation Commission for all of
| this someday, and a lot of people will need to be behind
| bars, if there be any healing to be done.
| belval wrote:
| > Even the temporarily embarrassed founders that populate
| this message board do it openly.
|
| Not a wannabe founder, I don't even use LLMs aside from
| Cursor. It's a bit disheartening that instead of trying to
| engage at all with a thought provoking idea you went
| straight for the ad hominem.
|
| There is plenty to disagree with, plenty of counter-
| arguments to what I wrote. You could have argued that human
| connection is special or exceptional even, anything really.
| Instead I get "temporarily embarrassed founders".
|
| Whether you accept it or not, the phenomenon of using LLMs
| as a friend is getting common because they are good enough
| for human to get attached to. Dismissing it as psychosis is
| reductive.
| jmcgough wrote:
| https://en.wikipedia.org/wiki/Deaths_linked_to_chatbots
|
| If you read through that list and dismiss it as people who
| were already mentally ill or more susceptible to this...
| that's what Dr. K (psychiatrist) assumed too until he looked
| at some recent studies:
| https://youtu.be/MW6FMgOzklw?si=JgpqLzMeaBLGuAAE
|
| Clickbait title, but well researched and explained.
| isoprophlex wrote:
| My dude/entity, before there were these LLM hookups, there
| existed the Snapewives. People wanna go crazy, they will, LLMs
| or not.
|
| https://www.mdpi.com/2077-1444/5/1/219
|
| _This paper explores a small community of Snape fans who have
| gone beyond a narrative retelling of the character as
| constrained by the work of Joanne Katherine Rowling. The
| 'Snapewives' or 'Snapists' are women who channel Snape, are
| engaged in romantic relationships with him, and see him as a
| vital guide for their daily lives. In this context, Snape is
| viewed as more than a mere fictional creation._
| quitit wrote:
| There are plenty of reasons why having a chatbot partner is a
| bad idea (especially for young people), but here's just a few:
|
| - The sycophantic and unchallenging behaviours of chatbots
| leaves a person unconditioned for human interactions. Real
| relationships have friction, from this we develop important
| interpersonal skills such as setting boundaries, settling
| disagreements, building compromise, standing up for oneself,
| understanding one another, and so on. These also have an effect
| on one's personal identity and self-value.
|
| - Real relationships have the input from each participant,
| whereas chatbots are responding to the user's contribution
| only. The chatbot doesn't have its own life experiences and
| happenings to bring to the relationship, nor does it instigate
| autonomously, it's always some kind of structured reply to the
| user.
|
| - The implication of being fully satisfied by a chatbot is that
| the person is seeking a partner who does not contribute to the
| relationship, but rather just an entity that only acts in
| response to them. It can also be an indication of some kind of
| problem that the individual needs to work through with why they
| don't want to seek genuine human connection.
| nostrademons wrote:
| These are only problems if you assume the person later wants
| to come back to having human relationships. If you assume AI
| relationships are the new normal and the future looks kinda
| like The Matrix, with each person having their own
| constructed version of reality while their life-force is bled
| dry by some superintelligent machine, then it is all working
| as designed.
| AI_rumination wrote:
| Love your thoughts about needing input from others! In
| Autistic / ADHD circles, the lack of input from other people,
| and the feedback of thoughts being amplified by oneself is
| called rumination. It can happen for many multiple ways--
| lack of social discussion, drugs, etc. AI psychosis is just
| rumination, but the bot expands and validates your own ideas,
| making them appear to be validated by others. For vulnerable
| people, AI can be incredibly useful, but also dangerous. It
| requires individuals to deliberately self-regulate, pause,
| and break the cycle of rumination.
| josh-sematic wrote:
| I hadn't heard of that until today. Wild, it seems some people
| report genuinely feeling deeply in love with the personas
| they've crafted for their chatbots. It seems like an incredibly
| precarious position to be in to have a deep relationship where
| you have to perpetually pay a 3rd party company to keep it
| going, and the company may destroy your "partner" or change
| their personality at a whim. Very "Black Mirror".
| jmcgough wrote:
| There were a lot of that type who were upset when chatGPT was
| changed to be less personable and sycophantic. Like, openly
| grieving upset.
| venturecruelty wrote:
| What's going on is that we've spent a few solid decades
| absolutely destroying normal human relationships, mostly
| because it's profitable to do so, and the people running the
| show have displayed no signs of stopping. Meanwhile, the rest
| of society is either unwilling or unable (or both) to do
| anything to reverse course. There is truly no other outcome,
| and it will not change unless and until regular people decide
| that enough is enough.
|
| I'd tell you exactly what we need to do, but it is at odds with
| the interests of capital, so I guess keep showing up to work
| and smiling through that hour-long standup. You still have a
| mortgage to pay.
| blurbleblurble wrote:
| The whiplash of carefully filtering out sycophantic behavior from
| GPT-5 to adding it back in full force for GPT-5.1 is dystopian.
| We all know what's going on behind the scenes:
|
| The investors want their money.
| ACCount37 wrote:
| OpenAI fought 4o, and 4o won.
|
| By now, I'm willing to pay extra to avoid OpenAI's atrocious
| personality tuning and their inane "safety" filters.
| sunaookami wrote:
| GPT-5 was so good in the first week, just a raw chatbot like
| GPT-3.5 and GPT-4 were in the beginning and now it has this
| disgusting "happy" and "comforting" personality and "tuning" it
| doesn't help one bit, it makes performance way worse and after
| a few rounds it forgets all instructions. I've already deleted
| memory, past chats, etc...
| stavros wrote:
| Even when you tell it to not coddle you, it just says
| something cringeworthy like "ok, the gloves are off here's
| the raw deal, with New Yorker honesty:" and proceeds to feed
| you a ton of patronizing bullshit. It's extremely annoying.
| helpfulclippy wrote:
| I've had some limited success attributing ideas to other
| people and asking it to help me assess the quality of the
| idea. Only limited success though. It's still a fucking
| LLM.
| stavros wrote:
| The issue is not that it's an LLM, the issue is that it's
| been RLHFed to hell to be a sycophant.
| venturecruelty wrote:
| Yeah, this is why a lot of us don't use these tools.
| stavros wrote:
| Yeah but baby, bathwater, throw.
| venturecruelty wrote:
| Remarkable that you're being downvoted on a venture capital
| forum whose entire purpose is "take venture capital and then
| eventually pay it back because that's how venture capital
| works".
| Peritract wrote:
| "Profited".
| leoh wrote:
| Anthropic was founded by exiles of OpenAI's safety team, who quit
| en masse about 5 years ago. Then a few years later, the board
| tried to fire Altman. When will folks stop trusting OpenAI?
| kotaKat wrote:
| When the justice system finally catches up and puts Sam behind
| bars.
| JumpCrisscross wrote:
| > _When the justice system finally catches up and puts Sam
| behind bars_
|
| Sam bears massive personal liability, in my opinion. But
| criminal? What crimes has he committed?
| venturecruelty wrote:
| I'm sure we could invent one that sufficiently covers the
| insane sociopathy that rots the upper echelons of corporate
| technology. Society needs to hold these people accountable.
| If the current legal system is not adequate, we can repair
| it until it is.
| chris-vls wrote:
| It seems quite probable that an LLM provider will lose a major
| liability lawsuit. "Is this product ready for release?" is a very
| hard question. And it is one of the most important ones to get
| right.
|
| Different providers have delivered different levels of safety.
| This will make it easier to prove that the less-safe provider
| chose to ship a more dangerous product -- and that we could
| reasonably expect them to take more care.
|
| Interestingly, a lot of liability law dates back to the railroad
| era. Another time that it took courts to rein in incredibly
| politically powerful companies deploying a new technology on a
| vast scale.
| JumpCrisscross wrote:
| > _a lot of liability law dates back to the railroad era.
| Another time that it took courts to rein in incredibly
| politically powerful companies deploying a new technology on a
| vast scale_
|
| Do you have a layman-accessible history of this? (Ideally an
| essay.)
| thot_experiment wrote:
| Caelan Conrad made a few videos on specifically AI encouraging
| kids to socially isolate and commit suicide. In the videos he
| reads the final messages aloud for multiple cases, if this isn't
| your cup of tea there's also the court cases if you would prefer
| to read the chat logs. It's very harrowing stuff. I'm not trying
| to make any explicit point here as I haven't really processed
| this fully enough to have one, but I encourage anyone working in
| this space to hold this shit in their head at the very least.
|
| https://www.youtube.com/watch?v=hNBoULJkxoU
|
| https://www.youtube.com/watch?v=JXRmGxudOC0
|
| https://www.youtube.com/watch?v=RcImUT-9tb4
| venturecruelty wrote:
| "Sure, this software induces psychosis and uses a trillion
| gallons of water and all the electricity of Europe, and also it
| gives wrong answers most of the time, but if you ignore all that,
| it's really quite amazing."
| throwaway48476 wrote:
| It would be helpful to tell users that it's just a model
| producing mathematically probable tokens but that would go
| against the AI marketing.
| jsheard wrote:
| > The chatbot now encourages users to take breaks during a long
| session.
|
| However, they've employed the Microsoft School Of Dark Patterns
| when implementing it.
|
| https://bsky.app/profile/kashhill.bsky.social/post/3m6exw5lx...
___________________________________________________________________
(page generated 2025-11-24 23:00 UTC)