[HN Gopher] A failed AI girlfriend product, and my lessons
___________________________________________________________________
A failed AI girlfriend product, and my lessons
Author : mazzystar
Score : 166 points
Date : 2023-11-16 09:17 UTC (13 hours ago)
(HTM) web link (mazzzystar.github.io)
(TXT) w3m dump (mazzzystar.github.io)
| vouaobrasil wrote:
| You should not even try to create these sorts of technologies.
| Take a break from technology, spend time outdoors with plants and
| animals. These types of technologies will only lead to a
| worsening of this already sick society.
| HPsquared wrote:
| Could say the same about antidepressants. "Fix the causes, not
| the symptoms!" And yet, here we are.
| vouaobrasil wrote:
| I agree, 100%. Here we are indeed. But that doesn't mean we
| should not try and change things. One step at a time, right?
| toomuchtodo wrote:
| We are here because of free will. What steps would you
| take? Convince people who have opted out of the dating pool
| to opt back in? Convince women who would rather be single
| that their potential partner expectations are unrealistic
| (even if they genuinely are)?
|
| https://www.mintel.com/press-centre/all-the-single-
| ladies-61...
|
| https://www.psychologytoday.com/us/blog/why-bad-looks-
| good/2...
|
| https://www.nytimes.com/2023/11/11/opinion/marriage-women-
| me...
|
| https://freddiedeboer.substack.com/p/the-demographic-
| dating-...
|
| https://link.springer.com/article/10.1007/s12110-022-09422-
| 2
|
| You can patch over the outcomes but there is no going back
| to the before times. Dating app dates and relationships for
| some, AI companions for others, single life for the rest.
| It's not great, it just is.
| mejutoco wrote:
| Saying that something exists is not an argument about why
| something _should_ not exist.
|
| But I agree, "should" discussions are rarely the most
| productive. Shoulda, woulda, coulda, I guess :)
| throw18376 wrote:
| antidepressants can help people fix root causes. take them
| for 6 months to 2 years or so, break out of a bad cycle, get
| some space without depression symptoms to build up healthy
| habits and restore a social life.
|
| there are some bad profit motives to keep people on SSRIs
| indefinitely. however at least they don't compete with other
| aspects of your life. and now that they are dirt cheap (at
| cheapest, $4 for a month supply at Walmart) the profit
| motives are much less than before.
|
| hard to imagine an "AI girlfriend" that could be similarly
| short-term and bring positive changes. it's just dystopian.
| the most profitable way to run such a product is not to help,
| it's to eat up as much user time as possible and outcompete
| healthy activities and real human relationships.
| HPsquared wrote:
| Ok, maybe not the best analogy... Painkillers then? Other
| forms of entertainment in general? It's the relationship
| version of playing videogames instead of getting a job.
| dspillett wrote:
| _> antidepressants can help people fix root causes_
|
| Nit-picking rather than disagreeing, but feel it worth
| making more obvious in the wording: antidepressants can
| help people _deal with the symptoms_ while they find /fix
| root causes. Being on them for an extended time can bring
| its own problems.
| esclerofilo wrote:
| > hard to imagine an "AI girlfriend" that could be
| similarly short-term and bring positive changes.
|
| Feeling lonely has negative consequences, there's a famous
| study comparing it to smoking cigarettes. Lonely people can
| get fixated on that feeling. Mitigating it may help people
| look for other hobbies, that they kept putting off because
| they were so focused on their need of a girlfriend.
|
| Hell, a realistic enough "woman AI" (not "girlfriend AI")
| could make you lose the fear of flirting, be useful as
| practice, to have more confidence when you have a real
| date.
|
| > the most profitable way to run such a product is not to
| help, it's to eat up as much user time as possible
|
| If you charge per-month (like OP), it's more profitable is
| to eat up less time.
| phatskat wrote:
| > "Fix the causes"
|
| How, exactly? Yes, some people experience depression as a
| symptom of something than can be addressed through various
| therapies, but what about people who just have it wired into
| their brains? How do we "fix" that?
| vouaobrasil wrote:
| Even if it is wired into certain brains, that wiring may
| only cause depression when it interacts with our modern
| technological society.
| phatskat wrote:
| I don't think that's how that works. If your depression
| "goes away" when you stop using Instagram, that's not
| what I'm talking about. I'm talking about that "wake up
| on a warm sunny day in a house with your loving family
| and feel like jumping off the roof" kind of depression.
| That "everything can be going great and you can't get out
| of bed" depression. The kind of depression that isn't
| "cured" by turning off your phone or going off the grid.
|
| And it certainly isn't "cured" by antidepressants,
| however, they can make a huge impact on quality of life
| for many people.
| piva00 wrote:
| Have you or a close loved one ever had clinical
| depression? Diagnosed by a professional which required
| help? It's a very strange take if you ever had any
| personal and direct contact with it, it's not only caused
| by "modern technological society" even though it's pretty
| aggravated by it.
| InfiniteRand wrote:
| I think it's helpful to look at depression as a group of
| diseases like cancer. There are some cases of depression
| where we know the root cause: thyroid issues, sleep apnea-
| related depression, low testosterone related depression,
| brain tumors, etc. In those cases, addressing the root cause
| is probably the best treatment method. This is similar in
| some ways to cancer, where certain cancers have very definite
| causes and can be treated by dealing directly with those
| causes (like cervical cancer).
|
| Then there are all the other cases, some of which likely have
| simple causes that are just unknown, some of which have
| complex causes that are intermeshed deeply with the biology
| of human thinking. In that case, you have blunt tools, like
| anti-depressants, to address the pathway of the disease
| without knowing the exact cause, much like chemo-therapy can
| be used to attack cancer cells without knowing what prompted
| the cancer originally.
|
| So in short, yes treat the causes, but when you don't know
| the cause it's not irrational to treat the symptoms as best
| you can
| krzat wrote:
| If the AI had a secondary goal of improving customer's mental
| health, it could work very well. Usually you can't fuck your
| therapist.
| vouaobrasil wrote:
| I think in theory, you are right. However, because AI is so
| easily accessible and it is being developed in our current,
| highly commoditized society, there is really no change that
| it will ever only be used for good in this way.
| chpatrick wrote:
| You can't really fuck your imaginary shoggoth girlfriend
| either.
| dkjaudyeqooe wrote:
| You can if you have the appropriate peripheral(s) attached.
| u32480932048 wrote:
| Dongles!
|
| https://www.youtube.com/watch?v=-XSC_UG5_kU
| mensetmanusman wrote:
| Yes, if you redefine the meaning of words.
| dkjaudyeqooe wrote:
| All you need is a USB orifice. How is that not a
| peripheral?
| SoftTalker wrote:
| > Usually you can't fuck your therapist.
|
| Isn't that what prostitution is though?
| Der_Einzige wrote:
| For many it is, but it's very illegal in the USA. I
| actually expect that a real movement from women (of both
| the left and the right) which tries to criminalize sexbots,
| and AI erotica be here soon. This kind of technology is
| extremely destructive to an already low birth rate western
| world. I fully expect the USA to have birth rates closer to
| Korea within 20 years.
| bambax wrote:
| Who are we to judge the users of such a product?
|
| Some people suffer from a variety of disabilities which makes
| them unable to find friends and engage in meaningful
| relationships with others, etc.
| optimalsolver wrote:
| >spend time outdoors with plants and animals
|
| Which animals? Blood ticks?
| pixl97 wrote:
| Bot flies. Or a bunch of those things we've driven extinct
| because they tried to eat us.
|
| Naturalistic fallacies are common here :/
| icepat wrote:
| Had a similar moment with playing around using smart contracts.
| As a joke I wrote a crypto for the company I worked at, which
| then got added to employment contracts, and an internal DAO
| styled administration layout.
|
| After coding this out for a few hours I realized I had re-
| created the company scrip, except on a blockchain. I ended up
| turfing the project and spending some time thinking about how
| horrific a future technology like this could create.
| Dylan16807 wrote:
| I don't get it. Yes, you re-created scrip. That's the
| opposite of a new technology coming in and causing problems,
| isn't it? It's a known issue with a known solution.
| icepat wrote:
| Not really, I saw it as technology that was "new and
| trendy" rebranding a very old and exploitive employment
| paradigm.
|
| "You can work at a DAO and be paid on the blockchain"
|
| "You can work at a company and be paid in script"
|
| It's the same thing, but the modern technology masks what
| exactly is happening.
| Dylan16807 wrote:
| Do the relevant laws about how people get paid care about
| the difference between company scrip and company token?
| kobalsky wrote:
| I don't know why studies are even being made on the effects of
| these novel technolgies. We should just point the scientists to
| your email for answers.
| kridsdale3 wrote:
| Not a lot of Hackernews posts are so depraved that they get a
| "Jesus Christ, touch grass" response.
| Der_Einzige wrote:
| God forbid that ugly people get to have the same kinds of
| positive relationship experiences that attractive people get. I
| can't believe how the moment that a technology comes out which
| gives incels real options it's seen as being terrible for
| society.
|
| It's awesome for society.
| technics256 wrote:
| _I even tried it myself, lying in bed and creating sexual
| fantasies with Dolores through text, culminating in ejaculation._
|
| This was an excellent read. And I applaud the creator for
| experimenting with this to the fullest.
| swyx wrote:
| mustve been hard to admit publicly. i wish the author hadnt
| edited it out subsequently. just own it bro. you wanted to
| understand your customer.
| Traubenfuchs wrote:
| So, all in all, OpenAIs puritanism ruined this product, made the
| developer sad and took away a virtual porn toy from lonely, horny
| men. Who were they doing this for? Who is the winner? They lost a
| customer and made a lot of people sad.
| immibis wrote:
| You aren't the customer, you're the product.
| Traubenfuchs wrote:
| People paying for their API must be their customers, I mean,
| if there is no one paying for their API, how do they make
| money? (Besides ChatGPT+)
| ReactiveJelly wrote:
| What I'm seeing these days is that you're always the
| product, because advertisers are one more customer, and
| everyone wants more customers, and the individuals who care
| about privacy don't "weigh enough" to win tug-of-war with
| the majority of people who don't care.
| FeepingCreature wrote:
| > Who were they doing this for?
|
| VISA probably. Card processing is weirdly puritanical.
| papa-whisky wrote:
| I've read that payment processors don't like dealing with
| porn companies because of the very high rate of chargebacks
| they attract, not because the processors are Puritans...
| ReactiveJelly wrote:
| If that were the case, and it was B2B where pricing tiers
| are more acceptable, would they not simply have a "porno
| tax"?
| anthony100 wrote:
| Interesting. Not sure how this is different from Onlyfans and
| lots of money is going there.
| numpad0 wrote:
| VISA/Mastercard goes after those as well. ebooks too.
| Penalizing short people, forcing verbiage changes, etc.
| mensetmanusman wrote:
| They want chatgpt in schools to own the next gen of workers.
| ReactiveJelly wrote:
| "Hook 'em while they're young, like the tobacco industry.
| God, I wish we had their numbers." -- Cardinal Glick,
| _Dogma_, accidentally explaining Apple's outbound marketing
| model, later copied by Microsoft and Google.
| feintruled wrote:
| Well this went places. The dev really 'ate his own dogfood' so to
| speak. I thought it was going to be a despairing look at how his
| companion AI was continuously turned to sleazy uses by a pervy
| userbase but that's pretty much what he was going for it seems. I
| have to say, I hated this part: "...gained a lot of Chinese
| users. I immediately realized the political risk: users could set
| Dolores' background, personality, and attributes, which might
| lead to political issues. I quickly took Dolores off the shelves
| in China."
|
| Just pre-emptorily censors himself to appease the Chinese
| government. They've done their job well.
|
| Also it seems like he had full access to the replies via the
| voice generator, which sure isn't great from a privacy
| standpoint.
| globular-toast wrote:
| > Also it seems like he had full access to the replies via the
| voice generator, which sure isn't great from a privacy
| standpoint.
|
| Whose privacy is being violated? Dolores's?
| feintruled wrote:
| You joke, but I can forsee issues. If he has the replies to a
| particular user, and he has a means of IDing that user, he
| could cause them a lot of hurt. They are getting intensely
| sexual replies back, they aren't getting that without
| supplying sexual prompting. Could also be used to out gay
| users.
| globular-toast wrote:
| Oh, yeah, there's a huge privacy problem here and the part
| you referenced gave me pause too. But there's also the part
| where OpenAI flagged the replies as being sexual in nature.
| There are multiple third parties able to intercept these
| communications and it's not just the replies.
|
| I was being facetious, but I found it interesting to think
| that Dolores's privacy was being intruded too and that an
| intelligence like us probably wouldn't engage in such a
| conversation knowing full well it is being eavesdropped by
| the author. Unless the intelligence considers the author to
| be God, I suppose.
| Kye wrote:
| This was mostly just amusing, but it's the kind of thing
| that can happen and leak people's deepest secrets:
| https://news.ycombinator.com/item?id=38191470
|
| All it would take is one of those people genuinely putting
| in a query without noticing what's happened to put in some
| personal info.
| cnity wrote:
| Have you never listened to one half of a phone call before?
| You can glean a great deal of information about the person on
| the other end.
| butlike wrote:
| At the risk of sounding like a space case: I think we need to
| make sure we know what consciousness is before we make a
| bunch of abusable models.
|
| I, for one, wouldn't necessarily like to be a depraved-sex
| slave bot if I was able to reason about it.
| ryanackley wrote:
| To your point, if you were able to reason about it,
| couldn't you make some self-determination and decide for
| yourself or at least communicate what you wanted to your
| "slaver"?
|
| LLM's like GPT are just predicting the next token in a
| sequence of tokens. It's not magic. It's still just a
| computer program.
| ygjb wrote:
| > LLM's like GPT are just predicting the next token in a
| sequence of tokens. It's not magic. It's still just a
| computer program.
|
| Frankly, that approach is problematic, since any computer
| software that might achieve consciousness or self-
| awareness, and should be accorded rights, could have
| their rights dismissed using the argument "it's just a
| computer program".
|
| I know they are probably a long way off, but AGI, and
| other adjacent or related technologies, such as a human
| to machine copy of a consciousness are explicitly stated
| goals of both businesses and extremely wealthy tech
| oligarchs that will have fundamental rights issues
| attached. We are bad enough at recognizing those rights
| in humans, in most of the world that we shouldn't wait
| until abuses start piling up to consider them.
| ryanackley wrote:
| Your argument is the slippery slope not mine. Should we
| give Microsoft Word the right to vote since we can never
| be too sure we are infringing on a consciousness' rights?
| It is "just a computer program" after all.
|
| Presenting these things as moral conundrums makes no
| sense whatsoever at this stage in the game. Sure, let's
| make sure the rights of tech oligarchs who've achieved
| immortality 300 years in the future are protected. We
| gotta lay the groundwork in the morality group think.
| _tom_ wrote:
| You are correct.
|
| But are you sure you are anything more?
| ryanackley wrote:
| Nope but my programming was created by millions of years
| of evolution rather than intelligent design.
|
| The concept of human rights and human feelings in general
| is based on the human condition. That is we're born and
| we die and in between we need to find a way to take care
| of ourselves, create offspring etc. Our programming has
| evolved thoughts and feelings to manage these things.
|
| If at some point, there were a true AGI, why would it
| care about human society or it's own rights we assigned
| it? It wouldn't have feelings and it would essentially be
| immortal. It could just wait for us to die out (or murder
| us) then do whatever it wanted.
| scyzoryk_xyz wrote:
| Pretty much came into the comments to agree with the self-
| censorship part there.
|
| But then the author did use 'Her' as the ideal example. That's
| a pretty good indicator they didn't get the point.
| persnickety wrote:
| What's the point?
| scyzoryk_xyz wrote:
| The point of the film ,,Her"? Pretty sure you can look that
| up or watch the movie on your own.
|
| My point being, that it's like someone reading Frankenstein
| and then thinking it would be a good idea to make a monster
| of their own.
| duskwuff wrote:
| "At long last, we have created the Torment Nexus from
| classic sci-fi novel _Don 't Create The Torment
| Nexus_..."
|
| (https://twitter.com/AlexBlechman/status/1457842724128833
| 538)
| ShamelessC wrote:
| Samantha from that movie was the opposite of a monster.
| She was a metaphor for a human relationship and even
| though she betrays Theo in the end, Theo has learned "how
| to love".
|
| I didn't find this that difficult to parse.
| notpachet wrote:
| "Well did it work for those people?" "No. It never does.
| I mean, these people somehow delude themselves into
| thinking it might, but... but it might work for _us_. "
|
| https://www.youtube.com/watch?v=Po4adxJxqZk
| luke-stanley wrote:
| Their name is "Ke Fang", and they say Chinese is their first
| language...
| whywhywhywhy wrote:
| > Just pre-emptorily censors himself to appease the Chinese
| government.
|
| Every major tech company does the exact same for China, Russia
| and UAE. Business as usual
| narinxas wrote:
| every major business company has also done this for the
| USA... not just tech companies
|
| so I agree, business as usual
| telotortium wrote:
| He's Chinese - makes sense to pre-censor himself,
| unfortunately.
| FartyMcFarter wrote:
| This guy created and lived a whole Black Mirror episode by
| himself:
|
| > I even tried it myself, lying in bed and creating sexual
| fantasies with Dolores through text, culminating in ejaculation.
|
| > However, I started to feel a sense of loss: if every Dolores
| user was engaging in anonymous, NSFW role-play, what real
| significance did it hold for me? This was drifting away from the
| essence of 'Her'.
| sva_ wrote:
| Maybe he shouldn't have named it after the most hated Harry
| Potter character.
| skummetmaelk wrote:
| He was inspired by Westworld.
| amelius wrote:
| More likely:
|
| https://en.wikipedia.org/wiki/Dolores_Abernathy
| aidenn0 wrote:
| Wow, I totally missed they made a TV Show out of that Yul
| Brynner movie, which I remember most for being kind of a
| proto Jurrasic Park.
| delecti wrote:
| Or more creepily: https://en.wikipedia.org/wiki/Lolita
| muwtyhg wrote:
| Incredibly likely given this line:
|
| > On May 4, the Dolores iOS app went live, named after the
| oldest android host in the park from Westworld.
| HPsquared wrote:
| I always think of that Seinfeld episode.
| cuuupid wrote:
| Is that you Mulva?
| amelius wrote:
| Well, the meaning of the name is the plural of "sorrow" or
| "pain".
|
| So he had it coming :)
|
| https://en.wikipedia.org/wiki/Dolores_(given_name)
| nwoli wrote:
| Never get high off your own supply
| red-iron-pine wrote:
| > This was drifting away from the essence of 'Her'.
|
| "choke me with that dead cat"
|
| (note: dead cat usually means some sort of microphone cover)
| ReactiveJelly wrote:
| I think he edited that line out when it got big on HN lol. And
| clarified that she was named for Westworld, not... "She was
| Dolores on the dotted line"
| Michelangelo11 wrote:
| Yeah, I read the comments and I was like damn ... that's not
| in the linked version.
| averageRoyalty wrote:
| Yeah, I think he's edited a few things. A lot of comments
| here no longer make sense.
|
| Here's the original:
|
| https://web.archive.org/web/20231116091926/https://mazzzysta.
| ..
| Octokiddie wrote:
| Looks like that part was edited out.
| bloodyplonker22 wrote:
| It was too cringey and creepy.
| bitwize wrote:
| > However, I started to feel a sense of loss: if every Dolores
| user was engaging in anonymous, NSFW role-play, what real
| significance did it hold for me? This was drifting away from the
| essence of 'Her'.
|
| That is indeed exactly the essence of _Her_. Had the author
| watched the movie to the end?
| globular-toast wrote:
| Yeah, it's like the author read the blurb for _Her_ and didn 't
| even read a synopsis.
| theGnuMe wrote:
| Or Blade runner 2049
| laurent_du wrote:
| Eventually, a virtual girlfriend app that will put together sound
| and image will be released an empty the pockets of sad, lonely
| men on a large scale. Depressing stuff but this is the world we
| live in. Thank you to the author for this interesting read.
| louthy wrote:
| > Eventually, a virtual girlfriend app that will put together
| sound and image will be released an empty the pockets of sad,
| lonely men on a large scale.
|
| Sounds indistinguishable from an actual girlfriend ;)
| orng wrote:
| Although probably intended as a joke, this is perhaps the
| saddest comment I have read on HN.
| phatskat wrote:
| Glad to see some boomer humor on HN /s
| wruza wrote:
| You mean snatch their money out of hands of twitch streamers.
| It's not in their pockets already.
| em500 wrote:
| If the barriers to entry to the tech are low, virtual
| girlfriends should not be expensive. Probably much cheaper than
| real ones.
| xethos wrote:
| _If_ the barriers to entry are low. But I 'd imagine that
| even if they are, switching costs will be extremely high.
| stevage wrote:
| Eventually? Months away.
| phatskat wrote:
| To imagine these apps aren't already doing such is a hard sell
| imo.
|
| Full disclosure: I've paid for several AI services so far, most
| in the realm of "virtual girlfriend" experiences, and they all
| generally are more than subpar. Mind you, I have a couple of
| IRL partners. I also enjoy the act of sexual roleplaying, and
| finding reasonably expressive human partners is time-consuming.
| The "Girlfriend AI" options are pretty terrible though - I'll
| take character.ai with its heavy moderation any day.
|
| But if I'm paying to try these and explore and see what works,
| I'm certain that wallets are being drained already. Is this a
| problem? Could be, at least for some - to think otherwise feels
| disingenuous. The question will come down to how we address it:
| do we have frank conversations with people about the risks of
| devoting potential social time to AI engagement, or do people
| clamor for the AI backing tech to button up and become even
| more prudish?
| mrshadowgoose wrote:
| There are very many people (of both genders) that decide to
| stay in mediocre relationships because the alternative of
| loneliness is worse.
|
| The effects on societal structure will be profound once virtual
| partners pass a certain threshold of fidelity.
| mustacheemperor wrote:
| That would be almost exactly Replika, right? I don't think that
| was really the intent of the creators - their original vision
| seems similar to OP's - and they apparently confirmed as much
| when they restricted its use for 'erotic content' last year .
| But the screeching anguish from their community (still visible
| in the history on their subreddit) resulted in a quick pivot
| back towards NSFW capabilities. I think there was some coverage
| of it on HN at the time, because the Replika AI is frankly
| pretty primitive, but passionate users became deeply connected
| to "their" partners, and it raises questions about what's going
| to happen as these tools become more powerful.
|
| Interesting, in context to what the OP remarks about the odd
| feeling of realizing all their users were anonymously
| interacting with "Dolores" too - Replika allows the user to
| name 'their' AI, set some criteria around its personality,
| etc...and will even sell you personality traits as 'upgrades.'
|
| OP's experience seems to track rather closely with the
| trajectory of Replika, actually, which has also upgraded with
| voice synthesis, etc. The difference seems to be Replika has
| buckets of VC funding to throw at alternative models.
| iwillremember wrote:
| > I even tried it myself, lying in bed and creating sexual
| fantasies with Dolores through text, culminating in ejaculation.
|
| TMI bro!
| shaan7 wrote:
| Looks like the author removed that phrase eventually, I can't
| find it.
| mazzystar wrote:
| Yes, my post was collapsed on Hacker News (disappeared
| suddenly from #1), which led me to think about whether I
| wrote something inappropriate for the times. I don't find it
| shameful, but it might not be suitable for the majority of
| readers to view.
| barcode_feeder wrote:
| While I understand why you'd remove it, I appreciate the
| initial inclusion. It's humanizing and genuine. Cool post
| even without it!
| bradley13 wrote:
| Interesting read. Others have also created virtual friends using
| ChatGPT.
|
| What I find unsavory, is OpenAI imposing content restrictions
| like NSFW. That can easily extend to censorship of non-PC
| content, unpopular political ideas, and more.
|
| The solution will (I hope) ultimately be open-source LLMs that
| you can run locally, on your own hardware.
| bambax wrote:
| Yes. But those open-source LLMs already exist. It's surprising
| the OP didn't at least explore the possibility of using them to
| get around the limitations of "Open" AI (which sounds more and
| more like a Victorian matron).
| wruza wrote:
| Are the risks worth it for OpenAI to allow that at what is
| basically the start of their business?
| phatskat wrote:
| And this doesn't just cover sexually NSFW content - anything
| trending towards topics like violence and suicide is also off
| the table for most AI backends from what I understand.
|
| I struggle to see the benefit of this kind of censorship as
| well, it has a very "video games make people murders" vibe.
| That we're being protected, for our own good, from...horny
| robots?
| dontupvoteme wrote:
| It's largely to protect their brand from bad press (this has
| hit even Google/Microsoft in the past), but established
| finance institutions absolutely hate porn, even the "less
| shady" ones like onlyfans or pornhub have issues with
| banking.
| jstarfish wrote:
| > I struggle to see the benefit of this kind of censorship as
| well, it has a very "video games make people murders" vibe.
| That we're being protected, for our own good, from...horny
| robots?
|
| These broad strokes are always the result of poor reporting
| extrapolated to all humans. See also: escalation of porn use.
|
| In all of these "___ makes people ___," there are a minority
| of people for which it's true. The media just presents it
| backward, as though _most_ people are this way.
|
| The question then becomes, do we care. Let a maladjusted
| 12-year old boy play with a sexbot for the next few years and
| they will come to some deeply-flawed conclusions about how
| sexual interactions and healthy human relationships work. If
| he's distracted and banging away at a flashlight, that's one
| less competitor for me on the dating market.
|
| Where does that end though? What happens when he wakes up,
| realizes he's been duped, and goes looking for retribution
| against someone? It won't be PornHub; they're an ephemeral
| concept. It won't be me either; I never did him wrong or
| denied him anything. The victim is more likely to be our
| wives and daughters-- the objects of his envy.
|
| Entertainment is supposed to be a distraction-- maybe you
| drag out the sexbot after an unsuccessful night of trying to
| hook up. But that's not how these things get used. People
| clearly spend 8-12 hours at a time with these things and form
| parasocial relationships with them _instead_ of forging
| actual human connections. Most of us don 't need the
| handholding to be told this isn't going to end well, but some
| people do, and these are the ones who slip through the cracks
| and become the 40-year-old creeps following your wife to the
| car or trying to solicit your kids on Roblox.
|
| I suspect sexbots (and porn) are particularly exploitative of
| the autistic. They're an easy trap to fall into when you have
| to work 3x as hard to connect with people. You get the
| benefits of sexual release without any of the effort. There's
| no incentive to go the hard route and learn to work around
| your issues when you can just engage with a bot you can rape
| to death once offended. You don't learn any skills that
| transfer to useful human interactions this way.
| Sir_Burpalot wrote:
| 1. Some people are so far to the left side of the bell
| curve that they're going to be screwed out of a human
| relationship no matter what they do, and it's unkind to
| withhold sex bots from them just to provide increased
| motivation to the segment of the population that's still
| salvageable.
|
| 2. Society seems to have collectively decided that romantic
| intimacy, unlike food and shelter, is optional and that
| experiencing an absence of it should not entitle you to
| sympathy or support. Therefore, the kindest thing they can
| do is leave the autists alone so they can cope as best they
| can in peace.
|
| UPD:
|
| 3. Replacing productive activities with substitutes like
| video games is obviously bad both for individual survival
| and for society at large, but what are the negative effects
| of being in a relationship with a fictional partner instead
| of a real one? You could argue the AI companies would have
| an incentive to exploit their users, but this is a thing
| that happens in real relationships as well!
|
| Fertility might be a concern, but my understanding is that
| the current mainstream view is that the world needs fewer
| children, not more of them.
| akomtu wrote:
| 3 - this is technodrugs; its users get high on their
| sexual addiction dopamine loop, and lose the remaining of
| their will and life energy. It's not our business to stop
| those who consciously engage in such self destruction,
| but luring the naive users into this pit, exploiting
| their loneliness and making money off it isnt any better
| than what cartels do.
|
| The author of this app must be doing this unknowingly:
| his overdeveloped mind sees just an interesting problem
| to solve, and the thick layer of nihilism obscures the
| sight of what he's really doing.
| stef25 wrote:
| If you're able to let your imagination run wild you'll see that
| the depth of depravity of NSFW AI content generated by user
| input probably has no end.
| dontupvoteme wrote:
| Even worse, some people turn it into a competition.
|
| Leak an API key on github or somewhere else and heaven help
| you if you get picked up by their scraping.
| Invictus0 wrote:
| It is inevitable
| chankstein38 wrote:
| 100% this. I'm not interested in these kinds of usages of LLMs
| but if someone is paying you to access your API directly you
| should be letting them do simple stuff like have sexual
| conversations.
|
| OpenAI's filtering is probably the most annoying part of
| ChatGPT and it makes it useless in so many cases. I can't tell
| you how many times I've uploaded a picture of a a place from
| Google Maps or of a location somewhere (that happens to contain
| a person) and tried to ask for details about the location to
| find out more info about it just to be shutdown... "I can't
| help with that" well then you're slowly making yourself useless
| and not worth the $20/mo.
| PoignardAzur wrote:
| > _What I find unsavory, is OpenAI imposing content
| restrictions like NSFW._
|
| I mean, the fact that people's NSFW conversations were being
| processed in a way where both OP and multiple companies could
| monitor them and do absolutely anything they want with them is
| a hint that something went very wrong there.
|
| I consider OpenAI pre-emptively pulling the plug a good thing
| in this case. Maybe there's a case for an AI erotica service,
| but OP sure as hell shouldn't be the one operating it.
| com2kid wrote:
| > What I find unsavory, is OpenAI imposing content restrictions
| like NSFW. That can easily extend to censorship of non-PC
| content, unpopular political ideas, and more.
|
| I'm guessing they don't want payment processors cutting them
| off.
|
| Everyone, even MS backed companies, are afraid of Visa.
| cornholio wrote:
| > GPT-based products, if not priced per usage, would fall into a
| dilemma: 1% of users consume 99% of tokens. A user from Sweden
| (seen from Cloudflare's call volume) chatted with Dolores for 12
| hours straight
|
| This, my friend, is a captive customer, that will pay anything to
| get his girlfriend back. I cringe at the potential of unethical
| behavior and abuse, where people fall in love with virtual
| entities fully owned by unscrupulous corporations, which can the
| legally "kidnap" or "torture" the characters, and generally tune
| their AI learning loop for profit maximization.
|
| "It was not extortion, the user willingly purchased a $50,000
| Kidnapping Roleplay package."
| HPsquared wrote:
| This is something even Stallman probably didn't think about.
| john-radio wrote:
| I'd take that bet, given that Stallman is in fact still alive
| solardev wrote:
| How we do know he isn't just an AI by this point?
| malicka wrote:
| I'd wager he has thought of it, based on his interesting AI-
| girlfriend short story: https://stallman.org/articles/made-
| for-you.html
| optimalsolver wrote:
| It's going to be an interesting century for sure.
|
| The human mind is simply not evolutionary adapted for what's
| coming up.
| ekianjo wrote:
| > The human mind is simply not evolutionary adapted for
| what's coming up.
|
| This can be said of pretty much everything humans ended up
| creating with technology so... not sure there's anything
| really new down the road. Humans adapt.
| pizza234 wrote:
| Humans adapt within boundaries.
|
| A couple of examples of things humans did not adapt to:
|
| - emphathy at distance; nowadays, a drone operator will
| push a button, kill a dozen people, and feel like it was a
| videogame
|
| - prioritizing long-term and abstract rewards, over short
| term ones; this is the reason why we have phenomenons like
| global warming
|
| - adjusting hunger to virtually unlimited availability of
| food
|
| All the above are concrete and very serious problems for
| humanity.
| krzat wrote:
| The fact that you point this out, is a proof that
| humanity can improve, what needs to change is our
| culture, especially education, ofc it may take a lot of
| time but it will happen eventually.
| lynx23 wrote:
| Regarding long-term rewards and global warming: What is
| the actual long term reward here? As I see it, there is
| no way to collect any sort of reward for helping to
| prevent global warming by changing your habits. Before
| the problem gets really problematic, most of us will be
| dead and burried. I fail to see how "caring for the
| future generation" has any kind of reward attached to it.
| Its an act of kindness, but there are no rewards attached
| to it.
| krzat wrote:
| Probably a question to the guy above, but there is
| immediate psychological reward in any act of kindness or
| generosity (at least in healthy, thriving individuals).
| lynx23 wrote:
| However, OP argued that global warming is suffering from
| people _NOT_ getting immediate rewards for their actions.
| Your argument basically says OP is wrong, and there is no
| need to counteract peoples tendency to prefer short-term
| rewards over long-term because they are supposed to get
| enough incentive by just knowing they have been kind. I
| doubt that.
| krzat wrote:
| What I think OP said, is that humans won't adapt to these
| challenges due to our biological limitations.
|
| IMO humanity will adapt, and the first sign of this is
| that those problems are getting recognized and given more
| and more traction.
| pizza234 wrote:
| > Before the problem gets really problematic, most of us
| will be dead and burried.
|
| The situation lies on a spectrum between "not
| problematic" and "really problematic".
|
| There is actually plenty of evidence that tangible impact
| already started, independently of the perception of those
| whoe live in areas where the effect is neither perceived
| nor obvious.
| nerdbert wrote:
| For many people, feeling like one has exhibited kindness
| is a strong reward.
| butlike wrote:
| What if war became just a couple of countries
| intelligence bots crunching digits of pi until one came
| up with a new one. That country would be the winner and
| they could ask one thing from the 'losing' side that
| didn't result in destruction or terror.
| notpachet wrote:
| You're not far from describing Bitcoin.
| antod wrote:
| Humans being humans, they'd then want to start destroying
| the other sides ability to
| design/make/run/maintain/afford pi digit crunching
| intelligence bots. Then ways to defend against those
| attacks on their people and economy by attacking the
| aspects of the other sides ability to attack etc. After a
| few rounds of that, soon the pi digit crunching element
| is completely replaced by a traditional war.
| cwillu wrote:
| Those who try to end warfare, but who do not understand
| warfare, are doomed to reinvent it.
| jollyllama wrote:
| Drone operators experience trauma.
| https://archive.li/j5urs
| horrible-hilde wrote:
| One more: attacking people who are familiar and
| comfortable to you rather than the people causing your
| actual problems.
|
| As a manager in tech, people who were on performance
| management rarely attacked me, they would find someone on
| the team to harass instead. Same thing with blaming a
| downturn in the economy on women painting their hair blue
| or mini skirts. Anything but blaming the powerful.
| bumby wrote:
| Pure conjecture on my part, but I wonder how much of this
| is risk-based status-mongering. Challenging the powerful
| is obviously risky. But knocking someone weaker down a
| few pegs can solidify your status in the hierarchy at a
| much lower risk. From that perspective, it's arguably
| rational behavior to maintain your status within a group
| when you feel vulnerable.
|
| (Pardon my reach here, I watched _Chimp Empire_ not too
| long ago...)
| fallingknife wrote:
| We never seemed to have a problem killing back when you
| had to do it face to face with a spear, so I don't think
| the drone really changes anything. Agree on the others
| though.
| bumby wrote:
| > _a drone operator will push a button, kill a dozen
| people, and feel like it was a videogame_
|
| This might depend on the context. RadioLab recently did a
| podcast titled "Toy Soldiers." Using low-flying "toy"
| drones rather than the high-flying "predator" drones,
| they make the case that drone operators get an oddly
| intimate portrait of their enemies. They go into how they
| reference them by their attire ("the 'red shoe' guy) and
| witness on an up-close and personal level how they grieve
| over their comrades.
| amanaplanacanal wrote:
| As a species, I don't think those matter. They could kill
| off 90% of the population and humans will go on.
| Evolution is a bitch in that way, individuals matter not
| at all.
| mensetmanusman wrote:
| The survivors adapt.
| butlike wrote:
| it depends.
| mordechai9000 wrote:
| I can't help but think of those beetles that preferred a beer
| bottle over the bumpy carapace of an appropriate mate.
|
| https://www.npr.org/sections/krulwich/2013/06/19/193493225/t.
| ..
| datadrivenangel wrote:
| Nothing human makes it out of the near future.
| eqvinox wrote:
| > The human mind is simply not evolutionary adapted for
| what's coming up.
|
| Maybe we're in fact evolutionarily adapted to not being
| evolutionarily adapted. We have successfully dealt with a
| whole sequence of hard societal pivots by now...
| ceejayoz wrote:
| That's literally
| https://en.wikipedia.org/wiki/Survivorship_bias.
| nerdbert wrote:
| All the humans who couldn't deal with tiktok are dead?
| stevage wrote:
| The author doesn't appear to even consider any kind of
| obligation to their users.
| mazzystar wrote:
| I'm sorry you feel that way. The primary intention of writing
| this article was to discuss it from the perspective of a
| 'failed product,' so I didn't mention my personal feelings. I
| am also aware of the potential harm it could cause to users,
| which is why I don't resent OpenAI's review process. I hope
| that during the period without review, it didn't cause any
| psychological harm to users.
| suoduandao3 wrote:
| "I'd like to correct the record on this story. We do NOT sell
| kidnapping roleplay packages. The user paid for a premium
| Daring Rescue Package. These do not become available unless the
| user spends two consecutive months as a net cost to our
| company. The user was given multiple reminders that he might
| want to cut back on usage or purchase a premium plan. Even
| then, he could have opted for a free Daring Rescue package and
| taken his chances with the base 1% chance of untraumatized
| recovery this package offers. Shuffling user responses under
| these high-pressure conditions is vital to improving our
| training data, and helps our users learn to deal with loss of
| important relationships. We are a relationship training
| service, not a replacement for them, and this user wanted the
| equivalent of a college degree.".
|
| On a more serious note, anyone know why unethical business
| plans are so much more fun to write? I always find myself
| giggling when these ideas come up.
| red-iron-pine wrote:
| ethical ones are fairly straightforward, and will often be
| the same.
|
| unethical ones can range from not-so-nice to mind-bogglingly
| evil, and in all sorts of "fun", different, unique ways.
|
| "Happy families are all alike; every unhappy family is
| unhappy in its own way."
| WalterBright wrote:
| > unscrupulous corporations
|
| I wonder why everyone is afraid of unscrupulous corporations,
| but are not concerned about use by unscrupulous governments.
| The latter are a lot more dangerous.
| jacquesm wrote:
| In the long term, but in the short term unscrupulous
| corporations _far_ outnumber unscrupulous governments and
| they act much, much faster to adapt to new technology.
| WalterBright wrote:
| Unscrupulous governments can easily take everything you
| have and destroy you, literally.
|
| History is full of examples. Heck, you'll see examples in
| the newspaper every day.
|
| The rookiest cop has more power over you than Bill Gates.
| jacquesm wrote:
| Absolutely. But the rookiest cop isn't currently trying
| to track my whereabouts, abuse my privacy, stick a bunch
| of AI down my throat and in general isn't trying to make
| my wallet any thinner. Bill Gates and the more modern
| versions of Bill Gates are doing that and more.
| WalterBright wrote:
| The rookiest cop can:
|
| 1. track you with license plate readers
|
| 2. pull up any information about you
|
| 3. fine you
|
| 4. force you to stop
|
| 5. detain you
|
| 6. confiscate your car and property
|
| 7. search you and your property
|
| 8. take your phone and search it
|
| Bill Gates can:
|
| 1. give you a free browser when you buy his operating
| system
| jacquesm wrote:
| This is getting silly.
| WalterBright wrote:
| It is? I suggest reading the newspaper.
| amanaplanacanal wrote:
| I suspect the issue here is that most of HN folks live in
| western countries with mostly working democracies. For
| the most part we haven't experienced how bad governments
| can get.
|
| Whether or not western style democracies are immune to
| going really bad, remains to be seen. Some days I'm
| optimistic, some days not so much.
| jacquesm wrote:
| I've lived in dictatorships and non-functional
| democracies, in spite of that corporations have done me
| _far_ more harm than those states ever did. I do realize
| they have that power and I do realize it gets abused and
| regularly so. But the fact is that people get abused by
| corporations all the time and by their governments less
| frequently if ever. But when they are abused the
| consequences are likely much worse.
| horrible-hilde wrote:
| I think the former is far more dangerous particularly if a
| monopoly because there is no feedback loop.
| danShumway wrote:
| HN does skew towards skeptical of government regulation and
| control of AI. People are concerned about government usage of
| AI, particularly in the realms of facial recognition,
| automated sentencing, algorithmic bias, and security (LLMs
| are pretty insecure right now and there are companies trying
| to sell the US government on using them in the military).
| There's obviously a lot to be concerned about on the subject
| of government abuse of AI, and that gets conversation on HN.
|
| However, given that this specific article is about selling
| access to an AI girlfriend, I think we're _probably_ OK to
| talk more about the corporate angle than the government
| angle. Unless you 're predicting that the 2024 election is
| about to get real weird, "what if the US government starts
| offering AI girlfriends" is not going to be at the top of my
| worry list any time soon.
| CamperBob2 wrote:
| _HN does skew towards skeptical of government regulation
| and control of AI._
|
| Which HN is that? Certainly not this one.
| danShumway wrote:
| Quite genuinely, I don't think I've ever posted a comment
| on HN where I've expressed support for open, locally
| running LLMs and had it been downvoted. Am I missing
| something here? You want to point me to all of the
| articles where people criticize Huggingface?
|
| But regardless of how you interpret HN sentiment towards
| uncontrolled AI access, it doesn't change the fact that
| "what about the government" is a profoundly weird thing
| to ask in the middle of a conversation about people doing
| erotic roleplays with a text bot. Is that something
| people are worried about? Do we think that the US Post
| Office is going to suddenly start advertising a sex bot
| service?
|
| Yes, government abuse of facial recognition exists,
| regulatory capture exists, etc. But that's not really
| relevant to a conversation about companies offering AI
| girlfriends; AI girlfriends do seem to be mostly a
| corporation thing.
| api wrote:
| IMHO this is industrialized automated abuse of lonely people
| and especially those with mental health issues. It's really
| truly gross.
|
| At least Joi in Bladerunner 2024 was a local model the user
| could apparently download onto their own portable device. She
| also never presented an upgrade dialog requiring payment.
| Tade0 wrote:
| Funny how a fantasy dystopia is in some aspects already less
| dystopian than the real world, where you already get such
| notifications.
| api wrote:
| Pros: it's not raining all the time, pollution isn't as
| bad, and I don't live in a slum where I have to step over
| half-dead drug addicts just to get to my apartment.
|
| Cons: the e-waifus are much more manipulative and
| exploitative.
| ThrowAway1922A wrote:
| I don't care if it's abuse, I want it. I want to retreat from
| the disgusting world and society around me as much as
| possible and AI friends would just be another option towards
| that point.
| mustacheemperor wrote:
| > This, my friend, is a captive customer, that will pay
| anything to get his girlfriend back
|
| Or potentially _do_ anything. I 'd be a little scared of having
| folks like this convinced I am the personal arbiter gatekeeping
| their access to their 'lover,' that I 'took them away.'
|
| When Replika.ai restricted erotic chat from their product, the
| apoplectic anguish on their subreddit was unlike any emotional
| reaction I've ever witnessed from a group of people about a
| consumer technology. And their anguish was genuine - there are
| Replika users who truly consider themselves married to their AI
| companion.
|
| And frankly, the Replika AI is not even that smart. After
| watching that unfold, I am convinced that these tools don't
| need to be much more sophisticated for many people to start
| forming what they feel to be deep and genuine emotional
| connections with them.
|
| Edit: Brings to mind the Nature paper[0] posted this week about
| how the CASA theory seems obsolete today, that we are less
| prone to personify computing systems now.
|
| > A recent study investigated whether we could be friends with
| a social computer, in which participants were asked to converse
| with a chatbot over a period of three weeks and constantly rate
| their relationship. The results showed that initially
| participants were enthusiastic and engaging with their chatbot
| friend, but quickly this diminished, with scores for intimacy,
| believability, and likability decreasing with each interaction
|
| It would seem this definitely does not apply for everyone, like
| our user in Sweden.
|
| https://www.nature.com/articles/s41598-023-46527-9
| toss1 wrote:
| THIS
|
| "Those who can make you believe absurdities, can make you
| commit atrocities." -- Voltaire
| reducesuffering wrote:
| We aren't far from people believing God is talking to them
| through AI generated text and voice, and commanding them...
| AussieWog93 wrote:
| There was a guy a couple of years back whose AI
| girlfriend went off the rails and convinced him to try
| and assassinate Queen Elizabeth
| robotnikman wrote:
| And another example to add to the other user, the one guy
| who killed himself when the AI he was talking to
| encouraged it.
|
| https://www.euronews.com/next/2023/03/31/man-ends-his-
| life-a...
| polishdude20 wrote:
| What about the same but in a different light:
|
| "Those that are most likely to commit atrocities are those
| that are most likely to believe in absurdities."
| lmm wrote:
| Isn't that putting the causation backwards? The point is
| that believing absurdities leads to committing
| atrocities, not that committing atrocities leads to
| believing absurdities.
| mensetmanusman wrote:
| Psychologists recognize that nearly all such folks also
| suffer from massive mental health issues, and that is where
| some of the danger is (in terms of irrational violent
| response).
|
| Maybe these systems will be useful as a honeypot for finding
| these people and helping them?
| bobthepanda wrote:
| The reason we have so many strict laws around mental health
| is because in the past this was more likely to lead to
| hoovering up people into jail or institutions. I'm not
| _super_ confident that this also wouldn 't be the case
| today.
| Blackthorn wrote:
| > When Replika.ai restricted erotic chat from their product,
| the apoplectic anguish on their subreddit was unlike any
| emotional reaction I've ever witnessed from a group of people
| about a consumer technology. And their anguish was genuine -
| there are Replika users who truly consider themselves married
| to their AI companion.
|
| There's a second layer to this though -- Replika's marketing
| was _heavily_ centered around that erotic chat element. I 'm
| trying to think of a good car simile and actually coming up
| blank. It's, uhh, like advertising the incredible off-road
| ability of this vehicle, and then when you show up and
| purchase the vehicle someone comes and takes off the tires
| and replaces them with tiny balds? I'm bad at similes.
| Dylan16807 wrote:
| Also, my understanding is that when they made that change
| they goofed up companionship talk in general, by virtue of
| how messy AI censoring is.
| duskwuff wrote:
| IIRC, they screwed it up so massively that, for a while,
| the chatbot would still send "thirsty" automated messages
| inviting users to sexually explicit conversations, but
| would refuse to follow through.
| drewzero1 wrote:
| Or like advertising that a car will be able to drive itself
| without any human intervention, and then deactivating or
| removing the sensors that might allow anything close to
| that capability...?
| kridsdale3 wrote:
| "The Lifecycle of Software Objects" by Ted Chiang was a
| really good exploration of people bonding deeply with their
| AI companions (in this instance, pet animals in a metaverse).
| But it goes all the way in to the topic.
| r2_pilot wrote:
| Amusingly, that specific short story (short may be
| misleading) is what stopped my completion of that book of
| short stories because I just couldn't get through it.
| robotnikman wrote:
| We can already see this with Replika when they took away
| many... roleplay... capabilities that originally came with the
| AI. The communities on Reddit and Facebook were absolutely
| devastated. People were genuinely attached to these AI's like a
| real relationship, and were feeling the resultant heartbreak.
|
| The kind of emotional manipulation which can be done with these
| products is insane, and I can see things going very wrong very
| quickly.
| wholinator2 wrote:
| It's particularly terrifying when you think about how
| Facebook and others probably already have projects in the
| works to befriend children with AI. little kids who don't
| know the difference will become Weaponized with constant
| nudges towards whatever motives grant the corporate owners
| more power. The ability to persuade children is near infinite
| and we can observe in those who've grown up in cults/strict
| religious compounds that breaking that programming can be
| nearly impossible and leaves scars for the rest of your life.
|
| The thing is, how do we outlaw this without also preventing
| children from using AI to learn at a faster rate than their
| teachers can give them. There's such incredible, paradigm
| shifting power for good. But you know the people who've made
| the evils of today are already working on the evils of
| tomorrow.
| carapace wrote:
| > Facebook and others probably already have projects in the
| works to befriend children with AI.
|
| Yah, they are already rolling them out!
|
| > Meta Platforms is planning to release artificial
| intelligence chatbots as soon as this week with distinct
| personalities across its social-media apps as a way to
| attract young users, according to people familiar with the
| matter.
|
| https://www.wsj.com/tech/ai/meta-ai-chatbot-younger-users-
| da...
| jprete wrote:
| There's no way to separate them. Education is influencing
| someone towards things that we believe are true. If an
| educational AI believes that propaganda is true, then
| education and propaganda are the same thing.
| dorkwood wrote:
| > The thing is, how do we outlaw this without also
| preventing children from using AI to learn at a faster rate
| than their teachers can give them.
|
| Can you teach me, an adult, to do that?
| andrewstuart wrote:
| I wonder if people were pirating his voice synthesis feed.
| rahidz wrote:
| Why not use a local model instead of relying on OpenAI? Even
| something like Mistral or Mytholite will probably satisfy 90% of
| people using your app, and I bet hosting it would be cheaper than
| GPT anyway.
| stevage wrote:
| > AI Friends inevitably turn into AI Girlfriends/Boyfriends
| because you and the Character in your phone are not equal: she
| can't comfort you when you're hurt (unless you tell her), she
| can't actively express emotions to you, and all this is because
| she lacks external vision.
|
| That is such a curious lesson to draw. So specific, and seemingly
| so wrong.
|
| Surely long distance friendships work just fine even if only
| through text, without "eyes".
| chankstein38 wrote:
| My girl and I have lived together for more than a year now
| after meeting online and talking exclusively through text for
| months before eventually graduating to voice chats before we
| even knew what each other looked like. I'm with you, long
| distance relationships (romantic or otherwise) can work great!
| ReactiveJelly wrote:
| Yeah I thought that was odd. You could use some random timers
| to have the AI "check in" on you if you don't talk to it for a
| while.
|
| Is it eerily similar to the gambling gacha games that expect
| you to play a couple hours a day _every day_? Yes. Is it also
| similar to a real friendship or long-distance relationship?
| Also yes.
|
| It may cost a few tokens, or you could generate a few canned
| responses and use those. For my real friends I usually use "Hi"
| agomez314 wrote:
| Wow this read more like Frankenstein than anything I've read
| recently: A talented scientist is driven by curiosity to create
| ever more sophisticated creations, culminating in a monstrous
| entity that leads to the scientist's downfall. But the scientist
| doesn't realize this until too late.
|
| Really, if our top tinkerers had a little more contextual
| knowledge of history and less naivete about people we could save
| ourselves from ever more dangerous Frankenstein tragedies.
| andrewstuart wrote:
| This is an interesting post and the person achieved a huge
| amount.
|
| They should not have given up.
|
| I would have pivoted to personal pet companions. If people want
| virtual girlfriends then they certainly want personal dogs, cats
| and birds that talk to you.
|
| Enough was achieved here to raise money.
|
| Some careful redesign and development might have allowed some
| caching and reuse of expensive AI API interactions, reducing
| costs.
|
| Add in further gameplay elements so it is not just about making
| the AI API do stuff, that would also reduce costs and maybe even
| engage the users more.
|
| Perhaps AI should be the icing and accents on a game rather than
| the focus.
|
| As an aside, this is the first time I have read anyone tell about
| how they wanked to their own software.
|
| Altogether this was perhaps a missed opportunity for this
| founder.
|
| BUT it is now an opportunity for one of the HN readers to follow
| this playbook and replicate what this founder did.
| bogwog wrote:
| > I would have pivoted to personal pet companions. If people
| want virtual girlfriends then they certainly want personal
| dogs, cats and birds that talk to you.
|
| Then you have the same problem, only with a different (arguably
| worse...) fetish
| DrNosferatu wrote:
| This business venture was completely dependent on a main service
| from a single 3rd party - risky!
|
| Why didn't the author hosted his own uncensored open source LLM
| on the cloud?
| mazzystar wrote:
| I will try to make it open source.
| DrNosferatu wrote:
| Have you considered running your own uncensored LLaMA2
| instance via something like llama.cpp on a cloud for hire
| like, say, Azure?
|
| Some examples - there's probably a whole zoo of these:
|
| https://ollama.ai/blog/run-llama2-uncensored-locally
|
| https://huggingface.co/TheBloke/Luna-AI-Llama2-Uncensored-
| GG...
|
| https://ollama.ai/library/llama2-uncensored
| lynx23 wrote:
| Reading this, I really wonder why the dev hasn't forseen this
| outcome. It was already obvious in March that OpenAI will censor
| all sorts of content. Building a sexually charged bussiness on
| the OpenAI API was always a dead end for those that have a feel
| for american prudish approach to sexuality. If you had asked me
| to do this sort of product 6 motnsh ago, I would have declined
| because it was (to me) rather obvious content filtering will kill
| you at some point.
|
| Heck, there are a bunch of legitimate use cases that are being
| blocked by OpenAI. A blind friend of mine wanted to use Be My AI
| to have explicit photos of his _girlfriend_ described. Guess
| what, no way. Consent was given, actually, _she_ asked him to do
| this because she was also interested in the result. Everyone
| involved consented. But the sex filter from the USA decided that
| even though the technology is available, a blind man is not
| supposed to know how his own girlfriend _looks_ naked... This is
| weird beyond description. Plain patronisation.
|
| Another example regarding Be My AI: A blind woman I know was
| trying to have something described which she held in her hand to
| present it to the camera. A kind of selfie situation. Guess what?
| OpenAI declined to describe the thing in her hand, because the
| photo also contained parts of her (covered) breats. Imagine how
| hilarious this must feel? "Censoring just declared my own boobs
| to be off-limits for me." If you think about this long enough,
| its a form of discrimination. Because women will likely hit the
| filter more often then men when they are (accidentally) part of
| the picture they are trying to get described.
|
| And then there is this blog post by Sam Hartman:
| https://hartmans.dreamwidth.org/100499.html
| mustacheemperor wrote:
| On the other hand, Sudowrite appears to have been granted some
| kind of specialized license from OpenAI. They are _definitely_
| not a 'sexually charged business', but they are a creative
| writing tool, and their output is somewhat milquetoast but
| generally not restricted from "NSFW" concepts like sex and
| violence.
|
| They do incorporate multiple models in their system, so maybe
| generation falls back from OpenAI for those cases. But I dimly
| recall reading something from a team member indicating they
| have some special permissions with OpenAI. Take that bit as
| worthless hearsay, just mentioning it in case someone else has
| better receipts.
| spacemadness wrote:
| At this point in the hype cycle, I am completely unimpressed by
| everyone slapping something on top of GPT and calling it a
| product. There are at least 4 projects on the front page that do
| this today. The interesting technology is GPT itself, but these
| projects just feels like people trying to scoop up attention for
| themselves using the psychology of the hype cycle. Please HN,
| enlighten me on why the middle-man approach here is so
| interesting.
| wilsonnb3 wrote:
| The startup/"entrepreneur" contingent on hacker news generally
| considers anything that makes money inherently good, aside from
| extreme immoral cases.
|
| There is a subgroup in there that is especially interested in
| low effort ways to make money, with the ultimate goal being to
| scrape off enough of the money flowing around to either live a
| good life or get rich.
|
| In summary, the middle-man approach is so interesting on hacker
| news because a significant portion of the readership aspires to
| be the middle man.
| chankstein38 wrote:
| Yeah I'm getting tired of "Show HN: I built a wrapper for
| ChatGPT and wrote a prompt that probably isn't very good"
| posts.
|
| EDIT: Just want to add, this write-up was interesting and was
| not something I considered to be the above.
| cedws wrote:
| GPT wrappers will come and go. It's the new hype, just as
| shitcoins were a few years ago.
|
| Don't hate the players, hate the game.
| m4rtink wrote:
| Also a definition of single point of failure & third party
| actually running the thing messing up the project.
| gumballindie wrote:
| Yup, sums up the ai crowd.
| dangerwill wrote:
| Yeah, and the comments on this thread seriously worry me. We
| have people arguing that it is good for society to satisfy
| lonely men with virtual porn and other people saying that this
| is great, scrappy product development and the author should go
| further/use other models not constrained by openAI's policies.
| It feels like I'm discovering a significant portion of our
| field are sociopaths
| ReactiveJelly wrote:
| Well, it might be harm reduction to let lonely men have AI
| girlfriends.
|
| Sure, I wish reality was not a dystopia that constantly let
| me down, but all I can do is take anti-depressants, make
| money, keep voting, and hope things get better the longer I
| live.
|
| If the hypothesis that isolated men cause trouble is true,
| and if we don't have a long-term solution, at least this
| could be a stop-gap. Assuming it does not radicalize anyone
| further. (Hard to say what will come from uncensored models,
| but it's a real genie-is-out-of-the-bottle sitch)
| bilater wrote:
| This is such a fascinating topic. I do think we will see the plot
| of "Her" playing out in the next 12-24 months. For your specific
| case, I agree OpenAI moderation is an annoying overreach and
| maybe switching to Mistral could be a possible way to move
| forward. The 7B is perhaps not good enough currently but we are
| fast getting there and will likely have a GPT-4 level capability
| in an open source < 20B model soon.
| brucethemoose2 wrote:
| Mistral 7B finetunes are extremely good. More than good enough
| for an AI friend app, with long context to boot. I would
| suggest Toppy-M or OpenHermes 2.5, or one of the new YARN
| extended models.
|
| And, there is the potential to run the model on the user's
| device with MLC-LLM and whisper, to save on API costs (and
| privacy concerns) for users with newer phones. This could give
| the app a true "free" tier to help it proliferate while making
| charging for API requests kinda doable. I'm not sure if any
| good TTS models have been ported yet, but I'm sure its a matter
| of time.
| bilater wrote:
| Yes - ASR on browser/device has reached the quality threshold
| pretty much with distil whisper/ whisper 3.
|
| Quality TTS with fast latency is still not there but getting
| better (you can use Tortoise but its slow and compute
| expensive as far as I know). Bark is another option but has
| mixed results.
|
| I played around with a recent transformers.js one (using
| SpeechT5) you can run in your browser and am optimistic where
| we can go with some improvements:
|
| https://tinyllms.vercel.app/dashboard/tts
| brucethemoose2 wrote:
| I was recently pointed to xtts:
|
| https://coqui.ai/blog/tts/open_xtts
|
| I dunno about performance, but the model is 1.86GB.
| thriftwy wrote:
| I wonder if you could create an AI assist to help you hold on to
| an existing relationship, though?
| rpigab wrote:
| I still don't understand why people are building things on top of
| OpenAI's APIs.
|
| It's obvious that in the early days, they'll operate at a loss,
| with many free features and cheap premium features. It's a good
| time to try it out and see what it can do, but you can't build
| anything that relies on it, it's a rugpull waiting to happen.
|
| They'll add moderation, raise prices, and you'll have to do the
| same, your users will leave and the ones who stay will hate you.
| This is only the beginning.
|
| Use self-hostable open source alternatives even if they're less
| easy to use or require infrastructure, and plan for high
| electricity and storage bills. Whatever you build will last, as
| long as you respect your users and don't peek at the cleartext
| logs of users who didn't opt-in to that. It won't be easy.
|
| If OpenAI keeps a monopoly on LLMs, they're gonna become like
| Microsoft did with Windows, but the enshittification process will
| be a thousand times faster.
| gwern wrote:
| You build it on top of the API to bootstrap and prototype with
| a very high quality model that takes minutes to get started
| with. Then once you have operated for a while, you have a ton
| of logged data to train/retrieve/analyze and can swap in a
| competitor, downgrade the OA model, finetune instead, or
| something.
|
| In OP case's, this worked perfectly: he built a working product
| quickly and easily, and... discovered that it was a product he
| didn't want to work on regardless of token cost, cut his
| losses, did the post-mortem, and moved on with his life.
|
| Imagine if he had wasted a year & a ton of money building his
| own LLM "because I'll need to economize on tokens and have to
| build a custom LLM" _before_ he could even launch!
| urbandw311er wrote:
| I'm torn on this.
|
| Yes, it's never clever building a business on top of a single
| platform dependency (eg Twitter, Facebook, Play Store, etc)
|
| Equally, if you take the example of the App Store, market
| pressures have actually forced Apple to _reduce_ their cut.
|
| Time will tell I guess. And also whether OpenAI end up with a
| monopoly, part of a duopoly etc.
| pixl97 wrote:
| People still build on Microsoft platforms. It turns out
| actually having a usable platform is more important to some
| businesses than having freedom and no usable platform.
| grahamgooch wrote:
| 6.9 dollars!
|
| Cheeky my friend
| halfjoking wrote:
| Grok has an API looking for use cases.
|
| Just let Elon know the pricing... "I'm all in"
| gscott wrote:
| Maybe record and save these AI NSFW responses then intercept the
| request and spit out of the saved responses once you have enough
| of them stored. This would save from behind banned from the API.
| urbandw311er wrote:
| My hunch is that they're too contextually dependent upon the
| conversation / tokens that have come before. It's not simply a
| "if user inputs A, output B" situation.
| chrisbrandow wrote:
| This is not meant as criticism of this author, but boy oh boy, do
| I find this future depressing.
| Smar wrote:
| Maybe better to call it AI doll rather than AI girlfriend...
| Octokiddie wrote:
| > But, does humanity really need an AI friend?
|
| This reminds me of the Fredrick Pohl line "A good science fiction
| story should be able to predict not the automobile but the
| traffic jam."
|
| So I wonder what the traffic jam is going to look like given that
| the car seems to already be parked outside with the engine
| running.
|
| Faster depopulation across the developed world? Probably, but
| maybe there's something even more important.
|
| I'd put what the author describes into the category of "pleasure
| trip," which is what the car was mainly used for initially.
|
| As infrastructure to support the car was built, it became a
| necessity. And that's what created the traffic jam: network
| effect. The suburb and the car fed on each other. You don't get a
| traffic jam worth talking about without a suburb and a network
| effect.
|
| So I'm curious about the network effects something like an AI
| friend might have. What kinds of new infrastructure and new
| lifestyles will they enable/encourage? And how will this
| infrastructure fuel future demand?
| dpflan wrote:
| Hm, let's try an extreme: the AI friend slowly becomes a cult
| leader to you, hijacking your mind into subservience to the
| owner of the service to which AI belongs, and further up some
| social/political hierarchy to a few controlling people/groups
| of people.
|
| The traffic jam is society stymied by individuals mind-
| controlled at a level closer to connection to a human than any
| other way possible before this.
| ericmcer wrote:
| Kinda feel like you have that backwards, why would an AI that
| sophisticated care about controlling your mind? Why would the
| group of people who control that AI care about you as
| anything other than a liability? What do they get out of
| manipulating average individuals? Votes?
|
| The scarier prospect to me is human labor and intelligence
| become almost worthless and those at the top start to view
| the masses as superfluous.
| nemothekid wrote:
| I think the pleasure trip is just porn. We have that already
| and it's possible to discuss the effects of widespread porn on
| people.
|
| I think the next step is, what does it mean to have an
| emotional relationship with someone in your pocket who is for
| all definitions perfect - can provide perfect advice and does
| so with judgement, ridicule, pity or anger? What will be my bar
| for human relationships, those imperfect messy human beings,
| when I have a partner I can be perfectly vulnerable with? What
| happens to interpersonal relationships when you are competing
| with a virtual perfect significant other?
| narrator wrote:
| I got koboldcpp (https://github.com/LostRuins/koboldcpp) running
| locally on my M1 mac with 16gb. Use the guides on
| http://4chan.org/g/ in the AI Chatbot Thread.
|
| In the future, you will own and control access to your very own
| AI girlfriend running on your local machine, and you will be
| happy.
|
| Think of all the millions of guys in China and elsewhere who, due
| to sex selective abortion, will never have a wife. They deserve
| to have their own Waifu running on their own computer that serves
| them alone, and not the interests of the state or some
| corporation.
| Der_Einzige wrote:
| Agreed. This is why Civit.AI and huggingface and the entire
| rest of the open models movement need to be cherished and
| protected. We are living in a golden age and don't know it yet,
| but this is the era before big copyright and regulators come
| shitting down on the beautiful thing the AI community has done
| vivekd wrote:
| For this to work the AI girlfriend has to be better than
| videogames or online chat forums
|
| I don't see AI partners overtaking those
| ReactiveJelly wrote:
| No reason she can't participate in chat forums or text games
| with you, right? Imagine seeing a comment and recognizing her
| username, and then flirting a little in a public thread.
| sangnoir wrote:
| > I even repeatedly modified the system prompt, such as changing
| _try to attract {user}_
|
| So it was a virtual _girlfriend_ app, after all. Cashing-in on
| loneliness gives me the heebie-jeebies; in a dystopian cyberpunk
| way - extremely horny but 0% sexy.
| akomtu wrote:
| "the Dolores iOS app went live" - his subconscious suggested a
| good name. Mater Dolorosa is Mother of Sorrows in Latin.
| QuercusMax wrote:
| Reminds me of this article: https://bloodknife.com/everyone-
| beautiful-no-one-horny/
| nerdbert wrote:
| Cashing in on loneliness is big business. Singles bars.
| OKCupid. Half of the self-improvement/self-help world.
| vivekd wrote:
| To quote slate star codex, horny men want sex, horny women want
| attention. AI is much better at the second.
|
| Maybe there's more potential for AI boyfriends
| dangerwill wrote:
| Tech people, I beg you, BEG YOU, to learn media literacy. Her,
| Black Mirror, Ready Player One, Cyberpunk, etc are dystopias not
| something to aspire to
| dangerwill wrote:
| For those that are defending this on the basis of providing
| "help" for lonely people, how does this not result in those
| lonely people isolating from actual human society more? This
| feels like it is going to create more hikikomoris more than
| anything else
___________________________________________________________________
(page generated 2023-11-16 23:01 UTC)