[HN Gopher] Jan Leike Resigns from OpenAI
___________________________________________________________________
Jan Leike Resigns from OpenAI
Author : Jimmc414
Score : 45 points
Date : 2024-05-15 05:05 UTC (17 hours ago)
(HTM) web link (twitter.com)
(TXT) w3m dump (twitter.com)
| aresant wrote:
| Co-head of super-alignment (I had to look it up)
| Jimmc414 wrote:
| Jan and Ilya were the leads of the superalignment team set up in
| July of 2023.
|
| https://openai.com/index/introducing-superalignment/
|
| "Our goal is to solve the core technical challenges of
| superintelligence alignment in four years.
|
| While this is an incredibly ambitious goal and we're not
| guaranteed to succeed, we are optimistic that a focused,
| concerted effort can solve this problem:C There are many ideas
| that have shown promise in preliminary experiments, we have
| increasingly useful metrics for progress, and we can use today's
| models to study many of these problems empirically.
|
| Ilya Sutskever (cofounder and Chief Scientist of OpenAI) has made
| this his core research focus, and will be co-leading the team
| with Jan Leike (Head of Alignment). Joining the team are
| researchers and engineers from our previous alignment team, as
| well as researchers from other teams across the company."
| learkonas wrote:
| With Ilya and two other superalignment team members gone,
| safetyism in OpenAI is getting decimated, after taking their (EA)
| money for more half a decade. Brutal.
| lkoolma wrote:
| Related: https://news.ycombinator.com/item?id=40361128
| Pet_Ant wrote:
| In case people haven't noticed, this is the second resignation in
| as many days.
|
| https://news.ycombinator.com/item?id=40361128
| the_mitsuhiko wrote:
| And entirely predictable from the first one:
| https://openai.com/index/introducing-superalignment/
| uLogMicheal wrote:
| Imagine trying to keep something so far above us in
| intelligence, caged. Scary stuff...
| binarymax wrote:
| I'm genuinely curious - do you actually believe that GPT is
| a super intelligence? Because I have the opposite
| experience. It consistently fails to be correct on
| following even the most basic instructions. For a little
| while I thought maybe I'm doing it wrong, and I need better
| prompts, but then I realized that its zero shot and few
| shot capabilities are really hit and miss. Furthermore, a
| superior intelligence shouldn't need us to conform to its
| persnickety requirements, and it should be able to adapt
| far better than it actually does.
| uLogMicheal wrote:
| GPT does not need super-alignment. This refers to
| aligning artificial general and super intelligence.
| btown wrote:
| Makes me wonder if that 20% compute commitment to
| superalignment research was walked back (or redesigned so as
| to be distant from the original mission). Or, perhaps the two
| deemed that even more commitment was necessary, and were
| dissatisfied with Altman's response.
|
| Either way, if it's enough to cause them both to think it's
| better to research _outside of_ the opportunities and access
| to data that OpenAI provides, I don 't see a scenario where
| this doesn't indicate a significant shift in OpenAI's
| commitment to superalignment research and safety. One hopes
| that, at the very least, Microsoft's interests in brand
| integrity incentivize at least some modicum of continued
| commitment to safety research.
| dontupvoteme wrote:
| Ironically Microsoft is the one that's notoriously terrible
| at checking their "AI" products before releasing them.
|
| Besides the infamous Tay there was that apparently un-
| aligned Wizard-2[or something like that] model from them
| which got released by mistake for about 12 hours
| buildbot wrote:
| As an MS employee working on LLMs, that entire saga is
| super weird. We need approval for everything! Releasing
| anything without approval is quite weird.
|
| We can't just drop papers on arxiv. There is no way
| running your own twitter, github, etc as a separate group
| allowed.
|
| I checked fairly recently to see if the model was
| actually released again, it doesn't seem to be; I find
| this telling.
| fzzzy wrote:
| I was able to download a copy of that before they took it
| down. Silly.
| pfist wrote:
| I have noticed, and I am concerned that they were the leaders
| of the Superalignment team.
| zer00eyz wrote:
| Sam Altman superalinged them right out the door...
| transcriptase wrote:
| On the other hand, they clearly they weren't concerned enough
| about the issue to continue working on it.
| dontupvoteme wrote:
| One could argue that at this point openai is being Extended
| and Embraced by Microsoft and is unlikely to have much
| autonomy or groundbreaking impact one way or another.
| llamaimperative wrote:
| Ah yes, a scientist refusing to work on the hydrogen bomb
| couldn't have been all that concerned about it.
| scarmig wrote:
| If your ostensible purpose is being sidelined by decision
| makers, trying to fight back is often a good option, but
| sometimes you fail. Admitting failure and focusing on other
| approaches is the right choice at that point.
| HarHarVeryFunny wrote:
| The Anthropic folk were concerned enough that they left,
| and are indeed continuing to work on it [AI safety].
|
| Now, we've have the co-leads of the super-alignment/safety
| team leaving too.
|
| Certainly not a good look for OpenAI.
|
| There really doesn't seem to be much of a mission left at
| OpenAI - they have a CEO giving off used car salesman vibes
| who recently mentioned considering allowing their AI to
| generate porn, and now releasing a flirty AI-girlfriend as
| his gift to humanity.
| treme wrote:
| Reads like beginnings of a good dystopian movie script
| dontupvoteme wrote:
| Turns out we already have alignment, it's called capitalism.
| mbgerring wrote:
| This is true and we do not talk about it enough. Moreover,
| Capitalism is itself an unaligned AI, and understanding it
| through that lens clarifies a great deal.
| lowmagnet wrote:
| oh no, it's just a real world reinforcement model
| panarky wrote:
| People experience existential terror from AI because it
| feels like massive, pervasive, implacable forces that we
| can't understand or control, with the potential to do
| great harm to our personal lives and to larger social and
| political systems, where we have zero power to stop it or
| avoid it or redirect it. Forces that benefit a few at the
| expense of the many.
|
| What many of us are actually experiencing is existential
| terror about capitalism itself, but we don't have the
| conceptual framework or vocabulary to describe it that
| way.
|
| It's a cognitive shortcut to look for a definable villain
| to blame for our fear, and historically that's taken the
| form of antisemitism, anti-migrant, anti-homeless, even
| ironically anti-communist, and we see similar corrupted
| forms of blame in antivax and anti-globalist conspiracy
| thinking, from both the left and the right.
|
| While there are genuine x-risk hazards from AI, it seems
| like a lot of the current fear is really a corrupted and
| misplaced fear of having zero control over the foreboding
| and implacable forces of capitalism itself.
|
| AI is hypercapitalism and that is terrifying.
| kridsdale1 wrote:
| Ted Chiang on the Ezra Klein podcast said basically the
| same thing:
|
| AI Doomerism is actually capitalist anxiety.
| exolymph wrote:
| Nick Land type beat
| sgregnt wrote:
| You mean freedom is an analigned AI?
| throwaway5959 wrote:
| How does capitalism work if there aren't any workers to buy
| the products made by the capitalists? Not being
| argumentative here, I really want to know.
| Atotalnoob wrote:
| Honestly don't know if these kinds of people have thought
| that far ahead
| lrvick wrote:
| All that matters are quarterly profits.
| SamoyedFurFluff wrote:
| The machines can buy the products. We already have HFT,
| which obviously has little to do with actual products
| people are buying or selling. Just number go up/down.
| hn_throwaway_99 wrote:
| They resigned together on the same day - people are just
| announcing this like it's some type of "drip drip" of people
| leaving to build suspense.
|
| While Jan's (very pithy) tweet was later in the evening, I was
| reading other posts yesterday at the time of Ilya's
| announcement saying that Jan was also leaving.
| dawoodee wrote:
| A few more people involved in the alignment efforts have left
| recently:
| https://x.com/ShakeelHashim/status/1790685752134656371
| bloopernova wrote:
| It appears that there are Interesting Times in OpenAI right now.
| prab97 wrote:
| It's always interesting times at OpenAI!
| ganzuul wrote:
| I bet superalignment is indistinguishable from religion (the
| spiritual, not manipulative kind), so proponents get frequency-
| pulled into the well-established cult leader pipeline. It's a
| quagmire to navigate so we can't have both open and enlightening
| discussions about what is going on.
| uLogMicheal wrote:
| I thought the whole point of making a transparent organization
| to lead the charge on AI was so that we could prevent this sort
| of ego and the other risks that come with.
| alephnerd wrote:
| Nonprofits are not really that transparent, and do bend to
| the will of donors, who themselves try to limit transparency.
|
| That's why Private Foundations are more popular than Public
| Charities even though both are 501c3 organizations, because
| they don't need to provide transparency into their
| operations.
| uLogMicheal wrote:
| I was more referencing the name and origin intentions than
| the non-profit status :)
| ganzuul wrote:
| Say I have intelligence x and a superintelligence is 10x,
| then I get stuck at local minima that the 10x is able to get
| out of. To me, the local minima looked "good", so if I see
| the 10x get out of my "good" then most likely I'm looking at
| something that appears to me to be "evil" even if that is
| just my limited perspective.
|
| It's one hell of a problem.
| buckley_1991 wrote:
| Your words sound like something from a joke: Human: How to
| achieve human peace. AI: Eliminate all humans.
| ganzuul wrote:
| Well I know I'm not good at explaining what I mean.
| Please do ask what I should clarify.
| esmevane wrote:
| Short response:
|
| I agree it's a problem but it isn't incumbent on the 'x'
| peers to solve it. The burden of that goes to any supposed
| '10x'.
|
| Long version:
|
| I agree with you, though I would add that a superintellect
| at '10x' that couldn't look at the 'x' baseline of those
| around it and navigate that in an effective way (in other
| words, couldn't organize its thoughts and present them in a
| safe or good seeming way), is just plain not going to ever
| function at a '10x' level sustainably in an ecosystem full
| of normal 'x' peers.
|
| I think the whole point of Stranger in a Strange Land is
| about this. The Martian is (generally) not only completely
| ascendant, he's also incredibly effective at leveraging his
| ascendancy. Repeatedly, characters who find him abhorrent
| at a distance chill out as they begin to grok him.
|
| The reality is that this is an ecosystem of normal 'x'
| peers and the '10x', as the abnormality, needs to have
| "functional and effective in an ecosystem of 'x' peers" as
| part of its core skill set, or else none of us (not even
| the '10x' itself) can never recognize or utilize its
| supposed '10x' capacity.
| ganzuul wrote:
| That's what I meant, once you apply what happens in
| practice to the theory. It's a response to a comment
| about ego and cults so I tried to be as political as I
| can... which just isn't sufficient. My entire premise is
| that this subject is something familiar and controversial
| in a new guise so there is going to be a lot of knee-jerk
| reactions as soon as you bring up something that looks
| like a pain-point.
|
| For reference, I think most of us are '10x' in a
| particular field and that is our talent. Society-in-
| scarcity rewards talents unequally so we get status and
| ego resulting in a host of dark patterns. I think AI can
| ease scarcity so I keep betting on this horse for solving
| the real problem, which is ego.
| camgunz wrote:
| To focus on something I don't think gets a lot of play:
|
| > To me, the local minima looked "good"
|
| AI's entire business [0] is generating high quality digital
| content for free, but we've never ever ever needed help
| "generating content". For millennia we've sung songs and
| told stories, and we were happy with the media the entire
| time. If we'd never invented Tivo we'd be completely happy
| with linear TV. If we'd never invented TV we'd be
| completely happy with the radio. If we'd never invented the
| the CD we'd be completely happy with tapes. At every local
| minima of media, humanity has been super satisfied. Even if
| it were a problem, it's nowhere near the top of the list.
| We don't need more AI-generated news articles, music,
| movies, photos, illustrations, websites, instant summaries
| of research papers, (very very bad) singing. No one's
| looking around saying, "God there's just not enough
| pictures of fake waves crashing against a fake cliff". We
| need help with stuff like diseases and climate change. We
| need to figure out fusion, and it would be pretty cool if
| we could build the replicator (I am absolutely serious
| about the replicator). I remember a quote from long ago,
| someone saying something like, "it's lamentable that the
| greatest minds of my generation are focused 100% on getting
| more eyeballs on more ads". Well, here we are again
| (still?).
|
| So why do we get wave after wave of companies doing this?
| Advances in this area are insanely popular and create
| instant dissatisfaction with the status quo. Suddenly radio
| is what your parents listened to, fast-forwarding a
| cassette is super tedious, not having instant access to
| every episode of every show feels deeply limiting, etc.
| There's tremendous profits to be had here.
|
| You might be thinking, "here we go again, another
| 'capitalism just exploits humanity's bugs' rant", which of
| course I always have at the ready, but I want to make a
| different point here. For a while now the rich world has
| been _OK_. We reached an equilibrium where our agonies are
| almost purely aesthetic: "what kind of company do I want to
| work for", "what's the best air quality monitor", "should I
| buy a Framework on a lark and support a company doing
| something I believe in or do the obvious thing and buy an
| MBP", "how can I justify buying the biggest lawnmower
| possible", etc. Barring some big dips we've been here since
| the 80s, and now our culture just gasps from one "this
| changes everything" cigarette to the next. Is it Atari? Is
| it Capcom? Is it IMAX? Is it the Unreal Engine? Is it
| Instagram? Is it AI? Is it the Internet? Is it smartphones?
| Is it Web 2.0? Is it self-driving cars? Is it crypto? Is it
| the Metaverse and AR/VR headsets? I think us in the know
| wince whenever people make the leap from crypto to AI and
| say it's just the latest Silicon Valley scam--it's
| definitely not the same. But the truth in that comparison
| is that it _is_ just the next fix, we the dealers and
| American culture the junkies in a codependent catastrophe
| of trillions wasted when like, HTML4 was _absolutely fine_.
| Flip phones, email, 1080p, all totally fine.
|
| There is peace in realizing you have enough [1]. There is
| beauty and discovery in doing things that, sure, AI could
| do, but you can also do. There is joy in other humans.
| People listening to Hall & Oates on Walkmans teaching kids
| Spanish were just as happy (actually, probably a lot
| happier) as you are, and assuredly happier than you will be
| in a Wall-E future where 90% of your interactions are with
| an AI because no human wants to interact with any other
| human, and we've all decided we're too good to make food
| for each other or teach each other's kids algebra. It is
| miserable, the absolute definition of misery: in a mad
| craze to maximize our joy we have imprisoned ourselves in a
| joyless, desolate digital wasteland full of everything we
| can imagine, and nothing we actually want.
|
| [0]: I'm sure there's infinite use cases people can come up
| with where AI isn't just generating a six fingered
| girlfriend that tricks you into loving her and occasionally
| tells you how great you would look in adidas Sambas. These
| are all more cases where tech wants humanity to adapt to
| the thing it built (cf. self-driving cars) rather than
| build a thing useful to humanity now. A good example is
| language learning: we don't have enough language tutors, so
| we'll close the gap with AI. Except teaching is a
| beautiful, unique, enriching experience, and the only
| reason we don't have enough teachers is that we treat them
| like dirt. It would have been better to spend the billions
| we spent on AI training more teachers and paying them more
| money. Etc. etc. etc.
|
| [1]: https://www.themarginalian.org/2014/01/16/kurt-
| vonnegut-joe-...
| dontupvoteme wrote:
| >Frequency-pulled
|
| You mean like injection locking with oscillators? Or is this a
| new term in the tweetosphere
| ganzuul wrote:
| Injection locking. This:
| https://www.youtube.com/watch?v=e-c6S6SdkPo
|
| I mean it hides nuance in conversation.
| marricks wrote:
| It's also making sure AI is aligned with "our" intent and that
| "our" is a board made up of large corporations.
|
| If AI did run away and do it's own thing (seems super unlikely)
| it's probably a crapshoot as to whether what it does is worse
| than the environmental apocalypse we live in where the rich
| continue to get richer and the poor poorer.
| ben_w wrote:
| It can only be "super unlikely" for an AI to "run away and do
| it's own thing" when we actually know how to align it.
|
| Which we don't.
|
| So we're not aligning it with corporate boards yet, though
| not for lack of trying.
|
| (While LLMs are not directly agents, they are easy enough to
| turn into agents, and there's plenty of people willing to do
| that and disregard any concerns about the wisdom of this).
|
| So yes, the crapshoot is exactly what everyone in AI
| alignment is trying to prevent.
|
| (There's also, confusingly, "AI safety", which includes
| alignment but also covers things like misuse, social
| responsibility, and so on)
| root_axis wrote:
| "Run away" AI is total science fiction - i.e, not anything
| happening in the foreseeable future. That's simply not how
| these systems work. Any looming AI threat will be entirely
| the result of deliberate human actions.
| ben_w wrote:
| We've already had robots "run away" into a water feature
| in one case and a pedestrian pushing a bike in another,
| the phrase doesn't only mean getting paperclipped.
|
| And for non-robotic AI, also flash-crashes on the stock
| market and that thing with Amazon book pricing bots
| caught up in a reactive cycle that drove up prices for a
| book they didn't have.
| root_axis wrote:
| > _the phrase doesn 't only mean getting paperclipped._
|
| This is what most people mean when they say "run away",
| i.e. the machine behaves in a surreptitious way to do
| things it was never designed to do, not a catastrophic
| failure that causes harm because the AI did not perform
| reliably.
| ben_w wrote:
| Every error is surreptitious to those who cannot predict
| the behaviour of a few billion matrix operations, which
| is most of us.
|
| When people are not paying attention, they're just as
| dead if it's Therac-25 or Thule airforce base early
| warning radar or an actual paperclipper.
| root_axis wrote:
| No. Surreptitious means done with deliberate stealth to
| conceal your actions, not a miscalculation that results
| in a failure.
| ben_w wrote:
| To those who are dead, that's a distinction without a
| difference. So far as I'm aware, none of the "killer
| robots gone wrong" actual sci-fi starts with someone
| _deliberately aiming to wipe themselves out_ , it's
| always a misspecification or an unintended consequence.
|
| The fact that we don't know how to determine if there's a
| misspecification or an unintended consequence _is the
| alignment problem_.
| root_axis wrote:
| "Unintended consequences" has nothing to do with AI
| specifically, it's a problem endemic to every human
| system. The "alignment" problem has no meaning in today's
| AI landscape beyond whether or not your LLM will emit
| slurs.
| 8note wrote:
| To those who are dead, it doesn't matter if there was a
| human behind the wheel, or a matrix
| adverbly wrote:
| I think you're on to something, but to me it has more to do
| with being part of the set of issues that intersect political
| policy and ethics. I see it as facing the same "discourse
| challenges" as:
|
| abortion
|
| animal cruelty laws/veganism/vegetarianism
|
| affirmative action
|
| climate change(denial)
|
| These are legitimate issues, but it is also totally possible to
| just "ignore" them and pretend like they don't exist.
| ganzuul wrote:
| This time we have a genie in a lamp which will not be
| ignored. This should mean that a previously unknown variable
| is now set to "true" so discussion is more focused on
| reality.
|
| However the paranoid part of me says that these crises and
| wars are just for the sake of letting people continue to
| ignore the truly difficult questions.
| cma wrote:
| Did he sign the letter for keeping Sam?
| runjake wrote:
| Yes.
|
| https://twitter.com/janleike/status/1726599944029753750
| kamikaz1k wrote:
| who is this and why is it important? [1]
|
| super-alignment co-lead with Ilya (who resigned yesterday)
|
| what is super alignment? [2]
|
| > We need scientific and technical breakthroughs to steer and
| control AI systems much smarter than us. Our goal is to solve the
| core technical challenges of superintelligence alignment by 2027.
|
| [1] https://jan.leike.name/ [2]
| https://openai.com/superalignment/
| jvanderbot wrote:
| My honest-to-god guess is that it just seemed like a needless
| cost center in a growing business, so there was pressure
| against them doing the work they wanted to do.
|
| I'm guessing, but OpenAI probably wants to start monetizing,
| and doesn't feel like they are going to hit a
| superintelligence, not really. That may have been the goal
| originally.
| for_i_in_range wrote:
| I agree. Not everything has to be a conspiracy. Microsoft
| looked at a $10m+/year cost center, and deemed it unnecessary
| (which it arguably was), and snipped it.
| threetonesun wrote:
| I assume a lot of companies want in on the AI-to-purchase
| pipeline. "Hey [AI] what kind of car is this?" with a
| response that helps you buy it at the very high end, or
| something as simple as "hey [AI] I need more bread, it's
| [brand and type]" and who it gets purchased from and how it
| shows up is the... bread and butter of the AI company.
|
| Super intelligent AI seems contrary to the goals of
| consumerist Capitalism, but maybe I'm just not smart enough
| to see the play there.
| imzadi wrote:
| I think what companies want is to replace as many human
| employees as possible. I don't think they really care what
| the consequences of that are.
| icapybara wrote:
| This is the simplest explanation.
| mjr00 wrote:
| Yeah, OpenAI is all-in on the LLM golden goose and is much
| more focused on how to monetize it via embedding
| advertisements, continuing to provide "safety" via topic
| restrictions, etc., than going further down the AGI route.
|
| There's zero chance LLMs lead to AGI or superintelligence, so
| if that's all OpenAI is going to focus on for the next ~5
| years, a group related to superintelligence alignment is
| unnecessary.
| stewarts wrote:
| How can you be so certain there is 0 chance LLMs lead to
| AGI/Superintelligence? Asking curiously, not something I've
| heard prior.
| guhidalg wrote:
| I'm 100% certain that I need to do more than just predict
| the next token to be considered intelligent. Also call me
| when ChatGPT can manipulate matter.
| soulofmischief wrote:
| > Also call me when ChatGPT can manipulate matter.
|
| You mean like PALM-E? https://palm-e.github.io/
|
| Embodiment is the _easy part_.
| mypalmike wrote:
| Are you 100% certain that the human brain performs no
| language processing which is analogous to token
| prediction?
| guitarlimeo wrote:
| Pure LLM based approach will not lead to AGI, I'm 100%
| sure. A new research paper has shown [0] that no matter
| what LLM model is used, it exhibits diminishing returns,
| when you would be wanting at least a linear curve when
| looking for AGI.
|
| [0] https://www.youtube.com/watch?v=dDUC-LqVrPU
| sebzim4500 wrote:
| Based on the abstract this is about image models not LLMs
| guitarlimeo wrote:
| Ah fair point, should've read it more carefully.
|
| I'm tuning my probabilities back to 99%, I still don't
| believe just feeding more data to the LLM will do it. But
| I'll give the chance a possibility.
| barlines wrote:
| LLMs are gigantic curves fitted to civilizational scale
| datasets. LLM predictions are based on this. A language
| model is a mathematical construct and can only be as
| intelligent as that Algebra book sitting on your shelf.
| hollerith wrote:
| >LLMs are gigantic curves fitted to civilizational scale
| datasets
|
| >A language model is a mathematical construct
|
| That is like telling someone from the Middle Ages that a
| gun is merely an assemblage of metal parts not too
| different from the horseshoes and cast-iron nails
| produced by your village blacksmith and consequently it
| is safe to give a child a loaded gun.
|
| ADDED. Actually a better response (because it does not
| rely on an analogy) is to point out that none of the
| people who are upset over the possibility that most of
| the benefits of AI might accrue to a few tech titans and
| billionaires would be in the least bit re-assured by
| being told that an AI model is just a mathematical
| construct.
| dwaltrip wrote:
| An algebra book is a collection of paper pages with ink
| on them. An LLM is... nothing like that at all. LLMs are
| complex machines that operate on data and produce data.
| Books are completely static. They don't do anything.
|
| Do you have a better analogy? I'd like to hear more about
| how ML models _can 't_ be intelligent, if you don't mind.
|
| I'm pretty skeptical of the idea that we know enough at
| this point to make that claim definitively.
| andsoitis wrote:
| > Books are completely static. They don't do anything.
|
| Books (and writing) are a big force in cultural
| evolution.
| llamaimperative wrote:
| So you mean the things the charter foresaw and was intended
| to make impossible is in fact happening? Who could've thunk
| it (other than the creators of the charter and nearly anyone
| else with a loose grasp on how capitalism and technology
| interact).
| hollerith wrote:
| >it just seemed like a needless cost center in a growing
| business
|
| To some of us, that sounds like, "Fire all the climate
| scientists because they are needless cost center distracting
| us from the noble goal of burning as much fossil fuel as
| possible."
| mjr00 wrote:
| It's more like you started a company to research new fuel
| sources and hired climate scientists to evaluate the
| environmental impact of those new fuel sources, but during
| the course of the research you invented a new internal
| combustion engine that operates 1000% more efficiently so
| you pivoted your entire business toward that, removing the
| need for climate scientists.
|
| This is a tortured analogy, but what I'm getting at is, if
| OpenAI is no longer pursuing AGI/superintelligence, it
| doesn't need an expensive superintelligence alignment team.
| hollerith wrote:
| You're much more confident than I am that the researchers
| at OpenAI (or anyone else currently alive) are masters of
| their craft to such an extent that they _would even be
| able to predict_ whether the next big training run they
| do will result in a superintelligence or not. Another way
| of saying the same thing is to say that the only way
| anyone knows that GPT-4 is not dangerously capable is
| that it has been deployed extensively enough by now that
| if it was going to harm us, it would 've done so by now:
| not even the researchers that designed and coded-up GPT-4
| or watched it during training could predict with any
| confidence how capable it would be. For example, everyone
| was quite surprised by its being able to score in the
| 90th decile on a bar exam.
|
| Also, even if they never produce a superintelligence,
| they are likely to produce insights that would make it
| easier for other teams to produce a superintelligence.
| (Since employees are free to leave OpenAI and join some
| other team, there is no practical way to prevent the flow
| of insights out of OpenAI.)
| mariusor wrote:
| > if OpenAI is no longer pursuing AGI/superintelligence
|
| What leads you to believe that's true?
| mjr00 wrote:
| Microsoft dumping $10 billion into them to commercialize
| LLM tech, primarily.
| mariusor wrote:
| If anything, in my opinion, the more runway they have,
| the better chances to actually hit an inflexion point in
| AGI development. But maybe you're right.
| root_axis wrote:
| You can't build a business on AGI, which is an unbounded
| research project without any foreseeable end or path to
| revenue. However, LLMs definitely have some commercial
| value and OpenAI has a first-mover market advantage that
| they'd be insane to squander from a business perspective.
| I'm sure they will continue to do research in
| advancements of AI, but AGI is still science fiction.
| legohead wrote:
| What is the "intelligence" behind a word predictor?
| TiredOfLife wrote:
| > We need scientific and technical breakthroughs to steer and
| control AI systems much smarter than us. Our goal is to solve
| the core technical challenges of superintelligence alignment by
| 2027
|
| Can somebody translate this to human?
| brandall10 wrote:
| That by 2027 they will figure out how to control Skynet so it
| doesn't kill us all when it awakens.
| colibri727 wrote:
| What are these core technical challenges ?
| willsmith72 wrote:
| Why can't openai just be a competent drama-free org?
|
| Is it impossible to build really cool tech without the upheaval?
| CSMastermind wrote:
| Probably because if you collect a bunch of highly motivated and
| extremely smart people who are good at overcoming obstacles and
| eliminating problems they eventually start to see each other as
| obstacles or problems.
| llm_trw wrote:
| Having been on founding teams that imploded like open AI and
| ones that succeeded I've found that waving a big wad of cash
| before every meeting does wonders to motivate people.
|
| >This is what we all want, and we want it as fast as
| possible. Let's get through this together and enjoy our
| tropical islands alone.
|
| You can trust people's greed, you can't trust their morals.
| Anyone who says otherwise hasn't seen a large enough wad of
| cash yet.
| baobabKoodaa wrote:
| This is a bit confusing in the context of OpenAI drama. The
| employees at OpenAI have never really lacked monetary
| incentives. It seems that you... _can 't_ trust people's
| greed when they have ulterior motives like "safe AGI".
| llm_trw wrote:
| Open AI was founded on morals. It was a not for profit
| there for the benefit of humanity. Then M$ came around
| with a lot of cash and we got the soap opera we're seeing
| here.
| baobabKoodaa wrote:
| Right. But everybody didn't play along nicely, did they?
| Even though MS flaunted wads of cash in front of their
| faces. So we can't really trust everybody to choose greed
| over morals or whatnot.
| chambers wrote:
| > You can trust people's greed, you can't trust their
| morals.
|
| I've lived this experience. It's a time old tale. Judas
| Iscariot sold out his best friend for a few silver pieces.
|
| However, I've learned I can trust someone's morals so long
| as it comes from outside themselves. In small ways, that
| outside is society, school, company or circumstance. But I
| think the big one is more important, i.e., Jeremiah 17:9-10
| Nullabillity wrote:
| You can trust greed to produce _something_. You can also
| trust that something to be utterly worthless, at best.
| makk wrote:
| > Why can't openai just be a competent drama-free org?
|
| Because it is run by humans, for the time being.
| jlarocco wrote:
| That's funny.
|
| If OpenAI's tech is so great, why aren't they using it to
| direct the company yet?
| Apocryphon wrote:
| This sort of thing happens even in FOSS projects all of the
| time, you can't have people without drama. And perhaps you
| won't have AI without drama, either.
| llm_trw wrote:
| Foss projects are drama. I've found it much worse in the non-
| profit space than the for profit space. Which openai showed
| us ironically.
| jejeyyy77 wrote:
| because these are 2 people who are at the top of the field and
| can work on literally anything they want.
| motoboi wrote:
| You can't remove politics (human beings interacting) from human
| beings interactions.
| JimDabell wrote:
| Where are you seeing the drama? Two people have left a startup,
| that's all. It happens all the time.
| gaudystead wrote:
| If you think those are the only points of drama, you may not
| have seen news about OpenAI from a couple months ago where
| Sam got ousted from the board, got in bed with Microsoft,
| then eventually came back to OpenAI, just to name one of the
| bigger dramas off the top of my head. It's definitely more
| than just the two data points from yesterday that you
| mentioned.
| jimberlage wrote:
| Less powerful inventions have caused more drama than this,
| historically speaking.
| summerlight wrote:
| You have limited resources on both computations and
| engineering. And everyone have different, conflicting
| priorities. The group needs to make a decision where to put
| their resources, so there is politic.
| dr_ wrote:
| This was from the NYTimes yesterday (5/14) " Jan Leike, who ran
| the Super Alignment team alongside Dr. Sutskever, has also
| resigned from OpenAI. His role will be taken by John Schulman,
| another company co-founder."
| nicklecompte wrote:
| It is easy to point to loopy theories around superalignment,
| p(doom), etc. But you don't have to be hopped up on sci-fi to
| oppose something like GPT-4o. Low-latency response time is fine.
| The faking of emotions and overt references to Her (along with
| the suspiciously-timed relaxation of pornographic generations)
| are not fine. I suspect Altman/Brockman/Murati intended for this
| thing to be dangerous for mentally unwell users, using the exact
| same logic as tobacco companies.
| simonw wrote:
| I don't understand what you're referring to with that tobacco
| reference.
| ChicagoBoy11 wrote:
| Not the parent comment, but I think he means something like
| "we know folks will be addicted to this pseudo-person and
| that is a good thing cause it makes our product valuable",
| akin to reports that tobacco companies knew the harms and
| addictive nature of their products and kept steadfast
| nonetheless. (But I'm speculating as to the parent's actual
| intent)
| rvnx wrote:
| I miss Sydney :'
| all2 wrote:
| I'm confused. Context?
| scrollaway wrote:
| Sydney was an early version of Bing GPT that was more
| than a little nuts.
| atlasunshrugged wrote:
| I read it as the economics of tobacco (and alcohol and a few
| other 'vice' industries) that there will invariably be
| superusers who get addicted and produce the most economic
| value for companies even while consuming an actively harmful
| product
| renewedrebecca wrote:
| Purposely making an addiction machine, most likely.
| incahoots wrote:
| I mean it's par for the course. What better business model
| exists than the turning a want into a need. Caffeine comes
| to mind
| nicklecompte wrote:
| https://nida.nih.gov/publications/research-
| reports/tobacco-n... A larger proportion of
| people diagnosed with mental disorders report cigarette
| smoking compared with people without mental disorders. Among
| US adults in 2019, the percentage who reported past-month
| cigarette smoking was 1.8 times higher for those with any
| past-year mental illness than those without (28.2% vs.
| 15.8%). Smoking rates are particularly high among people with
| serious mental illness (those who demonstrate greater
| functional impairment). While estimates vary, as many as
| 70-85% of people with schizophrenia and as many as 50-70% of
| people with bipolar disorder smoke.
|
| I am accusing OpenAI (and Phillip Morris) of knowingly
| profiting off mental illness by providing unhealthy solutions
| to loneliness, stress, etc.
| djohnston wrote:
| I've also heard of studies, admittedly I don't have the
| link on hand, that schizophrenic patients benefit from
| smoking. When did big tobacco actively target them? How do
| you know these people don't naturally seek out cigarettes
| as a means to manage some of their symptoms?
| nicklecompte wrote:
| I have schizophrenia. I have struggled with nicotine
| addiction since high school. In 2015 I had three heart
| attacks in a month, even though I was only 28 and seemed
| physically fit. Two weeks ago I had a minor stroke.
|
| It is not just me and it is not just the smoking: https:/
| /www.cambridge.org/core/blog/2020/08/19/physically-he...
| We have known for many years that people who suffer from
| schizophrenia die younger than expected, as much as 20
| years younger than the general population. This appears
| unfair, and it was the inspiration of this work. Most
| people thought that this added risk of death was mostly
| due to the higher prevalence in schizophrenia of smoking,
| obesity and to other lifestyle differences.
| For this reason, we recruited 40 patients with
| schizophrenia and an equal number of healthy controls,
| and scanned their hearts using a state-of-the-art
| approach, called cardiac magnetic resonance. This was
| performed at the state-of-the-art Robert Steiner MRI
| unit. [...] Surprisingly, in our study we
| found that even after matching patients and healthy
| controls for age, sex, ethnicity and body mass index
| (BMI, deriving from height and weight); and after
| excluding any participants with any medical conditions,
| and other risk factors for heart disease, people with
| schizophrenia show hearts that are smaller and chunkier
| than controls. These changes are similar to those found
| in aging.
|
| I was able to move to the gum and patch, but I am very
| high-functioning. People sicker than me have less
| options. Smoking is very bad for everyone, including
| people for schizophrenia. We do not in any way benefit
| from terrible heart/lung damage in exchange for minor
| cognitive clarity - our hearts need all they help they
| can get. I have no tolerance for this sort of ignorant
| paternalism, and I'm ignoring your bad-faith question
| about "actively target them" because that's not what I
| said.
| bambax wrote:
| Oh well... It seems at least one of those two things have to be
| true: either AGI is so far away that "alignment" (whatever it
| means) is unnecessary; or, as you suggest, Altman et al. have
| decided it's a hindrance to commercial success.
|
| I tend to believe the former, but it's possible those two
| things are true at the same time.
| nicklecompte wrote:
| Specifically I am supposing the superalignment people were
| generally more concerned about AI safety and ethics than
| Altman/etc. I don't think this has anything to do with
| superalignment itself.
| Liquix wrote:
| or C) the first AGI was/is being/will be carried away by men
| with earpieces to a heavily fortified underground compound.
| any government - let alone the US government - isn't going to
| twiddle their thumbs while tech that will change human
| history is released to the unwitting public. at best they'll
| want to prepare for and control the narrative surrounding the
| event, at worst AGI will be weaponized against humans before
| the majority are aware it exists.
|
| if OAI is motivated by money, uncle sam can name any figure
| to buy them out. if OAI is motivated by power, it becomes "a
| matter of national security" and they do what the gov tells
| them. more likely the two parties' interests are aligned and
| the public will hear about it when _It 's Time(tm)_. not
| saying C) is what's happening - A) seems likely too - but
| it's a real possibility
| wins32767 wrote:
| Why do you think that the US government has the state
| capacity to do anything like that these days?
| Gud wrote:
| By observing reality.
| ToucanLoucan wrote:
| The use of LLM's as pseudo-friends or girlfriends for people as
| a market solution for loneliness is so incredibly sad and
| dystopian. Genuinely one of the most unsettling goddamn things
| I've seen gain traction since I've been in this industry.
|
| And so many otherwise perfectly normal products are now
| employing addiction mechanics to drive engagement, but somehow
| this one is just even further over the line for me in a way I
| can't articulate. I'm so sick of startups taking advantage of
| people. So, so fucking gross.
| swatcoder wrote:
| It's a technological salve that gives individuals a minor and
| imperfect remedy for a profound failure in modern society.
| It's of a kind with pharmaceutical treatments for depression
| or anxiety or obesity -- best seen as a temporary "bridge"
| towards wellness (achieved, perhaps, through other
| interventions) -- but altogether just trying to help troubled
| individuals navigate a society that failed to enable their
| deeper wellness in the first place.
| reducesuffering wrote:
| These type of techno-solutions are some of the root cause
| of those "profound failure of modern society!" The
| technological salve is just a further extreme causing these
| people's problems! Much like there exists some societal
| problems, alcohol is a tiny relief, but can further
| exacerbate those problems, and you advocate that they drink
| even more alcohol because society has issues and they
| should escape it.
|
| Idk how we've gotten away from such a natural human
| experience, but everyone knows damn well that the happiest
| children are out playing soccer with their friends in a
| field or eating lunch together at a park bench, and not
| holed up in their room watching endless YouTube.
| swatcoder wrote:
| I don't disagree in the least. I'm just saying it's the
| in same bucket as many commercial products that are
| designated as therapeutic and that they should all be
| looked at with a similar kind of celebration/skepticism.
| ToucanLoucan wrote:
| I think celebration of any sort should be belayed until
| we have actual evidence of these things having positive
| effects on people. Like this is just me reacting as a
| human to a human issue but: a fake friend in an LLM is
| not a friend. It's never going to crawl out of the phone
| and help you put the donut on your car when your get a
| flat tire. It's not going to take you out for a drink if
| you go through a rough breakup. It's not going to have
| difficult conversations with you and call you out on your
| bullshit because it cares about you.
|
| LLM friends have the same energy to me as video game
| progression: it's a homeopathic version of a real thing
| you need, social activation and achievement respectively.
| But like homeopathy, you don't actually get anything out
| of it. The placebo effect will make the symptoms of your
| lack feel better, for awhile, but it will never be
| _solved_ by it, and because of that whatever is selling
| you your LLM girlfriend or phony achievement structure
| will never lose you as a customer. I 'm suspicious of
| that.
| fullshark wrote:
| I'll just point to the theory that they didn't want to work for
| a megacorp creating tools for other megacorps (or worse) and
| actually believed in OpenAI's (initial) mission to further
| humanity. The tools are going to be used by deep pocketed
| entities for their purposes, the compute resources necessary
| require that to be the case for the foreseeable future.
| lghh wrote:
| > along with the suspiciously-timed relaxation of pornographic
| generations
|
| Has ChatGPT's censoring (a loaded term, but idk what else to
| use) been relaxed with GPT-4o? I have not tested it because I
| wouldn't have expected them to do it. Does this also extend to
| other types of censorship or filtering they do? If not, it
| feels very intentional in the way you're alluding to.
| ziml77 wrote:
| I don't see anything that says they've changed their policies
| yet. Just that they're looking into it. I also tested 4o and
| it still gives me a content policy warning for NSFW requests.
| nicklecompte wrote:
| Sure, I was being sloppy, I meant "suspiciously timed
| announcement."
| bnralt wrote:
| One could also say that therapists prey on lonely people who
| pay them to talk to them and seem like they're genuinely
| interested in them, when the therapist wouldn't bother having a
| connection with these people once they stop paying. Which I
| suppose is true from a certain point of view. But from another
| point of view, sometimes people feel like they don't have close
| friends or family to talk to and need something, even if it's
| not a genuine love or friendship.
| kettro wrote:
| This is implying that therapy is nothing more than someone to
| talk to; if that's your experience with therapy, then you
| should get another therapist.
| baobabKoodaa wrote:
| Evidence points in this direction, though.
|
| Different methods of therapy appear to be equally effective
| despite having theoretical foundations which are
| conflicting with each other. The common aspect between
| different therapies seems to be "having someone to talk
| to", so I'm inclined to believe that really is what's
| behind the success.
| pphysch wrote:
| Having someone to talk to, who is somewhat emotionally
| intelligent, who doesn't have strong biases against you,
| and so on...
|
| If you are fortunate, you have people like that in your
| immediate circle, but increasingly few people do.
| AmericanChopper wrote:
| What part of the therapist training regimen tests for
| emotional intelligence? What test do they use to measure
| this?
| pdabbadabba wrote:
| They don't attempt to measure it, but they do teach
| approaches like "unconditional positive regard" and other
| techniques that allow a practitioner to demonstrate (or
| at least seem to demonstrate) a higher level of emotional
| intelligence.
|
| A big part of therapy is also rapport. Many people go
| through many therapist before finding one that works for
| them. In part, you can think of this as the market
| performing the assessment your'e referring to.
| AmericanChopper wrote:
| They don't attempt to measure it because it not something
| that's even properly defined with any rigour. Any person
| who seriously uses the phrase is going to have their own
| completely individual idea of what it means, and there's
| no reason the think any therapist would have this
| nebulous quality, or even that their idea of what it
| means has any similarity to your idea of what it means.
| pdabbadabba wrote:
| I suppose I agree -- "emotional intelligence" is probably
| not the word I would have used, writing on a blank slate.
| I think the idea is better captured in the concept of
| rapport, which is really just a function of clients'
| subjective experience working with a given therapist. A
| therapist can learn techniques to increase the chances of
| establishing a good rapport with a given client, but I'd
| be inclined to leave it at that.
| detourdog wrote:
| It is covered in the curriculum. They study emotional
| intelligence and with luck they are able to self-reflect
| using their education.
|
| Actual maladaptive personalities are the result of low
| emotional intelligence.
| pas wrote:
| > "having someone to talk to"
|
| it's a bit more complicated than that
|
| https://www.youtube.com/watch?v=Z37i8-FnAh8
|
| and on top of this the method of therapy is to find
| better copings, not just to vent.
| baobabKoodaa wrote:
| I'm not going to watch a 45 minute video in an effort to
| decipher what you are implying with this comment.
| pas wrote:
| I'm implying that data shows that efficacy of therapy
| depends on concrete factors (~5 of them discussed in the
| video). it's not "just someone to talk to".
| baobabKoodaa wrote:
| Thank you.
| kridsdale1 wrote:
| You could dump the YouTube link in to Gemini Advanced and
| ask it for the point.
| cbsmith wrote:
| > Evidence points in this direction, though.
|
| >
|
| > Different methods of therapy appear to be equally
| effective despite having theoretical foundations which
| are conflicting with each other. The common aspect
| between different therapies seems to be "having someone
| to talk to", so I'm inclined to believe that really is
| what's behind the success.
|
| Just because talking is the common trait, doesn't mean
| that that's evidence that that is all it is. Paying
| someone to help you with the problem is also a common
| trait (and ironically, that is, no doubt, a contributory
| factor), but that isn't all that therapy is.
|
| Let's say that there are three ways to solve a problem,
| and depending on context that we're not terribly good at
| determining, one of those ways will work quite often, one
| will work some of the time, and the other will be a
| disaster... but there's an equal probability that each of
| those ways are equally likely to fall in to each of those
| categories. Statistically, one could claim that how you
| solve the problem is not behind the success. In a sense,
| that would be correct, because the real determinant of
| success would be being lucky with the solution you chose
| to employ. While one could imply though that really it's
| nothing more than being lucky at choosing the solution,
| in reality without all of what's involved in that choice,
| the problem will remain.
| pdabbadabba wrote:
| There may be a kernel of truth in this, but it depends on
| why you're seeing a therapist. For treatment of OCD, for
| example, or phobias, there are specific protocols that
| yield results, but they do not respond to just "having
| someone to talk to."
|
| Other kinds of conditions, like depression and anxiety,
| respond to a wider range of therapy styles. But those
| aren't the only conditions that people seek to treat
| through talk therapy. (And it's also an exaggeration to
| say that just having _any_ conversation will help to
| treat anxiety and reopression. But it is probably true
| that treatment of these conditions is less technical and
| responds to a much wider range of styles.)
| burnte wrote:
| > Different methods of therapy appear to be equally
| effective despite having theoretical foundations which
| are conflicting with each other. The common aspect
| between different therapies seems to be "having someone
| to talk to", so I'm inclined to believe that really is
| what's behind the success.
|
| This isn't true. Different methods work better for
| different problems. I've been in behavior health for 7
| years now. It's having someone with a lot of education to
| talk to, someone with education in social and
| psychological problems and healthy coping mechanisms.
| lo_zamoyski wrote:
| From what I understand, therapy success rates are quite
| low, with only cognitive behavior therapy showing notable
| progress. That isn't to say all the others are
| categorically useless, only that in the majority of
| cases, they seem to be ineffective or harmful.
| bnralt wrote:
| It's implying that this is the case for many people, not
| all. Which it is, in my experience. Particularly since the
| advice you gave:
|
| > then you should get another therapist
|
| Seems to be fairly ubiquitous. "Find a therapist you
| like"/"shop around"/etc. leads a lot of people to find
| people who will tell them what they want to hear. Sometimes
| what people want to hear is how to practice CBT - but in
| that case, such people are probably going to be using AI to
| work on CBT.
| helboi4 wrote:
| Yeah I have found there is very little you get from
| therapy that you can't get from a mixture of journalling,
| learning CBT methods, having a routine (which includes
| regular exercise) and trying lots of different methods of
| making friends that you assess maturely for their
| reliability. Maybe meditation if you're into that. All of
| these things are free and require effort, personal effort
| and intention being what will actually improve your life
| anyway, whether you use therapy or not. This makes
| therapy seem like a scam for anything other than dealing
| with a very dire short period of isolation.
| naasking wrote:
| I think the preying part of therapy is that there's just no
| defined stop condition. There's no such thing as "healthy"
| in mental health. You get chemo until you go into remission
| or you die. You take blood pressure meds until you have a
| better lifestyle and body composition and don't need them
| anymore, etc. There's no analogue for "you're healthy now,
| go away so I can help others", and so therapy goes on
| forever until the patient stops for whatever reason.
| cbsmith wrote:
| > I think the preying part of therapy is that there's
| just no defined stop condition.
|
| There's no defined stop point for physical development
| either... Top performing athletes still have trainers,
| and nobody sees that as a problem. If it's mental
| development though, it must have a stop point?
| detourdog wrote:
| The stop point is obvious to the individual and the
| therapist. I'm dealing with someone that prefers to stop
| rather than actually self-reflection.
|
| There is nothing stopping them from exiting therapy. The
| therapist may be aware that the person is still a basket
| case but if they are non-violent they are free to roam.
| cbsmith wrote:
| Exactly.
| naasking wrote:
| > The stop point is obvious to the individual and the
| therapist.
|
| It's not at all. There are plenty of stories of people
| who realized that therapy was just causing them to
| ruminate on their problems, and that the therapist was
| just milking them for _years_ before they wised up and
| walked away. That 's not what I call "obvious".
| detourdog wrote:
| I think either the therapist or the patient was not
| devoted to therapy.
| naasking wrote:
| No true Scotsman eh? Classic for a reason!
| detourdog wrote:
| Considering we are discussing a licensed professional I
| think your argument is weak. Second I allowed for the
| failure of the therapist in their duty.
| naasking wrote:
| The trustworthiness of a license in a field with a poor
| replication rate and whose best therapy is at best 50%
| effective is what's weak.
| naasking wrote:
| What a ridiculous analogy. "Athlete" is a career. Is
| someone making a career of being in therapy?
|
| > If it's mental development though, it must have a stop
| point?
|
| What is being developed, exactly?
| cbsmith wrote:
| > What a ridiculous analogy. "Athlete" is a career.
|
| The athlete is the extreme example, but there are
| obviously people who are not career athletes that don't
| have a defined stop point with employing a trainer (maybe
| you could say "death" is the stop point).
|
| Most everyone who goes to spinning class isn't a career
| athlete. Some of them are terribly out of shape, and some
| of those people just want to get in shape. Others may
| already be in shape, but see the spinning class as a way
| to either improve or maintain their conditioning. None of
| this is deemed ridiculous.
|
| I'm curious, it's considered the norm to regularly see a
| doctor or dentist, do you think they're preying on their
| patients?
|
| > What is being developed, exactly?
|
| Mental health. There's obviously a more involved answer,
| but if you don't know it already, it's unlikely I'll be
| able to educate you with a comment on social media.
| naasking wrote:
| > but there are obviously people who are not career
| athletes that don't have a defined stop point with
| employing a trainer
|
| And many of them are being bilked as well. The fitness
| industry is notoriously filled with hucksters and scams,
| and "trainers" rarely have any real training in
| kinesiology or exercise science.
|
| > I'm curious, it's considered the norm to regularly see
| a doctor or dentist, do you think they're preying on
| their patients?
|
| Once a year for a health checkup. Is that the norm for
| therapy?
|
| > Mental health. There's obviously a more involved answer
|
| The more involved answer is that "mental health" is not
| well-defined, so it's not developing anything. The only
| therapies that have shown to have any empirical validity,
| like CBT, train the user in tools to change their own
| behaviour and thinking, then it's on the user to employ
| the tools. Does a family doctor call you in once a week
| and watch you take the pills that address your physical
| ailment?
|
| The best analogy for psychiatric therapy is physical
| therapy for recovering from an injury or surgery, except
| _physical therapy has a well-defined end condition_ ,
| which is when you understand how to do the exercises
| yourself. Then it's on you to do them. This is just not
| the norm for "mental health" therapy.
| sdwr wrote:
| Nail on the head, thanks. I'm deeply uncomfortable with
| anything that combines paying for a service with a social
| element. Feels like an unstable equilibrium.
|
| I guess the skill is riding the line, but that doesn't
| feel very enjoyable.
| renewiltord wrote:
| Top performing athletes are better than me at being
| athletes.
|
| Meanwhile, the more therapy someone does, the more
| miserable they are compared to me. I'm the Usain Bolt of
| mental health compared to them. Makes me think their
| trainer is an idiot.
| jodrellblank wrote:
| I sometimes recommend Dr David Burns' Feeling Good
| podcast[1], and he is big on measuring and testing and
| stop points. Instead of 'tell me about your mother' his
| style of Cognitive Behavioural Therapy (CBT) is called
| TEAMS in which the T stands for Testing, and it involves:
|
| - Patient choosing a specific mood problem/feeling they
| want to work on.
|
| - A mood survey, where the patient rates their own level
| of e.g. anxiety, depression, fear, hopelessness. (e.g.
| out of 5 or 10).
|
| - Therapy session, following his TEAMS CBT structure.
| Including patient choosing how much fear they'd like to
| feel (e.g. they want to keep a little bit of fear so they
| don't endanger themselves, but don't want to be
| overwhelmed by fear, 5% or 20%, say).
|
| - A repeat of the mood survey, where the patient re-
| assesses themselves to see if anything has improved.
| There's no units on the measures because it's self-
| reported, the patient knows if the fear is unchanged, a
| little less, a lot less, almost gone, completely gone,
| and that's what matters.
|
| That gives them feedback; if there is improvement within
| a session they know something in the session helped, if
| several sessions go by with no improvement they know it
| and can change things up and move away from those
| unhelpful approaches in future with other patients, and
| if there is good improvement - patient is self-reporting
| that they are no longer hopeless about their relationship
| status, or afraid of social situations, or depressed, to
| the level they want, then therapy can stop.
|
| He's adamant that a single 2hr session is enough to make
| a significant change in many common mood disorders[2],
| and this "therapy needs to take 10 years" is a bad
| pattern and therapists who don't take mood surveys and
| before and after every session are flying blind. With
| feedback on every session and decades of experience, he
| has identified a lot of techniques and ways to use them
| which actually do help people's moods change. I liken it
| to the invention of test cases and debuggers (and looking
| at the output from them).
|
| [1] Quick list: https://feelinggood.com/list-of-feeling-
| good-podcasts/ more detailed database:
| https://feelinggood.com/podcast-database/
|
| [2] no, internet cynic, obviously not everything and
| presumably not whatever it is you have.
| naasking wrote:
| I agree some therapists are starting to come around on
| this, but even what you describe is somewhat flawed due
| to placebo effects, eg. some always feel better from any
| kind of talk, probably as a result of someone paying
| attention to their problems.
|
| You might be able to overcome this effect by
| quantitatively tracking this across many sessions to some
| degree, but I think it's still always the patient that
| has to walk away, and never the therapist who says,
| "you're good, go on now".
| fatbird wrote:
| I think you're mistaken, at least in a lot of cases. All
| CBT based therapies I've had have started with a clear
| discussion about what the problem is, and what the
| solution looks like in terms of my happiness and mental
| well-being. In all cases, my therapist has "graduated"
| me, telling me that they don't think I need to continue
| (or having me say that I'm comfortable now stopping
| regular therapy).
|
| CBT and its derivatives very strongly attend to
| individual effectiveness and view therapy that goes on
| endlessly as a sign that the real problem isn't being
| addressed, and that no therapy is considered effective
| unless it ends. Individual therapists might be bad
| actors, but the field itself is now admirably focussed on
| finite, positive results.
| fatbird wrote:
| This is very true, and I would add to it that the dominant
| paradigm in most therapy these days (at least those forms
| coming from a Cognitive Behavioural Therapy background)
| have "graduation" as an explicit goal: the client feels
| like they've addressed what they want to address and no
| longer need the ongoing relationship.
|
| This is largely due to a crisis in the field in the late
| 70s/early 80s when several studies demonstrated that talk
| therapy had outcomes no different than no therapy. In both
| cases, some got better, some got worse, some didn't change.
| CBT was a direct result of that, prioritizing and tracking
| positive outcomes, and from CBT came a lot of different
| approaches, all similarly focussed on being demonstrably
| effective.
|
| Talk therapy isn't a cure-all, but it's definitely more
| results-oriented than it was 50 years ago.
| solardev wrote:
| I feel like this is easier said than done. There's not a
| great way (that I know of) to evaluate the
| quality/potential helpfulness of therapists... if only
| there were a Steam-like review system for them! There's
| ratemds.com, but not a lot of people use it, since there's
| not a central marketplace to find therapists to begin with
| (that I know of). I would love to be able to find good
| therapists locally and/or online. It just seems like such
| an expensive gamble every time.
|
| When I was younger, I went through many therapy sessions
| with multiple professionals of different kinds
| (psychologists, psychiatrists, MFT (marriage and family
| therapists, social workers, etc.).
|
| A couple of them were wonderful: thoughtful, caring,
| helpful, providing useful guidance with a compassionate
| ear.
|
| Another couple tried to be helpful but were still in
| training themselves (this was at a college) and couldn't
| really provide any useful guidance.
|
| One was going through a divorce of her own at the time and
| ended up crying in many of our sessions and having to abort
| them to deal with her own emotions - it was a tough time
| for her, and she's only human. I often tried to console
| her, but she wouldn't let me, so it made for a very awkward
| situation lol.
|
| One of them had one a single session with me, charged me
| for it, and then told me she couldn't help me and to go
| somewhere else.
|
| But the worst of them was an older guy who, despite the
| referrals and my history, thought I was faking mental
| illness. He dared me to attempt suicide, and when I
| eventually did (not because of him, but a separate romantic
| failure), he chuckled in my face and said, "Heh, you
| finally tried it, huh? Didn't think you would." This was an
| older psychiatrist in a small town - either the only one
| there, or one of very few - the kind of sleazy place that
| had a captive market and a whole bunch of pharma ads in the
| lobby, with young female pharma reps going in and out all
| day. What a racket =/ If I were wiser then, I would've
| reported him to the board and news media.
|
| So, anecdotally, my success rate with therapists was only
| 2/7. To be fair, I was a pretty fucked up teenager and
| young adult, but still... the point is that "just find a
| better therapist" is often a difficult process. Depending
| on your insurance and area, there may not even be any other
| therapists with a waiting list of less than a few months,
| and even if you can get in, there's no guarantee they are
| good at their jobs AND a good fit for your personality and
| issues.
|
| Think it's hard to find good devs? At least our line of
| work produces some measurable output (software/apps that
| run, or not, according to specs). How do you even measure
| the output of a therapist? Improvements to someone's life
| aren't going to happen overnight, and many never report
| back; the best successes may not bother to leave a review,
| the worst failures may end up dead before spreading the
| word. The rest probably just run out of sessions allowed by
| their insurance and try to move on with their lives, with
| unknown levels of positive or negative change.
| moralestapia wrote:
| GP's not saying that, what GP is saying is "good luck
| trying to talk to your therapist if you stop paying $$$".
|
| I do think therapists are one of the professions that will
| be naturally displaced by LLMs. You're not paying them to
| be your friend (and they are usually very clear on that),
| so any sort of emotional connection is ruled out. If
| emotions are taken away, then it's just an input/output
| process, which is something LLMs _excel_ at.
| fatbird wrote:
| I would argue the opposite: a good therapist isn't just
| offering back-and-forth conversation, they're bringing
| knowledge, experience and insight into the client after
| interacting with them. A good therapist understands when
| one approach isn't working and can shift to a different
| one; they're also self-reflective and very aware of how
| they're influencing the situation, and try to apply that
| intelligently. This all requires reflective and
| improvisational reasoning that LLMs famously can't do.
|
| Put another way, a good therapist is professionally
| trained and consciously monitoring whether or not they're
| _misleading_ you. An LLM has no executive function acting
| as a check on its input /output cycle.
| smt88 wrote:
| Therapists are educated and trained to help alleviate mental-
| health issues, and their licenses can be revoked for
| malpractice. Their livelihood partially depends on ethics and
| honest effort.
|
| None of those safeguards are in place for AI companies.
| graphe wrote:
| And most of them suck. Imagine if you bought a class of
| hardware that needed to be swapped constantly, has no QC
| and dubious reviews, with many "works on my machine"
| comments saying it worked for them. Little did they know it
| just reported it was fine and was also broken for them.
|
| The liability for a bad therapist or psychologist is very
| low, I have never heard of them getting revoked for being
| bad therapists. If if was true then bad therapists wouldn't
| exist. I would not be surprised if they ceased to exist in
| the near future with AI being consistent, and much better
| quality.
| smt88 wrote:
| "Works on my machine" for therapists is not a bug or a
| problem. People's needs are highly individual, and the
| best therapist will be, too.
| hn_throwaway_99 wrote:
| Exactly. I know there has been research on the
| effectiveness on different types of talk therapy and _by
| far_ the most important factor (much more than any
| specific theory the practitioner uses) is the "fit"
| between therapist and patient.
| tech_ken wrote:
| > One could also say that therapists prey on lonely people
| who pay them to talk to them
|
| It is indisputable that one could say this
| mistrial9 wrote:
| ok - you could say "the rapist" too.. many have.. guess
| what, people in crisis sometimes attack the first line
| helpers.. this is well known among trained health
| professionals
| detourdog wrote:
| Ideally a therapist is an uninvolved neutral party in one's
| life. They act as sounding board to measure one's internal
| reactions to the outside world.
|
| The key is neutral point of view. Friend's and family come
| with bias. The bias's can be compounded by mentally ill
| friend's and family.
|
| Therapist must meet with other therapists about their patient
| interactions. The second therapist acts a neutral third party
| to the keep the first therapist from losing their neutrality.
|
| That is the ideal and the real world may differ.
|
| I'm struggling with someone that looks to be having some real
| mental issues. The person believes I'm the issue and I need
| to maintain a therapist to make sure I'm treating this person
| fairly.
|
| I need a neutral third party that I gossip with that is bound
| to keep it to themselves.
| hn_throwaway_99 wrote:
| > One could also say that therapists prey on lonely people
| who pay them to talk to them and seem like they're genuinely
| interested in them, when the therapist wouldn't bother having
| a connection with these people once they stop paying.
|
| As another commenter said, if that's your experience with a
| therapist, you have a shitty therapist and should switch.
|
| Most importantly, a _good_ therapist will very clearly
| outline their role, discuss with you what you hope to
| achieve, etc. I 've been in therapy many years, and I know
| exactly what I'm paying for. Sure, some weeks I really do
| just need someone to talk to. But never have I, or my
| therapist, been unclear that I am paying for a service, and
| one that I value much more than just having "someone to talk
| to".
|
| Using the terminology "prey on lonely people" is ridiculous
| (again, for any good therapist). If they were actually
| preying on me, then their goal would be to _keep_ me lonely
| so I become dependent on them (and I 'm not saying that never
| happens, but when it does it's called malpractice). A good
| therapist's entire goal is to make people self-sufficient in
| their lives.
| llm_trw wrote:
| >dangerous for mentally unwell users
|
| It's not our job to make the world safe for fundamentally
| unsafe people.
| kettro wrote:
| I would argue that it is society's job to care for its most
| vulnerable.
| bongodongobob wrote:
| That doesn't mean we pad all the rooms or ban peanuts. Yes,
| we should care for them but not at the detriment of the
| other 99%.
| earthling8118 wrote:
| Well, conveniently, this is benefitting the 1% much more
| than the 99%
| llm_trw wrote:
| And a nanny state benefits a different 1% much more than
| the 99%.
| fipar wrote:
| That's a false dilemma.
| llm_trw wrote:
| Having seem far too many orgs implode because of that new
| 1%, no it really isn't. Replacing greed with self-
| righteousness as the original sin of those in power does
| not help anyone.
| bongodongobob wrote:
| Do you mean the richest 1%? How? Sounds pretty woo woo to
| me.
| llm_trw wrote:
| Yes, not openai's.
| woopsn wrote:
| "Our primary fiduciary duty is to humanity."
| dns_snek wrote:
| It depends. I don't think OpenAI (or anyone else selling
| products to the general audience) should be forced to
| make their products so safe that they can't possibly harm
| anyone under any circumstance. That's just going to make
| the product useless (like many LLMs currently are,
| depending on the topic). However that's a very different
| standard than the original comment which stated:
|
| > I suspect Altman/Brockman/Murati intended for this
| thing to be dangerous for mentally unwell users, using
| the exact same logic as tobacco companies.
|
| Tobacco companies knew about the dangers of their
| products, and they purposefully downplayed them,
| manipulated research, and exploited addictive properties
| of their products for profit, which caused great harm to
| society.
|
| Disclosing all known (or potential) dangers of your
| products and not purposefully exploiting society
| (psychologically, physiologically, financially, or
| otherwise) is a standard that every company should be
| forced to meet.
| swat535 wrote:
| Corporations should benefit the society and avoid harming
| it in some shape or form, this is why we have regulations
| around them.
| limpbizkitfan wrote:
| Okay this is a weird philosophy to have lol
| anthonypasq wrote:
| no it isnt, its how everything in society currently
| operates. We put dangerous people in jail away from
| everyone else
| itishappy wrote:
| The crime of the "dangerous people" in OP's statement was
| loneliness and suggestibility.
| baobabKoodaa wrote:
| And it's not OpenAI's job to safetify the world for
| gullible loners.
| itishappy wrote:
| Who else can? OpenAI makes the tool.
|
| Are you suggesting we need government intervention or
| just saying "damn the consequences"?
| baobabKoodaa wrote:
| Society can't be built with the idea that everything has
| to work for the most troubled and challenging
| individuals.
|
| We build cars, even though some alcoholics drive drunk.
| We could make cars safer for them by mandating a steering
| wheel lock with breathalazyer for every car, but we
| choose to not do that because it's expensive.
|
| We have horror movies, even though some people really
| freak out from watching horror movies, to the point where
| they have to be placed in mental asylums for extended
| periods of time. We could outlaw horror movies to reduce
| the strain on these mentally troubled individuals, but we
| choose to not do that because horror movies are cool.
| incahoots wrote:
| >Society can't be built with the idea that everything has
| to work for the most troubled and challenging
| individuals.
|
| But it is, nearly every product, procedure, process is
| aimed at the lowest common denominator, it's the entire
| reasoning warning labels exist, or fail safe systems
| (like airbags) exist.
| baobabKoodaa wrote:
| If every product or process was truly aimed at the lowest
| common denominator, then we wouldn't have warning labels
| on hot coffee, we would instead have medium-heated
| coffee.
| incahoots wrote:
| The label doesn't confirm if the coffee is hot, it warns
| that _it might be_.
| baobabKoodaa wrote:
| My point is that hot coffee is still being sold
| everywhere, even though we know for a fact that it's
| dangerous for our most vulnerable individuals. Mentally
| unstable people will sometimes spill coffee and when the
| coffee is hot it causes burns. If we really wanted to
| make coffee safe for our most vulnerable individuals, we
| would outlaw hot coffee, and just have medium-heated
| coffee instead. So the existence of "warning labels on
| hot coffee" is really evidence for my point, not evidence
| for your point.
| incahoots wrote:
| then you would agree that warning labels are the lowest
| common denominator solution to a well known fact, vis-a-
| vis all processes, products, & procedures are aimed at
| the lowest factor.
| baobabKoodaa wrote:
| I don't know what that sentence means. But I know it
| doesn't mean "warning labels solve the problem that
| everything has to work for the most troubled and
| challenging individuals", which is what this discussion
| was about at least a few messages ago.
| itishappy wrote:
| > Society can't be built with the idea that everything
| has to work for the most troubled and challenging
| individuals.
|
| That's a far cry from saying the sellers are free from
| any responsibility.
|
| Cars are highly engineered AND regulated because they
| have a tendency to kill their operators and pedestrians.
| It does cost more, but you're not allowed to sell a car
| that can't pass safety standards.
|
| OpenAI have created a shiny new tool with no regulation.
| Great! It can drive progress or cause harm. I think they
| deserve credit for both.
| baobabKoodaa wrote:
| > Cars are highly engineered AND regulated because they
| have a tendency to kill their operators and pedestrians.
| It does cost more, but you're not allowed to sell a car
| that can't pass safety standards.
|
| But you _are_ allowed to sell a car without a mechanical
| steering wheel lock connected to a breathalyzer.
| Remember, this discussion isn 't about "should technology
| be made safe for the average person", this discussion is
| about "should technology be made safe for the most
| vulnerable amongst us". In the context of cars,
| alcoholics are definitely within this "most vulnerable"
| group. And yet, car safety standards do not require
| engine startup to check for a breathalyzer result.
|
| > OpenAI have created a shiny new tool with no
| regulation. Great! It can drive progress or cause harm. I
| think they deserve credit for both.
|
| I didn't make an argument for "no regulation", so this is
| not really related to anything I said.
| digging wrote:
| > We put dangerous people in jail away from everyone else
|
| This is a very naive understanding of what prisons are
| and who goes in them and why.
| lantry wrote:
| This is literally everyone's job. It's the whole point of
| society. Everyone is "fundamentally unsafe", and we all rely
| on each other.
| kiba wrote:
| We may not be responsible for people's behaviors but it's
| certainly not going to get better if nobody do anything
| about it.
| baobabKoodaa wrote:
| > This is literally everyone's job. It's the whole point of
| society.
|
| This is the most extreme left viewpoint I have ever heard
| in my life. If you talk with people and find out what other
| people believe besides you and your tiny bubble, you will
| find out that almost every other person in the world
| disagrees with you. Myself, I believe people have different
| jobs in society, everyone doesn't have the same job. It's
| okay if there's, for example, a carpenter who focuses on
| making furniture out of wood, and doesn't try to solve
| mental issues of people who have a tendency to bash
| furniture when they are angry.
| hengheng wrote:
| > This is the most extreme left viewpoint I have ever
| heard in my life.
|
| In that case I suggest more exposure to viewpoints that
| aren't similar to your own.
| baobabKoodaa wrote:
| If that's all you have to say on this, then I whole
| heartedly recommend the same to you as well.
| CooCooCaCha wrote:
| Please enumerate some right wing perspectives that we
| should learn about. I'm curious.
| baobabKoodaa wrote:
| I'm not here to promote "right wing" ideology or any
| other political agenda. I'm stating - as a matter of fact
| - that different people have different jobs and different
| responsibilities. I'm also stating - as an opinion - that
| that's okay.
| m_fayer wrote:
| I agree that it's an outlier viewpoint but I'd love it if
| we moved past categorizing everything along this
| cartoonish left-right axis.
|
| How about: "deep within a humanist paradigm"
| categorization, or something like that.
| baobabKoodaa wrote:
| I'm tired of this insane political activism that
| permeates everything, and I don't want to play dress up
| with it. I rather call it out when I see it.
| itishappy wrote:
| You'd fight political activism using more political
| activism?
| baobabKoodaa wrote:
| If it's political activism to say "different people can
| have different jobs and that's okay", then so be it. What
| a controversial take, I know...
| jpdenford wrote:
| I think you're taking the "job" bit too literally. As
| someone else said, "responsibility" might be a better
| term. We all act (vote, talk, help, purchase etc) and
| have the ability to do so in a way which is ethically
| informed. You could say it's everyone's responsibility to
| make sure that the important jobs are being done by
| someone.
| baobabKoodaa wrote:
| Yes, we all have a various moral obligations. Do all of
| us have a "responsibility" to make the world safe for our
| most vulnerable individuals? No. A barista will make a
| hot coffee when asked, knowing it can hurt people who
| might spill it. A barista does not have a
| "responsibility" to make only medium-heated coffee. At
| the same time, there are people in the world who try to
| make good tradeoffs with regards to coffee temperature
| (e.g. MacDonalds corporate, or lawmakers setting an upper
| limit to allowed coffee temp). So at the same time, one
| person might make an effort to make coffee safe for
| vulnerable persons, while another person might just have
| a responsibility to serve coffee at whatever temperature
| was decided by the other people. Different people can
| have different responsibilities in the world.
| m_fayer wrote:
| I'm tired of the activism too. I would love to have less
| of that and more complex discussion that acknowledges the
| myriad takes on political organization, political
| economy, economics, basic ethics, etc. that are out
| there. Forcing everything into left-vs-right isn't
| calling out activism, it's turning discussion into
| ideological conflict.
| baobabKoodaa wrote:
| I think you have a point here and maybe my conduct here
| wasn't the best.
| m_fayer wrote:
| Thanks for saying that, I'll try to remember for when
| it's me in your place.
| CooCooCaCha wrote:
| You realize that is also a political position? You can't
| escape politics. Being against political activism just
| means you're pro status quo.
| Zardoz89 wrote:
| Carpenters try to solve those problems. When it's their
| mother or their son. Unless you believe empathy is a
| political alignment, a sad and confusing world for you.
| baobabKoodaa wrote:
| I never said "no carpenter" tries to solve mental
| problems for mentally troubled people. I said it's fine
| if "a carpenter" just focuses on carpentry instead of
| people. Of course some carpenters can focus on both,
| that's great. But _it 's not everybody's job_. This is
| just a factually accurate statement about the world.
| Every single person on this planet does not have the same
| job.
| 8note wrote:
| Everyone shares many jobs, like clothing themselves,
| breathing, eating, and using toilets.
|
| Being a carpenter is not the only job a carpenter has
| baobabKoodaa wrote:
| Everyone needs to do the "job" of "breathing", yes.
|
| Everyone does not need to do the job of "sheltering
| mentally unstable people", no.
| itishappy wrote:
| We can have more than one job:
|
| If you have kids, that's a job. If you have pets, that's
| a job. If you maintain a social circle, that's a job. If
| you have a job, that's also a job. I'd argue it's your
| job to restrain from stabbing random strangers in the
| street.
|
| Maybe "responsibility" is a better term, or maybe that's
| too strong too. You're not going to be paid for many of
| these things or penalized if you don't, but living a life
| outside of any expectations towards other people does not
| a healthy society create.
| baobabKoodaa wrote:
| > I'd argue it's your job to restrain from stabbing
| random strangers in the street.
|
| Sure, I agree with you, we all have a responsibility to
| restrain ourselves from stabbing random people in the
| street. Do we all have a responsibility to make the world
| "safe" for the most vulnerable people amongst us? No.
| It's great that we have some people working on that, like
| mental health professionals, or city councilmembers, but
| every single person does not and should not work on that
| particular thing.
| llm_trw wrote:
| >I'd argue it's your job to restrain from stabbing random
| strangers in the street.
|
| It's not my job to restrain others from doing that.
| woodruffw wrote:
| Only on HN would an essentially Hobbesian view ("society
| is an optimal solution to individual human weaknesses")
| be labeled "extreme left" :-)
|
| (Leviathan, XIII).
| baobabKoodaa wrote:
| > Only on HN would an essentially Hobbesian view
| ("society is an optimal solution to individual human
| weaknesses") be labeled "extreme left"
|
| Now you're just trying to twist my words to make it
| appear as if I said something which I didn't say.
|
| I don't agree with that statement. I would not label the
| words "society is an optimal solution to individual human
| weaknesses" as "extreme left".
|
| Society can be an optimal solution to individual human
| weaknesses while different people perform different
| functions within society.
|
| Don't smear me again.
| woodruffw wrote:
| It's not a "smear." Besides, you're the one bandying
| "extreme left" around.
|
| I think if you read "job" to mean "social obligation" or
| "responsibility," you'll understand that the GP's
| statement is essentially the Hobbesian one. If you read
| it to mean "the thing someone pays you to do," you'll get
| a very silly (and obviously incorrect) argument.
| baobabKoodaa wrote:
| > I think if you read "job" to mean "social obligation"
| or "responsibility," you'll understand that the GP's
| statement is essentially the Hobbesian one
|
| Let's read it as "responsibility". No, I don't agree with
| you on the statement being equivalent to what you wrote
| earlier. Different people can have different
| responsibilities in the world. "Making things safe for
| our most vulnerable individuals" is not a responsibility
| that belongs to literally every person in the world. It's
| okay if someone just makes furniture that's safe for the
| average person, while it has a sharp edge that can hurt a
| mentally unstable person.
| claytongulick wrote:
| > This is the most extreme left viewpoint I have ever
| heard in my life.
|
| I hope this is hyperbolic.
|
| >> This is literally everyone's job. It's the whole point
| of society.
|
| If viewed through a political lense, I can see how you'd
| interpret it as a left-leaning ideology.
|
| If viewed objectively with the best possible
| interpretation, I think the statement is factually
| correct - we banded together, like most pack animals, for
| mutual security.
| baobabKoodaa wrote:
| > we banded together, like most pack animals, for mutual
| security
|
| Every animal in our pack does not share the same
| responsibilities. Different animals can perform different
| functions within the pack.
| claytongulick wrote:
| > This is literally everyone's job. It's the whole point of
| society.
|
| To a degree, yes - but I think if it's taken too far it
| becomes a trap that many people seeking power lay out.
|
| Benjamin Franklin said it best: "Those who would give up
| essential Liberty, to purchase a little temporary Safety,
| deserve neither Liberty nor Safety."
|
| That being said, I do agree with part of your point. The
| purpose of having a society is that collective action lets
| us do amazing things like build airplanes, that would be
| otherwise impossible. In order to succeed at that we need
| some rules that everyone plays by, which involve giving up
| some freedoms - or the "social contract".
|
| The more of a safety net a society provides, the more
| restrictive the society must be. Optimizing for this is
| known as politics.
|
| I think history has shown us that the proper balance is one
| where we optimize for maximum elbow room, without letting
| people die on the streets. Trying to provide the illusion
| of safety and restrict interesting technology to protect a
| small percentage of the population is on the wrong side of
| this balance.
|
| Maybe we try it, and see what the effect actually are,
| rather than guessing. If it becomes a major problem, then
| address it - in the least restrictive way possible.
| iudqnolq wrote:
| Fun fact, that quote has been entirely misinterpreted.
|
| > He was writing about a tax dispute between the
| Pennsylvania General Assembly and the family of the
| Penns, the proprietary family of the Pennsylvania colony
| who ruled it from afar. And the legislature was trying to
| tax the Penn family lands to pay for frontier defense
| during the French and Indian War. And the Penn family
| kept instructing the governor to veto. Franklin felt that
| this was a great affront to the ability of the
| legislature to govern. And so he actually meant purchase
| a little temporary safety very literally. The Penn family
| was trying to give a lump sum of money in exchange for
| the General Assembly's acknowledging that it did not have
| the authority to tax it.
|
| > It is a quotation that defends the authority of a
| legislature to govern in the interests of collective
| security. It means, in context, not quite the opposite of
| what it's almost always quoted as saying but much closer
| to the opposite than to the thing that people think it
| means.
|
| https://www.npr.org/2015/03/02/390245038/ben-franklins-
| famou...
| itishappy wrote:
| I'm guessing your work isn't sanitation either. Do you throw
| your trash straight on the ground?
|
| Some things are everyone's responsibility if we want to live
| in a pleasant society.
| nicklecompte wrote:
| "fundamentally unsafe people" is probably the grossest thing
| I've read on here in years.
| poulpy123 wrote:
| Actually yes, it's our job
| yinser wrote:
| What a wild accusation for someone light years away from the
| board room.
| nicklecompte wrote:
| I wasn't making an accusation about why Leike/Sutskever left,
| though I definitely understand why you read my comment that
| way.
|
| The actual accusation I am making is that someone at OpenAI
| knew the risks of GPT-4o and Sam Altman didn't care. I am
| confident this is true even without spies in the boardroom.
| My guess is that Leike or Sutskever also knew the risks and
| actually did care, but that is idle speculation.
| qarl wrote:
| > The faking of emotions
|
| HEH. In previous versions, when it told jokes, were those fake
| jokes?
| awkwardpotato wrote:
| Those are fundamentally different things. You can tell a joke
| without understanding context, you can't express emotions if
| you don't have any. It's a computation model, it cannot feel
| emotion.
| terse-broccoli wrote:
| Are they fundamentally different? Couldn't you make the
| argument that it's advanced from a probabilistic
| determination of the most likely next token, to a
| probabilistic determination of the next token AND a
| probabilistic determination of the inflection that that
| token should be transmitted with? How is one any more or
| less fake than the other?
| qarl wrote:
| > It's a computation model, it cannot feel emotion.
|
| HEH. I'd love to see your proof for that statement.
| jobs_throwaway wrote:
| what is your theory of how human emotions arise?
| itsnotlupus wrote:
| > you can't express emotions if you don't have any
|
| That feels off. When I watch an actor on screen conveying
| emotions, there's no actual human being feeling those
| emotions as I watch their movie. Very dumb machines have
| already been rendering emotions convincingly for a while in
| that way, and their rendering impacts our own emotional
| state.
|
| Emotions expressed through tone of voice are just one mean
| of nonverbal communication. We should expect more of those
| to develop and become more widely available next.
|
| In a way, we're lucky all that gpt-4o seems to be hell bent
| on communicating how cheerful and happy it is so far,
| because it's certainly not the only option.
|
| Humans can be manipulated through nonverbal communications,
| in a way that's harder to consciously spot than through
| words, and a model that's able to craft its "emotional
| output" would not be far from being able to use it to
| adjust its interlocutor or audience's frame of mind.
|
| I for one look forward to the arrival of our increasingly
| charismatic and oddly convincing LLMs.
| sebzim4500 wrote:
| >I suspect Altman/Brockman/Murati intended for this thing to be
| dangerous for mentally unwell users
|
| Isn't it much more likely that they are just trying to make a
| product that people want to use?
|
| Even Tobacco companies don't go out of their way to give people
| cancer.
| talldayo wrote:
| But tobacco companies _are_ still complicit in distributing
| addictive carcinogens to people even if only in trace
| amounts. The same could be said about predatory business
| models /products.
| smt88 wrote:
| > _just trying to make a product that people want to use?_
|
| Sure, but you can do that ethically or unethically.
|
| If you make a product that's harmful, disincentivizes healthy
| behavior (like getting therapy), or becomes addictive, then
| you've crossed into something unethical.
|
| > _Even Tobacco companies don 't go out of their way to give
| people cancer._
|
| This is like saying pool companies don't go out of their way
| to get people wet.
|
| While it isn't their primary goal, the use of tobacco causes
| cancer, so their behavior (promoting addiction among
| children, burying medical research, publishing false
| research, lobbying against regulation) are all in service of
| ultimately giving cancer to more people.
|
| Cancer and cigarettes are inseparable, the same way casinos
| and gambling addiction are inseparable.
| GaggiX wrote:
| There has been no relaxation of pornographic generations on
| OpenAI products.
| qingcharles wrote:
| They announced an intention to allow porn generations.
| GaggiX wrote:
| An OpenAI spokesperson recently stated explicitly that "We
| have no intention to create AI-generated pornography".
| shmatt wrote:
| Realistically its all just probabilistic word generation.
| People "feel" like an LLM understands them but it doesn't, its
| just guessing the next token. You could say all our brains are
| doing are just guessing the next token but thats a little too
| deep for this morning
|
| All these companies are doing now is taking an existing
| inferencing engine, making it 3% faster, 3% more accurate, etc.
| per quarter, fighting over the $20/month users
|
| One can imagine product is now taking the wheel from
| engineering and are building ideas on how to monetize the
| existing engine. Thats essentially what GPT-4o is, and who
| knows what else is in the 1,2,3 year roadmaps for any of these
| $20 companies
|
| To reach true AGI we need to get past guessing, and that
| doesn't seem close at all. Even if one of these companies gets
| better at making you "feel" like its understanding and not
| guessing, if it isnt actually happening, its not a breakthrough
|
| Now with product leading the way, its really interesting to see
| where these engineers head
| DiogenesKynikos wrote:
| > People "feel" like an LLM understands them but it doesn't,
| its just guessing the next token. You could say all our
| brains are doing are just guessing the next token but thats a
| little too deep for this morning
|
| "Just" guessing the next token requires understanding. The
| fact that LLMs are able to respond so intelligently to such a
| wide range of novel prompts means that they have a very
| effective internal representation of the outside world.
| That's what we colloquially call "understanding."
| cmiles74 wrote:
| I would argue it's an understanding of the relationship
| between the words; an effective internal representation of
| those relationships. IMHO, it's still quite a ways from a
| representation of the outside world.
| carom wrote:
| I disagree. I use ChatGPT daily as a replacement for
| Google. It doesn't understand or have logic, it can spit
| out information very well though. It has a broad knowledge
| base. There is no entity there to have an understanding of
| the topic.
|
| This becomes pretty clear when you get to more complex
| algorithms or low level details like drawing a stack frame.
| There is not logic there.
| DiogenesKynikos wrote:
| > It doesn't understand or have logic
|
| I can ask ChatGPT questions that require logic to answer,
| and it will do just fine in most cases. It has certain
| limitations, but to say it isn't able to apply logic is
| just completely contrary to my experience with ChatGPT.
| jobs_throwaway wrote:
| give us an example please
| DiogenesKynikos wrote:
| I deliberately asked ChatGPT a logical question with a
| false premise: "If all snakes have legs, and a python is
| a snake, does a python have legs?"
|
| ChatGPT answers:
|
| > Yes, if we assume the statement "all snakes have legs"
| to be true and accept that a python is a type of snake,
| then logically, a python would have legs. This conclusion
| follows from the structure of a logical syllogism:
|
| > 1. All snakes have legs.
|
| > 2. A python is a snake.
|
| > 3. Therefore, a python has legs.
|
| > However, it's important to note that in reality,
| snakes, including pythons, do not have legs. This logical
| exercise is based on the hypothetical premise that all
| snakes have legs.
|
| ChatGPT clearly understands the logic of the question,
| answers correctly, and then tells me that the premise of
| my question is incorrect.
|
| You can say, "But it doesn't really understand logic.
| It's just predicting the most likely token." Well, it
| responds exactly how someone who understands logic would
| respond. If you assert that that's not the same as
| applying logic, then I think you're essentially making a
| religious statement.
| root_axis wrote:
| Indeed. It's also obvious when the "hallucinations"
| create contradictory responses that a conceptual
| understanding would always preclude. For example, "In a
| vacuum, 100g of feathers and 100g of iron would fall at
| the same rate due to the constant force of gravity, thus
| the iron would hit the ground first". Only a language
| model makes this type of mistake because its output is
| statistical, not conceptual.
| gortok wrote:
| it requires calculation of frequency of how often words
| appear next to each other given other surrounding words. If
| you want to call that 'understanding', you can, but it's
| not semantic understanding.
|
| If it were, these LLMs wouldn't hallucinate so much.
|
| Semantic understanding is still a ways off, and requires
| much more intelligence than we can give machines at this
| moment. Right now the machines are really good at frequency
| analysis, and in our fervor we mistake that for
| intelligence.
| DiogenesKynikos wrote:
| > it requires calculation of frequency of how often words
| appear next to each other given other surrounding words
|
| In order to do that effectively, you have to have very
| significant understanding of the world. The texts that
| LLMs are learning from describe a wide range of human
| knowledge, and if you want to accurately predict what
| words will appear where, you have to build an internal
| representation of that knowledge.
|
| ChatGPT knows who Henry VIII was, who his wives were, the
| reasons he divorced/offed them, what a divorce is, what a
| king is, that England has kings, etc.
|
| > If it were, these LLMs wouldn't hallucinate so much.
|
| I don't see how this follows. First, humans hallucinate.
| Second, why does hallucination prove that LLMs don't
| understand anything? To me, it just means that they are
| trained to answer, and if they don't know the answer,
| they BS it.
| woodruffw wrote:
| To my understanding (ha!), none of these language models
| have demonstrated the "recursive" ability that's basic to
| human consciousness and language: they've managed to
| iteratively refine their internal world model, but that
| model implodes as the user performs recursive
| constructions.
|
| This results in the appearance of an arms race between
| world model refinement and user cleverness, but it's really
| a fundamental expressive limitation: the user can always
| recurse, but the model can only predict tokens.
|
| (There are a lot of contexts in which this distinction
| doesn't matter, but I would argue that it _does_ matter for
| a meaningful definition of human-like understanding.)
| johnthewise wrote:
| Supposedly that was Q* all about. Search recursively,
| backtrack if dead end. who knows really, but the
| technology is still very new, I personally don't see why
| a sufficiently good world model can't be used in this
| manner.
| 8crazyideas wrote:
| I've seen this idea that "LLMs are just guessing the next
| token" repeated everywhere. It is true that accuracy in
| that task is what the training algorithms aim at. That is
| not however, what the output of the model represents in
| use, in my opinion. I suspect the process is better
| understood as predicting the next concept, not the next
| token. As the procedure passes from one level to the next,
| this concept morphs from a simple token to an ever more
| abstract representation of an idea. That representation
| (and all the others being created elsewhere from the text)
| interact to form the next, even more abstract concept. In
| this way ideas "close" to each other become combined and
| can fuse into each other, until an "intelligent" final
| output is generated. It is true that the present
| configuration doesn't offer the LLM a very good way to look
| back to see what its output has been doing, and I suspect
| that kind of feedback will be necessary for big
| improvements in performance. Clearly, there is an
| integration of information occurring, and it is interesting
| to contemplate how that plays into G. Tononi's definition
| of consciousness in his "information integration theory".
| qq66 wrote:
| Doesn't have to be smart to be dangerous. The asteroid that
| killed the dinosaurs was just a big rock.
| abeppu wrote:
| I think the comparison to tobacco companies is misleading
| because tobacco is not good for anyone, poses a risk of harm to
| everyone who uses it regularly, and causes very bad outcomes
| for some of those users. I.e. there's not a large population
| who can use tobacco without putting themselves at risk.
|
| But hypothetically if a lot of people _would_ benefit from a
| GPT with more fake emotions, that might reasonably
| counterbalance concerns about harm for a mentally unwell
| minority. If we build a highway, we know that eventually it
| will lead to deaths from car crashes -- but if the highway is
| actually adding value by letting people travel, those benefits
| might reasonably be expected to outweigh that harm. And the
| people getting into their cars and onto the highway agree, that
| the benefits outweigh the costs, right up until they crash.
|
| None of this is to say that I think OpenAI's choices here were
| benevolent rather than a business choice. But I think _even if_
| they were trying to do the ethically best thing for the world
| overall, it would be plausible to move forward
|
| I for one found the fake emotions in their voice demos to be
| really annoying tho.
| alxjrvs wrote:
| Playing devils advocate for a moment - have you ever _had_ a
| cigarette? It does plenty of good for the user. In fact, I
| think we do make this risk calculation that you describe in
| the exact same way - there are plenty of substances that are
| so toxic to humanity that we make them illegal to own or
| consume or produce, and the presence of these in your body
| can sometimes even risk employment, let alone death.
|
| We know the risks from cigarettes, but it offers tangible
| benefits to its users, so they continue to use the product.
| So too cars and emotionally manipulative AI's, I imagine.
|
| (None of this negates your overall point, but I do think the
| initial tobacco comparison is very apt.)
| abeppu wrote:
| > We know the risks from cigarettes
|
| Hmm, the tobacco industry is also famous for actively
| trying to deny and suppress evidence about its harms. They
| actively didn't want people to be in a position to make a
| fully informed decision. In cases where jurisdictions
| introduced policies that packaging etc had to carry factual
| information about health risks, the tobacco industry pushed
| back.
| alxjrvs wrote:
| Wholeheartedly agreed!
|
| Please don't mistake my post as an endorsement of the
| tobacco industry - I was only saying that while we do not
| have extensive proof of the dangers of social AI, wink-
| and-nodding at the audience about AI intimacy (sexual or
| otherwise) strikes me as irresponsible, and so I thought
| the tobacco comparison was apt.
| vasco wrote:
| > The faking of emotions and overt references to Her (along
| with the suspiciously-timed relaxation of pornographic
| generations) are not fine.
|
| Are you not aware of how many billions are getting spent on
| fake girlfriends on Only Fans, with millions of people chatting
| away with low paid labor across an ocean pretending to be an
| american girl? This is just reducing costs, but the consumers
| are already wanting the product.
|
| I'm not sure I get the outrage / puritanism. Adults being able
| to chat to a fake girlfriend if they want to seems super bland.
| There's way more stuff out there that's way wilder you can do
| online, potentially also exploiting the real person on the
| other side of the screen if they are trafficked or whatever. I
| don't have any mental issues (well, who knows? ha) and
| genuinely would try it the same way you try a new porn category
| every once in a while.
| Retr0id wrote:
| There's a lot of harmful stuff already in the world, but most
| of us would rather not add to the pile on an industrial
| scale.
| jobs_throwaway wrote:
| I for one think consenting adults should be able to do
| mildly harmful things like waste their time/money on a fake
| girlfriend if they so choose, and find it offensive that
| others would try to impose their values on others by
| restricting them from doing so
| Retr0id wrote:
| Nobody's restricting anything, we're discussing someone
| choosing not to work for a particular company.
| Sirizarry wrote:
| I agree. I don't care for and actually dislike most
| "features" of AI including and especially chat bots but
| it's also none of my business to go and stand on the soap
| box of my own subjective morality to tell others how to
| live. If you don't want people ruining their lives over
| this, go after root causes not this piddly shit.
| dwaltrip wrote:
| How do you know having an AI girlfriend is only mildly
| harmful? No one has ever had one before.
| Retr0id wrote:
| Supposedly plenty have already, although I don't think
| anyone's studied it clinically
| https://knowyourmeme.com/news/replika-ai-shuts-down-
| erotic-r...
| dwaltrip wrote:
| I'm talking more about potential "Her"-like AI partners
| (the movie). Ones that would be a much more compelling
| replacement for actual human relationships.
|
| We might want to be a little cautious about blindly going
| down that path. The tech isn't there yet, and probably
| won't be there next year, but it feels far more possible
| than ever before.
| screye wrote:
| The 2nd most visited GenAI website is character ai.
|
| The net sum of all LLM B2C usecases (effectively Chatgpt) is
| competing with AI girlfriends for rank 1.
|
| It isn't just huge. It is the most profitable usecase for gen
| AI.
|
| "A goal of a system is what it does"
| vasco wrote:
| I mean I didn't want to say it in my original reply but
| your comment makes it even more difficult to resist:
|
| "The internet is for porn"
| https://youtu.be/LTJvdGcb7Fs?si=8H1OzeyG5XzU-Qe8
| poulpy123 wrote:
| I saw the faking of emotions, and it's already visible in
| previous LLM and I find that extremely annoying indeed.
| wseqyrku wrote:
| This trail of resignations and drama only signals that whoever is
| left had to decide whether or not they're ok if they ended up
| wherever they're going.
| whazor wrote:
| Honestly, OpenAI does not look that far ahead of open-
| source/competitors. Sure they are leading, but for alignment it
| might be a lost cause.
| renegade-otter wrote:
| "Superalignment" sounds so much cooler than "output moderation".
| reducesuffering wrote:
| Like Emmett Shear says[0], in this discourse we crucially need
| to separate out the "AI ethics" (like output moderation) from
| the real problem of "AI NotKillEveryoneism". People are pattern
| matching "safety" as "those people concerned about AI racism
| and censoring" when they're orthogonal.
|
| I encourage everyone who is skeptical of superalignment
| (there's a reason very smart people like Ilya were heading that
| team) watch Emmett Shear (former OpenAI and Twitch CEOs) talk
| about this problem in the discourse.
|
| https://youtu.be/jZ2xw_1_KHY?si=Jog7ULmjjNW21ZIf&t=800
| AlexAndScripts wrote:
| 100%. I see them conflated a lot, particularly outside of HN,
| and I think it really damages the discourse. Censorship for
| political or brand reasons (and I'm not making a moral
| judgement by my choice of language) is very different to
| aligning for safety reasons.
| reducesuffering wrote:
| Yep. And it's lazy to think that the pro-cryogenics, pro-
| nuclear, pro-engineering people are the same as the ones
| storming a Tesla factory or causing wildly unexpected
| Gemini output.
|
| "No no no, this thing though actually, may be a problem. If
| you're rejecting it because we sound like a bunch of
| crazies, Just notice that some number of people who are
| worried about this, I'm on your team, I'm on the techno-
| optimist team. Its not obvious why its true. It takes a
| good deal of engagement with the material to see why its
| true. Because at first, it seems like it shouldn't be that
| big of a deal, it shouldn't be that big of a problem. but
| then the more you dig in, actually, oh wait a second, and
| it is." -Emmett
| nicklecompte wrote:
| "People who support cryonics are sober-minded, reasonable
| individuals who would surely never be misled by
| unscientific techno-mysticism."
|
| I am amazed at the Dunning-Kruger syndrome on display
| here.
| reducesuffering wrote:
| A large part of that Emmett Shear interview is about the
| fallacy of pattern-matching beliefs like you're doing
| right now. "oh those people have X belief, which is
| stupid, so I just tune the rest of them out." I guarantee
| you haven't done an in-depth study of the merits of
| cryogenics. Neither have I yet, but there are many smart
| people who do think the tradeoff is worth it. The point I
| and Emmett make is that AGI X-risk people are dismissed
| prematurely based on vaguely sounding like other stuff
| the subject already believes isn't worth investigating.
| Whether that's mistakenly believing they're not pro-tech
| along most axes, "dismissed", they generally have some
| other quirky belief like cryonics, "dismissed", or they
| think we could be headed for a very dangerous place,
| "sounds like other doomsday people, dismissed."
| nicklecompte wrote:
| > I guarantee you haven't done an in-depth study of the
| merits of cryogenics. Neither have I yet, but there are
| many smart people who do think the tradeoff is worth it.
|
| This is literally the strategy of conspiracy theorists!
| "You didn't do the research, but Emmett did and he's
| smarter than you." The scientific consensus is that
| cryonics is pseudoscience. It does not matter if I have
| personally done a study. It is entirely rational for me
| to trust the Wikipedia article on cryonics + common
| sense. And lots of people way smarter than Shear believed
| in stranger things.
|
| Emmett Shear - and yourself! - uncritically described
| cryonics as being a rational, techno-optimist thing to
| support, and proof of Shear's credentials as a
| reasonable, grounded thinker, not like these Tesla-
| smashing wackos. That strongly implies that Shear is a
| bad judge of what counts as reasonable, because cryonics
| is ridiculous quackery. Hence why he has Dunning-Kruger
| syndrome. He's operating in an ignorant Silicon Valley
| bubble where sci-fi and reality are interchangeable as
| long as you have a thoughtful affectation.
|
| One thing ChatGPT has made very obvious is that the tech
| community is vulnerable to the tactics of psychics and
| conspiracy theorists.
| qxfys wrote:
| So, what do Ilya and Jan know that we don't know?
| light_triad wrote:
| Both super-alignment leads resigned. One way to interpret this is
| that it's an interesting research topic that has wide
| repercussions the closer they get to AGI, but it's more of a
| extra cost that doesn't directly help with monetisation and
| productisation. The real killer applications are in business
| verticals with domain specific expertise and customised (and
| proprietary) datasets, not so much the 'pure' academic research.
|
| OpenAI dedicated 20% of compute to the effort which sounds kind
| of like Google's 20% side project time :)
| w10-1 wrote:
| It looks like "Super-alignment" was to automate monitoring, and
| their job was to find AI researchers who wanted not to build
| new things but find problems.
|
| But there's really zero glory or profit in doing QA, much as
| users complain about quality issues.
|
| So perhaps they succeeded and built a team, or perhaps they
| found they couldn't recruit a decent team, or perhaps they
| failed to produce anything useful (since the goal of not
| offending anyone or embarrassing the company or doing harm is
| relatively difficult to concretize).
|
| Regardless, I suspect there's no good answer yet to how to
| manage quality, in part because there's no good answer on how
| these things work. That problem will likely remain long after
| people have forgotten about Ilya, Jan, or even Sam.
|
| Though it might be solved by rethinking the idea that
| "attention is all you need".
| nemo44x wrote:
| It's funny when people that have been managed out of an
| organization don't realize it and then resign with indignation. I
| guess clues aren't for sale.
| ilaksh wrote:
| Maybe that means that Jan and Ilya feel they have achieved some
| definition of AGI.
|
| What's going on with the Musk lawsuit (Morgan Chu etc) against
| OpenAI (Wachtel Lipton) claiming that they abandoned their
| original open vision?
| option wrote:
| I have a simpler explanation though - Sam reversed a decision to
| dedicate 20% of OAI compute to superalignment team.
| OutOfHere wrote:
| Good riddance, considering the employee's focus on moderation.
| After all, most users don't like any moderation for their own
| personal activities, only sometimes for other people's
| activities.
___________________________________________________________________
(page generated 2024-05-15 23:02 UTC)