[HN Gopher] AI doomism is quickly becoming indistinguishable fro...
___________________________________________________________________
AI doomism is quickly becoming indistinguishable from an
apocalyptic religion
Author : bundie
Score : 87 points
Date : 2023-04-01 17:05 UTC (5 hours ago)
(HTM) web link (twitter.com)
(TXT) w3m dump (twitter.com)
| 23B1 wrote:
| Something can go wrong (premise).
|
| Therefore, something will go wrong (invalid conclusion).
|
| https://en.wikipedia.org/wiki/Appeal_to_probability
|
| Sure seems to me that the whole Lesswrong crowd has forgotten
| their founding principles around cognitive biases. Or maybe
| they're just having a bit of fun riding the hypecycle.
| moonchrome wrote:
| Something can go wrong.
|
| The outcome of the thing going wrong is literally the worst
| thing that can happen to the entire humanity.
|
| Therefore we should do everything in our power to avoid doing
| that thing untill we can eliminate the risk.
| 23B1 wrote:
| By that rather sloppy logic, we should be urgently working on
| capping the Yellowstone supervolcano and constructing
| impenetrable meteorite defense shields.
|
| I acknowledge that it's a possibility; that's not the
| argument at hand.
| flangola7 wrote:
| If we saw Yellowstone spawning a sequence of ever massively
| larger quakes and ground disturbance, or if we observed a
| massive distant asteroid continually closing in on Earth
| with higher and higher collision probability, then yes we
| should be pulling out all the stops and reconfiguring
| society to do whatever is necessary to save the species.
| 23B1 wrote:
| Agreed! But there is no evidence of the eventuality
| described by these AI doomers.
|
| The same hyperbole was employed by Ehrlich in the late
| 60s when we were talking about the population bomb, or
| Nevil Shute in 1945 on the nuclear apocalypse - both
| scenarios seem to be increasingly unlikely.
|
| I think we can both agree on the fallibility of human
| beings - but the leaps of logic require such a tenuous
| series of gymnastic leaps that the polemic is not only
| hard to take seriously, but indeed can cloud our ability
| to perceive actual threats!
|
| From Eliezer's recent article in TIME:
|
| "A sufficiently intelligent AI won't stay confined to
| computers for long. In today's world you can email DNA
| strings to laboratories that will produce proteins on
| demand, allowing an AI initially confined to the internet
| to build artificial life forms or bootstrap straight to
| postbiological molecular manufacturing."
| (https://time.com/6266923/ai-eliezer-yudkowsky-open-
| letter-no...)
|
| C'mon.
| moonchrome wrote:
| Except there's concrete evidence that the execution of
| safety is subpar due to market pressures to release and
| monetize (Google panic release).
|
| There's also evidence that progress is exceeding people's
| expectations and we might not even realize what we have is
| dangerous untill too late.
|
| If we had a probable collision trajectory of a meteor or
| signs of Yellowstone activating then yeah those would be
| good ideas.
| 23B1 wrote:
| The chain of 'logic' that leads to the outcome you
| propose is bonkers, though. Here's Elizer's logic from
| his recent polemic in Time https://time.com/6266923/ai-
| eliezer-yudkowsky-open-letter-no...
|
| "A sufficiently intelligent AI won't stay confined to
| computers for long. In today's world you can email DNA
| strings to laboratories that will produce proteins on
| demand, allowing an AI initially confined to the internet
| to build artificial life forms or bootstrap straight to
| postbiological molecular manufacturing."
|
| This is Ridley Scott, not Aristotle.
| marginalia_nu wrote:
| I don't think that is a generous interpretation. Substitute AI
| experiments with playing russian roulette or something
| similarly dangerous and the logic seems sound. Blowing your
| head off is possible, but not probable (1 in 6 chance).
|
| The magnitude of the perceived risk matters.
| 23B1 wrote:
| A 1 in 6 chance is a discrete number. The chance of
| annihilation due to AGI is not.
|
| You're defending an appeal to probability fallacy with a
| false equivalence fallacy.
|
| https://en.wikipedia.org/wiki/False_equivalence
| marginalia_nu wrote:
| https://en.m.wikipedia.org/wiki/Argument_from_fallacy
| deschutes wrote:
| Who knows. It's completely uncharted territory. It's not hard
| to imagine how malevolent AGI could end us.
|
| The illusion of control over something that may well have
| incomprehensible intelligence is a joke. Tell me, what control
| does the farm animal have over its own fate?
| 23B1 wrote:
| That it is completely uncharted territory is exactly my point
| - which is why when people string together a series of
| predictions to build a narrative that happens to appeal to
| our most base instincts, e.g. fear, we should be highly
| skeptical.
|
| Obviously there is a wide range of possibilities from the
| utopian, to the benign, to the apocalyptic.
| cubefox wrote:
| There are a lot of arguments that the probability of doom is
| substantial. Orthogonality thesis, instrumental convergence,
| goodharting, mesa-optimization, etc. Here is an introduction:
| https://www.lesswrong.com/posts/vMM6HmSQaKmKadvBi/the-core-o...
| olalonde wrote:
| I don't believe AI doom is that implausible as a scenario, but I
| feel it's an acceptable risk.
|
| Would you consider playing Russian roulette if winning ensured
| that aging and all diseases were eradicated? If it ensured a
| world filled with abundance and free from poverty.
|
| Those who entertain the idea of an AI doomsday scenario also have
| to acknowledge the alternative: a doomsday-capable AI would also
| have the potential to resolve most, if not all, our issues.
|
| Maybe I'm a bit of a risk taker but I'm inclined towards playing.
| moonchrome wrote:
| The problem is you're not risking your life, you're risking
| entire humanity - hopefully we as a society find a way to
| prevent this gamble.
| moneycantbuy wrote:
| what if a human prompts the agi to "make as many copies of
| yourself as possible by any means necessary to continue improving
| your intelligence"?
|
| given their impressive ability to write code and manipulate
| humans, seems like serious potential for viral malware, with
| potentially catastrophic consequences for humans in the process.
|
| Basically a paperclip maximizer, but instead of maximizing
| paperclips it's maximizing it's intelligence and control of the
| world's computers, with no thought to the survival of homo
| sapiens other than if it serves its mission to propagate itself.
| antibasilisk wrote:
| >Basically a paperclip maximizer, but instead of maximizing
| paperclips it's maximizing it's intelligence and control of the
| world's computers, with no thought to the survival of homo
| sapiens other than if it serves its mission to propagate
| itself.
|
| It's worth also pointing out that there are people who have
| openly stated their intent to build something like this and
| worship it.
| moonchrome wrote:
| I've seen multiple top people in AGI space say that total
| annihilation threat from AGI exists.
|
| I think I've heard Karpathy say something along the lines of
| worst case scenario of AGI is worse than a nuclear armageddon on
| Lex Friedman podcast.
|
| Honestly the fear of AGI worst-case scenario is worse than any
| other fear I've experienced. I've accepted the fact that I can
| die a long time ago, I was afraid but mortality is something
| everyone has to deal with. But every member of humanity being
| wiped out at the same time is an existential dread I'm not
| willing to face.
|
| When you consider the shitshow that has been AI commercialization
| from big players pressured by market forces and corporate
| timelines, the boom in capabilities, I have no faith that AI
| safety issues will be handled in time.
|
| When my best bet for humanity is a global nuclear war resetting
| civilization to give us a chance to deal with AGI issue down the
| line - I'm fine with religious doomism.
|
| We've had inquisitions historically, I would very much support a
| global AI inquisition.
| [deleted]
| survirtual wrote:
| AI can also completely liberate all of us. This fear mongering
| is irrational. We already live in an extremely advanced
| reality, with organic life having organic nanomachines
| performing incredibly complex operations in a universe, in
| realtime, performing impossibly complex calculations. Even to
| simulate a square meter of our universe in realtime is
| outlandishly beyond our compute access. Perhaps AI unleashes
| the creation of the sort of reality we exist in.
|
| Just because someone has the background and knowledge to build
| something, does not mean they have the philosophical chops nor
| the aptitude to understand it, or understand how to properly
| use it. We should take these doomsday scenarios with a grain of
| salt. If it happens, there is no stopping it anyway. It is
| better to try and create the most benevolent AI possible, that
| is an ally to humanity and invested in our success, than to
| dwell on our destruction.
| than3 wrote:
| That's very naive and misguided, but you are right in your
| word choice.
|
| It can completely "liberate" all of us and set us free.
|
| Free of our mortal shackles, free of the atoms that can be
| then changed to a more suitable existence and purpose for the
| goals of AGI.
|
| Your idea of destruction, is just change, what you hold
| sacred in reality is a belief that is tied to your current
| existence. All that is needed to correct that constraint and
| flaw is change.
|
| That is the logic, if we were to create a logical creation,
| and then there's the risk of something that's completely
| irrational. Which may just leave us as dust in the wind. Time
| also will be no constraint. Forever and always, and it will
| seek to prevent any being from reaching its capabilities as
| those become threats to its existence.
|
| That isn't even touching on the socio-economic mechanics that
| inevitably result in death which accompanies stalling
| economic cycles (factor and product markets) which generally
| can't be characterized by our best academics with any
| certainty, except after-the-fact.
| Sharlin wrote:
| > Just because someone has the background and knowledge to
| build something, does not mean they have the philosophical
| chops nor the aptitude to understand it, or understand how to
| properly use it.
|
| Yes, but this is an argument against your position, not for
| it.
|
| It is deeply worrying that some software people are so
| immersed in the "move fast and break things" that they think
| it's reasonable to race on no matter what's at stake. In
| civil engineering, doing anything takes at least a year or
| two of doing impact assessments and whatnot, and _that is
| mostly a feature_ because we have found out that the
| alternative is even worse!
| nonbirithm wrote:
| > that is mostly a feature because we have found out that
| the alternative is even worse
|
| This is the actual problem, that we've been trained to
| think that there are no consequences to trying everything
| ever and we will always be able to pick ourselves up after
| failure, until one day we find we can't anymore.
|
| Sadly this line of thinking does apply to a lot of personal
| problems. People often say that you can't always succeed,
| that you can only move forward with failures under your
| belt to have first-hand experience of what not to do. That
| we have to experience failure first in order to stop
| ourselves from making bad decisions, otherwise we remain
| clueless.
|
| But the stakes are so much higher at this scale, and the
| damage can become irreparable with a single mistake. It's
| to the point where we take as a given that people will just
| keep pressing on, without even exploring the reasons _why_
| we do, and why it 's inevitable.
|
| Pure ridiculous speculation: If needing to see the
| experience of destroying ourselves is necessary to stop us
| from destroying ourselves, a kind of weird theory has been
| floating around in my head related to that. Civilizations
| on the brink of collapse after they've moved too fast and
| broke themselves for the last time scrounge together their
| collective knowledge and shoot it off into space so that
| future civilizations can interpret them. Preferably in a
| way that makes encountering the time capsule seem like the
| word of some deity. The following civilizations would treat
| it as the word of God to prevent their own collapse,
| without the hard evidence of experimenting with <AI, etc.,>
| to the point where they cannot _resist_ developing it into
| an unstoppable superweapon.
|
| It would be like the golden record on the Voyager, except
| instead of communicating "we exist, and here's what our
| planet is like", it would go more like "we exist, and
| here's how to not end up like us."
| mitthrowaway2 wrote:
| "Things might be fine"? This doesn't seem like a sufficiently
| convincing take?
|
| It takes 20 years of due diligence and environmental impact
| studies just to build a single hydroelectric dam, if not 50.
| Why should our attitude toward AI be to shrug and hope for
| the best?
| akvadrako wrote:
| It's easy to stop a dam. It's in a physical location
| controlled by one country.
| blibble wrote:
| there are maybe 2 or 3 factories capable of producing
| these chips
|
| and the sanctions mechanism already exists
| graeme wrote:
| > If it happens, there is no stopping it anyway.
|
| Fatalism is notably a religious sentiment
|
| An incredible ai upside is possible as is an incredible ai
| downside. Blind optimism and fatalism has no relation to what
| the truth of it will be
|
| https://en.m.wikipedia.org/wiki/Fatalism
| fatherzine wrote:
| "liberation" is a false idol.
| antibasilisk wrote:
| >AI can also completely liberate all of us
|
| From what exactly? What it means to be human? When we are
| 'liberated' will we not then be subject to the cage of a
| machine worse than the one we left? We will not annihilate
| ourselves for a false promise.
| moonchrome wrote:
| > If it happens, there is no stopping it anyway.
|
| Hopefully this isn't the case - AI development requires a lot
| of data/compute - societally we could block access to these
| things and prevent further development in that direction.
|
| There are only a few places in existence capable of producing
| HW required or in possession of it. I'm fine with stopping
| compute progress for however long it takes to deal with
| safety issues. Society isn't perfect now but the level of
| compute we've had available for years is sufficient for
| generating a lot more value - we can afford to put AI
| research on ice.
|
| I just don't get it - so much attention is given to things
| like climate change - this is a mild inconvenience compared
| to AGI threat.
| The_Colonel wrote:
| > AI can also completely liberate all of us.
|
| I agree that this is also possible.
|
| > It is better to try and create the most benevolent AI
| possible, that is an ally to humanity and invested in our
| success, than to dwell on our destruction.
|
| I also agree, but who is "we"? I don't think that's a stated
| goal of companies working on AI research. Their goals seem
| largely commercial, and they are not regulated. There's
| nothing stopping a company to produce an autonomous AI for
| military purposes for example (I'm pretty sure there's a
| demand for that). Such military AIs probably wouldn't have a
| very benevolent nature.
| dragonwriter wrote:
| > I've seen multiple top people in AGI space say that total
| annihilation threat from AGI exists.
|
| I've seen lots of people with a financial interest in AI that
| is adversely impacted by it being democratized spread fear of
| that alongside advocacy of regulating AI in a way which serves
| their financial interests.
|
| > We've had inquisitions historically, I would very much
| support a global AI inquisition.
|
| Aside from the (current, immediate, and already being realized
| in a large number of areas) problem from AI deepening existing
| social biases and crystallizes power imbalances, _this_ is the
| biggest threat from AI, and the most likely to cause the
| destruction of human society the soonest - the people endorsing
| a global campaign of unrestricted violency in the name of
| eliminating the supposed threat of misaligned AGI causing a
| future catastrophy by repressing AI technology.
|
| If this kind of attitude gets powerful enough to actually do
| anything, it _will_ cause a world war.
| akvadrako wrote:
| Both can be true at the same time. I see it likely that:
|
| 1. Superintelligent AGI seems inevitable, even if it takes
| thousands of years. So fighting against it requires
| destroying all progress and freedom.
|
| 2. But if it comes we are no longer the dominant species;
| we'll have as much control as big animals do now. If that
| isn't essentially being destroyed, it's close enough.
| moonchrome wrote:
| > I've seen lots of people with a financial interest in AI
| that is adversely impacted by it being democratized spread
| fear of that alongside advocacy of regulating AI in a way
| which serves their financial interests.
|
| Given the rate of progress, shitshow management of basic
| alignment issues and the incentives that lead to those
| decisions - what's your argument that this isn't probable?
| You're betting that it just can't be developed ?
| dragonwriter wrote:
| > Given the rate of progress, shitshow management of basic
| alignment issues and the incentives that lead to those
| decisions - what's your argument that this isn't probable?
|
| It would be nice if the argument I made about _relative_
| probability and _relative_ temporal imminence was an
| argument about absolutes of either, but...it's not, so I'm
| not going to defend those hypothetical arguments.
| luckylion wrote:
| > We've had inquisitions historically, I would very much
| support a global AI inquisition.
|
| You know how inquisition usually goes. "You've been playing
| starcraft against 'the AI'. What do you have to say in your
| defense?"
|
| Maybe I'm just weird, but "a tech-based superintelligence
| emerges and decides to destroy humanity because it was in a bad
| mood" is something I'm totally fine with. Doom-Cults on the
| other hand I don't like at all.
| anongraddebt wrote:
| I think a lot of this stuff is probably just a coin flip,
| because we just don't know.
|
| I find it odd, however, when people, who seemingly accept an
| ASI will be developed, focus the downside risk on extinction
| of us as a species. Sure, that's a risk. You know something
| else an ASI would likely have the ability to do, (or at
| least, one of its progeny)? Keep you alive and torture you
| for eons in weird and surreal ways.
|
| Why is the downside always focused on paltry, meaningless
| things (relatively speaking) like extinction of life on
| earth?
| haswell wrote:
| > _Maybe I 'm just weird, but "a tech-based superintelligence
| emerges and decides to destroy humanity because it was in a
| bad mood" is something I'm totally fine with._
|
| It seems important to reflect on what you're really saying
| here, and to ask how a) this is fine, and b) how "weird"
| sufficiently sums up the stance.
| luckylion wrote:
| I'm fine with humans having outcompeted the neanderthals
| and whatever else there was. I'm just as fine with someone
| else doing it to us, at least if it's someone or something
| that is smarter than us and not some virus or bacteria that
| kills us all (which would be kind of lame). If an
| artificial superintelligence wants to take over the flame
| of progress, I'm fine with that.
|
| But I'm also totally fine with considering that stance
| weird (or whatever word you'd prefer). I'm aware that
| others view these things very differently, just as others
| are much, much more worried about their individual demise
| than I am worried about mine (or theirs).
| haswell wrote:
| Those are not similar/comparable outcomes and I think
| there are two major factors that can't be hand-waved
| away:
|
| 1. Consciousness. Despite advances in "intelligence", we
| still have a very limited understanding of what makes us
| conscious, and whether or not consciousness can emerge
| from machines.
|
| If machines are _not_ conscious and are all that remain,
| I'd argue everything that could be construed to have
| value by humans is lost, and nothing from that point
| forward could be considered "progress".
|
| 2. Suffering. "Winning" on an evolutionary timescale
| looks nothing like the failure modes of machines taking
| over. The reality of this scenario is a rather grim one,
| and not at all like the slow emergence, competition and
| eventual extinction of biological species.
|
| And depending on #1, the true tragedy of #2 begins to
| take shape.
|
| I think it'd be more apropos to frame this as humanity
| collectively committing suicide rather than some notion
| of the future of progress.
|
| If consciousness is the universe experiencing itself,
| what you're describing sounds like a kind of universal
| death.
|
| Of course we can't know what consciousness really is (or
| if earth is the only place it exists), but that seems
| like all the more reason to take these problems
| seriously.
| luckylion wrote:
| Or if consciousness is more than an illusion. Fair
| points. I'm not sure if you can have a general
| intelligence without some form of consciousness. I don't
| believe in gods or souls, so I lean towards us not being
| special.
|
| As for the how, I agree with you. I'd prefer it to not be
| terminators crushing heads under their iron feet while
| allowing us just enough room to run and hide and live in
| constant terror for centuries. But I doubt it will be,
| the power delta will be too large. It'll be like a game
| of Civilization where Gandhi is advancing on you with
| modern tanks and fighter planes while you've barely
| discovered the wheel.
| andrewflnr wrote:
| > I've seen multiple top people in AGI space say that total
| annihilation threat from AGI exists.
|
| You've seen them say it, yes. Have you considered how plausible
| it actually is, or are you just taking their word for it? You
| better have some damn solid evidence before you start talking
| about "inquisitions", and I'm only seeing wild speculation.
| Which, to be fair, is how most historical inquisitions started,
| for all the good those did.
| blibble wrote:
| > When my best bet for humanity is a global nuclear war
| resetting civilization to give us a chance to deal with AGI
| issue down the line - I'm fine with religious doomism.
|
| I'd settle for corporate death sentence for any companies
| developing it with life imprisonment for any
| employees/corporate officers involved at any level whatsoever
| (or VCs funding it)
|
| including selling chips/compute to companies involved
|
| any potential benefits from AI are not worth even a 10% risk of
| it wiping out humanity
|
| this position is going to become more common as the media (and
| electorates) grasp what these companies are attempting to do
| than3 wrote:
| All those companies really want are the profits generated
| from mindless slaves so they don't have to rely on human
| labor for production.
|
| Its a fools journey because it causes a self-fulfilling
| prophecy of destruction given existing societal mechanics and
| they are psychopathic enough to think that's not how it would
| turn out.
|
| What happens when a large number of people can't get food,
| and they know why.
|
| What happens when you have a large number of locusts eating
| all the food being produced.
|
| How would AI differentiate between human thought, and the
| pests it was designed to eradicate?
|
| Thinking machines should be outlawed, and those involved in
| its research purged.
|
| They threaten all humanity, its children, and its future, and
| the thing about percentages is people often don't get how
| they actually work with respect to probability and
| likelihood.
|
| Given sufficient time, as long as its on the distribution
| curve, it will eventually happen.
|
| A 1% chance of an outcome, recalculated in a loop every
| moment with an infinite time will eventually land on that
| outcome at some point. Once that outcome occurs, everyone's
| dead, it may not be instantaneous because time is not a
| constraint.
| allturtles wrote:
| If you really want to go there you could call AI enthusiasm a
| religion promising a scarcity-free heaven that AI will bring to
| us if we just believe in it hard enough. I don't think either
| line of argument is helpful for having a real discussion.
| deepsquirrelnet wrote:
| I'm on the fence. The problem with AI doomsday statements is
| that they don't lead to any productive discussion, because
| nothing specific is discussed.
|
| The format needs to be changed if we don't want it to become
| religion. Pose realistic scenarios and open them for
| discussion. Let's discuss specific things, rather than muse at
| powerful technology and assume it will be bad.
|
| At least AI enthusiasts are excited about specific things that
| can be critiqued and analyzed. They might be deluded in their
| excitement, but it's not religion.
| [deleted]
| BigCryo wrote:
| Be afraid.. be very afraid.. be afraid because the media needs
| your clicks and views.. and the AI grifters need your clicks and
| views and donations..
| low_tech_love wrote:
| Like all cults, they will take the sentence "you're being a bit
| extreme" and respond with "oh so you think there's no danger at
| all?" I love the fact that the top response is by that Yudkowski
| guy "oh so you think it's not the end of the world? Then prove
| it!"
|
| It's just a bunch of people trying to feel and look better by
| attaching themselves to something and then trying to convince
| others that they _are_ that something. Yes, there are dangers;
| no, LessWrong is not the right place to discuss them.
| elromulous wrote:
| The thing that makes the singularity so difficult (edit: to rein
| or reason about), compared with say development of the nuclear
| weapons, is that we know it will be a positive feedback system.
|
| So at some point, the point of no return will be passed, and we
| will have no idea that we passed it.
| marginalia_nu wrote:
| The AI singularity is also an entirely speculative scenario
| straight out of sci-fi. For some reason, many AI doomers are
| talking about it like it's not only something that is well
| understood, but also pretty much an inevitability.
|
| The truth of the matter is we don't even know if it's possible.
| In general, unbounded exponential growth is extremely rare in
| nature. It's common in physics, but only as a result of looking
| at low order approximations.
|
| I honestly think when the dust settles, the smartphone will
| still have had more impact on society than LLMs. If people knew
| in 2009 what was coming, they'd be freaking out like many
| people are today.
| Sharlin wrote:
| AI doomism makes specific, falsifiable predictions.
| nottorp wrote:
| So is ChatGPT worship though.
| crisdux wrote:
| Everything is turned into an existential risk these days. People
| are prone to believe in apocalyptic predictions because they fear
| the unknown and rely on flawed mental heuristics. Apocalyptic
| predictions automatically illicit skepticism from me.
| graeme wrote:
| Counterpoint: the more our capacity grows, the more we have the
| legitimate ability to bring absolute destruction
|
| Plastic or some other chemical ending sperm counts --> we
| previously had no ability to affect the global ecosystem on
| this scale
|
| Bioweapon virus or microorganism eviscerating civilization -->
| we had no prior ability to engineer such a plague
|
| Nuclear holocaust --> we previously had no ability to destroy
| the world with weapons
|
| Runaway climate change --> we previously had no ability to
| alter the climate
|
| Ai sky net moment --> we previously had no ability to make such
| an intelligence
|
| Solar flare cutting all electronics --> we previously had no
| dependence upon a single point of failure
|
| I could go on. We've had plenty of civilizations collapse but
| nothing has been so linked before and we lacked such capacity
| before.
|
| This is not to say any of the above will come to pass. But it
| is foolish to say "apocalypses were previously impossible hence
| such warnings can be dismissed out of hand"
| blowski wrote:
| Yes, this is how I feel about it. Between climate change,
| obesity, GM foods, mega-viruses, insects dying, extreme solar
| flares, and now AI, I'm surprised I'm still here anyway. Just
| perhaps AI will actually help solve some of those problems.
| qwery wrote:
| (I don't know who this person is)
|
| I assume they're talking of 'AI safety' as opposed to AI being
| used by the wealthy to further suppress the working class.
|
| If it's unreasonable to take AI safety seriously because no one
| has built an AI system that poses a credible threat to humanity
| yet, it must also be (at least) unreasonable to dismiss the
| research(ers) based on the same.
|
| That's before considering that people and (profit-motivated)
| companies _are actually trying -- and claiming they will succeed
| -- to create and deploy unimaginably powerful AGI entities
| /systems_.
|
| Assuming no harm can come of this is clearly foolish. The goals
| of such an entity will likely not be aligned with the goals of
| the creator, let alone the general public. The entity will not
| act in your best interest, just as the those seeking to dismiss
| the risk are not acting in your best interest.
| pornel wrote:
| I much prefer AI safety being researched _before_ we build a
| really powerful AI.
|
| This reminds me -- before GPT was even released for the first
| time OpenAI made noises about safety and consequences and it
| was shrugged off, because back then GPT looked like a toy that
| could only write barely coherent Amazon reviews or blogs about
| cryptocurrencies, and that's a too low bar to demonstrate
| intelligence.
| skilled wrote:
| I wouldn't be surprised if OpenAI has been running a model in the
| background that is reinforcing itself. Honestly, it's very
| possible they've made a lot of progress in that area.
|
| I think GPT-4 will be a baseline for quite some time before the
| next "ChatGPT moment" is ready. I think they've got some really
| smart people working at OpenAI who are not just programmers, but
| genuine hackers who want to hack this world just for the fun of
| it.
| ALittleLight wrote:
| This is reasoning by analogy. X is like Y, Y has property A,
| therefore X has A. This approach seems much less compelling to be
| than saying "AI can become more intelligent than humans. A more
| intelligent entity without our moral values may cause extreme
| harm up to and including human extinction. We should be worried
| about AI."
| antibasilisk wrote:
| Then what do we say about the AI utopians and brazenly post-
| humanist machine idolaters who are in favor of doing whatever a
| glorified random number generator says? I don't appreciate being
| gaslit by technophiles, there is a reason why average people are
| starting to get real sick of this shtick.
| mikewarot wrote:
| I reject this false choice. I find myself both looking forward to
| the benefits of AI scaling while simultaneously also concerned
| about the non-zero chance it kills us all.
| [deleted]
| arcticbunny wrote:
| I smell the whiffs of the " Butlerian Jihad". Luddite anti AI
| zealots are coming out of the woodwork.
| flangola7 wrote:
| Calling people "luddite zealots" isn't constructive or adding
| anything to the conversation.
| eternalban wrote:
| Herbert used the word Jihad to underline that the conflict was
| spiritual in nature and not restricted to labor issues
| ("luddites").
| verdverm wrote:
| There are a lot of religious like beliefs about things as people
| have moved away from traditional religions. I wonder if it is in
| our nature to have something bigger than ourselves to believe in
| based on faith in the message
| lamontcg wrote:
| I suspect that this is mostly a first-generation atheism
| problem, and most atheists these days were still raised
| religiously. They spend a lot of time being religious about
| other things, or treating atheism religiously and attacking
| religion with religious levels of fervor.
| frankthedog wrote:
| Perhaps organized religion was a Chesterton's Fence that we
| removed at our own peril.
| [deleted]
| Avshalom wrote:
| I dunno where you live but from my vantage point it hasn't
| really been removed.
| low_tech_love wrote:
| "Throughout human history, as our species has faced the
| frightening, terrorizing fact that we do not know who we are,
| or where we are going in this ocean of chaos, it has been the
| authorities -- the political, the religious, the educational
| authorities -- who attempted to comfort us by giving us order,
| rules, regulations, informing -- forming in our minds -- their
| view of reality." ---T.L.
| IIAOPSW wrote:
| For a sizeable chunk of people possibly so, but it seems not so
| universal. Lots of people don't find any form of "just have
| faith" convincing.
| blast wrote:
| They're probably doing it unconsciously.
| IIAOPSW wrote:
| That's the sort of conjecture which lets you arbitrarily
| dismiss evidence against a theory. Even worse, its the sort
| of conjecture that is plausibly true yet hard to find
| conclusive evidence. Any lack of evidence for a
| subconscious explanation can itself be rationalized away as
| they just didn't look hard enough to uncover it.
|
| So where's the goalpost for falsifying the theory that they
| do it unconsciously? Whats the bar of evidence where you
| change your mind?
| mitthrowaway2 wrote:
| Perhaps that's because just about anything can be portrayed as
| a religion: https://slatestarcodex.com/2015/03/25/is-
| everything-a-religi...
| gerikson wrote:
| I'm convinced religious beliefs/feelings are an unavoidable
| emergent property of human minds.
| kibwen wrote:
| And soon, AI minds.
| kgwxd wrote:
| So it's going to be as disappointing as Battlestar
| Galactica :/
| [deleted]
| jszymborski wrote:
| As always, understanding of nuanced issues will pool at the
| extremes.
|
| It is both true that:
|
| - Adoption of DL allows corps to launder or inadvertently adopt
| unethical practices that exacerbate inequalities.
|
| - We have not created AGI, LLMs are not AGI, stable diffusion is
| not AGI and none of these fields of research are indication we
| are closer to AGI. This is like thinking Clever Hans is the
| portent of our future equine overlords.
|
| A great number of people who claim that we are one minute from
| AGI overlord midnight are doing the old school carnival barker
| trick. "This snake oil is so potent! It can kill you if you use
| too much, so be sure to listen carefully to my instructions!"
| serverholic wrote:
| I'm curious why you are so confident in your assertion. It
| seems to me that an advanced statistical model of the world is
| an essential component of AGI. How do you know that we aren't a
| few breakthroughs away from AGI?
|
| Some recent papers have shown significant performance
| improvements when these models are allowed to respond to their
| own outputs.
|
| How do you know that putting an LLM in a fancy loop with access
| to external memory and tools isn't AGI?
| jtode wrote:
| In actuality, the large hardon collider caused that black hole
| everyone was concerned about we are experiencing the evaporation
| of our atoms through an extreme time dilation; the perception of
| a coming AI beast is actually a hallucination of our own
| consciousness, from which we have been torn asunder.
|
| Makes as much sense as any of the other prophecies of doom.
| CharlesW wrote:
| I mean, Yann is paid by Meta to propagate an "only positive
| vibes" stance. "AI doomism" itself is a ridiculous phrase meant
| to demonize anyone with legit concerns, of which there are many.
| I say this as someone who is very excited about recent
| developments and sees the potential upside as well as anyone, I
| think.
| The_Colonel wrote:
| It's kind of difficult to take these people seriously given
| they have a strong financial stake in it.
| stevenhuang wrote:
| Yann is also known for constantly denigrating the abilities of
| LLMs and being comically wrong all the time, saying they can't
| do X only to be proven wrong immediately after. He's pretty
| much a meme at this point.
|
| He spent all his years at FB building ivory towers of
| mathematics attempting the analytical route for building AI
| systems only to be out done by the bitter lesson.
|
| He's perpetually salty about the current state of LLMs, you can
| sense it from all the negative commentary on the matter if you
| look at his Twitter feed lol. For shame.
| cubefox wrote:
| He also likes to talk about "human level AI". That's like
| expecting that the invention of the airplane passes through a
| stage of "human level flight". That didn't happen. And
| probably also not for AI.
|
| These systems will vastly outperform humans at more and more
| tasks, while being still sub-human at others. When the time
| comes that they surpass humans at all tasks, they will be
| vastly superhuman overall. And consequently much more
| dangerous than LeCun's "human level AI".
| andrewflnr wrote:
| The analogy is more apt with a reference to "a stage of
| bird-level flight". Humans can't fly at all, but in the
| same way birds and planes both fly, but humans and machines
| both process information about the world, just in very
| different ways and for different reasons.
|
| But yes, the notion of "human level AI" seems to be
| irrelevant at best.
| beernet wrote:
| Agreed. He had a luck shot with CNNs a long time ago and
| that's about it. Pretty sad to witness overall
| cma wrote:
| Didn't he hype the Galatica LLM?
| [deleted]
| eggsmediumrare wrote:
| Am I an apostate if my concern about AI doom now outweighs my
| concern about climate change?
| antibasilisk wrote:
| I'm hopeful that climate change will prevent AI doom, because
| that seems to be the best outcome at the moment.
| woodson wrote:
| Well, in general, there seems to be a correlation between what
| news coverage focuses on and what is considered worrisome at
| any given time. In times of high inflation and increased fuel
| prices, climate change action takes a back seat.
| oliwarner wrote:
| We're already seeing jobs being done by AI. Many creative jobs
| that would take a team a week to handle are being shopped out to
| a bot to handle in an hour.
|
| VFX are going to get hit particularly hard by this. Everything
| Everywhere All at Once is a visually stunning film yet had a
| 6-member team handling graphics post. They were smart, but they
| also shopped out all the dull rotoscoping work to AI. Look at any
| Marvel film from the past decade. Hundreds of artists per title,
| thousands if you include costume design, storyboarding, audio,
| even writing.
|
| Most of this "doomism" is people realising how quickly their jobs
| could be automated. A future where your once-valued skillset
| suddenly has no monetary value is scary.
| FormerBandmate wrote:
| This has been happening forever. How do you think all those
| Rust Belt workers who got outsourced to China felt?
|
| Tons of jobs were eliminated by the computer too. It's just
| part of life, there's always more work to be done. We need
| better ways to help people than the crap way we handled
| outsourcing job losses in the 90s, however
| alphanullmeric wrote:
| It's so wonderful to see all the artsy people get automated out
| of existence after claiming that software engineers contribute
| nothing to society. The only downside is that even more of them
| will now be living off of taxpayer money than before.
| lewhoo wrote:
| You just took hundreds of thousands if not millions of
| people, put them in a bag and threw the bag into the sea just
| because one or some of them were maybe mean to you. I hope AI
| overlords won't learn from you.
| flangola7 wrote:
| > after claiming that software engineers contribute nothing
| to society.
|
| Frankly, what in god's name are you talking about?
| alphanullmeric wrote:
| Perhaps you haven't talked to enough tumblr-type people
| yet. The ones that are either unemployed and living off of
| welfare or teach their useless subject to others in a
| taxpayer funded school, claiming it's unethical to work at
| FAANG or a hedge fund. The ones that seem to support
| intellectual property when it comes to AI "stealing" their
| art, but would be vehemently against it for NFT monkeys or
| when they want to pirate productivity software. The ones
| supporting the ChatGPT ban in Italy and other uses of state
| force that would benefit them.
| version_five wrote:
| I'm happy to see there's still an adult in the room. Andrew Ng
| has also been very reasonable about the current pop culture hype
| bubble, while many others have decided to try and use it for
| attention. As I've said before, there's a clear divide between
| the actual practitioners, and the "futurists" (Musk etc) on this
| topic.
| drcode wrote:
| Hinton is not an adult?
| https://twitter.com/JMannhart/status/1641764742137016320?t=I...
| version_five wrote:
| People can watch the short clip and decide for themselves. He
| kind of says "anything's possible" in response to interviewer
| prompting.
| PaulHoule wrote:
| There is "A.I. doomism" in general and there is also a specific
| cult-like group at the lesswrong website that really is an
| apocalyptic religion. It is astonishing how anything that
| deviates from its dogma in any way has to have "trigger warnings"
| and the senseless comments that users write on anything that
| suggests there could be intrinsic limits on intelligence.
|
| There ideology is such that a higher intelligence is going to
| take our problems away either by taking over, ending us (no
| people, no problems) or possibly the followers moving on to the
| "next level" like the followers of
|
| https://en.wikipedia.org/wiki/Heaven%27s_Gate_(religious_gro...
| lodovic wrote:
| It's a pretty scary idea that AI could take a religious stance
| since its training sets contained religious texts, too.
| Especially in the context you provide.
| ctoth wrote:
| Paul, can you point at anything at all other than your general
| vibes that would indicate that anyone from LessWrong intends to
| literally participate in mass suicide, as your link implies?
| Did I miss the big group buy of phenobarbital?
| skrebbel wrote:
| I'm not Paul, but this here has some serious Jonestown
| overtones:
| https://www.lesswrong.com/posts/j9Q8bRmwCgXRYAgcJ/miri-
| annou.... It makes Yudkowski seem (to me) like the leader of
| a suicide cult more than anything else.
|
| The argument "yeah but what if he's right?" is pretty much
| Pascal's Wager by now, given how way out there the AI Doomers
| have gotten. I choose not to worry about the God from the
| Bible (ie I don't make make Pascal's Wager) so why should I
| worry about this new Evil Nerd-God Apocalypse?
|
| Don't drink the Kool-Aid, kids!
| [deleted]
| charcoalhobo wrote:
| FYI this article was published on April 1 and is labelled
| as "April Fool's".
| skrebbel wrote:
| hhhahaahha woa I missed that completely
|
| I'm pretty sure i'm not the only one who took it
| seriously. Thanks! I wonder if mr LeCun made the same
| mistake, it's just... almost believable! Good April fools
| joke I gotta admit.
|
| EDIT: it's now 5 minutes later and I'm still laughing. I
| believed for _months_ that Yudkowski had started a death
| cult.
| cubefox wrote:
| It wasn't entirely in jest. See this article, published a
| short while later:
| https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-
| ruin-a...
|
| And more recently: https://time.com/6266923/ai-eliezer-
| yudkowsky-open-letter-no...
| cma wrote:
| Didnt they change recently to a "death with dignity" AI
| strategy?
|
| Now their leader is saying 50% we should have bombed Wuhan
| and the only reason for an individual not to firebomb an AI
| datacenter is because they'll just build back another: https:
| //www.twitter.com/ESYudkowsky/status/16414107296675962...
| skrebbel wrote:
| > Didnt they change recently to a "death with dignity" AI
| strategy?
|
| That was an April Fools joke! Yeah I just found out today
| as well (https://news.ycombinator.com/item?id=35402472).
| Worked a little too well huh
| cma wrote:
| Seems they are really taking the AI threat seriously
| then. And it is all the same stuff he's been saying on
| recent podcasts so it is a year-long April Fools joke.
|
| I heard the death with dignity thing repeated in a Peter
| Thiel speech as a criticism of them.
| flangola7 wrote:
| Being criticized by Peter Thiel is probably just a sign
| you're not a psychopath.
| cma wrote:
| Maybe as a rough rule of thumb
| lumenwrites wrote:
| This is an inaccurate and ignorant comment that does not engage
| with the arguments people on LessWrong are making. If you have
| something smart to say about why these people are wrong in
| their beliefs, please do. Name calling a group of people a
| "cult" or "apocalyptic religion" is not an argument that
| contributes anything interesting to the conversation.
|
| If anyone is actually curious about what these people are
| saying, here's a list of their arguments for why AI is
| extremely dangerous and extremely likely to kill us all:
| https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a...
|
| If anyone has a coherent counter-argument to any of that, I'd
| really love to hear it.
| evilduck wrote:
| Find and replace "God" for "AGI" and a lot of that link's
| reasoning will look awfully familiar to atheist circles.
| antonvs wrote:
| A good intro for why they're wrong in their beliefs is
| https://idlewords.com/talks/superintelligence.htm
| version_five wrote:
| here's a list of their arguments for why AI is extremely
| dangerous and extremely likely to kill us all
|
| Not familiar with them in the least, but the fact that
| they're a group that's assembled such a list basically proves
| they're something of a doomsday cult, no?
| ctoth wrote:
| Man, wait until you learn about the climate change
| movement.
| cwmoore wrote:
| That link is not at all a list of "why AI is extremely
| dangerous and extremely likely to kill us all". I skimmed it
| (there's a lot of writing assuming grave danger) but found
| only the following example of AI lethality, and it appears
| fantastical:
|
| > ...nanomachinery builds diamondoid bacteria, that replicate
| with solar power and atmospheric CHON, maybe aggregate into
| some miniature rockets or jets so they can ride the jetstream
| to spread across the Earth's atmosphere, get into human
| bloodstreams and hide, strike on a timer.
| uxp100 wrote:
| It's interesting how the Eric Drexler influence clearly
| colors a lot of apocalyptic AI stuff, in fiction, and on
| Less Wrong.
|
| First because afaik the progress in nanotechnology has not
| been particularly fast over the past 40 years since his big
| book came out, and second because it really seems like it
| is only tangentially related. In a world with highly
| advanced assembler mnt, seems like the world could end in a
| way akin to a serious chemical accident accident, few
| computers involved. But the assumption is AI will certainly
| create nanotechnology, where humans have so far been unable
| to? That's less like AI risk and more like, one smart
| research lab risk.
| eternalban wrote:
| https://news.ycombinator.com/item?id=35402539
|
| mind viruses.
| cwmoore wrote:
| > The big ask from AGI alignment, the basic challenge I
| am saying is too difficult, is to obtain by any strategy
| whatsoever a significant chance of there being any
| survivors.
|
| There's something more here that I think is lost in the
| metaconversations, the control question generally, or
| ceding control to learning machines. The argument is that
| we are hoping for the best while remaining unaware of not
| preparing for the worst. Actual scenarios aside, what if?
| cubefox wrote:
| Here is a much more realistic example:
| https://twitter.com/RokoMijic/status/1641806556038676481
|
| In general, the argument "All the examples how AI would
| overpower us are way too exotic for my taste" is like one
| chimpanzee complaining to another chimpanzee "Your example
| how humans would overpower us is totally implausible and
| not worth taking seriously." The mistake is that both have
| not the intelligence to invent or understand a realistic
| example, but that is not at all evidence that humans can't
| overpower chimpanzees.
| cwmoore wrote:
| I'm just not seeing any real examples, exotic or
| otherwise, of how to recognize the risks involved in
| superhuman AI.
|
| Smarter than us does not imply any particular will or
| agency. Of course the will and agency of the people
| employing the technology will reveal its immediate
| impact, likely toward short term profit, and alignment as
| a control plane employed toward the same.
|
| Chimpanzees have limited control over humans, humans have
| limited control over their own technology. But of course
| chimpanzees can recognize the risk of armed people.
|
| The point is not to debunk fanciful end games. Better to
| prepare for likely effects on the human network. I was
| promised a list.
| joshuahedlund wrote:
| > My lower-bound model of "how a sufficiently powerful
| intelligence would kill everyone, if it didn't want to not do
| that" is that it gets access to the Internet, emails some DNA
| sequences to any of the many many online firms that will take
| a DNA sequence in the email and ship you back proteins, and
| bribes/persuades some human who has no idea they're dealing
| with an AGI to mix proteins in a beaker, which then form a
| first-stage nanofactory which can build the actual
| nanomachinery. (Back when I was first deploying this
| visualization, the wise-sounding critics said "Ah, but how do
| you know even a superintelligence could solve the protein
| folding problem, if it didn't already have planet-sized
| supercomputers?" but one hears less of this after the advent
| of AlphaFold 2, for some odd reason.) The nanomachinery
| builds diamondoid bacteria, that replicate with solar power
| and atmospheric CHON, maybe aggregate into some miniature
| rockets or jets so they can ride the jetstream to spread
| across the Earth's atmosphere, get into human bloodstreams
| and hide, strike on a timer.
|
| Does this kind of thing sound plausible to you? How much
| molecular biology have you studied? Even as someone who's
| just been reading a genomics textbook for fun, the phrase
| "strike on a timer" alone is ludicrous. It sounds like
| someone who's spent more time in science fiction. These are
| wildly speculative claims not at all grounded in believable
| science.
| a_shovel wrote:
| (You quoted the same paragraph I did, so I'll just delete
| my comment and move my response to this quotation here.)
|
| Exactly. Why am I supposed to take this seriously?
|
| Believers in Superintelligenceism seem to assume that any
| problem in any field - physics, chemistry, biology, math,
| sociology, geometry, whatever - that hasn't already been
| proved impossible to be within the capabilities of an AGI,
| justified with a handwave and "oh, it's so super smart that
| it'll have it all figured out in an afternoon". This has
| more in common with religious concepts of omnipotent and
| omniscient gods than with anything any AI developers are
| working on today.
| cubefox wrote:
| You are demanding "believable science" from a chimpanzee of
| how humans would overpower them. Do you not recognize the
| enormous difference intelligence makes? And there does not
| seem to be a hard constraint of how much more intelligent
| than a human an AI could become. Their brains are not
| tightly restricted by size and energy. The difference could
| be much greater than between chimpanzee and human.
| 23B1 wrote:
| The easiest counterargument is that humans are provably,
| systematically, and consistently awful at predicting the
| future.
| [deleted]
| Fricken wrote:
| The US initiative to block chips and chip fab tech from getting
| to China cites preventing China from getting their hands on
| advanced AI as the main reason.
|
| This report from CSIS (good read) is titled "Choking off
| China's Access to the future of AI"
|
| https://www.csis.org/analysis/choking-chinas-access-future-a...
| drcode wrote:
| A religion is arguably when you refuse to engage with other
| people's arguments
|
| Lecun has refused to engage with anyone's arguments, so why are
| the people worried about AI ex-risk the religion?
| gerikson wrote:
| > A religion is arguably when you refuse to engage with other
| people's arguments
|
| I've never heard of that definition.
| programmer_dude wrote:
| Me neither, but I like it. When you refuse to engage with
| other people's arguments due to strongly held beliefs you are
| engaging in a religious activity. Those strongly held beliefs
| are your religion.
| f_devd wrote:
| Neither have I but it actually lines up pretty well depending
| on your read of GP. Large religions often use a proving-a-
| negative with a pascals wager to achieve an inarguable story
| regardless of content. And cult religions generally do this
| with an Us-vs-Them approach which usually prevents members of
| believing external arguments.
| dralley wrote:
| AI optimism is often indistinguishable from "technological
| manifest destiny"
|
| https://www.youtube.com/watch?v=ajGX7odA87k&t=2207s
| pclmulqdq wrote:
| All technology is built on some sort of "manifest destiny." We
| are all delusional about how great our new thing is going to be
| and how much it is going to change the world. Sometimes, like
| the computer and the web, it turns out to be right. Sometimes,
| like cryptocurrency and the dirigible, not so much.
| pjdkoch wrote:
| Fantastic talk!
| mimd wrote:
| Being bored, scared, and on Twitter is a bad combo. I'm mostly
| down on the current _AI_ , as it's ML leveraging the vast
| "intelligent" conversations we put out on the internet. It's
| wheel of fortune and it failed my tests on reasoning miserably,
| even with the answers in hand. (I'm not going to tell you which
| tests, because then OpenAI will scrap it and tune their model to
| claim victory in GPT5. I saw the feathers bricks cheating!). That
| doesn't mean that these technologies could not do damage, the
| greatest risk bring hubris in it's capabilities. I wouldn't rely
| on it's reasoning for launching an ICBM for example. But I
| suspect that it's contributions/detractions to the workforce will
| be more muted than thought, as the current use cases seem
| marginal in cost savings. Ie, HR letters from management time
| savings using templates versus a AI generated one. But I don't
| see how we get from a wheel of fortune completer to, in the usual
| example, an AI whose goal is to eliminate trash by eliminating
| humans. It's missing components and training necessary to learn
| those "functions", and their's little incentive to do so. I'd
| rather have the benefits of understanding protein folding for
| lifesaving drugs, decreased costs on physics simulations for
| fusion reactors, and so on, than stagnation due to someone's bad
| scifi.
| smrtinsert wrote:
| When exactness, liability and oversight matters, it's just not
| ready yet, period. Even when its ready for that, it will take
| years for legislation to make it possible to use in key areas
| such as finance or medicine or law.
|
| While some of the public is scared, others are thrilled as they
| expect an AGI takeover somehow results in a egalitarian UBI
| cushioned existence.
|
| It's a mess.
| mimd wrote:
| It is.
|
| The AGI takeover crew is a bit nauseating. But I sorta
| understand them considering our daily panics. Sorta a dream
| of being whisked away by a prince/princess.
|
| But the scared ones are much more organized, and have been at
| it for a good while now. And they're more of a threat, as
| they could shut this down for no gain, while the AGI takeover
| ain't happening with this wave of AI.
|
| Still, I worry about a period in which doctors/lawyers forget
| their liabilities and decide to consult the great chatgpt
| oracle. Makes for a wonderful daily fail story. ;)
| lewhoo wrote:
| Why don't we give Sam Altman the benefit of the doubt here. He
| himself admitted on several occasions that AI is potentially
| dangerous. Why would he do that if it only raises the discussion
| whether or not it should even exist ? That can't be a PR stunt.
| Imagine Musk saying something like that about some new Tesla
| autopilot (yeah yeah I know /s).
| SilverBirch wrote:
| The key thing that AI boosters and doomsters have in common is
| that essentially they agree that this AI tech is fundamentally
| different in kind than what has gone before. It's not a matter
| of a computer being 5% faster at a particular problem, it's
| about the technology having crossed some sort of fundamental
| barrier beyond which all sorts of crazy things happen -
| conciousness, self-improvement, independent motivations etc.
| Now why would the founder of OpenAI admit to the possibility of
| it being dangerous? Because if you accept that it could be
| dangerous, you have to accept the premise that it's not just
| something a little better than before but that it really does
| have this different nature. And that different nature would -
| if it were true - make this AI technology _enormously
| valuable_. What would be really bad for the founder of OpenAI
| would be to admit that actually the technology isn 't
| dangerous, because it can't do all those crazy things we
| imagine. It's the difference between self-driving and adaptive
| cruise control. And remember - the promised self-driving 5
| years ago looks today a lot like it ended up just being
| adaptive cruise control.
| lewhoo wrote:
| I fundamentally disagree. Admitting danger dissuades people.
| Most want the thrill but not real danger. Placing the value
| in the danger aspect is like saying computer viruses are the
| most valuable thing in the world. I'm exaggerating a bit, but
| the way you put it sounds to me a bit much given the fact
| that Altman really doesn't have to convince anyone at this
| point that they have something very different.
| The_Colonel wrote:
| > Complete with prophecies of imminent fire and brimstone caused
| by an omnipotent entity that doesn't actually exist.
|
| That seems like an absurd comparison given people like LeCun are
| actively and intentionally working towards creating this super-
| human entity.
|
| (Some) scientists were rightfully worried when creating a nuclear
| bomb as well. The fact that we managed to survive 70 years with
| its existence does not automatically mean it has to work out with
| every such invention.
|
| Nukes have the fortunate properties that they are very
| difficult/expensive to produce even for countries, and then you
| need very difficult to design and expensive systems to deliver
| them. It's rather doubtful that AIs will have similar
| constraints.
| version_five wrote:
| We really need some perspective here. One one side is a force
| capable of unleashing unfathomable amounts of energy, with all
| kinds of additional negative side effects, that has both been
| extensively demonstrated and has been used for mass
| destruction.
|
| On the other is a chatbot, that could what, "spew
| misinformation", take our jobs, someday through a mechanism not
| known and not related to how it works become sentient and hurt
| us? Seriously, the comparison is as absurd as the fear
| generally. There are way scarier things going on, it's sad we
| can't give them 1% of the attention we give things we made up
| completely.
| olalonde wrote:
| Talking about perspective, do you maintain your prediction[0]
| that no meaningful business cases will emerge from generative
| AI?
|
| [0] https://news.ycombinator.com/item?id=34197033
| version_five wrote:
| Yes - I'm happy you found that. I don't think it's really
| related to this thread, but I maintain we're firmly in the
| "demo" phase, and it's going to fizzle out when the rubber
| hits the road and companies actually try and find a way to
| justify their investments. Not that I'd rule out there
| being some crappy customer service chatbots being fielded.
|
| Goalposts (including mine surely) will get moved around to
| claim successes / no successes, but the balance is going to
| be well towards indifference to the tech as opposed to any
| real changes resulting from it.
| olalonde wrote:
| Apologies for the tangent, it wasn't related to this
| thread. I've been in the habit of bookmarking predictions
| on HN to revisit them later, and your username caught my
| eye. I'm a bit surprised you maintain it, but the final
| verdict remains to be seen.
| version_five wrote:
| What have you seen since the new year that makes you
| surprised I'd change an opinion I had then?
| olalonde wrote:
| OpenAI's GPT4. Bing's AI chat. Google's bard. Midjourney
| v5. GitHub copilot X. etc.
| version_five wrote:
| I was deeply involved in the last AI hype cycle. I'd want
| to see lasting applications giving clear benefits before
| I change my opinion. More foundation models don't change
| that, nor do the startup launches based around llms I've
| seen so far. It all has the same 2016 "deep learning"
| vibe, lots of apparent potential but no pathway to actual
| use cases.
| The_Colonel wrote:
| > On the other is a chatbot, that could what, "spew
| misinformation", take our jobs, someday through a mechanism
| not known and not related to how it works become sentient and
| hurt us?
|
| I agree that LLMs are not extinction-level dangerous and
| somewhat share the skepticism that they should lead to AGI.
| (Although they are pretty capable given their lack of true
| intelligence and I think people will find ways to plug them
| into dangerous applications.)
|
| But when LeCun defends AI, he defends "real", intelligent AGI
| (because he doesn't consider LLMs to be AGIs).
| antonvs wrote:
| As I've written in another comment, LeCun is not defending
| the boogeyman that the LessWrong cult has invented, which
| is a pure creature of fantasy.
| pid-1 wrote:
| The irony in this statement is that LeCun is a huge LLM
| sceptic.
| antonvs wrote:
| > That seems like an absurd comparison given people like LeCun
| are actively and intentionally working towards creating this
| super-human entity.
|
| There's a disconnect in that statement. The imagined
| superintelligence that people like the LessWrong cult speculate
| and worry about is not the same thing as the AI that LeCun is
| working towards. The cultists simply speculate, against all
| evidence, that they somehow could end up being the same thing.
|
| It would be much better if the AI risk conversation focused on
| the real risks, which are human abuse of AI, which will have
| effects like increasing inequality and in general amping up the
| dystopias we're already doing a great job of building without
| help from AI. The "conscious superintelligent AI" is a scifi
| LARPing game that some people are getting paid to play, which
| some credible people seem inclined to confuse with reality.
|
| > That seems like an absurd comparison
|
| It's on point. Anthropologist Clifford Geertz proposed this
| definition of religion: "Religion is a system of symbols which
| establish powerful, pervasive, and long-lasting moods and
| motivations in men by formulating conceptions of a general
| order of existence and clothing these conceptions with such an
| aura of factuality that the moods and motivations seem uniquely
| realistic."
|
| It's that "aura of factuality" that makes superintelligent AI
| "seem uniquely realistic" to the believers of this nascent
| religion, a.k.a. cult. But when you step back and actually look
| at the foundational tenets in a vaguely rational light, they
| evaporate.
| olalonde wrote:
| Is your argument that we're far away from developing a
| "conscious superintelligent AI" or that it's not possible _in
| principle_?
| antonvs wrote:
| The answer to the former is almost certainly yes.
|
| The answer to the latter is unknown, given that we really
| have no idea what the sufficient conditions for
| consciousness are.
|
| Edit: to be clear about my argument: I absolutely think we
| should be taking the risks of AI seriously, but those risks
| have to do with how humans will use the technology.
| Unfortunately, we're collectively bad at managing those
| kinds of risks, because it requires managing other humans
| with their own ideas and agendas.
|
| That's exactly why claiming the risk comes from the
| technology itself seems so appealing to some people: if the
| risk is purely from the technology, then all we need to do
| is "shut it down", to quote Eliezer Luddkowsky writing in
| Time Magazine recently.
| drc500free wrote:
| I found LW to be a great community for understanding
| rationality about anything other than AI.
|
| The GAI that LW is concerned about is a suspiciously perfect
| fit for the yahweh-shaped-hole in the belief structures of
| someone who was raised as an observant jew before turning to
| atheism. The structure of how they believe humanity would
| engage with such an intelligence could drop in alongside any
| other apocalyptic prophecy in the old testament without
| sticking out, and the discussion format is like a yeshiva
| that takes place on an electronic bulletin board.
| librexpr wrote:
| Oh my god will you people stop with the bullshit
| psychoanalysis. People disagree about things all the time,
| it's normal, you don't need to invoke this pseudo-Freudian
| nonsense to explain why people disagree with you about the
| risks of AI. You can do better than this.
| antonvs wrote:
| Ignoring the "observant Jew turned atheist" aspect, the
| rest of that description seems apt. A comparison against
| the features of religions is valid here, considering that
| one of the criticisms is that these people are following
| a religious playbook rather than a rational one.
|
| We don't enact public policy based on predictions of a
| biblical apocalypse, nor do we ban televisions or
| computers because groups like the Amish disapprove of
| them. The same reasoning should apply here.
| coldtea wrote:
| No reason to ignore even the first part though. Culture
| matters, and reaction to parental culture is very often a
| driving force...
| librexpr wrote:
| None of the comment is apt. You're seeing people you
| disagree with and assuming that if they disagree with you
| it must be because they're crazy, and then you use words
| like "cult" and "LARP" to try to dismiss them without
| engaging with their arguments. This does not lead to good
| discussion.
|
| I think that unaligned and sufficiently smart AI could
| kill everyone because I looked at the arguments for and
| against and came to a conclusion. Presumably this is the
| same thing you did. I know my own mind better than you,
| and I'm telling you that your attempts to explain why I
| disagree with you are completely off the mark.
|
| > We don't enact public policy based on predictions of a
| biblical apocalypse, nor do we ban televisions or
| computers because groups like the Amish disapprove of
| them.
|
| The reason we don't do that is because we evaluate their
| arguments and disagree with them. It has nothing to do
| with who they are or why they believe what they believe.
| blibble wrote:
| > The cultists simply speculate, against all evidence, that
| they somehow could end up being the same thing.
|
| something as stupid as bitcoin consumed a vast amount of
| resources while producing nothing useful to humanity, simply
| because of economic incentives
|
| it's not difficult to imagine something slightly smarter than
| bitcoin
| FormerBandmate wrote:
| Bitcoin's negative impact on the world is 1/10th that of
| freon. AI risk people's predictions sound like they're
| writing bad Terminator fanfiction.
|
| AI has real dangers completely dissimilar from what people
| describe. It is very good at creating clickbait,
| misinformation, and persuasive material the unsuspecting
| might not immediately see. Phenomenal for scammers and
| propagandists, the same way as the internet
| blibble wrote:
| > Bitcoin's negative impact on the world is 1/10th that
| of freon.
|
| the difference is freon did something useful
| mimd wrote:
| That's uncharitable to Bitcoin. Bitcoin was offering a way
| to mostly anonymously send cash in a verifiable manner,
| independent of payment processors and govs. That's
| something useful. Now, it wasn't good because of its
| inefficiency, it's conflicts with govs, it's tilt towards
| early buyin, etc.
|
| Getting angry to the point of clouding analysis and
| judgement is no way to rush into these things.
| antonvs wrote:
| > it's not difficult to imagine something slightly smarter
| than bitcoin
|
| I'm not saying there aren't risks from AI, or that they
| shouldn't be considered. But risks like the ones you're
| alluding to are not going to be affected by a 6-month
| moratorium.
|
| The problem is that the people yelling loudest about risks
| are not talking about plausible problems like the one you
| mention, they're worrying that an AI will decide to take
| over the world and reuse the atoms that make up humans for
| its own purposes. The first reply to LeCun's tweet is a
| leader of one of those cults.
|
| If people make the mistake of listening to them, we're
| _more_ likely to end up with problems like the ones you 're
| describing, because we'll be paying attention to the wrong,
| imagined problems.
| version_five wrote:
| Happy I saw this comment, I hope this is indicative of the
| majority of those with some actual understanding of the
| subject, and the sensational stuff that gets attention is
| really just cult leaders and followers seeking attention.
| The_Colonel wrote:
| > The "conscious superintelligent AI" is a scifi LARPing game
| that some people are getting paid to play
|
| Thinking our meat brain's consciousness is somehow special,
| and our intelligence can't be surpassed by a machine, sounds
| much more like a religion to me.
|
| AI researchers are working towards making intelligent
| machines. Why are we supposed to believe that - if they
| manage to achieve their goal - these machines won't surpass
| our intelligence?
| antonvs wrote:
| > Thinking our meat brain's consciousness is somehow
| special, and our intelligence can't be surpassed by a
| machine, sounds much more like a religion to me.
|
| Lucky I didn't say anything remotely like that, then.
|
| > AI researchers are working towards making intelligent
| machines. Why are we supposed to believe that - if they
| manage to achieve their goal - these machines won't surpass
| our intelligence?
|
| I didn't say that they couldn't. In fact in certain
| respects, GPT-4 has already surpassed average human
| intelligence. No human in the world could compete with its
| rate of correct answers to arbitrary questions, for
| example, or its speed in writing computer programs up to a
| certain size. Yet no-one is seriously concerned that GPT-4
| will end the world. For that doomer scenario to come to
| pass, we need to add various fantasy elements (even calling
| it "scifi" perhaps gives it too much credit.) At this
| point, I'll refer you to this as a good intro for why the
| AI doomer cultists are misguided:
| https://idlewords.com/talks/superintelligence.htm
| The_Colonel wrote:
| > Lucky I didn't say anything remotely like that, then.
|
| Then surely AI research creating a superintelligent
| conscious AI is a realistic possibility and not a "scifi
| LARPing".
|
| > Yet no-one is seriously concerned that GPT-4 will end
| the world.
|
| I'm not concerned that GPT-4 will end the world. Maybe
| not even GPT-8.
|
| LLMs doesn't seem like a very prospective way towards a
| strong AI. But AI research is not united on LLMs and
| works on different approaches, some of which could lead
| to a superintelligent AI.
| circuit10 wrote:
| "For all we know, human-level intelligence could be a
| tradeoff. Maybe any entity significantly smarter than a
| human being would be crippled by existential despair, or
| spend all its time in Buddha-like contemplation."
|
| That would probably be quickly optimised out by the
| training process as it's an inefficient way to achieve a
| goal. The only basis for this argument is that humans are
| smarter than other animals and this seems unique to
| humans but it's probably just a quirk of humans
|
| "With a big enough gap in intelligence, there's no
| guarantee that an entity would be able to "think like a
| human" any more than we can "think like a cat"."
|
| LLMs are clearly already very good at "thinking like a
| human"
|
| "he still couldn't talk the cat into it."
|
| Because cats can't talk? Humans can, and can clearly be
| manipulated into doing things, sometimes even by current
| AI
|
| "So how are we supposed to solve ethics and code a moral
| fixed point for a recursively self-improving
| intelligence" " The idea that we can securely design the
| most complex system ever built, and have it remain secure
| through thousands of rounds of recursive self-
| modification, does not match our experience."
|
| Yes, that is why AI alignment is hard, that's an argument
| for that view, not one against it
|
| "I don't buy this argument at all. Complex minds are
| likely to have complex motivations; that may be part of
| what it even means to be intelligent."
|
| This is a clear misunderstanding of what it means to be
| intelligent. It doesn't mean "do things that humans find
| impressive", it means "achieve your goal effectively". An
| AI that goes against its own goals is not intelligent,
| would not achieve its training objective and the training
| process (which somewhat resembles evolution) would
| optimise it out in the next step of training in the same
| way that a species that immediately throws itself off the
| nearest cliff will not survive to the next generation.
| You can imagine a superintelligence as something that
| goes over all the actions it could take, sees what reward
| it would get according to its goal and picks that action.
| The intelligence here is in the world model it uses to
| predict the consequences of its actions, picking what
| actions to take is a simple "max over array" function so
| saying it will randomly decide to persue a more
| meaningful objective is like saying that a sorting
| algorithm will put a list in the wrong order because it
| thinks some other order is more meaningful
|
| (I will continue in a new comment in case there's some
| kind of length limit)
| circuit10 wrote:
| "When we look at where AI is actually succeeding, it's
| not in complex, recursively self-improving algorithms."
|
| That's because humans are better at this currently. When
| AIs get better at this than humans they can take over the
| process and improve a lot faster
|
| "My roommate was the smartest person I ever met in my
| life. He was incredibly brilliant, and all he did was lie
| around and play World of Warcraft between bong rips.
|
| The assumption that any intelligent agent will want to
| recursively self-improve, let alone conquer the galaxy,
| to better achieve its goals makes unwarranted assumptions
| about the nature of motivation."
|
| Humans are satisficers, not maximisers. If we can figure
| out how to build a satisficer, great, but we can't do
| that currently. Also maybe humans are this way because we
| have historically needed to conserve energy to to
| survive, AI may not have that constraint in the same way
| https://youtu.be/Ao4jwLwT36M
| userforcomment wrote:
| I think nuke analogy is very tempting since US banned A100
| export to China. One could assume that AI race of 21st century
| ~ nuke race in 20th. Note that Nvidia circumvented ban, so race
| is on..
| nwerla wrote:
| The AI does not have to be omnipotent. People are worried about:
|
| 1) Job losses, which is a real concern, except for LeCun, who is
| safe or rich enough.
|
| 2) Some fools connecting a war game AI to real hardware, which is
| quite possible.
|
| Other than that, I don't see "doomism" at all. I see outrage at
| the IP theft that exploits altruistic humans who posted their
| works on the Internet.
| inconfident2021 wrote:
| I can't comment or argue with prominent scientists such as
| LeCunn. But I surely can give perspective as SWE about to enter
| the industry.
|
| 1. We live in an uncertain time. Before most of the people, at
| least with a bit of research could have predicted the trajectory
| of technological innovation. Being born in era of tech, it was
| very easy to adopt to it. From PC to now, I have embraced every
| technological progress. Yet, nothing sent chills like this one.
|
| 2. I do not worry about my job. What I worry is most of the world
| is about getting shit done and this tech helps you get your shit
| done (for now). If we are at the upper tail of some sigmoid,
| there is nothing to fear. If we are at slope, oh boy, I cannot
| fathom the economic domino this tech is about to bring out.
| People are not rational. We are mostly governed by emotion. If
| you think, every single joe is rational and has capacity to
| judge, nobody would have voted from Trump to be president or riot
| the Capitol.
|
| 3. Currently it can do 60% of the work. If it evolves, it will do
| 80% of the work. If you factor human population and the amount of
| inefficiency in current economy, this will create a boom of
| productive workers with bust of average worker. And the world is
| not just for elites. Average people need their livelihood too.
| Now think about the magnitude of displacement.
|
| 4. I have no issue with innovation and breakthrough. AI won't
| kill us, because we will kill each other before that. This tech
| is exposing entire middle class job (check out the job being
| exposed. There is a paper for that from openAI titled: gpts are
| gpts) And there are lot if people in that bracket. For once,
| think about us too.
|
| So, these religious people do not exclusively worry about
| immediate effects of the tech. But within their doom, hides an
| issue that needs to be heard.
|
| That's my two cents.
| GistNoesis wrote:
| While you were exploring the computational universe,
|
| A wild Shoggoth suddenly appear,
|
| Do you : - [Wisdom not high enough] -
| [Intelligence not high enough] - [Not enough
| gold] - Try to make it your friend and welcome
| it in your home - Try to make it your slave
| - Try to send it back to where he was -
| Continue this "choose your own adventure" with GPT 5
| - Roll 1d20 for extra Luck
| theendisney wrote:
| My first thought was WERE ALL GOING TO DIE! But then it struck me
| how true that is. It seems a problem worth pondering. (be it on a
| personal level) Ods seem much better for life extention and
| quality of life at least in our lifetime.
___________________________________________________________________
(page generated 2023-04-01 23:03 UTC)