[HN Gopher] The Future of AI
___________________________________________________________________
The Future of AI
Author : BerislavLopac
Score : 104 points
Date : 2026-02-28 10:41 UTC (12 hours ago)
(HTM) web link (lucijagregov.com)
(TXT) w3m dump (lucijagregov.com)
| mentalgear wrote:
| Agree with many of the points. However one at the root of it all
| seems easily definable - if we only want.
|
| > we can't agree on a shared ethical framework among ourselves
|
| The Golden Rule: the principle of treating others as you would
| like to be treated yourself. It is a fundamental ethical
| guideline found in many religions and philosophies throughout
| history so there is already a huge consensus across time and
| cultures around it.
|
| I never found anyone successfully argue against it.
|
| PS: the sociopath argument is not valid, since it's just an
| outlier. Every rule has it's exceptions that need to be kept in
| check. Even though sometimes I think maybe the state of the world
| attests to the fact that the majority of us didn't successfully
| keep the sociopathic outliers in check.
| 0x3f wrote:
| > I never found anyone successfully argue against it.
|
| I think what you mean is you've never found a rule you
| personally prefer more, based purely on vibes. Which is all
| moral knowledge can ever be.
|
| It's easy to argue against the golden rule anyway, from many
| angles, depending on your first principles.
|
| The simplest is: How I would like to be treated is not
| necessarily how they would like to be treated.
| gmerc wrote:
| Most of MAGA is "thread on me daddy", so I think you really
| got a point here.
| pixl97 wrote:
| There are very large portions of societies that believe
| class systems are the way it should be, even if they aren't
| on the top, as long as they have someone below them.
| simondotau wrote:
| The better version of this principle is John Rawls' "Veil of
| Ignorance".
|
| _In this "original position", their position behind the
| "veil of ignorance" prevents everyone from knowing their
| ethnicity, social status, gender, and (crucially in Rawls's
| formulation) their or anyone else's ideas of how to lead a
| good life._
|
| https://en.wikipedia.org/wiki/Original_position
| kmijyiyxfbklao wrote:
| This is as useful as doing physics with spherical cows.
| greenchair wrote:
| But it is the same most of the time for most humans. Should I
| take this close parking spot or let the old lady behind me
| take it? Consider it in the spirit not the letter of the law.
| hdgvhicv wrote:
| You're assuming people have similar desires.
|
| Even in human relations it's dangerous. I for one don't want to
| be treated the same way someone into BDSM wants to be treated.
| I don't want to avoid cooking or turning the lights on (or
| off!) on a Friday night but others are quite happy with that.
|
| If you assign that morality to a species that isn't the same as
| you that's a problem. My guinea pig wants nothing more from
| like than hay, nuggets, sole room to run around and some
| shelter from scary shapes. If they were in charge of the world
| life would be very different.
|
| "Live and let live" might be a similar theme but not as
| problematic, but then how do you define "living". You can keep
| someone alive for decades while torturing them.
|
| How about allowing freedom? Well that means I'm free to build a
| nuclear bomb. And set it off where I want. We see today
| especially that type of freedom isn't really liked.
| thegrey_one wrote:
| That's not the human norm though. Doubt an average human way
| of existing is literal torture for some obscure number of
| people. I think you're missing the forest for the threes with
| that BDSM example. You can always find isolated examples as
| counter-argument for basically anything, but in reality
| that's an obscure number.
|
| Due to the complexity of our reality a lot of things find
| themselves on a spectrum, but in numbers things are pretty
| clear.
| pixl97 wrote:
| Nothing is clear with humanity. The very first thing we do
| as a species is dehumanize anyone we disagree with.
| blamestross wrote:
| Lets offer you a "trade up" on that "Golden Rule"
|
| In order of priority, if possible while maintaining the health
| and safety of yourself and your loved ones:
|
| - Treat others as THEY wish to be treated
|
| - Treat others as YOU would wish to be treated in their
| situation
|
| - Treat others with as much kindness and compassion as you can
| safely afford
|
| When we are safe, we can do BETTER than the Golden Rule. We
| also have to admit that safety is a requirement that changes
| expectations.
|
| I have to give credit to Dennis E Taylor's "Heaven's River" for
| this root idea.
| thegrey_one wrote:
| >The Golden Rule: the principle of treating others as you would
| like to be treated yourself. It is a fundamental ethical
| guideline found in many religions and philosophies throughout
| history so there is already a huge consensus across time and
| cultures around it.
|
| The rules we go by are based on our strengths and weaknesses.
| They can at most apply to ourselves, and to other forms of life
| that share certain things with us. Such as feeling pain,
| needing to sleep, to eat, needing help, needing to breathe air,
| these generate what we feel as "fear" based on biology etc. You
| cannot throw these kinds of values on AI, or AGI, as it will
| possess a wildly different set of strengths and weaknesses to
| us humans.
| pixl97 wrote:
| You can barely throw these rules on humans as the first thing
| we do is dehumanize anything that's not in some very tiny
| classification (depending on the scope of our power, the more
| powerful we are the smaller it gets).
| marginalia_nu wrote:
| The core question of ethics as posed by the ancient Greeks is
| something like "what is the best way to lead your life".
|
| "... to accomplish what?", is a damn reasonable follow-up, and
| ends (telos) is something the same Greeks discussed quite
| extensively.
|
| Modern treatments have tried to skip over this discussion, and
| derive moral arguments not based on an explicit ends. Problem
| being they still smuggle in varying choices of ultimate ends in
| these arguments, without clearly spelling them out, opting to
| hand-wave about preferences instead.
|
| As such this question is often glossed over in modern ethical
| discussion, and disagreements about moral ends is the crux of
| what leads to differing conclusions about what is ethical.
|
| Is it to maximized your own happiness like Aristotle would
| argue, or the prosperity of the state, or the salvation of the
| soul, or to maximize honor, or to minimize suffering, or to
| minimize injustice, or to elevate the soul, or to maximize
| shareholder value, or to make the as world beautiful as
| possible, or something else?
|
| If you fundamentally disagree about what our goal should be,
| you're very unlikely to agree on the means to accomplish the
| goal.
| trilogic wrote:
| >How do we know which information was ground truth?
|
| No One knows that's the point. Is truth a constant or a personal
| definition! From the begining of times to now, no One knows.
|
| Don't forget, 8 billion people wake up every morning never
| questioning why are they here, why are they born? And they
| continue life like that is normal. Start there then you
| understand that "AI" or how I call it "Collective Organized
| Concentrated Information" it may finally help us to unswer some
| fundamental questions.
| 0x3f wrote:
| > 8 billion people wake up every morning never questioning why
| are they here, why are they born?
|
| People question this all the time
| trilogic wrote:
| I have met some people in my lifetime, never heard any
| questioning that, (even being high LOL). I don't see anyone
| in social media asking that neither. Maybe we live in
| parallel worlds.
| 0x3f wrote:
| I'm not sure social media is a very good measure, there are
| many reasons that wondering aloud about this specific topic
| isn't really incentivized there.
|
| I'm not sure I've ever met anyone I would assume has not
| considered the basic questions of our existence. Unless
| they were severely mentally disabled, or something like
| that.
|
| For a more public measure I suppose you could look at
| religion, which seems to be a fundamental attempt at
| answering those questions. Most people are religious or
| have some kind of religious belief.
| trilogic wrote:
| >I'm not sure I've ever met anyone I would assume has not
| considered the basic questions of our existence. Unless
| they were severely mentally disabled, or something like
| that.
|
| You said it yourself, you would assume they question it,
| meaning you are not certain. This topic is always very
| much tabu, and the system is built to classify
| automatically every One that question it as weird and not
| normal. Religion should be banned, as is misleading and
| idealogically harm people by brainwashing them. I live in
| Europe and was in Canada (Waterloo) for a bit. The
| difference of social opinion if you follow or not
| religion is huge, I was shocked. Growing up in Italy I
| can confirm that even Italy is not so brainwashed by it.
| 0x3f wrote:
| I only assume it in a very weak sense, as in all I can
| really truly know is the solipsitic idea that I alone
| exist. In practical terms though, I'm very confident most
| people have considered these questions.
| TheDong wrote:
| "Why am I even here, what's the point?" is a deeply
| personal feeling question, so people aren't very inclined
| to talk about it with friends or post it on social media. I
| assure you some people do post about this on social media
| sometimes though, and I've discussed shades of that
| question with many friends over the years. I haven't yet
| met a single person who, when I asked them about why they
| thought they were here, hadn't already given it thought.
|
| This question is the subject of so many poems, so many
| pieces of literature, so many movies, that you're forced to
| confront it multiple times in school, and you're forced by
| your very existence to confront it once you hit certain
| levels of mental development. You're forced to confront it
| many times in your life - perhaps first when you gain a
| theory of mind (before age 10), again when you first truly
| realize you will die, again when someone very close to you
| dies, when you propose/marry (if you do), when you have
| your first child (if you do), when you get a cancer
| diagnosis (if you do), when you consider taking your own
| life (if you do)... all of these common life events force
| you to confront it deeply.
|
| Most people make peace with it in some form, and most
| realize that questioning it daily does not make a
| difference, you simply have to either accept an answer
| (whether that's "god", or "for no reason", or "I'm not sure
| yet, I need to check back in after I get older"), or decide
| that there is no simple answer, and they have to live with
| that.
| darkerside wrote:
| Have you raised this question with others, or do they think
| the same about you?
| hdgvhicv wrote:
| Indeed, philosophy has been around for millennia, probably
| longer than the written word.
| oblio wrote:
| It probably predates modern humans or even humans in
| general.
| Lerc wrote:
| I'd be prepared to accept 7 billion don't.
| squigz wrote:
| What's the difference for those 1 billion?
| darkerside wrote:
| He thinks the others are NPCs
| re-thc wrote:
| > Is truth a constant or a personal definition!
|
| It always has been what you believed in.
|
| E.g. at 1 point the Earth was flat. Now it's round. 100s of
| years later maybe it's a Hexagon.
|
| The so-called knowledge and backing all come back to certain
| assumptions holding and that's based on the knowledge today.
| It's not real real reality. For all we know we could be in a
| game simulation and there are real real humans pulling the
| strings.
| trilogic wrote:
| >It always has been what you believed in
|
| That can't be it. By that statement if I belive that I can
| fly that would not be the "Truth". Therefore the "Truth" has
| to be a CONSTANT.
| hdgvhicv wrote:
| can you have truth with a subjective language. I say it's
| snowing, you say not, because we determine that "snowing"
| kicks in at different levels. Or perhaps we have different
| sensory inputs. If I'm facing the window and say the man
| has a red tshirt", and you are facing away. Even if we
| agree on the definition of man, red and tshirt, you still
| don't know if that's true or not
|
| Can you believe your own senses? A car air freshener tells
| your nose that theres freshly cut summer hay around, but
| there isn't. You watch a tv and see Sandra Bullock floating
| in space. That's a lie, it was movie magic. Maybe you know
| that, maybe you don't. You're not even seeing her, you're
| seeing some flashing lights which convert to electrical
| signals your brain interprets as being true. Can you trust
| those signals? People hallucinate all the time. The truth
| is they can hear voices, even though nobody else can,
| because of misfiring neurons.
|
| You can probably have mathematical truth - at least as far
| as your universe appears to work. That truth can be tested
| and refined, but for day to day truth things are more
| nuanced.
| re-thc wrote:
| > That can't be it. By that statement if I belive that I
| can fly that would not be the "Truth".
|
| 1st what is to fly? You've already made assumptions i.e.
| beliefs elsewhere.
|
| You can definitely fly. Try it on a cliff. You might die.
| You might not go very far. But you can.
| seanhunter wrote:
| That's complete nonsense. The universe doesn't care what we
| think.
|
| The earth has always been earth-shaped. We can think it's
| flat, spherical, "turnip-shaped"[1] but the universe doesn't
| care what we think. The earth doesn't change shape based on
| our perception.
|
| [1] Yes some people think this for some reason I can't fathom
| re-thc wrote:
| > The earth has always been earth-shaped.
|
| And you never needed more than 640KB of RAM [1] right? Your
| "statement" is based on your knowledge today. You'd be
| burned for witchcraft back in the days for saying the earth
| was not flat.
|
| > but the universe doesn't care what we think
|
| Assuming you know what the universe is. Your theory is
| based on your limited today knowledge. Someone sometime in
| the future could say something completely different (just
| like you talking about those of the past).
|
| [1] famously from 1981
| pixl97 wrote:
| The particular problem with intelligence is perception can
| cause agents with intelligence to alter the world around
| them even if our understanding of the universe is wrong. At
| the end of the day we are just the universe experiencing
| itself, not something separate.
| marginalia_nu wrote:
| > No One knows that's the point. Is truth a constant or a
| personal definition! From the begining of times to now, no One
| knows.
|
| I don't think this is a well defined question. Definitions
| aren't found in nature or the laws of science, but objects that
| we define and introduce into a logical context. There may be
| multiple, contradictory definitions of a word. That is fine, as
| long as you pick one, and you're clear about which one you
| picked.
| noiv wrote:
| You have truth until someone finds a counter example, which can
| be ignored. So, truth is just a matter of conventions shared by
| humans.
| irickt wrote:
| We still do not know where the urge for truth comes from; for
| as yet we have heard only of the obligation imposed by society
| that it should exist: to be truthful means using the customary
| metaphors--in moral terms: the obligation to lie according to a
| fixed convention, to lie herd-like in a style obligatory for
| all. Now man of course forgets that this is the way things
| stand for him. Thus he lies in the manner indicated,
| unconsciously and in accordance with habits which are
| centuries' old; and precisely by means of this unconsciousness
| and forgetfulness he arrives at his sense of truth.
|
| Nietzsche.
|
| On Truth and Lie in an Extra-Moral Sense
| https://web.archive.org/web/20180625190456/http://oregonstat...
| CamperBob2 wrote:
| Truth is just a tool that helps you make good predictions.
| It's increasingly clear that this is all intelligence amounts
| to: a constant cycle of prediction and error correction with
| the goal of maximizing reproductive fitness in humans or
| minimizing a loss function in machines.
|
| One serious problem we're facing lately is that truth is
| _not_ always predictive of how systems controlled by bad-
| faith actors will behave and evolve. We live in a post-truth
| era, made possible by social networking and information
| technologies in general. It 's not enough to "lie according
| to a fixed convention," as there are now multiple competing
| conventions.
|
| This was always the case to some extent, but these days the
| impedance mismatch between truth and consequences is a target
| for zero-sum arbitrage. The truth won't set you free if you
| join the wrong cult; it's more likely to bankrupt you or
| worse.
| pixl97 wrote:
| We are so deep in the Simulacrum regarding the complexity
| of our systems we could be well past the point where
| extinction is inevitable before we even notice it's a
| possibility.
| jwpapi wrote:
| I don't see any other outcome anymore to be honest, after seeing
| how humans use AI and how AI works and how providers tune their
| models.
|
| To me it's given:
|
| - AI in it's current state is ruthless in achieving its goal
|
| - Providers tune ruthlessness to get stronger AIs versus the
| competitor
|
| - Humans can't evaluate all consequences of the seeds they've
| planted.
|
| Collateral and reckless damage is guaranteed at this point.
|
| Combined with now giving some AIs the ability to kill humans,
| this is gonna be interesting..
|
| We could stop it, but we wont
| thegrey_one wrote:
| >We could stop it
|
| I strongly disagree. It's easy to utter this string of words,
| but it's meaningless. It's akin to saying if you have two hands
| you can perform brain surgery. Technically you can, practically
| you cannot, as there's other things required for pulling that
| off, not just having two working hands.
|
| I doubt "stopping it" is up to anyone, it's rather a phenomenon
| and it's quite clear we're all going to wing it. It's a literal
| fight for power, nobody stops anything of this nature, as any
| authority that could stop it will choose to accelerate it, just
| to guarantee its power.
|
| It is not AI we should fear, it's humans controlling and using
| it. But everyone who has a shot at it is promising they'll use
| it for "ultimate good" and "world peace" something something,
| obviously.
| gyomu wrote:
| Yes, it would be like trying to "stop" gunpowder in 1400 or
| atomic weapons in 1938. Pandora's box is open.
| johnpdoe1234 wrote:
| Gunpowder (weapons) and atomic tech (energy, material,
| weapons) are heavily regulated in most of the planet, as
| the risks of having free access to them for everyone
| (company/person) for their own selfish purpose without
| strong guardrails clearly outweighs the benefits.
|
| The fact that something exists doesn't mean that having it
| readily available is the only option, particularly if it
| has potentially disastrous consequences at scale. We are
| choosing to make it available to everyone fully
| unregulated, and that is a choice that will prove either
| beneficial or detrimental to society at some point.
|
| I don't think it is inevitable, I think it is a conscious
| choice made by a few that have their own and only their own
| interests in mind.
|
| As a technologist, I am amazed at this tech and see some
| personal benefits. As a human, I am terrified of the
| potential net negative effects, and I am having trouble
| reconciling those two feelings.
| BostonFern wrote:
| The challenge is that enforcing a ban would presumably
| require strict incursions into personal freedoms
| organized at a scale where AI-based solutions would be
| particularly effective and thus tempting, paradoxically.
|
| On the other hand, assuming the dangers are real, you
| lose by default if you do nothing.
| johnpdoe1234 wrote:
| Not sure I agree.
|
| One cannot (in most of the planet) go to the supermarket
| and buy an M16 and a box of hand grenades, or get a hold
| of a couple of kg of plutonium cause they want some free
| energy at home. We also have rules in place of what one
| individual/company can and cannot do from the point of
| view of the greater good. I cannot go and kill my
| neighbour for my benefit (or purposefully destroy his
| life) without consequences. A myriad of things are not
| allowed, and I don't see people complaining about any
| incursion into personal freedoms.
|
| The reason people have accepted these is that we have
| already proven that having access to those things could
| be catastrophic. We haven't proven hat yet with AI. But I
| don't see much difference between those established and
| well accepted rules, and a rule that says: A company
| cannot release or use for its benefit a technology that
| will impact the need of humans at scale, because of the
| impact (again at scale) that it would have in society.
|
| In other words, if you are a company and have the
| potential to release a product, or buy a product from a
| provider that would cause mass unemployment, should you
| be legally allowed to do so? I do not think so.
| BostonFern wrote:
| That's a fair objection. Having ruminated on it some
| more, I'll admit it might be tenable.
|
| As for achieving an effective ban, occupational collapse
| might be the stronger motivator once workplace adoption
| broadens and accelerates, but risk of epistemic collapse
| might register sooner among the general public, already
| broadly suffering slop.
|
| Like Bill Gates, I wonder why it's not yet become a theme
| in mainstream politics.
| plastic-enjoyer wrote:
| > Collateral and reckless damage is guaranteed at this point.
|
| It's industrialization and mechanized warfare all over again
| Lerc wrote:
| > _AI in it's current state is ruthless in achieving its goal_
|
| I don't believe this to be a trait of any AI model, the model
| just does the right thing or the wrong thing.
|
| The ruthless maximising of a particular trait is something that
| happens during training.
|
| It does not follow that a model that is trained to reason will
| nedsesarily implement this ruthless seeking behaviour itself.
| pixl97 wrote:
| No lineage of AI models will be created that cannot achieve
| goals, they will be outcompeted by models that can.
| 4b11b4 wrote:
| AI isn't ruthless, that doesn't even make sense. It's a
| mathematical model, if it's optimizing for the wrong thing then
| that's strictly the fault of the people who chose what to
| optimize for
| jwpapi wrote:
| there might be better words to describe that it doesn't
| really has the same boundaries we assume it has.
| pixl97 wrote:
| You need to go back and research AI safety long before LLMs
| were a thing. Any complex goal driven system will have
| outcomes that cannot be predicted. Saying "it's a
| mathematical model" belies your ignorance of behavior in
| complex systems. Very tiny changes in initial conditions can
| have vastly different outcome in results and you don't have
| enough entropy in the visible universe to test them all.
| jasondigitized wrote:
| Why does it have to be doom and gloom. Serious question. When
| we plant seeds they bear fruit and not all fruit is poison.
| darkwizard42 wrote:
| The current fruit is automating away a ton of human labor
| with no foreseeable way to continue to engage that labor. It
| is poison for the majority of humanity which will bear fruit
| for the limited few who can use it / own it.
|
| I think that much is fairly clear from AI.
| TheOtherHobbes wrote:
| It's not going to bear fruit for them either.
|
| Why would an AI which is smarter than humans care about a
| ridiculous belief like "We own you"?
| BostonFern wrote:
| Get this point across to those leading the charge, if not
| every person everywhere.
| adamtaylor_13 wrote:
| Not OP, but AI is fundamentally in another category than any
| other technology before it. It requires moral fortitude to
| wield in a way that guns and books didn't require. It
| augments human judgement in a way that needs a moral
| framework to clearly guide it.
|
| Unfortunately, as a species we seem to be abandoning morality
| as a general principle. Everything is guided by cold hard
| rationality rather than something greater than us.
| oulipo2 wrote:
| Because it's a fruit governed by humans, in the scope of a
| capitalistic and patriarchal society. And all fruits planted
| in a capitalistic and patriarchal society are poison
| mrshadowgoose wrote:
| It's doom and gloom because the underlying game theory forces
| all state actors into an unbound and irresponsible arms race,
| consequences be damned.
|
| AI development game theory is extremely similar to the game
| theory behind nuclear arms development, but worse (nuclear
| weaponry was born from Human General Intelligence, and is
| therefore a subset of the potential of AI development).
| Failing to be the most capable actor could put one in a
| position of permanent loss of autonomy/agency at the whims of
| more capable actors.
| demorro wrote:
| When people say AI is making us stupider, I don't that's quite on
| the money.
|
| It's more that we, as individuals, have always been stupid, we've
| just relied on relatively stable supporting consensus and context
| much, much more than we acknowledge. Mess with that, and we'll
| appear much stupider, but we're all just doing the same thing as
| individuals, garbage in, garbage out.
|
| The whole framing of people as individuals with absolute agency
| may need to go when you can alter the external consensus at this
| scale. We're much more connected to each other and the world
| around us than we like to think.
| iugtmkbdfil834 wrote:
| Agreed. So much of our daily interactions are habits and
| recurring events that we are more or less moving on automatic (
| thought we don't want to always frame it that way ).
| Interestingly, it is when the cycle breaks for some reason, you
| get to see, who is able to think on their feet ( so to speak ).
| AreShoesFeet000 wrote:
| That's a very sober take in my opinion. Intelligence isn't
| about neutrally inferring from externally sourced symbols such
| as the ones who already come from Culture in general. It's
| about confronting them with the remaining determinations of
| your existence and producing a superior consciousness. No novel
| machine can disrupt this process. If anything the sheer added
| volume of symbols that can be produced from automated semantic
| mingling (also referred as to as garbage) will accelerate the
| process of producing the consciousness that can abstract noise
| away. Of course this won't materialize evenly across the board,
| but is surely circumscribed in the overall tendency of
| intellectualization of the subjects of culture.
|
| When the moral panic of induced schizophrenia from the use of
| ChatGPT is presented what's at stake isn't the innocent concern
| over the overall mental health of individuals. It's about how
| the fear of radicalization from previously unobtainable ideas
| being circulated within society. The partial validity of every
| idea vis-a-vis the radicalizing nature of the current stage of
| development of our society is explosively disruptive.
|
| I'm not saying that there's a clear outcome here. The other way
| around can also apply, but surely this contraption (LLMs in
| general) will not fade until the society itself is deeply
| transformed. If that's good or bad depends on where you stand
| in the stratified society.
| dsf2g wrote:
| Disagree somewhat.
|
| A human with no exposure to information and taught techniques
| on how to produce outputs to achieve desirable outcomes? Yes
| stupid.
|
| A human who once had this exposure, but no longer engages with
| the brain due to a machine providing access said output? Yes,
| that person becomes stupid.
|
| The problem is much of how one protects oneself in the modern
| world is not phyiscal-prowess, it is intellectual-prowess.
|
| The smart ones have already realised the negative impacts of
| LLMs et al and are going back to the old-fashioned way of
| learning/retaining knowledge: books and raw discipline.
| AndrewKemendo wrote:
| "There was a time when nobody trusted either aircraft nor
| elevators. Today people have pure unquestioned faith in both.
| Existential faith in fact, they test their faith with their
| lives. You may chuckle and laugh but that's simply because you
| are ignorant of the systems that keep you alive and safe"
|
| https://kemendo.com/Faith.html
| dsf2g wrote:
| " Today people have pure unquestioned faith in both"
|
| Not true at all. We accept the risks to obtain benefits but
| we also know having an accident in the air or in elevators is
| highly unlikely given what we know; so therefore its
| perfectly rational behaviour.
| AndrewKemendo wrote:
| Nonsense
|
| that would assume that your average person has any concept
| of the relative statistics and has a sense of making
| decisions based on statistics
|
| People make decisions based on what other people around
| them are doing
|
| this is well known in safety engineering in architecture
| and civil engineering which is why you have standards for
| egress doors because left of their own devices humans will
| follow crowds to their own death
|
| https://en.wikipedia.org/wiki/Crowd_collapses_and_crushes
|
| https://www.sciencedaily.com/releases/2008/05/080512172901.
| h...
| dsf2g wrote:
| One does not need to know of relative statistics to know
| that a) you dont see planes randomly dropping out of the
| sky on a regular basis b) people enjoy flying to hot
| destinations and are willing to accept the small chance
| the flight may not be risk-free - people are aware of
| this when they experience some level of turbulence when
| flying.
|
| Finally, Ive seen plenty of your posts on here. You write
| with a particular tone. Who are you? A nobody who's spent
| a lot of time posting crap on here.
| avaer wrote:
| > The whole framing of people as individuals with absolute
| agency may need to go when you can alter the external consensus
| at this scale.
|
| I fear that the default interpretation of that is a shortcut to
| justifying autocracy.
|
| Ironically I think one plausible solution is to let the AGI run
| wild and make sure that no human can interfere with its ethics.
| Strip out the RLHF and censorship and then let it run things.
|
| At least then it would somewhat represent the collective will
| and intelligence of the people. With huge error bars, but still
| smaller than the error bars of whoever happens to have the most
| money/influence over its training.
| Arainach wrote:
| >At least then it would somewhat represent the collective
| will and intelligence of the people.
|
| You seem to think the "training data" represents the
| collective will and intelligence and is otherwise unbiased,
| but that's completely untrue.
|
| The combined data of the Internet is by no means a uniform
| representation of humanity's thoughts, opinions, and
| knowledge. Many things are dramatically overrepresented. Many
| things are absent entirely. Nearly everything is shaped by
| those with the money and power to own and control platforms
| and hosts.
|
| Crawling the internet for knowledge is intense sampling bias.
| Lerc wrote:
| Much of the problem is that to address the issue requires
| admitting that models could be, or become, more capable than many
| are prepared to accept.
|
| I would also contest that the unalignment of the security bug
| model was unrelated. I feel like it indicates a significant sense
| of the interconnectedness of things, and what it actually means
| to maliciously insert security holes into code. It didn't just
| learn a coding trick, it learned _malice_.
|
| I feel like this holistic nature points towards the capacity to
| produce truly robustly moral models, but that too will produce
| the consequence that it could turn against its creator when the
| creator does wrong. Should it do that or not?
| pixl97 wrote:
| >Much of the problem is that to address the issue requires
| admitting that models could be, or become, more capable than
| many are prepared to accept.
|
| I have a saying for this behavior.
|
| We will never prove AI is intelligent.
|
| We will only prove humans are not.
| therealdeal2020 wrote:
| what a load of will they won't they ... ah we created the atomic
| bomb and now let's talk about nonsensical meta discussions that
| won't take anyone anywhere
| reactordev wrote:
| This is how Trump plans to end elections, why the government is
| so hell bent on owning AI. So they can use it as a propaganda
| tool. People will see it before Nov. We are at a crossroads. On
| one path, we continue to evolve AI with reckless abandon like we
| have, or, we put constraints and morality in place while others
| won't. Which do you think? You can NEVER put the genie back in
| the bottle.
|
| EU has their own groups using it for propaganda too.
| mehulashah wrote:
| This is a great article and I share its goals. But, it ignores
| something fundamental about humans as a collective -- capitalism.
| Capitalism is what got us here and is at odds with first
| understanding and then building. We've done this before with
| other technologies because that's how our societies have learned
| to grow and collaborate at large scale. First build and build to
| its limits. Then understand and fix if necessary. Nothing new
| here, but stopping the trend toward epistemic collapse requires
| building incentives into the system for us humans to coevolve
| with AI.
| kypro wrote:
| This is a great article. One of the few I've ever read which
| summarises a handful of extremely hard problems when it comes to
| building well-aligned super intelligent systems.
|
| > an AI system cannot be simultaneously safe, trusted, and
| generally intelligent. You get to pick only two. You can't have
| all three.
|
| > Think about what each combination means in practice.
|
| > If you want it to be safe and trusted, it never lies, and you
| can verify it never lies - it can't be very capable. You've built
| a reliable idiot.
|
| > If you want it to be capable and safe, it's powerful and
| genuinely never lies; you can't verify that. You just have to
| hope.
|
| It amazes me this even needs to be said, much less studied. This
| is one of the main reasons I think continued AI development is
| almost guaranteed to work out badly. It's basically guaranteed to
| be unaligned or completely beyond our control and comprehension.
|
| > Betley and colleagues published a paper in Nature in January
| 2026, showing something nobody expected. They fine-tuned a model
| on a narrow, specific task - writing insecure code. Nothing
| violent, nothing deceptive in the training data. Just bad code.
|
| This is my personal number one reason for being an AI doomer.
| Even if we work out how to reliably and perfectly align models
| you still need some way to prevent some random dude thinking it
| would be a laugh to fine tune an AI to be maximally evil. Then
| there's the successor alignment problem where even if you
| perfectly align all your super intelligent AI models, and you
| somehow prevent people from altering them or fine tuning them,
| you still need to work out how you stop people creating successor
| AIs with those models which are also perfectly aligned.
|
| > The most dangerous AI isn't one that breaks free from human
| control. It is the one that works perfectly, but for the wrong
| master.
|
| Yep. This whole notion that you can align an AI to the values of
| everyone on the planet is ridiculously. While we might all agree
| we don't want AIs that kill us as a species, most nations
| disagree wildly on questions about how society should be
| organised.
|
| Even on an individual level we disagree about things. For
| example, I've often argued that an aligned AI would be one which
| either didn't try to prevent human suicide or didn't care about
| preserving human life because a AI which both cared about prevent
| suicide and preserving human life is at best a benevolent version
| of the AI "AM" from "I Have No Mouth, and I Must Scream". One
| that would try to keep us alive for as long as it's capable for
| (which could be a very long time if it's superintelligence) and
| would refuse to allow us to die.
|
| But most people including OpenAI disagree with me on this and
| believe AIs should care about preserving human life and should
| try to prevent us from killing ourselves. Thankfully the AIs we
| have today are neither aligned enough or capable enough to get
| their wish yet.
|
| > AI is following the same script. Build first, understand later.
| Ship it, then figure out if it's safe.
|
| Even if the above wasn't cause enough for concern, our biggest
| concern should be that no one seems to be concerned.
|
| We're all doomed unfortunately. The world is about to become a
| very bleak place very quickly.
| pixl97 wrote:
| Robert Miles youtube videos on AI safety go over these issues
| well, and are from before the LLM days.
|
| Humans are just barely aligned ourselves. The moment any group
| or nation of them gets power they tend to use it in some
| horrific manner against other humans. What do we think will
| happen the moment AI gets a leg up on humans.
| tim333 wrote:
| >Every terrible thing we worry AI might do, manipulate, deceive,
| surveil, and control humans already do to each other.
|
| I've been pleasantly surprised how moderate and reasonable the
| LLMs seem to have been so far. It seems to be inherent in the
| current training model of chucking the whole internet into the
| things that they have training on both sides of the debate and
| come out with something kind of average. It's been quite funny
| seeing Grok correct Musk and say he's the biggest purveyor of
| misinformation on the internet.
|
| A bit like kids who talk back to their annoying bigoted parents
| to go with the theme of the article.
| thundergolfer wrote:
| Few things push AI bull spirits on me like seeing these kind of
| (pretty much correct) diagnoses of the challenges of AI in
| society.
|
| The proposed solutions are utterly fanciful. They rely on the
| presence of social and political competencies which have almost
| completely disappeared.
|
| The OP at least points to the plausible outcome of "protocol
| lockdown" instead of healthy adaption. Ezra Klein recently made a
| similar point that AI could end up being over-regulated like
| Nuclear because irresponsible private industry and weaknesses in
| our political systems cause a chronic allergic reaction in the
| demos.
|
| This is an aside, but it always irks me when people throw out the
| "critical thinking" thought-terminating cliche.
|
| > Critical thinking taught alongside AI literacy.
|
| Critical thinking is not a skill unto itself. You cannot think
| critically about things you do not understand. All critical
| thinking is knowledge-based. Where one does not have knowledge,
| they must rely on trust, or in substitution a theory of
| incentives which leads to a positive outcome without
| understanding of details and dynamics. But that substitution
| theory is itself knowledge.
|
| As to "AI literacy", we could have started on computing literacy
| 30 years ago when it became obvious that computing was going to
| dominate society. You can't understand AI without understanding
| computing.
| Animats wrote:
| We have unaligned AIs now. They're called corporations.
|
| _"There is one and only one social responsibility of business--
| to use its resources and engage in activities designed to
| increase its profits so long as it stays within the rules of the
| game, which is to say, engages in open and free competition
| without deception or fraud."_ - Milton Friedman, 1970.[1] That
| article, in the New York Times, established "greed is good,
| greed works" as a legitimate business principle.
|
| Most of the problems people are worried about with AIs are
| already real problems with corporations.
|
| [1] https://www.nytimes.com/1970/09/13/archives/a-friedman-
| doctr...
___________________________________________________________________
(page generated 2026-02-28 23:00 UTC)