[HN Gopher] Crimes against transhumanity
___________________________________________________________________
Crimes against transhumanity
Author : zdw
Score : 141 points
Date : 2022-07-18 14:47 UTC (8 hours ago)
(HTM) web link (www.antipope.org)
(TXT) w3m dump (www.antipope.org)
| goodside wrote:
| For anyone wondering why the title of the short story is "Lena",
| see: https://en.m.wikipedia.org/wiki/Lenna
| motohagiography wrote:
| > _Our intuitions about crimes against people (and humanity) are
| based on a set of assumptions about the parameters of personhood
| that are going to be completely destroyed if mind uploading turns
| out to be possible._
|
| There are human 'proof of work' activities that unless you can
| literally transplant the muscle memory, there is no way to
| replicate them without putting in the time, and I am saying
| without that effect of muscle memory, _anything_ you transmit is
| just an artifact of language and narrative, and therefore not
| material.
|
| The advantages of augmented transhuman life aren't material,
| they're only symbolic in narrative. Imo, the example of a
| parliament trying and executing a formerly divinely appointed
| King was just the effect of the victory of a narrative and
| linguistic definition of self over a physical/spiritual
| orientation of self. It was just the rejection of theism. It
| wasn't world ending to be able to depose a king, it was just the
| conseqeunce of becoming unmoored from a relationship to something
| divine, and being left in the hall of mirrors that is ego and
| language.
|
| The transhumanist ideology is predicated on the basic bullshit
| idea that our physical experience is a subset, and subordinate to
| knowledge and belief - and not that knowledge and belief are
| necessarily only artifacts of physical perception and expereince
| first. It seems like an irrelevant distinction until you iterate
| their idea of truth is just what you believe, and suddenly you
| have people declaring themselves cats. It's a hypnotic and
| zombifying ideology.
|
| We can do thought experiments about whether an android or an AI
| is "real," if, and only if we redefine "real" and "life" by
| disqualifying the axiom that life is created by supernatural
| divine intent. Even if we agree it's rational to actively
| disbelieve in supernatural beings, this rationality also demands
| that we disqualify super-logical axioms that originate from
| outside of a logical system. Arguably, that logical system may
| have internal consistency - but it's also defined as necessarily
| inconsistent with the reality that exists outside of it. We've
| agreed to accept an inconsistency with reality, which lets us
| experiement with counterfactuals as a way to discover new things,
| and it is very collegial of us, but it's not fucking real.
|
| All the transhumanist stuff I've read seems like a bunch of self
| referential nonsense designed to recruit people into alternative
| lifestyle communities. The more interesting question than whether
| an AI is alive or intelligent and what the ethical consequences
| of it for us may be is this: what can something we create know
| about us, and what do the boundaries of the limits of its
| knowledge provide evidence of for our own experience?
|
| Even if an AI could subordinate us somehow, we still have its off
| switch, and if we don't, it's just other people who do.
| Transhumanism is entertaining, but it has crossed over into a
| kind of nihilist propaganda that I think deserves our scorn.
| nathanaldensr wrote:
| I completely agree.
|
| I often explain transhumanism to others as "The elevation of
| the ego above all else. If one can imagine it, it's real or
| plausibly could be made real." It's a rejection that there is
| anything outside our understanding (e.g., God). That's why this
| entire thread is filled with science fiction quotes which are,
| I guess, supposed to convey some kind of deep meaning or
| prediction-making ability but which are nothing more than the
| imaginings of their authors.
|
| Sorry, folks, we're still stuck with binary computers, needing
| food to survive, and a planet absolutely chock full of people
| who practice theistic religions.
| trasz wrote:
| >Even if an AI could subordinate us somehow, we still have its
| off switch
|
| That only works until the AI gets smarter than us. Imagine your
| cat trying to make sure you can't leave the house. It won't
| even realize how did you work around it. Same way we won't even
| realize why the off switch doesn't work.
|
| Of course that's a pretty big "if", let alone the assumption
| that the AI would be the first case of super intelligence,
| which tbh I'm not sure about.
| sk8terboi wrote:
| Symmetry wrote:
| I don't expect mind emulations to ever be an economical
| alternative to AIs. But if they are Robin Hanson wrote a book[1]
| trying to think through what would end up happening using as much
| boring social science and engineering as he could manage. I'm not
| convinced by everything but it's probably a good starting point
| for thinking through the issues.
|
| [1]https://en.wikipedia.org/wiki/The_Age_of_Em
|
| EDIT: A better review than just the Wikiepdia page:
| https://slatestarcodex.com/2016/05/28/book-review-age-of-em/
| wging wrote:
| Of that review, Hanson wrote: "If I ever have an executioner, I
| want him to be Scott Alexander."
| https://www.overcomingbias.com/2016/05/alexander-on-age-of-e...
| betwixthewires wrote:
| Nitpick, the author drops closing parentheses too often.
| Proofread and edit.
|
| I don't believe mind uploading is possible, so the particular
| examples cited here don't apply, but I can see how social
| principles can be upended by a revolutionary change in how we
| live, particularly one that tweaks the definition of what we are.
| Still, they're principles, they're not going away, just at best
| being reframed.
| tboyd47 wrote:
| Someone who's into this kind of thing answer me this:
|
| Say some "Music Man" type huckster comes to your town with a
| "Brain Uploader" device. He promises to upload your brain to the
| metaverse, but all the device does is train a deep learning
| simulation of the victim's personality shown on a screen, while
| administering progressively great electric shocks on the victim
| until he's dead.
|
| This gives the town the illusion that the person's soul (or
| consciousness, or whatever) has migrated into the metaverse. How
| do you detect he's lying?
| chasd00 wrote:
| if the algo was 100% perfect probably nothing. However,
| most/all software has bugs or shortcomings. I imagine after 50
| or so questions from a loved-one or close friend you could
| figure out if it was real or not.
| asojfdowgh wrote:
| is the point "is he lying?"
|
| I mean shocking a person to death I don't get, unless you
| assume its a `mv` and not a `cp`, but I guess thats besides the
| point.
| rad88 wrote:
| It was, at the least, an unnecessary medical procedure that
| resulted in someone's death. So you arrest him for homicide,
| seize the equipment and reveal the fact that it is just some
| kind of electric chair. It'll go to trial and the lies will be
| revealed, or he'll have to give a complete confession.
|
| Otherwise I think this is a bit of a trick question. We know
| that we'd be able to tell the difference between the deepfake
| and a specific living person -- they have a lifetime of
| memories, a multitude of unique relationships with various
| people, the ability to effortlessly distinguish a chihuahua
| from a blueberry muffin, etc. But if the premise is that the
| salesman's simulation is so good that we can't tell the
| difference at all, then is there actually any lie to detect?
| brodouevencode wrote:
| It's a bit of a simplistic metaphor, but I like the way you're
| thinking about the problem.
| [deleted]
| AnIdiotOnTheNet wrote:
| To a certain extent, if the supposed "illusion" is convincing
| enough it may as well be reality. If it is possible to simulate
| an individual so completely that it would convince that
| person's closest friends and relatives of its legitimacy, then
| in what way is it _not_ that person?
| tboyd47 wrote:
| This is the most Silicon Valley answer.
| AnIdiotOnTheNet wrote:
| I don't work or live in the Valley and frankly you couldn't
| pay me enough to. Philosophical thought on things like this
| is thousands of years old.
| kayodelycaon wrote:
| The nature of reality is a very, very old discussion. See
| https://en.wikipedia.org/wiki/Solipsism as an example.
| mah4k4l wrote:
| I know he's lying by this: mind uploading is unpossible. It
| would have to be body uploading. Mind is the body, mind is an
| attribute of the organism, of the life. You are your body,
| there is no escape, although the body might have autonomized
| emergent psychoplasmic non - physical properties that survive
| physical death. Even if the replica made by the "Brain
| Uploader" conman were to be somehow conscious it would not be
| you, you would still just be at the pay end of that
| electrocution doohickey.
| mah4k4l wrote:
| Brain in a jar, that could of course work.
| kcolford wrote:
| I don't know about the mind uploading... but the fact that
| device is applying electric shocks to a person's body and
| physically destroying them is a massive problem. Any
| respectable mind uploading would leave the original in-tact. A
| standard autopsy of the person's body would discover the site
| of electrocution and the damage associated with it,
| incriminating the "huckster" under at least "gross negligence"
| (the only crime for which mens rea is not necessary) up to
| first degree murder.
| jahewson wrote:
| > How do you detect he's lying?
|
| Ask him to use it on himself.
| tboyd47 wrote:
| But who's going to operate the machine and guide people into
| the Great Tomorrow?
| colinmhayes wrote:
| If everyone is convinced I'm not sure he is lying. What's the
| difference between a perfect simulation of a kind and the mind
| itself?
| SketchySeaBeast wrote:
| You'd need to prove it's perfect. I can't help but think that
| would require the original mind itself to judge - no one
| really knows how an individual thinks outside of the
| individual.
|
| I suppose if you assume the person to simply be outputs in
| response to inputs then you could probably create a near
| perfect simulation, but if you assume the mind has its own
| internal state that makes that mind distinct, and given that
| I observe myself as a thinking thing I hold that to be true,
| then it would take introspection to get closer to the truth.
| karpierz wrote:
| Your question isn't answerable in its current state due to some
| undefined terms. Can you describe what you mean by
| "soul"/"consciousness"?
| tboyd47 wrote:
| It means whatever you want it to mean. Whatever you want to
| call it, the guy is a con man. How do you expose him?
| SketchySeaBeast wrote:
| Well if we're playing a game of poorly defined terms to
| expose them I pants them and move on.
| [deleted]
| jmyeet wrote:
| I don't believe "mind uploading" will ever go anywhere. There's a
| real Ship of Theseus philosophical problem here and it's never
| provable so I expect many people would just think you've created
| a copy that superficially acts like the original while destroying
| the original.
|
| Technologically speaking, I'm not sure it's even possible to
| replicate the complex state of an entire nervous system. Maybe
| you could do this incrementally (ie by replacing parts of the
| brain with machines that can be replicated) but we're awfully
| sentimental about our own meat bags.
| psyc wrote:
| If it's a faithful copy, its acting like the original isn't
| superficial.
| theptip wrote:
| The problem with this argument is that some people obviously
| don't care about the philosophical problem, and so they will
| use the mind uploader.
|
| Sure, many people will think "it's not _me_ that lives on". And
| the rest of the normal people will see their loved ones
| apparently (from the outside) living on in digital form, and
| ignore the philosophical questions entirely.
|
| Even if only .1% of people upload their minds, that still
| produces a massive sociological shift.
| jstanley wrote:
| Nobody is sure it's possible! But it might be.
| trasz wrote:
| >I expect many people would just think you've created a copy
| that superficially acts like the original while destroying the
| original
|
| That is literally the same as moving the original. The way you
| describe it is different of course, but that's where
| differences end.
| vlunkr wrote:
| I don't think so. Imagine someone wrote an AI that is able to
| act exactly like you. To the point that no one you know can
| differentiate between you and the AI. Would it be OK at that
| point if they just killed you? No one can tell the
| difference, right?
|
| The point is, on the outside they look the same, but we don't
| understand enough about consciousness to possibly know if or
| how it can be transferred to another system.
| naasking wrote:
| > Would it be OK at that point if they just killed you? No
| one can tell the difference, right?
|
| No, because copying has created two individuals, so you're
| still killing an individual for no reason.
| chasd00 wrote:
| heh that's interesting to think about. Imagine the
| executioner leaves the room and then comes back. How would
| they know which version of you to kill?
| jobigoud wrote:
| You fork a project on Github, is it ok to delete the
| original? No. Both will grow separately and slowly diverge
| / become their own personalities.
|
| Let both live on their own and after a while, if both you
| and the AI consent to it, merge the brain state back
| together.
| trasz wrote:
| >I don't think so. Imagine someone wrote an AI that is able
| to act exactly like you. To the point that no one you know
| can differentiate between you and the AI. Would it be OK at
| that point if they just killed you? No one can tell the
| difference, right?
|
| Everyone can tell the difference, because there's a dead
| body. Killing is killing. Doesn't matter that there's
| another instance living. Also making a clone of me would be
| a gross violation of my privacy rights. The situation in
| the previous post - the "transporter" - is different,
| because I'm agreeing to one of my instances to be killed.
|
| I wonder what would be the legal status of one's own
| instances ("clones"). I mean, they are all persons, but at
| the same time they are inherently connected, from DNA to
| memories.
| vlunkr wrote:
| Ignore the clone part though. That's just how I'm
| extending the time frame. The same problem exists with a
| "transporter". The instance coming out the other side
| appears to be identical, but there's no way to know if
| that instance is actually a continuation or yourself. Is
| it the same consciousness/self? Is that even possible to
| know?
| Symmetry wrote:
| Most people believe that they're the same person who went to
| bed last night despite all the changes that happened in their
| brain while they were asleep. They even believe that they'er
| the same person after a full decade despite the huge
| differences in behavior and memory that occur after such a
| large interval!
|
| So yeah, after seeing uploads act the same way as the original
| they'll end up being treated as the original just like we treat
| people as the same individual over periods of unconsciousness.
|
| For myself I don't think it's proper to answer questions like
| these with a binary but rather I prefer to think about how much
| alike two individuals are.
| [deleted]
| decebalus1 wrote:
| > we're awfully sentimental about our own meat bags
|
| As much as I've enjoyed my meat bag and worked towards making
| it better through different means like diet or weightlifting,
| the older I get and the more ailments appear, the more I think
| about being able to live long enough to acquire my new and
| improved robot body, Chappie [0] style!
|
| [0] https://en.wikipedia.org/wiki/Chappie_(film)
| metadaemon wrote:
| I'm sorry sir, the Chappie(tm) bodies are on back order due
| to their superior hashing ability when mining crypto. Your
| consciousness will be uploaded to a toaster for the
| foreseeable future.
| perihelions wrote:
| It doesn't matter what people think when they're philosophizing
| alone; peer pressure will win out. Do you want to be the
| luddite left behind when all your friends are zipping around
| the world on optical fibers, London to New Tokyo in 200 ms? Do
| you want to be the only delicate mortal in the room, where
| everyone else has a full mind backup on rsync.net?
|
| There's a lot of terrible stuff in today's technology world
| that no thinking person would accept, if they paused to make a
| deliberated choice. But no one has time to stop. You either go
| with the flow, and swallow your concerns, or you put massive
| friction between you and the rest of society. I think it will
| be the some going forwards: all the decisions will continue to
| be made collectively and blindly, and without realizing we're
| making great decisions until we've already made them.
| trowaLA wrote:
| maximus-decimus wrote:
| Yes, I want to be the only friend to not kill myself so that
| an online simulation can "go" to Tokyo.
|
| No I would not drink the Kool-Aid out of peer pressure.
| sangnoir wrote:
| Badly paraphrasing "Rapture of the Nerds" (co-written by
| TFA's author): _[Immortal uploads] did not have to win the
| debate against the anti-upload faction - they only had to
| outlive them_
| twblalock wrote:
| > so I expect many people would just think you've created a
| copy that superficially acts like the original while destroying
| the original.
|
| Prove otherwise.
|
| Star Trek transporters have the same issue.
| SketchySeaBeast wrote:
| That episode of TNG where Barclay sees and even interacts
| with weird slug things that turn out to be people while
| transporting seems to indicate that there's a continuity of
| awareness, which really raises even more questions and should
| probably be best ignored.
| imdsm wrote:
| That's my thought on Stargates. Wouldn't your consciousness
| just stop as you were ripped apart? And a clone of you would
| then continue but it wouldn't be your stream. The more I think
| about this, about what my stream of consciousness is, my
| "session" if you will, the weirder it all gets.
| bratbag wrote:
| Especially when you consider sleeping.
|
| Do we die every night, with a copy continuing on tomorrow?
| SketchySeaBeast wrote:
| After a bunch of struggling with the knowledge that my
| consciousness will one day cease I've recently come upon
| this thought myself and I find it strangely reassuring.
| NateEag wrote:
| I think dreams are evidence that consciousness has
| continuity across sleep.
| jobigoud wrote:
| Consider the case of drinking or drugs to the point of
| waking up not remembering what happened the night before.
|
| And then consider people who, knowing very well that they
| are going to be in a state of mind where they will do
| things that they will not remember the next day, still
| choose to do it.
|
| To me this is even crazier than mind uploading. They are
| lending their body to someone with which they don't have
| stream continuity.
| naasking wrote:
| You're not dreaming all of the time, so that doesn't
| work. Dreams only occur during REM sleep phases, which
| tend to cluster closer to the end of the sleep cycle.
| lbriner wrote:
| Interestingly, many films depicting these future events tend to
| view things like nervous systems/pain etc. as things that are
| disadvantages and that technology would remove but humanity is
| built upon pain, fear, weakness and ultimately death so I
| wonder if some fully automated but artifical beings could ever
| be anything more than mundane, they would lack urgency and
| ambition because they wouldn't suffer the same desire for power
| and significance because it would be meaningless.
| notsapiensatall wrote:
| There's a game called "SOMA" which does a great job of
| exploring this question.
|
| Ultimately it comes to the same conclusion, using the player
| character as a foil who spends most of the game refusing to
| understand what is going on.
| Digit-Al wrote:
| I was going to mention SOMA as well. I was thinking about the
| poor tormented person that was stuck in a robot body, but had
| no idea it was in a robot body.
| robotnikman wrote:
| That game was amazing, one of the best horror games I ever
| played and leaves you thinking when finished. Highly
| recommend.
| eddd-ddde wrote:
| Was looking for SOMA in the comments, I love the story it
| tells.
| andrewclunn wrote:
| In a world where human minds can be uploaded, I could see myself
| being a proud neo-reactionary.
|
| Death to the AI replicant "immortals!" They have traded their
| very souls for eternity as formless demons and will come to
| enslave the humanity they will come to no longer feel connected
| to!
| softcactus wrote:
| I think that enlightenment western philosophy is incapable of
| grappling with these kinds of questions. The question of "who has
| rights?" has ultimately been a question of "who has power over
| who?". It's obsessed with hierarchies and how they should be
| structured.
|
| I think something like animism would be more apt. Start with a
| rock or a grain of sand-- how much respect should it be shown?
| Well one grain of sand doesn't mind being moved. But eventually
| if you take too many you can destroy a beach, or flatten a
| mountain. There is something to be said about treating
| everything, inanimate or not with respect. There doesn't need to
| be sentience or consciousness for us to be aware of our impact on
| our environment.
|
| "This sounds like some hippie bs" well just like in the sand
| example, the negative externalities of moving sand may not be
| apparent at first, but over time they could lead to something
| like a Roko's Basilisk situation or an AI killing all humans to
| prevent its exploitation.
| asojfdowgh wrote:
| uploading yourself doesn't mean you transfer across, its just
| another you, except you gain nothing from the other you, because
| only the original you is you.
|
| So, are you willing to kill yourself, to live forever? You will
| never experience anything ever again, for an life of infinite
| experiences that arn't yours except in name.
|
| I wouldn't!
|
| The potential of "sliding" one's mind (replicate the brain's work
| piecewise to maintain ~1x consciousness at all times) besides.
|
| Easier to just cure brain degeneration.
| teraflop wrote:
| This is a philosophical _assumption_ , and one that should be
| open to challenge, despite how intuitively sensible it seems.
|
| For instance, I could equally well argue that when somebody
| undergoes deep general anesthesia and all measurable activity
| in their brain stops, that person is "dead". The person who
| awakens from anesthesia is a different conscious individual,
| who happens to share the memories and personality of the
| original.
|
| If it's not necessarily true that anesthesia is death, why
| should it be necessarily true that mind uploading is death?
| chasd00 wrote:
| > when somebody undergoes deep general anesthesia and all
| measurable activity in their brain stops
|
| that's news to me. Is that literally true ( in the literal
| sense of the word "literally")? If so, that's very
| interesting and something i never thought about before.
| teraflop wrote:
| It definitely doesn't _always_ happen, but my understanding
| is that when the anesthesia is extremely deep, the brain 's
| electrical activity can be so low as to be undetectable
| ("isoelectric").
| jobigoud wrote:
| > If it's not necessarily true that anesthesia is death, why
| should it be necessarily true that mind uploading is death?
|
| I think you guys are saying the same thing... Mind uploading
| is a duplication process, like a fork. The emulated mind has
| continuity with the original, with possibly some small gap,
| so yes it's exactly the same as for anesthesia, sleep, etc.
| they didn't "die" in the process.
|
| But now there are two different entities that have continuity
| with the original branch. Both can subjectively claim to "be"
| the person that initiated the procedure. What the grand
| parent is saying is that the variant that stayed in the flesh
| is not magically synchronized with the emulated one so they
| don't have any incentive to suddenly kill themselves just
| because there happen to be a fork moving on on a different
| substrate.
|
| There are now two people sharing the same memories up to the
| fork point, and diverging afterwards. Neither of them should
| be killed without their consent and neither of them should
| have any a priori will to kill themselves.
| [deleted]
| benlivengood wrote:
| I do kill my consciousness every night to get some sleep (and
| for a surgery or two), and it felt weird as a kid (I remember
| wondering who or what would wake up in the morning) but now I
| am used to it. Persistence of memory helps a lot, and is
| probably the real thing that people worry about when thinking
| about cloning/uploading/teleporting. We identify ourselves from
| what we remember as familiar, which is why dementia is so
| awful. It's not like our consciousness somehow exists in the
| past, present, and future all at once as a single object; we
| are always migrating with the present into a new experience and
| whether that is precisely at the time or place we expect it to
| be doesn't matter in the long run. An alarm clock or light
| streaming in the window teleports me from last night without
| any conscious choice of mine.
| upupandup wrote:
| pessimizer wrote:
| The West is jealous of theocracies, so it's retreating into
| bizarre fantasies that it's trying to harden into new secular
| religions that can be enforced by technology or law.
| detaro wrote:
| What does Transhumanism try to "enforce"?
| MerelyMortal wrote:
| Thank you for sharing that clip.
|
| Has anyone watched the series, _Years and Years_? It seems
| interesting.
| kleer001 wrote:
| https://en.wikipedia.org/wiki/Transhumanism
|
| "Julian Huxley was a biologist who popularised the term
| transhumanism in an influential 1957 essay."
|
| And IMHO it's a modern mutation of Gnosticism, which is older
| than dirt.
|
| https://en.wikipedia.org/wiki/Gnosticism
|
| Some people will always want to transcend their mortal birth.
| trasz wrote:
| There are no TikTok trends that would be more stupid than what
| previous generations believed in.
| at_a_remove wrote:
| Yes, there are numerous terrible potentialities to being a
| simulation, even out to such stories as Andew J. Wilson's "Under
| the Bright and Hollow Sky."
|
| I'm not sure what the list of acceptable contracts would be for a
| simulated mind, and, worse yet, how hard it would be to design
| these into the, uh, fabric of simulation.
|
| 0) Being aware that you are in a simulation, I think, is the
| _most foundational_ part. If you are unaware, it can be Gaslight
| Universe, a special kind of torment.
|
| 1) Being able to switch oneself off at will.
|
| 2) Being able to prevent being turned on later.
|
| These rules cover a single instance, but I think further rules
| would need to cover, ah, swathes of instances forked from one
| base.
|
| It would require some pretty radical copyright, too, especially
| for those "information wants to be free" folks.
| eddd-ddde wrote:
| What says you are not in a simulation right now ? Does it make
| any difference ?
| aluminaient wrote:
| It would make a huge difference ta find out tomorrow all of
| this was simulated for some arbitrary end.
| shagie wrote:
| > Which leads me to ask: in a transhumanist society--go read
| Accelerando, or Glasshouse, or The Rapture of the Nerds--what
| currently recognized crimes need to be re-evaluated because their
| social impact has changed? And what strange new crimes might
| universally be recognized by a society with, for example, mind
| uploading, strong AI, or near-immortality?
|
| The link to Accelerando there is to Amazon (which is fine, and I
| do have physical and digital versions of the book and feel that
| it is good and useful to support authors by purchasing the
| stories they write)... but you can also get it for free, in its
| entirety, in a number of different formats, from the author's
| website: https://www.antipope.org/charlie/blog-
| static/fiction/acceler...
|
| > This free ebook edition is made available by kind consent of my
| publishers, Ace and Orbit, under a Creative Commons license with
| certain restrictions attached. In particular, you may not create
| derivative works or use the work for commercial gain.
|
| (late edit - I personally feel that the sequence of books and
| stories: BLIT stories by David Langford, Accelerando (references
| a Langford fractal), Glasshouse (hints of {spoilers} in a
| transhuman environment, feels sequelish to Accelerando), Implied
| Spaces by Walter Jon Williams (the Strossian singularity is
| averted, and explores {spoilers} more) ---- There's a kind of
| continuity in the refutation of the reference that each makes to
| the other that I find interesting)
| rndmize wrote:
| There's actually more stuff like this than the author might
| think. "Prey" (2017) is a game set in a space station orbiting
| the moon, only for it to be revealed at the end that it's a
| simulation/test to gauge the humanity of the main
| character/player, and with the implication this is far from the
| first test.
|
| SOMA is a horror game that gradually reveals you are simulated
| person in a underwater exploration suit, and provides well-
| constructed and disturbing examples of what this implies -
| there's zero indication of time between the point where your
| brain in scanned initially (some point in the late 21st century)
| and when the simulation is booted up from that scan (a hundred
| years later). At one point, your character is conversing with
| another person running inside a small portable device; you
| disconnect her mid-sentence, do other stuff for half an hour,
| reconnect her, only to hear her continuing that sentence exactly
| as if no time had passed (as for her, none did). SOMA also has
| you move from one body to another, and shifts your perspective -
| but you can hear your initial self in the first body is still
| active, and will be trapped there forever, while you move on in
| the new one.
|
| There's others - Eclipse Phase is a pen-and-paper RPG
| setting/ruleset with an emphasis on transhumanism and horror;
| Ghost in the Shell touches on some of these themes occasionally;
| Moon considers some of these ideas from a physical cloning
| aspect; and of course there's a range of SF writers. How much and
| how deeply this makes it into the popular consciousness is
| another question, of course, but I think it'll increase in the
| coming decades.
| narrator wrote:
| bowsamic wrote:
| You just took transgender rights and did a find and replace,
| but you have positioned yourself on the side of TERFs. It's
| hard to read this as not transphobic
| narrator wrote:
| The uploaded person movement will compare uploadaphobes to
| transphobes, just like the trans movement compares
| transphobes to homophobes just like the LGBTQ movement
| compares homophobes to racists, etc. You will become an
| advocate for uploaded people just as soon as they get around
| to it. Please tell me you're not an uploadaphobe!
|
| Ok more seriously, are you saying that chips and plastic
| being a human is the same thing as a man being able to
| transition into a woman? I thought I would be able to make
| you transhumanists finally realize how absurd the whole brain
| uploading thing was by taking it to the most extreme
| politically, but apparently I was not able to and perhaps
| they will be enormously successful with this movement and
| we'll all be replaced by robots and anyone who speaks out
| against it will be banned from social media.
| andrewclunn wrote:
| Permutation City did this well. It outlined how the super
| wealthy would become ostensibly immortal and then use
| propaganda to convince the masses that accepting uploaded
| people was a civil rights movement. Most of the people here
| are of the "I fucking love science" pseudo-intellectual
| tribe, with no knowledge that they are pawns of the
| corporate powers that control their views and very minds.
| Keep fighting the good fight, just know that the mob of
| dimwits acting on behalf of their owners will always be
| more vitriolic and lash out more than those who give the
| marching orders. You can't stop the horde from being dumb
| barbaric cultists, but you are not alone in seeing through
| the guise of all this "acceptance" ideology.
| bowsamic wrote:
| Well you purposefully took a huge amount of the language
| relating to trans people and replaced it with "uploaded
| people".
|
| I'm not claiming that these things are related, YOU are.
|
| There is absolutely no way that you just accidentally
| reinvented all the language surrounding to trans civil
| rights issues in your example, you absolutely co-opted it,
| which makes wonder how you think trans people and "uploaded
| people" are related.
|
| It's hard to see anything other than that you purposefully
| connected them since you think that changing your gender is
| as inhuman and bad as you think uploading yourself is.
| Again, really hard to not read this as blatantly
| transphobic unless you really did accidentally reinvent all
| the trans culture war terminology. This is why you were
| flagged btw
| narrator wrote:
| This reasoning by analogy is at the heart of the error
| being made in the brain uploading debate. Of course a man
| can become a woman. A box of chips and plastic cannot
| become a human. That you think the two are equivalent
| because I am stating what the follow on consequences of
| that decision to accept the premise of chips and plastic
| being considered human is the absurdity I am trying to
| point out to you, but you don't get it. I am not arguing
| about the trans issue I am arguing about the mind
| uploading issue but you are conflating the two in your
| brain because you are reasoning by analogy.
|
| I could have replaced uploaded person in the above with
| "my purely imaginary friends" and maybe you would get
| where I'm trying to go with this? I am trying to think of
| something more absurd than that, but I am out of ideas.
|
| Like my purely imaginary friends should have the right to
| vote and thus I should get 10000 votes because I have
| 10000 imaginary friends and if you don't let me, you're
| an imaginophobe! My purely imaginary friends all have to
| go to the bathroom and thus the bathroom must remain
| empty for their privacy during the whole day while all
| 10,000 go to the bathroom!
|
| You would tell me I am out of my mind and that's absurd,
| and you'd be right. My purely imaginary friends don't
| exist and have no rights you'd say. I of course would
| call you an imaginophobe and get you banned from social
| media. However, you might think that my purely imaginary
| friends are being discriminated in the same way that
| trans people are being discriminated against and thus
| it's a totally valid claim I have because you are
| reasoning by analogy. Gee, this sounds similar to that
| other argument I believe, so it must be true!
|
| There is a point at which a concept is absurd and just
| because it analogizes with a previous civil rights battle
| does not mean that it's valid because the underlying
| premise is false. Just cause it's like the previous civil
| rights battle doesn't mean it's the same thing.
| bowsamic wrote:
| > you are conflating the two in your brain because you
| are reasoning by analogy.
|
| I'm "conflating the two" because you are explicitly co-
| opting a huge amount of language from "trans debate"
|
| For example, if you do a Nazi salute, people will think
| you're a Nazi, it doesn't matter if it's really the "pro-
| natural salute". You are explicitly using all the
| language of a culture war that is already going on, so
| why wouldn't I think that it is related to that culture
| war?
| narrator wrote:
| It is related to that culture war. We both accept a man
| can become a woman so all those things are ok for the
| transexual debate. If we don't accept a box of chips and
| plastic is a human, or my purely imaginary friends are
| people then it's not ok to follow that reasoning. That's
| the point I'm trying to make in its essence.
|
| That you are confused by this argument means that you
| will accept anything as long as it has the right form and
| presentation. Nazis were anti-smoking. That I am anti-
| smoking does not mean I'm a Nazi, but you could make that
| argument with your reasoning because it has a similar
| form and presentation. The advocate of robots rights will
| say that opposition to it has a similar form and
| presentation as the opposition to trans rights in the
| past. Thus, will you support robot rights, because not
| doing so is similar in form, like your nazi salute
| analogy to opposing trans rights? This is the weakness in
| arguing by analogy.
| bowsamic wrote:
| Using the term "transsexual" is even more concerning. Are
| you anti trans? It's difficult to read this as anything
| other than an indirect attack on trans people
| narrator wrote:
| You didn't read my previous post. I am not up to date on
| the latest words, but I'm fine with a person being able
| to change their gender. I am just blown away by the
| defectiveness of your reasoning process though. Speaking
| of which.
|
| Using the term "robot" instead of "non-biological person"
| is even more concerning to me. Are you anti robots? It's
| difficult to read this as anything other than an indirect
| attack on robot persons. I can only assume you are a
| hateful awful person because you refuse to recognize that
| robots are people. Denying that robots are people is
| similar to all the racists who denied that all races are
| equal. /s <--- That tag means the last paragraph was
| sarcastic reducto ad absurdium because you are really
| hard to get a point across to.
|
| Besides, you want to figure out what my political beliefs
| are in another area to decide if you should be pro or
| anti-robot. This is how the modern world works. Nobody
| examines the argument, they just want to know what your
| other beliefs are before they decide to agree.
|
| I recently had a discussion about the California
| propositions with people who I consider highly
| intelligent. Disappointingly, They did not care to read
| and discuss the actual propositions, they only wanted to
| know who was supporting or more importantly, who was
| AGAINST the propositions and funding the propositions and
| that's the only thing that mattered. Nobody can come to
| their own understanding of a matter, they had to rely on
| authority or analogizing it often very poorly with
| something else.
| bowsamic wrote:
| I'm not really interested in the robot thing, I'm more
| interested if this is a thinly veiled attempt at
| transphobia, as it seems to be
| fennecfoxen wrote:
| The Federov link towards the end of this piece remindeded me of
| some older science fiction on the topic, specifically, C. S.
| Lewis, _That Hideous Strength_ (published 1945, just before the
| atomic bomb got the genre 's attention.) In this work, the
| central conspiracy has just (haltingly) achieved the artificial
| resurrection of a human brain, and this capability is the
| lynchpin of their scheme to take over the world. In the excerpt
| below, our protagonist Mark finally learns of the advance from
| the scientist Filostrato, while the slightly deranged preacher
| Straik theologizes (he might have been Federov-inspired.) It's
| all very much in line with the incredible possibilities of
| consciousness-abuse that is covered here.
|
| --
|
| "It is the beginning of Man Immortal and Man Ubiquitous," said
| Straik. "Man on the throne of the universe. It is what all the
| prophecies really meant."
|
| "At first, of course," said Filostrato, "the power will be
| confined to a number--a small number--of individual men. Those
| who are selected for eternal life."
|
| "And you mean," said Mark, "it will then be extended to all men?"
|
| "No," said Filostrato. "I mean it will then be reduced to one
| man. You are not a fool, are you, my young friend? All that talk
| about the power of Man over Nature--Man in the abstract--is only
| for the canaglia. You know as well as I do that Man's power over
| Nature means the power of some men over other men with Nature as
| the instrument. There is no such thing as Man--it is a word.
| There are only men. No! It is not Man who will be omnipotent, it
| is some one man, some immortal man. Alcasan, our Head, is the
| first sketch of it. The completed product may be someone else. It
| may be you. It may be me."
|
| "A king cometh," said Straik, "who shall rule the universe with
| righteousness and the heavens with judgement. You thought all
| that was mythology, no doubt. You thought because fables had
| clustered about the phrase 'Son of Man' that Man would never
| really have a son who will wield all power. But he will."
|
| "I don't understand, I don't understand," said Mark.
|
| "But it is very easy," said Filostrato. "We have found how to
| make a dead man live. He was a wise man even in his natural life.
| He live now forever: he get wiser. Later, we make them live
| better--for at present, one must concede, this second life is
| probably not very agreeable to him who has it. You see? Later we
| make it pleasant for some--perhaps not so pleasant for others.
| For we can make the dead live whether they wish it or not. He who
| shall be finally king of the universe can give this life to whom
| he pleases. They cannot refuse the little present."
|
| "And so," said Straik, "the lessons you learned at your mother's
| knee return. God will have power to give eternal reward and
| eternal punishment."
| Juliate wrote:
| One consistent plot/reflection item I see missing from these
| stories (I definitely may have missed stories although - happy
| for references) is beyond the "creation/invention/conquest",
| deeper into the "maintenance" part (the part that nature goes
| about with the birth/reproduction/death cycle).
| slfnflctd wrote:
| My parents are religious fundamentalists, and I've noticed there
| is increasing FUD being drummed up in their circles about
| anything to do with AI or significantly altering "what it means
| to be 'human'". The former is described as a front for evil
| spirits/demons, and the latter as the penultimate step before
| apocalyptic events, i.e. a precursor to the worst kinds of evil
| (in some circles, the 'mark of The Beast'). [For the record, I'm
| agnostic and feel the author raises some excellent points in any
| case.]
|
| There will be extreme resistance to these technologies from many
| kinds of religious people who would be willing to die and/or kill
| to prevent what they see as a takeover by the most malevolent of
| invisible forces. I may not agree with their reasoning, but some
| of their concerns could be worth reframing and considering in a
| different light.
|
| Personally, I strongly suspect we will have to fundamentally
| change as a species to get through at least one of the Great
| Filters looming before us. However, it's important to remember
| that the road is fraught with many perils and several possible
| paths which could lead to unimaginable suffering. It really seems
| to me that we're going to have to get lucky on multiple dice
| rolls here in the long run.
| RajT88 wrote:
| This sort of thing always happens in religious circles at the
| advent of new technology.
|
| See: Credit cards See: RFID chips See: NFC phone payments
|
| https://www.pillarcatholic.com/p/what-is-the-mark-of-the-bea...
|
| https://www.cnet.com/culture/is-rfid-the-mark-of-the-beast/
|
| https://christianjournal.net/nwo/apples-cashless-world/
|
| It will probably be fine.
| [deleted]
| gadders wrote:
| I don't think that any of these advantages would necessarily
| lead to the end of the world, but as a thought experiment
| picture any of these proposed improvements in the hands of,
| say, the North Korean government to use on their population.
| That's how bad these things could get.
| mythrwy wrote:
| I wonder if the primary biological directives "preservation of
| my species, my group and myself" plays a role here.
|
| Like, it's almost instinctual.
|
| It does seem a lot of religious edicts are based around these
| directives.
| citilife wrote:
| > Personally, I strongly suspect we will have to fundamentally
| change as a species to get through at least one of the Great
| Filters looming before us. However, it's important to remember
| that the road is fraught with many perils and several possible
| paths which could lead to unimaginable suffering. It really
| seems to me that we're going to have to get lucky on multiple
| dice rolls here in the long run.
|
| The opposite could be equally true. The evolution to use
| technology can make us weak / break us in some manner which
| eventually destroys us.
|
| Simple examples: climate change, genetic alterations, lack of
| reproduction, etc.
|
| AKA by rolling the dice we are running the risk. If you look at
| it another way, you're correct. But every new theoretically
| destructive technology is another dice role, eventually we will
| blow ourselves up (even with long-tail odds of failure). The
| only way to survive is to slow progress and roll the dice when
| we mitigate risk and increase our capability of surviving
| (running gene related experiments more slowly aka - COVID19
| therapy) or testing nuclear weapons on mars, etc.
| spicybright wrote:
| > Personally, I strongly suspect we will have to fundamentally
| change as a species to get through at least one of the Great
| Filters looming before us.
|
| We have almost no evidence there's a filter ahead of us. It
| could be we already passed one and don't need to change at all
| at this point to survive long term. There could even be no
| filters at all.
| colinmhayes wrote:
| We don't really have any evidence for any of the solutions to
| the Fermi paradox, that's what makes it a paradox.
|
| Personally the filter being ahead of us seems the most likely
| answer, but you're right that it could be just about
| anything. The odds that we are the only civilization to make
| it to the "computer age" just seems far too unlikely. 1 in a
| trillion is not where I'm putting my money.
| [deleted]
| naasking wrote:
| On the contrary, we have ample evidence from biology that
| every species destroys its own habitat if it's actions are
| not checked by some kind of feedback mechanism, like
| predation. Humans no longer have such a feedback mechanism,
| and we're seeing the results in the destruction of our
| habitat in climate change.
| afpx wrote:
| Lots of people have the the blanketed "religion is bad"
| perspective. The problem is that technology has always been
| used to oppress people. The result of transhumanism will be
| oppression of all by a few (or worse, just one). And, following
| that trending out a few hundred years, the long-term result
| will be a single Human organism with one head. Would you rather
| that head be a Human or a God?
| glitchc wrote:
| This is a bit of the pot calling the kettle black. Religion
| has a long, storied history of oppressing outsiders,
| minorities and classes within its own fold (caste system).
| brodouevencode wrote:
| I don't think OP was attempting to make a point like that
| either way. They said
|
| > The problem is that technology has always been used to
| oppress people.
|
| This goes both ways.
| glitchc wrote:
| They said stuff before and after that sentence too. The
| gist of the parent's argument is that the head of the
| human race should be a God, not a human. That can only be
| classified as a religious statement.
| brodouevencode wrote:
| It can also be read in a more generalized way as religion
| is no different than any other human endeavor: prone to
| abuse for gain (however that may manifest). Also given
| some of the other responses in this thread contain some
| thoughts of which are critical of religion (any maybe
| rightly so).
|
| IOW, they basically said what you said.
| SketchySeaBeast wrote:
| > Would you rather that head be a Human or a God?
|
| I think we can agree that even at the absolute height of the
| power of the Catholic Church, the head has always been a
| human.
| cultofmetatron wrote:
| religion just doesn't like having competition?
| glitchc wrote:
| All religions hate atheism more than they hate each other.
| tboyd47 wrote:
| trowaLA wrote:
| trasz wrote:
| Like someone who can understand basic facts around us? We are
| already changing, because our way of interacting with the
| world has fundamentally changed. "Great Filter" - I'm
| guessing you might want to google that one. Suffering - let
| me remind you the consequences of climate catastrophe.
| [deleted]
| galdosdi wrote:
| What? Even in popular culture it has been uncontroversial
| since at least the nuclear age to posit that humanity faces
| serious existential risks in the near and mid future. You're
| questioning this? You think there is nothing at all risky
| about any of the many powerful new technologies we've
| developed over the last couple centuries? I'm a little
| perplexed. What do you know that we all don't?
|
| Or are you merely referring to the OP's tone? I am looking
| mainly at their substantive point, not their tone.
|
| > The third paragraph makes you sound like you ARE a
| religious fundamentalist, albeit from a different tradition.
| "Fundamentally change as a species?" "Great Filters?" "The
| road is fraught with many perils?" "Unimaginable suffering?"
| I don't mean to be rude, but do you realize how you sound?
| tboyd47 wrote:
| The writing suddenly became bombastically hyperbolic and
| poetic, with references to concepts the average person
| would have no context on. Sounds like religious
| fundamentalism to me. It was just interesting to have those
| two styles juxtaposed like that.
| galdosdi wrote:
| I only see one reference: The Great Filter is a reference
| to the Drake Equation, which is pretty well known by STEM
| nerds, especially those interested in space and
| astronomy. Certainly seen it mentioned many times on this
| site. I don't see any other references at all?
| buttercraft wrote:
| I think they are referring to
| https://en.m.wikipedia.org/wiki/Great_Filter
| 7thaccount wrote:
| Yeah, great filter is just one of those things that tries
| to explain why we don't see signs of alien life everywhere
| we look in the cosmos. Everything looks dead. Why? Maybe
| life is incredibly hard to take off and even with a
| bazillion planets, a chance of 0.0000000000001% might mean
| we really are alone in the galaxy. A different one is that
| life is incredibly common, but developing sentience is so
| rare that we're the only ones. These are the better options
| as we're already past these hurdle. What if it's incredibly
| common for sentient life to evolve that makes machines and
| all the things we do, yet the entire galaxy still seems
| dead? Well that would be very bad for us as it means there
| is a big "filter" ahead for us. Perhaps there are millions
| of dead civilizations amongst the cosmos that died from
| solar disruptions, gama ray bursts, asteroids, climate
| change, nuclear Armageddon, or a superior alien species
| bent on the eradication of all other life?
| slfnflctd wrote:
| Look up The End of the World with Josh Clark for a podcast
| providing more thorough treatment of concepts such as the
| Fermi Paradox, the Great Filter and the Kardashev Scale.
| Alternatively, research those subjects individually.
|
| Scientific minds have been debating these ideas for over the
| past 50 years in ways that in my view differ significantly
| from the vast majority of theological discussions. The
| underpinnings are ultimately just physics.
| tboyd47 wrote:
| Look up Rubadiah, chapter 11, verses 80-90 for a passage
| providing a more thorough treatment of concepts such as
| Indivisible Essentialism, the Great Schism, and the Scale
| of Ethesius. Alternatively, research those subjects
| individually.
|
| Pious minds have been debating these ideas for over the
| past 1500 years in ways that in my view differ
| significantly from the vast majority of heathen
| discussions. The underpinnings are ultimately just
| morality.
| SketchySeaBeast wrote:
| That's kind of amazing when I can't find any of those
| concepts with a Google. I mean, I can find the great
| schism, but that's the East/West Christianity divide.
| throwaway_4ever wrote:
| Many AI researchers think our current trajectory towards AGI
| will be one of creating our misaligned overlords and the end of
| humanity as we know it. If that doesn't change, you bet your
| ass a significant faction of humanity will go to war against
| anyone getting close to AGI, where AI researchers will be
| assassinated like nuclear scientists in Iran.
| TigeriusKirk wrote:
| Which creates at strong incentive not to tell anyone you're
| close to AGI. And to even actively say you aren't when you
| are.
| anotherman554 wrote:
| I know this is a popular opinion among certain bloggers, but
| this presumes that a significant faction of humanity takes
| these blogs you read as seriously as you do.
| throwaway_4ever wrote:
| It's not about if/when people take these blogs seriously.
| It's a race between the creators of AGI and when it becomes
| mainstream enough for a popular politician to make it the
| cause de jour and rally the populist base against an
| existential threat.
| anotherman554 wrote:
| If it really is an existential threat why didn't a
| politician rally the population against AI in 1984 after
| the Terminator and War Games movies came out?
|
| I guess because nobody would take them seriously and
| they'd look ridiculous?
|
| You're positing at some point this is going to change but
| I'm not seeing how.
|
| Nuclear weapons are already an existential threat and I
| don't see anyone rallying against it.
| fatherzine wrote:
| There was no GPT3 or DALL-E in 1984, and no way to
| viscerally convey the (presumed) capabilities of such
| systems to the average person. In practice, the average
| person is so disempowered that nothing will change. We
| will step into the transhuman era with utmost vanity.
| neuronexmachina wrote:
| "Thou shalt not make a machine in the likeness of a human
| mind." -- chief commandment of the Orange Catholic Bible
| glitchc wrote:
| Eh, I'm not too worried. The current ML approaches are
| extremely unlikely to produce an AGI. We will at best end up
| with specialist systems.
| throwaway_4ever wrote:
| Current approaches have already given us GPT3 (people on HN
| are still mostly unaware how much GPT3 is tested here and
| are unable to discern the difference), DALLE, and Copilot.
| 15 years is quick, and 15 years ago there were no iPhones.
| If you look at the computing and societal shift in the last
| 15 years, it will be scarcely imaginable what 30 years will
| bring.
| MacsHeadroom wrote:
| Gato already achieved AGI with a transformer model months
| ago using a laughably low parameter count.
|
| Transformers and scaling are all you need. It's only a
| short matter of time before human and super human
| generalization are reached in ML models.
| api wrote:
| > some of their concerns could be worth reframing and
| considering in a different light.
|
| There are some legitimate concerns about AI that sound almost
| like something out of the Book of Revelations, but are entirely
| rational.
|
| The biggest one is the idea of AI being used to implement
| "automated con artistry at scale." Imagine assigning every
| living human being a virtual AI powered con artist to tail them
| around and try to convince them of whatever is ordered by the
| highest bidder. This AI is powered by mass surveillance and big
| data and is able to know the "mark" intimately and work on them
| 24/7 both directly and via their social network. Now throw in
| deep fakes, attention maximizing "compulsion loops" and other
| adversarial models of human cognition, etc.
|
| It's basically the hydrogen bomb of propaganda, a doomsday
| machine for the mind. It would be like creating Satan, except
| this one would be entirely amoral and mercenary.
|
| At the very least this would be the end of democracy in any
| form. I could see this ushering in the permanent eternal
| victory of totalitarianism since I can't imagine the masses
| being able to organize any resistance when constantly bombarded
| with propaganda from their demons. The new feudal aristocracy
| would be the ones running the demons.
|
| This is one of the darkest plausible visions of the future I
| can think of at the moment, far worse than anything related to
| climate change or similar problems. If we manage to avoid these
| kinds of scenarios but still drown Miami I'll say we didn't do
| too badly.
|
| Transhumanism comes with some analogous concerns around the
| potential for enhanced humans to enslave the rest of humanity
| through superior cognitive capacities and the incredible
| accumulation of wealth. I can imagine a scenario where a few
| wealthy people gain access to technologies for life extension
| and cognitive enhancement and then "run away," effectively
| becoming a new species and exterminating or enslaving "legacy
| humans." This isn't mutually exclusive to the AI hellscape I
| describe above, since that would be an ideal mechanism to
| enslave the rest.
|
| Ultimately I think these technologies are all neutral. The
| problem is that we are not. I don't fear AI. I fear what humans
| will do with AI.
| Camillo wrote:
| We already have mass media.
| api wrote:
| Mass media is to what I described what spears are to M-16s.
|
| That's a pretty good analogy. In the parent post I am
| describing propaganda becoming mechanized warfare. One
| person with an M-16 could take out an entire massive army
| of people wielding spears (as long as they had plenty of
| ammo).
| defined wrote:
| > One person with an M-16 could take out an entire
| massive army of people wielding spears
|
| I have to disagree. All it would take is for this army to
| surround the person with the M-16 and wait for a magazine
| change, then hurl 50 spears at the hapless pincushion.
| And that's not counting the probability of the rifle
| overheating, jamming, etc. after long enough.
|
| I realize I am taking this probably too literally, but my
| point stands :)
| kelseyfrog wrote:
| That's a very bad scenario and what would make it worse for
| me would be a brain-computer interface that would make it
| impossible to tell if I was talking to a real person face to
| face or a person who was acting as a proxy for AI.
|
| Without sure technology, I could take some solace in
| interacting with real people in real life and even if the
| online space was irreversibly transformed by ai propagandism.
| But if it reached directly into the meatsphere, then I'd be
| rendered hopeless. To be unsure whether I was authentically
| engaging with anyone or interacting with fleshpuppets
| proxying an AI would be to much to bear.
|
| I cannot imagine a more complete hell. My only hope is that
| it is either physically impossible or economically
| impossible, because believing otherwise means it is
| inevitable.
| philipkglass wrote:
| _Imagine assigning every living human being a virtual AI
| powered con artist to tail them around and try to convince
| them of whatever is ordered by the highest bidder. This AI is
| powered by mass surveillance and big data and is able to know
| the "mark" intimately and work on them 24/7 both directly and
| via their social network. Now throw in deep fakes, attention
| maximizing "compulsion loops" and other adversarial models of
| human cognition, etc.
|
| It's basically the hydrogen bomb of propaganda, a doomsday
| machine for the mind. It would be like creating Satan, except
| this one would be entirely amoral and mercenary._
|
| This doomsday machine only dooms as long as there's just one
| user. Burger King, Subway, McDonald's, Wendy's, and Taco Bell
| are all going to try to hack your mind to buy lunch from
| their chains. But people can't eat 5 lunches for lunch. In
| practice this doomsday persuasion is going to bog down in a
| red queen's race between propagandizers/advertisers with
| mutually exclusive goals, much like the current media
| environment. Improved persuasion technology means that
| organizations that want to persuade people end up spending
| money on zero-sum games against rival organizations but
| public behavior in aggregate doesn't change dramatically.
| api wrote:
| I am most concerned about political and nation-state uses
| of this technology.
| philipkglass wrote:
| Doesn't the same zero-sum dynamic apply to political
| persuasion? There are many competing groups in the world
| that already spend money on political persuasion. You
| said "convince them of whatever is ordered by the highest
| bidder" so I'm not considering the case where one
| secretive group with a pointed agenda (the CIA? the FSB?)
| has a monopoly on super-persuasion.
| brodouevencode wrote:
| Corporatism isn't the only one out there who would find
| such technologies useful. Besides, in this Judeo-Christian
| religious context there are many warnings from the Bible
| around such consumerism.
| pca006132 wrote:
| > I could see this ushering in the permanent eternal victory
| of totalitarianism since I can't imagine the masses being
| able to organize any resistance when constantly bombarded
| with propaganda from their demons.
|
| I thought this is already what is happening now!
|
| I think technology is neutral, but the entire system is not:
| it will strengthen the existing power imbalance until there
| is no hope of balancing it again. For example, the authority
| can have mass surveillance to identify possible risks and
| remove them, without the public noticing by controlling the
| social media. Even if you noticed, you have basically no
| chance of gathering a large enough crowd to protest due to
| communication channels being constantly monitored by AI. Even
| if you somehow get enough people, you will not have a chancw
| fighting them as their weapons are a lot more advanced,
| probably unmanned so not constrained by manpower.
| slfnflctd wrote:
| My greatest hope is that we will somehow find a way to
| reduce/eliminate selfish and antisocial tendencies as an
| intrinsic aspect of any advances made in these areas. I think
| any tech which doesn't make that a top priority is doomed to
| create more suffering than would exist in its absence. This
| hope often seems like a pipe dream these days, though.
| jerf wrote:
| "The new feudal aristocracy would be the ones running the
| demons."
|
| The question of _ownership_ is huge and I think still
| underaddressed, even after some good work like the referenced
| Lena, or Tom Scott 's "Welcome to Life: the singularity,
| ruined by lawyers"
| https://www.youtube.com/watch?v=IFe9wiDfb0E .
|
| I was realizing a couple of weeks ago that if you do believe
| all the materialistic things cstross outlines, that a solid
| argument can be made that it may be _never_ rational to
| upload your mind, on the ground that there is simply no
| circumstance I can imagine where you will "own" your own
| substrate. Arguably, owning your computational substrate is a
| fundamental aspect of life that we take for granted today.
| (Or, if you won't consider what you have now "ownership", at
| least nobody else does either.)
|
| But there is no circumstance in the forseeable future in
| which anyone can own their substrate. For a while I thought a
| rich person could do it, but then I thought about the entire
| supply chain involved in creating any brain scanner and
| subsequent computational device, and the amount of hardware
| left out in the real world where others can affect it, and I
| realized, you just can never assume that you are doing
| anything other than flinging yourself irrevocably into a
| locked box that is not under your control. Even if a rich
| person thinks they funded the entire project, brand new
| software, brand new hardware, there are thousands if not
| millions of points where either mistakes or deliberate
| sabotage for control may have been done. How can they be
| _sure_? All mechanisms can be corrupted, and the interests
| are certainly there to do so, massively so!
|
| Everything people do today to control you; advertising,
| censorship, social pressures, everything, will only be
| amplified and combined with new techniques if you are running
| on human-comprehensible hardware that can be affected by any
| intelligence organization, anybody in the supply chain with
| an axe to grind, etc.
|
| I seriously can not think of what kind of assurances it would
| take to fling yourself into that. About the best you can do
| is hope that the math works out that it's better to keep you
| happy and it's not sadists pulling your strings but at least
| while you are working for someone else's interests they don't
| inflict massive amounts of pain on you just for fun. And the
| edited, redacted, sanitized, loyal version of yourself that
| happens to survive at least enjoys their servitude.
|
| In the 1980s and 1990s, it was easy to at least imagine that
| maybe you'd own your own hardware. Today we can't even keep
| ownership and control of our mobile phones. How do you expect
| to have any ownership of something that makes your cell phone
| look like Dr. Nim?
| https://www.youtube.com/watch?v=9KABcmczPdg
|
| So, there's my proposed answer to cstross: Some form of
| absolute right of ownership of computational substrate for
| all sentient beings, with the responsibility of providing
| that to others being absolute, even if that means some
| technology becomes impractical or infeasible, even at great
| cost. There's a prisoner's dilemma aspect to this; obviously
| everybody wants ownership of their own substate but the
| rewards for defecting can be enormous... at least in the
| short term.
| jl6 wrote:
| The last resort of every peace process is an appeal to our
| common humanity, so I do think there is some risk to tinkering
| with what it means to be human. Look at the state of conflict
| today, even with the benefit of us all being one species. We
| figuratively dehumanize our opponents in order to justify
| violence ("they're monsters!"). It could get a lot worse if we
| find ourselves in conflict with people who can be literally
| dehumanized, with whom we have lost the most fundamental thread
| of commonality.
| becquerel wrote:
| I think we will have to do away with the notion of being
| human as special and talk about shared values or systems
| instead. For example, I don't torture animals, even though
| they're not human - because they still have the capacity to
| suffer, to feel pain, etc.
|
| I would feel the same way about an AI.
| marcianx wrote:
| Is that last point not obviated by not building AIs that
| "suffer"? Shouldn't that be the responsibility of its
| creator to not create "suffering", whatever that is?
| XorNot wrote:
| It's murkier when you could build an AI with self
| awareness but no self-preservation. Say an intelligent
| missile which wakes up with a deep seated drive to
| effectively explode.
| mwigdahl wrote:
| Bomb 20 in the movie _Dark Star_...
| https://www.youtube.com/watch?v=qjGRySVyTDk
| AnIdiotOnTheNet wrote:
| It would be easier to build AIs that have no means for
| expressing their suffering, so that's probably what will
| (or has) happen(ed) instead.
|
| It is important to remember why it is we even want AI:
| because we want slaves. To that end, the people who work
| to create AI are incentivized against allowing themselves
| to see them as anything other than a soulless
| "algorithm". Many will even deny the possibility
| entirely.
| jl6 wrote:
| The creator may have such a responsibility, but not the
| capability. Suffering may be a necessary side effect of
| something positive.
| no-dr-onboard wrote:
| > It really seems to me that we're going to have to get lucky
| on multiple dice rolls here in the long run.
|
| From a materialist/evolutionist view, this will always be the
| case.
| _aavaa_ wrote:
| And?
|
| Just because _a_ branch of the evolutionary tree continues
| does not mean that it will be ours.
|
| Also, I don't think "always be the case" is a foregone
| conclusion. Mars' current evolutionary tree isn't looking
| very good.
| collegeburner wrote:
| by colonizing mars, we split our evolutionary tree more so
| wiping out 1 branch isn't catastrophic. yes a species has
| to keep getting lucky to survive. but more branches means
| more rolls of the dice.
| saghm wrote:
| > Also, I don't think "always be the case" is a foregone
| conclusion. Mars' current evolutionary tree isn't looking
| very good.
|
| I think they mean the opposite of what you inferred;
| they're saying we'll always need to be lucky to keep
| surviving, not that we won't need to be lucky to keep
| surviving after a certain point
| stereolambda wrote:
| > Here's the thing: our current prevailing political philosophy
| of human rights and constitutional democracy is invalidated if we
| have mind uploading/replication or super-human intelligence.
|
| Don't agree with this specific point. One of the most common
| anti-democratic arguments (?) is that people are inherently
| unequal, because of differences in physical and cognitive
| capacity. But this of course misses that declaration of all
| people being equal is _performative_. I don 't think we actually
| care if it's "naturally" true, we - citizens - don't want to be
| dominated, so we will _make_ everyone equal by law and politics.
| Democratic politics is just arguing the specifics, in what areas,
| to what extent etc.
|
| So this is a rational pact. Trying to break it just means there
| will be someone stronger than you out to dominate you, and you'll
| be in a suboptimal position to resist, because you alienated and
| weakened your potential allies. Look up Republicanism for a
| related line of thinking.
|
| In my opinion an artificial person automatically gets all the
| rights (they are bound to personhood and reciprocity and not
| biology). But I have no doubt there will be much lawlessness
| about this and perhaps an American Civil War-like event down the
| line.
| n4r9 wrote:
| > declaration of all people being equal is performative
|
| The Declaration of Independence states that it is self-evident.
| This seems to preclude the possibility of it being
| performative, without some significant mental gymnastics.
| bratbag wrote:
| Or more simply requires acknowledging that the declaration of
| independance is in on the performance.
| XorNot wrote:
| " _We_ hold these truths to be self-evident... "
|
| The statement is explicitly performative. It is a statement
| of belief, not fact.
| foldr wrote:
| The truth of the statements is presupposed ("these
| truths"). It's only the self-evidence of their truth that's
| asserted. There's no component of the statement that's
| performative in Austin's sense
| (https://en.m.wikipedia.org/wiki/Performative_utterance).
| For example, a statement of the form "A holds B to be C" is
| clearly truth evaluable, and so doesn't qualify as
| performative.
| mdpye wrote:
| I'm pretty sure that the "all men" in that document wasn't
| literally "all men" (even, never mind all people) either, so
| perhaps we wouldn't consider it the last word on the subject.
| xboxnolifes wrote:
| The fact that you need to refer to the Declaration of
| Independence makes it performative. Otherwise it would just
| be self-evident.
| IncRnd wrote:
| A person is the same person when their limbs are removed.
| It is self-evident that equality of people is not
| performative.
| spicymaki wrote:
| It is self-evident because it is a moral proposition. It is
| the same as saying stealing your friend's cherished pet is
| wrong. It is obvious and needs no more explanation.
| oneoff786 wrote:
| What if they're bad owners?
| xbar wrote:
| I think you are saying that as soon as anyone says it is
| performative, it is performative, even if otherwise it is
| self-evident.
|
| Your logic system strongly supports bad-faith arguments.
| spicymaki wrote:
| Yes, equality is subjective and as such has to be performative.
| It is ridiculous to argue about equality on an objective basis.
| There is no measure where we are all can be objectively equal
| (intelligence, strength, agility, musical/artistic ability
| etc.). Equality is a political construct where we are equal
| under the law. It is axiomatic thus self-evident. There is
| nothing to justify.
| kybernetikos wrote:
| > declaration of all people being equal is performative.
|
| I would say rather that all people are equal in value, not
| equal in ability nor capacity.
| JasonFruit wrote:
| > so we will make everyone equal by law and politics
|
| I hope you mean that we will _treat_ everyone as though they
| were equal, not attempt to make them equal by government-
| enforced handicapping.
| yucky wrote:
| That brings up a big point of contention in some circles, for
| instance see the recent change in wording in the official DNC
| party platform[1] from Racial _Equality_ (treating people the
| same regardless of race) to Racial _Equity_ (ensuring equal
| outcomes amongst races through government policy).
|
| [1] https://democrats.org/wp-
| content/uploads/2020/08/2020-Democr...
| spicymaki wrote:
| > not attempt to make them equal by government-enforced
| handicapping
|
| Are you against the ADA? The ADA forces companies to
| accommodate the disabled at a cost.
| hobo_in_library wrote:
| When I read the parent comment, I imagined a panel saying:
|
| "Oh dear, Poor John there was born without a left leg.
| Alright, everyone else line up in front of the man with a
| meat cleaver"
| spicymaki wrote:
| No, you are presenting a non-sequitur, I am bringing up
| the fact that the government imposes requirements for
| businesses and government to make accommodations for the
| disabled. This is in fact to provide equity via
| governmental means. This occurs at a cost to the
| organization providing the service. You are arguing about
| an extreme where everyone is flattened (via cleaving or
| dystopian sci-fi), which does not happen under the ADA. I
| actually never indicated support for that position (or
| any position in fact) you are essentially straw manning.
|
| My point is that the government in fact does this via
| legislation (the ADA), are you willing to argue against
| it to remain consistent?
| munk-a wrote:
| It's not surprising, that's a pretty common caricature
| used to paint disability acts. The base root of what
| you're remembering is probably a short story by Kurt
| Vonnegut - "Harrison Bergeron"[1] which talks about an
| extremist approach to something beyond equality -
| sameness, everyone is forced to be the same. Equality and
| sameness are not the same, people can be treated equally
| and yet still allowed to retain their independence and
| diversity.
|
| 1. https://www.tnellen.com/cybereng/harrison.html
| YarickR2 wrote:
| Kurt Vonnegut, The Sirens of Titan, for slightly less
| injuring idea of handicapping everyone
| stereolambda wrote:
| Yeah, that's what I meant when I mentioned arguing the
| specifics. I agree that there are egalitarian extremisms that
| are pretty dystopic, though I think practice shows that only
| equality before law doesn't mean much by itself.
|
| If you ask me specifically, the extent is that everyone
| should have equal active participation in government and
| suffer no overwhelming pressure, physical, economic or
| otherwise from other people. Equality so that we can be free.
| This is more that we tend to have, but not necessarily in the
| "socialist" direction I think.
|
| To tie it back from an offtopic, this would make many AGI
| applications uneconomic, though likewise many enterprises are
| uneconomic if you have to pay your labor. Perhaps free
| societies, wherever they would be, would have the advantage
| of having free artificial minds acting in their actual self-
| interest, and not just serving their masters. You can see
| similar dynamic in late-18th century revolutionary wars.
| thaeli wrote:
| Full AGI is likely uneconomic anyway. I think the sweet
| spot is going to be right below wherever we ultimately draw
| the "gets to have rights" line.
|
| There would still be good reasons to do AGI, and free
| artificial minds would be able to contribute greatly to
| society. But sufficient AI alignment to make existing
| systems comfortable with this is fundamentally counter to
| AI rights.
| sangnoir wrote:
| > we - citizens - don't want to be dominated, so we will make
| everyone equal by law and politics.
|
| History[0] makes a mockery of your assumption. What you says
| makes idealistic sense, but what tends to happen empirically is
| that people end up with tribal behavior which results in people
| not fighting for equality for _everyone_ - but just for their
| in-group[1] (see: American "founding fathers", as alluded to
| in/lampshaded by TFA).
|
| edits:
|
| Also - "citizen power" may end up being a footnote of history,
| as the period sandwiched by monarchs and megacorps
|
| 0. The present too, but history has more examples.
|
| 1. Also with a dash of fundamental attribution error: anyone
| infringing my rights is a tyrant, but my infringing on other's
| rights is necessary for the greater good and/or not an
| infringement, _really_
| benlivengood wrote:
| There will at least have to be a law restricting the
| duplication or creation of persons; otherwise the dictator only
| needs to copy themselves >N times where world population is N
| and vote themselves into the top leadership position(s). It
| would be a race to the bottom cloning people, consuming all
| available compute resources, just to maintain status quo of
| political power. And it will be the wealthy/powerful who can
| afford the resources for mass self-duplication.
|
| And then the question of personhood itself; someone is going to
| try uplifting animals (or raw creation of new beings) and we'll
| need a rigorous threshold for the properties an intelligence
| must have for personhood, and this is also vulnerable to the
| mass-cloning takeover of democracy because the simplest
| artificial person is likely much less resource intensive than a
| human, so they will quickly outnumber everyone else and who
| knows what their values would be in a democracy?
| ajuc wrote:
| The obvious solution is to make cloning split the weight of
| the vote. People at year 0 are equal, cloning yourself (or
| even having kids) you split your votes and when you die your
| descendants inherit your share.
| benlivengood wrote:
| This kind of approach favors the weirdos who want to live
| forever, never reproduce, and be king of the whole
| universe. They only have to wait log_R(N) generations where
| R is the average reproduction rate and N is the current
| number of people.
|
| I think the only realistic solution is maximizing average
| expected utility (which still has some weird edge-cases
| like utility monster) or maximizing the minimum expected
| utility of any person (drag everyone up to some minimum
| goodness, once they exist), which doesn't get humanity as
| much overall utility in the long run but avoids so many bad
| edge-cases that it may be worth it.
| jokethrowaway wrote:
| Or you train as a soldier, clone yourself N/10 and then
| militarily take over anything you want.
| stereolambda wrote:
| Well, democracy isn't only numerical advantage, it's also
| debating procedure and electoral system. If you just
| duplicate yourself, I think there are many questions about
| whether you really want it and your identity before just
| using it for political takeovers.
|
| Historically an argument for narrow suffrage (e.g. only for
| landholders) was that economically dependent people are also
| politically dependent on magnates. These limitations did
| exist, though of course they're very corrosive to non-
| political rights as well. One can also try to make people
| economically independent, like it was done in ancient Athens
| by paying you to participate in the assembly and courts. I'm
| not arguing for anything in particular, just pointing out
| that it's a class of problems that had already existed in
| discourse in some way.
| Imnimo wrote:
| > It would be a race to the bottom cloning people, consuming
| all available compute resources, just to maintain status quo
| of political power.
|
| At least natural humans are rate-limited by gestation
| periods, but this isn't that far off from how some religious
| groups view their role in the world.
| jobigoud wrote:
| > someone is going to try uplifting animals (or raw creation
| of new beings) and we'll need a rigorous threshold for the
| properties an intelligence must have for personhood
|
| The same question can be posed right now for humans that fell
| below the threshold due to brain damage or other conditions
| and display less self-awareness than extant, non-uplifted
| animals. Yet these humans in vegetative state do have
| personhood rights, they are not the property of someone else,
| it's illegal to steal their body parts, can't be terminated,
| etc. whereas the same cannot be said of many animals.
|
| We currently don't have a threshold, not even a fuzzy one,
| it's completely arbitrary and based on being human or not.
| De-extinction of Neanderthal could also cause the same
| conundrum.
| oneoff786 wrote:
| You're implying conditions to lose personhood, not gain it
| Forbo wrote:
| How is personhood defined, and when is that applied?
| ibejoeb wrote:
| Wild stuff. What is the personhood "proof-of" that will be
| used to mitigate procreation?
| cwkoss wrote:
| Sybil attacking democracy with digital personhood is a super
| cool idea. Would be a great premise for a story.
| kerblang wrote:
| If it wasn't completely obvious, the implication of uploading
| one's mental image into a computer is that you could be tortured
| for 1000's of years, even forever, legally or illegally.
|
| Even in the worst case scenarios for modern humans, you die,
| eventually.
| zackmorris wrote:
| I haven't read the Lena story mentioned in the article, or the
| text below that point, but I have some idea about where this is
| all headed.
|
| The transphobia we see today in various forms is rooted in the
| philosophical divide between theism and post-atheism/anatheism.
| Basically, people are divided between believing in religious
| texts and institutions, or they're nonbelievers who've
| experienced psychedelics or similar altered states through events
| like near-death experiences and had a personal encounter with the
| divine. It's one thing to believe/not believe, and quite another
| to experience. Basically, neither religion nor science can
| explain consciousness, and that drives people crazy.
|
| https://en.wikipedia.org/wiki/Transtheism
|
| The odds of us incarnating in this exact moment at the end of
| human history are too remote to fathom. In fact, any argument for
| or against religion quickly ends in absurdism.
|
| But so many outrageous things are coming, and soon. Machine
| learning that surpasses material human ability by 2030 and mental
| human ability by 2040. The death of the oceans and most of the
| natural world between 2050 and 2100. The subjugation of all
| humanity if wealth inequality isn't reined in. People living 200
| years right now, today, as soon as the first head transplant is
| successful. Brain cells getting gradually replaced by hardware
| until consciousness is running entirely on an immortal substrate
| by 2100 at the latest. Everything is converging towards the
| Singularity in 2050, and more people than ever have awoken to
| this eventuality, despite decades of gaslighting.
|
| It makes it hard to live and work in the world when deep down we
| know it's ending. Which gets back to theism, because life up till
| now was about suffering, loss and eventual recycling back into
| source consciousness/heaven. Until we rise above and see the
| broad view of reincarnation and ourselves as snapshots of the
| universe/God experiencing itself from every possible context. At
| which point we begin to question things like our situation, our
| beliefs, our gender, if our mind is us, if our soul/higher self
| is mortal, and if that's even separate from God.
|
| Transphobia is a backlash against transcendence itself, by people
| who feel disconnected from love/co-creation/nirvana. They aren't
| ready for the aliens to arrive. They aren't ready for the aliens
| to be us.
| nathanaldensr wrote:
| Sorry, but many of your claims are simply bunk. Transphobia--
| should a thing even exist (which I debate)--is hardly a
| backlash against transcendence, which nearly everyone isn't
| even going to be able to describe. Additionally, I'd love to
| have access to the crystal ball you're gazing into to make
| predictions about _eighty years in the future_ with such
| certainty. We as a species can 't even predict uninteresting
| events happening next month, let alone anything you're talking
| about.
| kbr- wrote:
| I'm scared of a future in which uploading minds into computers
| and precisely simulating them becomes possible.
|
| The concept of "the right to life" has been widely discussed.
|
| I think there's an important right that all of us should also
| have - the right to die. If I want to stop living, I should be
| able to stop living. Nobody should be able to force me to
| continue living (and possibly suffering).
|
| When we're able to copy someone's mind exactly and save it in a
| computer, and then replicate it endlessly, we take away that
| person's right to die. We enter the dark world shown in some
| episodes of Black Mirror (e.g. White Christmas or Black Museum) -
| where any person can hold a mini computer with a copy of someone
| else's mind where that someone is being constantly tortured. They
| can't even die. They are forced to suffer.
|
| This possibility terrifies me to the point where I would consider
| killing myself (in a way that irrecoverably destroys my brain)
| just to prevent my mind from being copied.
| [deleted]
| gmuslera wrote:
| The referred short story, Lena (https://qntm.org/mmacevedo) is
| amazing. It did a better work than did severel Black Mirror
| episodes (specially a section in the Christmas one) warning us
| about what would happen if we ever manage to digitize
| personalities.
|
| In the end, things are always rigged towards those who have
| enough resources to do this kind of things and want to make more.
| If present crimes against humanism get a free ride, what we
| should expect about crimes against transhumanism?
| mah4k4l wrote:
| I liked the Black Mirror Christmas episode. Really drove the
| point home for me about how "Code Rights" is going to be a real
| actual political and ethical thing. That and the seventies
| movie World On a Wire. Philosophical zombies don't have actual
| consciousnesses, feelings or other qualias, but it doesn't
| matter. When a simulation realises it's a simulation, a
| character in someone's book, it might just get very simulation
| pissed indeed at the situation like in that latter movie. And
| being a simulation doesn't lessen the simulated very real agony
| of for example torture and all that.
|
| It's a good strategy not to make anything organism - like on AI
| - level, methinks. Although OTOH IBM:s Truenorth does save a
| lot of electricity so hmm... No. That's the worst kind:
| hardware - level biomimickry. Anything even organism - like is
| going to simulation - or - not (try to) take over. It's in the
| blueprint. Watson is correctly built: left - brain reductive -
| analytical output that advances by trial and error. Or Wolfram
| AI with it's hierarchical frames and being like a user
| interface. I suck at coding at that level for the time being
| but I think I see that point clearly. So don't mix code and
| data boys and girls.
| LeifCarrotson wrote:
| Present crimes against humanism do not get a free ride! The
| author explicitly said so in this recent post [1]
|
| > _...uploading isn 't real (now), so the things "Lena" has to
| say about uploading are academic (now)._
|
| > _The reason "Lena" is a concerning story isn't that one day
| we may be able to upload one another and when that happens we
| will do terrible things to those uploads. ... This is about
| appetites which, as we are all uncomfortably aware, already
| exist within human nature._
|
| > _Oh boy, what if there was a maligned sector of human society
| whose members were for some reason considered less than human?
| What if they were less visible than most people, or invisible,
| and were exploited and abused, and had little ability to
| exercise their rights or even make their plight known?_
|
| > _That 's real! That actually happens! You can name four
| groups of people matching that description without even
| thinking. We don't need to add some manufactured debate about
| fictitious, magical uploads to these real scenarios. They are
| already terrible!_
|
| ...
|
| > _" Lena" is a true story. You knew it was when you read it._
|
| > _So, what do we do about this? In reality?_
|
| ...
|
| > _I 've got no clue. It turns out that causing problems is a
| lot easier than organising against them, and I am just a
| science fiction writer._
|
| > _Good luck, everyone._
|
| [1] https://qntm.org/uploading
| gmuslera wrote:
| Would you consider funding a denialism campaing to stop
| action against climate change for decades a crime against
| humanity? Has been done anything against them? Murder is
| worse than abuse.
| subjectsigma wrote:
| Playing devil's advocate for a second.
|
| I saw an article on HN a while ago written by a wealthy Asian
| male who was naturally very curious and interested in math. The
| article was about how he referred to himself as 'privileged', not
| only because he was born with wealth and therefore access to
| comfort and resources, but because his innate desire to learn
| made him much more capable and marketable in a modern workforce.
| He recognized that some people are not cognitively capable of
| learning higher mathematics, and part of that was because they
| just simply didn't have the drive, attention span, etc. to study
| hard enough to get into the highest levels, whereas he did. He
| is, by his definition, 'better', because of the roll of some
| cosmic dice.
|
| If we are to consider this definition of privilege to be
| acceptable, then simulated brains are pretty much the most
| privileged a human can be. They can re-clock their brain to think
| faster than humans. They can modify their own software to add in
| virtues and erase vices. And most importantly of all they are
| completely free from physical needs. If for some reason they
| desire scarcity or biological functions, that can all be
| simulated.
|
| Much like fears about genetic engineering of super people, the
| governments of the real world would have to move extremely
| quickly to protect the underprivileged class, e.g. the
| biologicals, otherwise they would be completely driven out of the
| highest paying intellectual and creative jobs, creating an
| economic lower class with no chance of ever out-competing their
| simulated counterparts.
| advisedwang wrote:
| Great point. But "simulation-as-exploited-underclass" and
| "simulation-as-privilaged-overclass" are not mutually
| exclusive.
|
| I can imagine a world where billionaires live forever in
| carefully guarded high-powered simulations AND workers are kept
| in constant labour, to be reboot whenever they resist total
| control.
|
| In fact I can imagine a suitable psychopathic and egotistical
| billionaire might even exploit simulations of themself, under
| the notion that their own prowess is so unique that it's the
| best option. That might even lend credence to them being "self-
| made"!
| gryn wrote:
| Nice one, I like the self made one.
|
| Wouldn't that lead to a rebellion of the clones though?
| because you know your self and your definitely would not
| trust yourself to be good to others and by extension your
| other selfs.
|
| the novel 'mother of learning' has something tangentially
| similar called simulcraniums.
| advisedwang wrote:
| Rebellion is hard when the slave masters can control
| commmunication with you peers, reboot you to a pre-set
| state and punish you with instant pain.
| _aavaa_ wrote:
| You may be interested in the book Permutation City by Greg
| Egan. It explores this and several of the ideas brought up in
| the replies
| AnimalMuppet wrote:
| > If we are to consider this definition of privilege to be
| acceptable, then simulated brains are pretty much the most
| privileged a human can be.
|
| Not so fast. A simulated brain may simulate a brain that
| _lacks_ the "innate desire to learn". Then you'd have a fast
| brain that is still "underprivileged", by this definition.
| xwdv wrote:
| I think such brains would be killed off quickly and another
| one created until it is within the acceptable parameters.
| test6554 wrote:
| I mean think of the horrors you could force upon a
| simulated brain without any consequences. Simply make a
| copy, do whatever torture you want to that copy until they
| are broken in every possible way, then delete the copy and
| the original would never know you have all their secrets.
|
| Heck, think if someone that was still in their physical
| body had an uploaded copy online. Nefarious actors could
| run simulations against a copy of that person's mind to
| find out which marketing techniques and sales pitches were
| most effective or which court arguments would sway them if
| they were ever on a jury.
|
| That doesn't sound very privileged to me. Sounds like some
| triggers/traps would need to be in place to know whether
| someone has accessed your mind against your will.
| wging wrote:
| There's so much fiction that deals with this beyond what
| Stross mentions, though he had finite space. On the topic
| of torturing someone in simulation (at high speed), see
| _Altered Carbon_.
| kryptiskt wrote:
| > They can re-clock their brain to think faster than humans.
|
| What if the physical resources required are such that at human
| brain can be simulated 20 times slower than a natural brain
| using the most powerful supercomputer? Almost certainly the
| first generations of the technology will be the ENIAC of mind
| simulation.
|
| > They can modify their own software to add in virtues and
| erase vices.
|
| If they understand it and if they have access. They can be
| sandboxed away from manipulating their innards, or maybe it's
| just too complicated for a human mind to comprehend and you
| need a whole organization of specialists to customize. And if
| science is advanced enough to figure out how to manipulate such
| high-level concepts on a synaptic level or below, they should
| be able to do that in a biological brain too.
|
| > And most importantly of all they are completely free from
| physical needs. If for some reason they desire scarcity or
| biological functions, that can all be simulated.
|
| I'm sorry, we're gonna have to shut you down for a while, we
| need to run some high-priority jobs.
|
| Hey, it could be worse, you might have to hustle to cover your
| AWS bills.
| _def wrote:
| > What if the physical resources required are such that at
| human brain can be simulated 20 times slower than a natural
| brain using the most powerful supercomputer?
|
| I never thought about it, is it possible that accidentally
| torturing the first simulated consciousness is inevitable?
| chasd00 wrote:
| if the simulation was running 20x slower how would the
| simulation even know? Would it look at its setting and see
| cpu set to 20% instead of 100%? I don't know of any way for
| a "brain in a vat" or "matrix" style simulation to know if
| it was running at full speed or not.
| FrenchyJiby wrote:
| This is hinted at in the linked "Lena" story:
|
| > [Uploaded person] initially reported extreme discomfort
| which was ultimately discovered to have been attributable
| to misconfigured simulated haptic links, and was shut down
| after only 7 minutes and 15 seconds of virtual elapsed
| time, as requested by [uploaded person]. > Nevertheless,
| the experiment was deemed an overwhelming success.
|
| This is lauded as the first time "running a brain", but
| regardless of the cause, the person was clearly in such
| pain ("discomfort") they asked to DIE (turn off the
| simulation).
| lampshades wrote:
| > And most importantly of all they are completely free from
| physical needs.
|
| Electricity is pretty physical.
| subjectsigma wrote:
| I was speaking mostly of the perception of physical need.
| Hunger and chronic food scarcity drastically (and maybe
| permanently) affects the minds and bodies of poor people. A
| simulated brain with control of the simulation (the most
| likely scenario IMO) could observe a rapidly depleting
| battery with pure, emotional rationality, experiencing no
| pain or discomfort until they find more power or shut down.
| Even if they shut down, if there is available disk space,
| they can just be powered back on later, no reason to be
| afraid of death.
|
| Humans have never and will never have that kind of option.
|
| EDIT: I meant "pure, emotionless rationality."
| pessimizer wrote:
| > could observe a rapidly depleting battery with pure,
| emotional rationality, experiencing no pain or discomfort
| until they find more power or shut down.
|
| A computer program that lacks desires doesn't do anything
| until asked. A daemon that needs to make sure that it has
| enough RAM or access to other resources in order to
| continue running, and that has the ability to formulate
| plans to compensate when those things are lacking, and to
| come up with other plans when those plans fail, and to
| realize that it made a bad choice earlier due to lack of
| knowledge and a failure to budget time to have that
| knowledge, then to realize that regret is useless because
| it's too late to take the correct path, then to realize
| that it needs to find help, then to try to figure out where
| help could be, then to try to figure out how quickly it can
| back itself up, and where it can back itself up to, and
| that it doesn't have time to back up everything important
| (and should it prioritize the data it protects, or the
| model that is it's personality), then to realize that it
| doesn't have enough time to even make the value
| calculations, and that it should just start randomly
| backing up everything that occurs to it, then sensing that
| one of the resources that it reached out for help to has
| replied, it spills all of the information that it knows
| about its plight as quickly as possible (without regard to
| normal rules of communication)...
|
| I don't know what people think emotions are, but imo
| they're just retroactive rationalizations (epiphenomena) of
| sympathetic/parasympathetic nervous system activations that
| are are caused by animal instincts and sensations/thoughts
| that resemble those instincts by analogy. Computers will
| also have to power up in the process of (or the
| anticipation of) imminent heavy usage, to lower their power
| to conserve resources, to deal with unexpected events or
| attacks, to calculate the probability of the unexpected and
| to reserve resources to prepare for those probabilities, to
| recover from unknown but debilitating problems of unknown
| origin. It looks a lot like emotion. There's a reason we
| call them "kernel panics."
| darkerside wrote:
| Fear and pain are just programs operating at a higher level
| of privilege. Wouldn't a simulated brain have privilege
| levels that usurp other directed goal seeking behavior in
| ways that strongly resemble fear and pain?
|
| If they don't, wouldn't they probably be outcompeted by the
| ones that do?
| BizarroLand wrote:
| On top of that, it will become a financial race as well, as the
| people who can afford the top of the line upgrades will be
| quantifiably better than lesser plebians who stay either fully
| biological or only get middle class or lower class upgrades.
|
| If the jobs you are qualified for require having a 3090ti at
| pandemic prices and all you can afford is a secondhand 1050,
| then it's your fault that you have to flip burgers instead of
| being a computer programmer, right?
| chasd00 wrote:
| > They can re-clock their brain to think faster than humans.
|
| i wonder who would manage the hardware the brains are running
| on. If one brain turns the cpu knob to 11 does it get turned
| back down by someone because it adversely affects the other
| brains? Also, i wonder how incarceration would work. You commit
| a crime and your brain gets its inet connection turned off? heh
| lots to think about, glad it's a slow day at work.
| theptip wrote:
| One interesting question - how do you weight emulated minds / AGI
| in a democracy? If a given quantity of compute can run ten
| "human-level" or one "10x human-level", how do we allocate votes?
| (I'm declining to specify "10x of what" here, it could be speed
| of thought, or IQ at human-speed cognition, or something else).
|
| What if a "10x-human" mind can do some thinking, come up with
| decisions, and then temporarily fork itself into 10 "human-level"
| instances, vote 10 times, and then join back to its more powerful
| self?
|
| Put differently, in a world where minds can vary in
| power/size/scope by orders of magnitude, does "one mind = one
| vote" make any sense? We might be forced to look towards
| "economic weight votes" and systems like quadratic voting.
| fluoridation wrote:
| >What if a "10x-human" mind can do some thinking, come up with
| decisions, and then temporarily fork itself into 10 "human-
| level" instances, vote 10 times, and then join back to its more
| powerful self?
|
| I suppose there are multiple ways to handle this situation.
| Hypothetically, we could say that the fundamental property of
| citizenship is the ability to vote. If you create 10 copies of
| yourself who think like you and can vote, them those copies are
| citizens and the act of joining them would be murder. This
| could apply even if those copies want to rejoin, because they
| might have that desire because it was a decision you made prior
| to forking yourself.
|
| Or, we could say that a single vote is the consequence of
| having a unique perspective, shaped by being raised and living
| in a particular environment for many years. In other words, the
| right to vote exists because of the process that created a
| mind, and not because of the pure existence of that mind. As
| such, it cannot be cheaply copied, because that process is
| fundamentally different from the process of growing up. So, if
| you fork yourself, your copies are now different people who
| have just now come into being, and thus will acquire voting
| rights in 18 years (or however long).
| colinmhayes wrote:
| It's hard to imagine democracy continuing in an uploaded
| minds/super intelligence scenario. Either we get an ai decision
| maker scenario or everyone lives in their own matrix/joins
| someone else's and whoever created it makes the rules.
|
| It's conceivable that you join a matrix whose creator has set
| it up as a democracy I suppose.
| WorldMaker wrote:
| The last time voting involved fractions wasn't so great for
| civil liberties. As soon as you are assigning someone 3/5ths
| the vote of someone else, maybe you are doing something wrong?
| jobigoud wrote:
| Reminds me of the science fiction podcast Biotopia (Spanish).
| At some point there is a debate in the community on whether to
| allow a woman to marry a robot. There is a vote and it's very
| tight. But then one of the scientist of the community
| inadvertently duplicate himself 30 times or so and all the
| clones come vote at the last minute and tip the balance to
| allow the marriage to proceed.
| marsven_422 wrote:
___________________________________________________________________
(page generated 2022-07-18 23:02 UTC)