http://www.antipope.org/charlie/blog-static/2022/07/crimes-against-transhumanity.html Charlie's Diary Being the blog of Charles Stross, author, and occasional guests ... [ Home ] [ FAQ ] [ Contact me ] [ Older stuff ] Back to: Books I will not write: BIGGLES!! Crimes against Transhumanity By Charlie Stross (Disclaimer: I am a transhumanist skeptic these days, not to mention a singularity curmudgeon and a critic of Mars colonization, but I still find these ideas nice to chew on sometimes.) Humans are social animals, and it seems reasonable to assume that any transhuman condition we can wrap our minds around will also be a social one for most of its participants. Society implies a social contract, that is: we grant one another rights and in return make the concession of respecting each others' rights, in order that our own rights be observed and respected. And violations of rights tend to be at the root of our concept of crime and injustice--at least, any modern concept of crime once we discard religious justifications and start trying to figure things out from first principles. Which leads me to ask: in a transhumanist society--go read Accelerando , or Glasshouse, or The Rapture of the Nerds--what currently recognized crimes need to be re-evaluated because their social impact has changed? And what strange new crimes might universally be recognized by a society with, for example, mind uploading, strong AI, or near-immortality? SF authors are paid to think our way around the outside of ideas, so it's always worth raiding the used fiction bin for side-effects and consequences. Here's qntm's take on the early years of mind uploading--the process of digitizing the connectome of a human brain in order to treat it as software: I strongly suggest you read Lena (if you haven't previously done so) before continuing. It's a short story, structured as a Wikipedia monograph, and absolutely horrifying by implication, for various reasons. Let me give you that link again: Lena. (Go read: it's short, good fiction, and the rest of this essay will still be here when you get back.) Mind uploading makes certain assumptions. (Notably: mind/body dualism is a bust, there is no supernatural element to consciousness, also that we can resolve the structures involved in neurological information processing with sufficient resolution to be useful, and that the connectivity and training of the weighted neural network in the wetware is what consciousness emerges from.) Uploading also implies that consciousness is replicable and fungible, which in turn implies our legal systems can't cope without extensive modification because we rely on an implicit definition of humanity which at that point will be obsolete, as the treatment of MMAcevedo (Mnemonic Map/Acevedo), aka "Miguel" in the story, demonstrates: MMAcevedo is considered by some to be the "first immortal", and by others to be a profound warning of the horrors of immortality. Historically, our identity has been linear: there is a start, there is a terminus, along the way we are indivisible, although we undergo change over time (and may lose or gain significant portions of our selves--for example, most people retain few or no memories of their life before a point some time between the ages of 3 and 5 years old). The premature termination of a human life is an irrevocable act, and to deliberately inflict it on someone is seen as a crime (various degrees of murder). Because our identity is indivisible and of limited duration, time is a rivalrous resource to us: we have to choose what to do with it, or be subject to someone else's choices. (One of the reasons why imprisonment is seen as a punishment--to which we are averse--is the total loss of opportunities to choose what to do with the time we lose. (Yes, there are other reasons: let's ignore them and focus on what this might signify for the posthuman condition.) There's a fascinating sequence early in Linda Nagata's space opera novel Vast that throws the implications of alienated labour for uploaded minds into stark relief: if you're confronted with a mind-numbingly tedious task that needs human-level cognitive supervision for a period of years or decades, why not divide your time up in chunks and discard the boring ones? You could set up a watchdog timer to reset your uploaded mind to a baseline state every 3 minutes, unless an exception occurs--an emergency that makes you hit the dead man's handle in your environment, at which point the subjective passage of time resumes. In Vast, a human mind is needed to supervise a slower-than-light starship on a voyage that takes centuries during which nothing much happens. The crew use this three minute reset cycle to avoid experiencing tedium: subjectively, they condense the entire voyage into 180 seconds. (If you've driven long distance you'll probably have wished for the ability to push a button and find yourself at your destination. Right?) Other authors found other angles on this question: the first book in Hannu Rajaniemi's Jean Le Flambeur trilogy (The Quantum Thief starts with the exact opposite--a thief sentenced to spend a subjective eternity in an escape-proof prison, as a punishment of sorts. Spoiler: he escapes. How he does it and why he was there is the start of yet more musing on what might constitute crimes in a realm populated entirely by uploaded minds. In particular Rajaniemi dives headlong into two really disturbing questions: firstly, the potential for eternal enslavement such a setting offers (never mind perpetual torment), and secondly, what it does to the post-Enlightenment social concept of human equality. We are all living in the afterglow of a sociological big bang that took place in 1649--the execution of Charles I, who was variously King of England and Wales, Scotland, and Ireland at the time of the Wars of the Three Kingdoms: his trial and execution by a court--appointed by a parliament of the people--shattered the then-prevalent understanding among European/Christian communities that Kings were appointed by God to rule on Earth. A corollary of the Divine Right of Kings is that some people really aren't equal--monarchs, and by extension, aristocrats, have more rights (by religious decree) than other people, and some categories (chattel slavery springs to mind: also the status of women and children) have less. But if the People could try the King for crimes against the state, then what next? "What next" turned out to be a troublesome precedent. Charles I's younger son James II tried to walk back the uneasy settlement with parliament and got yeeted into exile in 1688-90 as a result, with the resounding and lasting outcome that the powers of the Crown in English and Scottish law was now vested in Parliament, and the head beneath the fancy hat was merely a figurehead who could be sacked if he (or she) acted up. If the monarch wasn't divinely appointed, what set him apart? Numerous philosophical maunderings later it was the French king's turn, and also time for the US Bill of Rights--which, while based on the 1698 English Bill of Rights, implicitly adopted the pernicious logic that there could be no king, no nobility, only free citizens. (Pay no attention to the slaves--for now.) Here's the thing: our current prevailing political philosophy of human rights and constitutional democracy is invalidated if we have mind uploading/replication or super-human intelligence. (The latter need not be AI; it could be uploaded human minds able to monopolize sufficient computing substrate to get more thinking done per unit baseline time than actual humans can achieve.) Some people are, once again, clearly superior in capability to born-humans. And other persons can be ruthlessly exploited for their labour output without reward, and without even being allowed to know that they're being exploited. Again, see also the subtext of Ken MacLeod's The Corporation Wars trilogy: in which the war between the neoreactionaries and the post-Enlightenment democrats has been won ... by the wrong side. The second book in The Quantum Thief trilogy, The Fractal Prince, gives us a ghastly look at a world where genocide and enslavement are carried out by forcibly abducting and uploading the last born-human survivors--is it actually genocide if the body is dead but the mind is still there? (It's a new version of Caedite eos. Novit enim Dominus qui sunt eius, of course.) It may not be genocide in the currently accepted legal sense of the term--the forcible extermination of a cultural group or of the people who are members of such a group--but it's certainly a comparable abomination. Our intuitions about crimes against people (and humanity) are based on a set of assumptions about the parameters of personhood that are going to be completely destroyed if mind uploading turns out to be possible. And the only people I see doing much thinking about this (in public) are either SF authors or people pushing a crankish ideology based on 19th century Russian orthodox theology. Surely we can do better? Posted by Charlie Stross at 13:29 on July 18, 2022 | Comments (63) 63 Comments | Leave a comment Kevin Marks # Kevin Marks | July 18, 2022 14:06 | Reply 1: I've been rewatching Westworld (the TV series) the last couple of weeks as there's a new season out, and they do quite a lot with the topic of who is human, uploadability and the different worldview when you are revivable. Also Black Mirror has done a fair bit with these topics. These kind of big budget mainstream TV series are going to provide the cultural reference points for this in future. Greg Tingey # Greg Tingey | July 18, 2022 14:18 | Reply 2: firstly, the potential for eternal enslavement such a setting offers (never mind perpetual torment) - See also Ian Banks, um, err "Matter" (?) Re: The Fractal Prince - I still couldn't work out what was actually happening - I have all three books & am still "confused". As for "doing better" - we need ( I think ) to (re)-start by redefining what makes a human, in as broad a set of terms &/or descriptors as possible & then give them actual equal rights .... Charlie Stross # Charlie Stross replied to this comment from Greg Tingey | July 18, 2022 14:21 | Reply 3: Matter -- noted, and the uber-capitalist who makes bank by buying up obsolete hells and running them as a service for disreputable species who want to torment their dissidents for eternity was a striking image! But I'm leery of citing Banks as a reference point for plausible futures; his SF conceits are only very tenuously constrained. Justin Jordan # Justin Jordan | July 18, 2022 14:24 | Reply 4: The thing I'm pitching these next couple weeks takes the idea that for plot reasons, the world is now full of ghosts - essentially, the soul of person, mostly intangible EXCEPT that they can affect electronics. Capitalism immediately realizes how very useful a bunch of minds who can operate machines but have no legals rights is, and it goes just as badly as you might imagine. Germane to the topic, I've found lots and lots of new crimes and horrors to go with this. Elderly Cynic # Elderly Cynic | July 18, 2022 14:36 | Reply 5: Perhaps the most hopeful thing is that we still don't understand enough to know how to do this in theory. It strikes me that this is the same ethical problem as 'true AI', because I don't regard the origin of an intelligence as important, ethically. Yes, it's a horrific problem, and I agree that we would see slavery come back (not that it had ever really gone away). JReynolds # JReynolds | July 18, 2022 14:42 | Reply 6: Minor nitpick: Charles I's grandson tried to walk back the uneasy settlement with parliament and got yeeted into exile in 1688-90 as a result James II was Charles I's son, not grandson. Dave Lester # Dave Lester | July 18, 2022 15:11 | Reply 7: Mind uploading makes certain assumptions. (Notably: mind/body dualism is a bust, there is no supernatural element to consciousness, also that we can resolve the structures involved in neurological information processing with sufficient resolution to be useful, and that the connectivity and training of the weighted neural network in the wetware is what consciousness emerges from.) If you want a horrific thought about what it would mean if we did have mind/body duality, consider what would happen if someone mechanises a way to access "mind" computationally. Do we want IBM to own all our souls? Now, on with the rest of your caveats which are my specialist subject (helping lead the UK contribution to the Human Brain Project, before I retired due to Brexit). (*) According to what Henry Markram (look him up) told me, the biological neural network is configured at an early stage. What then happens is that babies start to lose connections at a fairly startling rate; I've heard of numbers such as 90% of all possible connections are lost in the first few years of life. (*) His thoughts on neural plasticity are that each synapse (connection between neurons) is one of five or six possible connections it might make to five or six other neurons. Plasticity then consists of the connection trying each of these six possibilities to find the "best" -- whatever that might mean. (*) Seth Grant (look him up) told me that this synapse is really a very complicated bio-machine-mediated connection. We know next to nothing about these bio-machines -- each about 1M Dalton in size -- and what they do. (*) Finally, and fatally undermining my area (neuromorphic computing: taking a simplified approach to it all), we now know that signals within the dendritic tree (a in-fanning structure connection to synapses and the neurons' soma or cell centre) are not additive. The effect of incoming spikes is greatly affected by other signals currently in the dendritic tree. So just knowing the connectome -- the 85 billion neurons in the brain along with the connectivity graph of 10,000 connections per neuron, is merely your starter for ten. You will also need to know something of the state in each individual synapse. That said, Katrin Amunts (look her up) has a basement in Julich where she slices up donor human brains in an attempt to construct a connectome. Mikko Parviainen (he/him) # Mikko Parviainen (he/him) | July 18, 2022 15:13 | Reply 8: This reminds me of a campaign idea I had for Eclipse Phase ttrpg. Maybe some day... Dave Lester # Dave Lester replied to this comment from Dave Lester | July 18, 2022 15:15 | Reply 9: Oh, yes, device variability. We have no way of knowing which sort of neuron is in which location. There are lots and lots of different sorts of neurons, but because even neurons of the same type vary wildly in their responses, we cannot usually determine which is which. Ithilanor # Ithilanor | July 18, 2022 15:22 | Reply 10: Quick note - Surface Detail is the Culture book that revolves around simulated hellish afterlives. I don't remember Matter nearly as well, it might touch on the topic, but IIRC it's not as central to the plot. Some of the other Culture books mention related issues - I think it's Hydrogen Sonata that touches on how the Culture doesn't like doing full human-level simulations for predicting future outcomes. I don't think that's as likely to be an immediate issue in reality, though, partly because of the computational resources, partly because it doesn't seem like you need full mind simulation for accurate modeling. Heteromeles # Heteromeles | July 18, 2022 15:38 | Reply 11: Here's a scenario, which I think some are trying to instantiate right now. There is no meritocracy, you have no inherent rights, even to existence. You have relative rights based on how your existence is valued by one or members of the super-rich, and civilization exists to grow and protect their freedoms, their power, and their continued existence. This "solves" the problem of dealing with an increasing number of beings with an increasingly large spectrum of skills by wiping out all their rights, and focusing only on their utility. Personally, I don't want to live in this type of world, and I'm increasingly unwilling to pay for works that promote such worlds too. It's more worth imagining plausible alternatives, no? Pigeon # Pigeon replied to this comment from Elderly Cynic | July 18, 2022 15:40 | Reply 12: "Perhaps the most hopeful thing is that we still don't understand enough to know how to do this in theory." ...although that does not eliminate the possibility of some bunch of aliens who do know how to do it and don't bother to ask first. For instance the Jarts in Greg Bear "Eternity", and the described experiences of Rhita Vaskayza (plus the uncounted undescribed other such instances). They are not evil, they just have an utterly unhuman value system, and the result is mindbendingly shit for any being who doesn't completely share that system. It's really a subject I basically avoid thinking about, because the thoughts invariably head so rapidly in the direction of horror beyond belief that I just shut down on it. Accordingly my response to the question of how the law would need to be changed is purely reflexive and to the effect that it should simply be banned absolutely without exception, and any computing devices, storage media etc. that could be used for that purpose are also banned and must be destroyed by at the minimum reducing them to plasma. I haven't seen any SF treatment of the idea that doesn't equate literally and precisely to hell, including the ones that are supposed to be some kind of paradise. "Matter" is initially shocking but is actually not as bad as what all the variants would end up being like eventually. @ Charlie: "...our legal systems can't cope without extensive modification because we rely on an implicit definition of humanity which at that point will be obsolete" I don't think it's just the legal systems, I think the state of humanity itself relies on that (in a sort of circular/bootstrapping way). Part of my extreme objection is that I do not want my personal state of humanity thus redefined. "Notably: mind/body dualism is a bust" Is experimentally demonstrated, surely? Pigeon # Pigeon replied to this comment from Ithilanor | July 18, 2022 15:44 | Reply 13: "Quick note - Surface Detail is the Culture book that revolves around simulated hellish afterlives." Ah, OK. "Surface Detail" is initially shocking then. I can never remember which of those two is which. Ithilanor # Ithilanor replied to this comment from Elderly Cynic | July 18, 2022 15:53 | Reply 14: That's roughly how I feel, especially with regards to uploading tech specifically. It's certainly an interesting issue to think about, but scanning an existing brain seems well out of reach from everything I've heard. Take that with a grain of salt, though, I don't know much about the various fields involved. Current ML-based tech doesn't seem likely to lead to anything like what we'd recognize as conscious minds, though it's worth thinking about what other models of thought it might lead to, and how we'd incorporate them into our framework of human/social rights. (See OGH's Rule 34, not to mention Peter Watts's books) Charlie Stross # Charlie Stross replied to this comment from Dave Lester | July 18, 2022 15:58 | Reply 15: Do we want IBM to own all our souls? I can think of worse people than IBM. IBM functionaries can at least probably be persuaded that they'll end up in the same place themselves eventually, so best be at least moderately non-evil. I'd be a lot more worried about the Southern Baptists, the Church of Latter-Day Saints, the Taliban, Vladimir Putin, and so on. On the buried complexity below the level of the connectome: yup. My crude metaphor for it is we're electronic engineers circa 1950-55, looking at a modern smartphone motherboard and scratching our heads. We can see the circuit board tracks with a microscope but those resin carriers with the impure silicon crystals inside are clearly magical black boxes ... Heteromeles # Heteromeles | July 18, 2022 16:01 | Reply 16: Another alternative starts with the concept of animism, that everything, without exception has some inherent rights. Obviously creation, destruction, and exploitation have to happen, but these are to be regarded as takings that need to be compensated for, not inherent rights given to the exploiters by their Creator(s). This again does away with the meritocracy, but it does so by scrapping intelligence and ability as critical qualities by assigning agency to everything. Note that this is an extremely old and widespread concept, known as animism. The more modern JCI systems and the sciences that spring from them are notable for being actively hostile towards it, but if you want more global inclusivity, it's probably worth exploring a system of law based on universal agency without regard to ability. One way to do it might be to riff on the Australian aboriginal concept of "Dreaming" (not what they call it, but a widely used term by ignorant white outsiders including myself, so...). The basic ideas are: --"Dreaming" is a kind of system. That system and its components have inherent rights of existence. --People belong to dreamings, and people includes transhumans. They are both members of Dreamings (Human Dreaming, AI Dreaming, etc.) and parts of other dreaming (the GHG dreaming, white oak dreaming, worm dreaming, etc.). In the latter role, they have the responsibility of caring for the other members of their Dreaming, not exploiting them. In tech terms Dreamers are Sysops, not users. Governments (which have their own Dreamings) exists to help the Dreamers keep all the Dreams in existence by providing a framework within which necessary exploitation can happen, while unnecessary exploitation can be remediated and punished. Dreaming systems are managed communally (meaning by communities of beings with differing capabilities), not by individual authorities. There's no Prince of Tides, just a lot of beach dreamers trying to keep sandy shoreline systems in existence in the face of sea level rise, using whatever governance structures trial and experience has shown works. I'm again being vague here: systems are diverse, and sewage, the internet backbone, particular crops, and rare species all require very different community structures and management systems. As an ecologist, I don't want to tell a computer programmer how SmallTalk Dreaming should work, but equally, I want them to realize that what worked for them may not work at all for me. Systems vaguely akin to this have been used by humans for millennia at low-density but wide scale. While I think there are huge problems with keeping exploitative authoritarians from doing "move fast and break things" attacks (cf Eurasian deep history), such attacks are only useful when the attacker has a substantial advantage, AND there's a substantial surplus to be gained from the attack. Going into the 21st century and beyond, I don't think surpluses of anything except garbage will be all that available. Charlie Stross # Charlie Stross replied to this comment from Pigeon | July 18, 2022 16:05 | Reply 17: I am pretty sure that to a first approximation, a universal declaration of posthuman rights should enumerate an absolute ban on the creation of hells (afterlife sims designed for punishment) for any purpose. (NB: this makes Roko's Basilisk a criminal.) Also probably something equivalent to the right-to-life for us squishy meatsacks (only more inclusive for folks who can fork()/exec () themselves), the right to suicide irrevocably (woth complete and permanent erasure of backups and offsite copies of one's mind), the right not to have your personality, beliefs, or memories modified without prior informed consent, the right not to be enslaved ... And some definition of "person" that focuses on individuals with a distinct identity and some features (ability to communicate and process information, consciousness) that are more expansive than "made out of meat derived from H. Sapiens Sapiens cell lineage". Heteromeles # Heteromeles replied to this comment from Charlie Stross | July 18, 2022 16:16 | Reply 18: Funny thing is that humans have always uploaded parts of themselves. I'd be crippled without my online and library selves, the medical records my doctors maintain, the legal records my government maintains, and so forth. Again, we're stuck with a system that instantiates JCI concepts, even though their reality is questionable. Are brains stuck in heads? Nope, neurons run through our entire body. There's no physical break between the neurons in our heads and the ones that extend through our bodies as nerves. The problem with an upload likely isn't someone uploading your connectome, but rather companies extending deep fake technology to the point where (at least online) the tech can convince anyone, including you, that it is you online, and it's far too complex under the hood for anyone to demonstrate otherwise. This metaphor is akin to the comments that submarines aren't artificial fish, and there's little reason to build them to swim. Similarly, why should we simulate people's brains online when a much smaller AI can literally impersonate them? And if even you can't tell the difference between it and you, why shouldn't it share all your rights? Graham # Graham | July 18, 2022 16:20 | Reply 19: One issue with uploading is the question of why we'd want to do it. A moderately dystopian version is something like The Tunnel under the World where the victims/subjects are used to test the effectiveness of advertising. It doesn't take too much imagination though to imagine a Microsoft Mengele experimenting on virtual concentration camp victims without even having to burn the bodies. Although perhaps BF Skinner would be a more apt comparison, considering these would be purely psychological experiments. Those issues apart though, if we've got enough processing to emulate a human brain in anything like real-time, then it seems clear we've got enough to do orders of magnitude better processing without consciousness. After all, all human analysis processes are notoriously crap, even under perfect test conditions. We're pretty good generalists, but give us anything more detailed and we really, really suck. If you add the requirement to concentrate on a task for long periods, it's beyond proven that the best way to get the task done is to remove the monkey from the loop. And unless we get to the point we can radically overclock our emulations, our reaction times are pretty awful. In comparison, if you want something done well, you can design or train something which isn't self-aware and doesn't have the built-in flaws of human processing. Accidents with self-driving cars are pretty well documented, but no self-driving car is going to do a Charlottesville or get drunk or high. The bar for replacing humans with something else isn't perfection, it's just being better than a human - and if you look at the data it's bloody hard to find something that we're good at. From driving to flying to face recognition to spotting cancer cells, we're running out of reasons to have humans do it. Scanning and replicating specific humans might be a win, if they can keep thinking new thoughts. If you could respawn multiple Einsteins, then yay for science. But your average human, doing general data-processing work as in Lena, it seems hard to justify. Which leads to a conclusion that the main reason to upload humans is to prolong personal existence. That being the case, we're going to want to keep interacting with the world in real-time - and in that case most of the existing laws about personal freedom (and how the law can take it off you) still apply. Of course if you know you're immortal then you can theoretically afford to wait 100 years for that jail term for murder to finish. But long-term imprisonment isn't something you just sit out without personal consequences, especially if that whole time is spent in isolation (you don't need food or exercise), so it still serves as a punishment. Short version, in practise I'm not sure all this would be quite as different as it potentially could be. Justin Jordan # Justin Jordan replied to this comment from Graham | July 18, 2022 16:40 | Reply 20: Even IF you go with the 'we can do this, we should also be able to do this' logic, which I wouldn't call a given, what needs to be true is to do something as well as a human....and cheaper*. You COULD develop a bunch of expert systems and automate them...or you could just torture a simulated mind to do it. It's essentially a general AI that you can apply to all kinds of things and the idea that it's cheaper, assuming you have the tech and the lack of laws to stop you, makes it plausible. I mean, we currently produce more than enough food to feed everyone and...don't. The assumption that because something is possible it's how people will do it is...dubious. *In the short term, anyway. sabik # sabik replied to this comment from Justin Jordan | July 18, 2022 16:59 | Reply 21: There's a passing comment in The Peripheral about using a human as some sort of boutique AI... the plot of the whole book kicks off with someone engaging a human to do a job that could've been automated sabik # sabik | July 18, 2022 17:05 | Reply 22: It may not be genocide in the currently accepted legal sense of the term--the forcible extermination of a cultural group or of the people who are members of such a group--but it's certainly a comparable abomination. The current definition of genocide already prohibits transferring children; the destruction of the group as a cultural and social unit, rather than its extermination. LAvery # LAvery | July 18, 2022 17:05 | Reply 23: Here is what makes me extremely skeptical about the way upload is usually understood in SF. For 31 years of my life I worked on the neuroscience of behavior in the nematode Caenorhabditis elegans. The normal Ce hermaphrodite has 302 neurons (always exactly that number), and every normal Ce herm is anatomically identical at the cellular level to every other. The nervous system was fully reconstructed from serial section electron micrographs. Here is the principle publication. We can monitor the activity of the nervous system in a living worm in real time using fluorescent probes. We can manipulate nervous system function genetically. It is either the best or second best (fruit fly researchers sometimes make the claim) animal. And STILL, in 2022 no one on Earth can tell you how that nervous system produces normal worm behavior. We certainly can't reproduce it in software. Not even close. LAvery # LAvery replied to this comment from LAvery | July 18, 2022 17:18 | Reply 24: *It is either the best or second best (fruit fly researchers sometimes make the claim) UNDERDSTOOD animal. LAvery # LAvery replied to this comment from Greg Tingey | July 18, 2022 17:38 | Reply 25: Re: The Fractal Prince - I still couldn't work out what was actually happening - I have all three books & am still "confused". I read all three because OGH stated on this blog that they presented a realistic SF depiction of uploading. I was disappointed. If I remember correctly an upload occurs exactly once in the entire series, and it is covered in two paragraphs with the same superficiality one finds in other SF accounts of uploading. I suspect what Charlie was thinking of was Rajaniemi's rather detailed description of the architecture of the hardware system on which brains were simulated. But it was a win, anyway -- I enjoyed them. Yes, I had trouble working out what was actually happening, but I have gotten so used to that in modern SF that I am disappointed now if I read a novel and it all makes sense on the first read. Duffy # Duffy | July 18, 2022 17:46 | Reply 26: You all keep talking about hells for uploaded minds. But what about the AI Devil that runs the place? Something like AM from "I have no mouth and I must scream"? SFReader # SFReader replied to this comment from Dave Lester | July 18, 2022 17:47 | Reply 27: Re: 'So just knowing the connectome -- the 85 billion neurons in the brain along with the connectivity graph of 10,000 connections per neuron, is merely your starter for ten. You will also need to know something of the state in each individual synapse.' Apart from identifying, pigeon-holing (mapping) individual neurons, I'm wondering how many potential combinations and permutations would then have to be mapped to result in a verifiably 'identical' mind? Neuronal plasticity can change connections plus it seems that new nervous system structures and functions are still being discovered. Basically: The more parts, the likelier something can and will go wrong. (Asimov's robot stories explored some weird results as a consequence of a much, much smaller set of factors, Three Laws.) The gut-brain axis plus microbiome - both concepts have become accepted as the way humans operate and most likely are necessary for human survival. So far the microbiome discussion has been limited to the gut but what if it turns out that there's an equivalent non-human microbe necessary for optimal human brain function. (Yeah - I know, there's a blood brain barrier between the brain and the rest of the human body but at the same time it's also been established that some viruses/parasites hijack insect/frog brains so IMO the potential therefore also exists for a virus/parasite to get humans to act in ways that are healthy/beneficial for humans.) My long-winded point: what if the uploaded brain doesn't properly function because the 'cerebrome' wasn't yet known/uploaded. Plus - a cerebrome adds more pieces and processes, complexicating things even further. My understanding is that emotions are the result of interactions either with people or the environment. This means input/feedback with the sensory systems is a must-have. If your intent for uploading your brain is to be eternally happy, how do you program/upload for emotionally-complete/evoking sensory data inputs and processes? Also - what is the risk that the program that edits and consolidates your current brain processes doesn't lop off something that upon 'awakening' turns you into a psychopath, e.g., Phineas Gage? Will potential uploaders be psych screened for psycho/sociopathy? [Thanks but I'll take a pass on an immortal DT.] Not sure how much sense the above makes - boils down to: do we know enough about anything? Maybe learnings from another branch of biotech might help us understand some of the likely ethical implications (CrisprCas9). tomoyo # tomoyo | July 18, 2022 17:48 | Reply 28: Since we're reaching into the annals of fiction, I will note that Clive Staples Lewis discussed this in 1945 in That Hideous Strength. (His mad preacher, Straik, might very well be based off Federov.) In this excerpt, our antagonists are trying to recruit the deuteragonist Mark into their scheme to take over the world forever, and he has learned of their key advance: keeping a human consciousness alive after what would otherwise be its natural death. The potential for horrific abuses was not just obvious, it was a key feature: "It is the beginning of Man Immortal and Man Ubiquitous," said Straik. "Man on the throne of the universe. It is what all the prophecies really meant." "At first, of course," said Filostrato, "the power will be confined to a number--a small number--of individual men. Those who are selected for eternal life." "And you mean," said Mark, "it will then be extended to all men?" "No," said Filostrato. "I mean it will then be reduced to one man. You are not a fool, are you, my young friend? All that talk about the power of Man over Nature--Man in the abstract--is only for the canaglia. You know as well as I do that Man's power over Nature means the power of some men over other men with Nature as the instrument. There is no such thing as Man--it is a word. There are only men. No! It is not Man who will be omnipotent, it is some one man, some immortal man. Alcasan, our Head, is the first sketch of it. The completed product may be someone else. It may be you. It may be me." "A king cometh," said Straik, "who shall rule the universe with righteousness and the heavens with judgement. You thought all that was mythology, no doubt. You thought because fables had clustered about the phrase 'Son of Man' that Man would never really have a son who will wield all power. But he will." "I don't understand, I don't understand," said Mark. "But it is very easy," said Filostrato. "We have found how to make a dead man live. He was a wise man even in his natural life. He live now forever: he get wiser. Later, we make them live better--for at present, one must concede, this second life is probably not very agreeable to him who has it. You see? Later we make it pleasant for some--perhaps not so pleasant for others. For we can make the dead live whether they wish it or not. He who shall be finally king of the universe can give this life to whom he pleases. They cannot refuse the little present." "And so," said Straik, "the lessons you learned at your mother's knee return. God will have power to give eternal reward and eternal punishment." Aardvark Cheeselog # Aardvark Cheeselog | July 18, 2022 17:57 | Reply 29: What LAvery said. LAvery # LAvery replied to this comment from SFReader | July 18, 2022 17:59 | Reply 30: The gut-brain axis plus microbiome - both concepts have become accepted as the way humans operate and most likely are necessary for human survival. One thing I liked about Stephenson's /Fall, or Dodge in Hell/ (which remains, at this date, the closest thing I seen in SF to an appreciation of the real difficulty of the upload problem) was that the uploaders eventually realized that it is not enough just to scan heads. To get a realistic reconstruction, you need the whole body. This would particularly be true for emotions. The main thing that distinguishes emotions from other mental states is that they are tightly linked to survival-related changes in body physiology. Thus, the sensation of ones heart speeding up is an essential part of the experience of fear. This is why beta-blockers, which have essentially no direct effect on the brain, reduce the emotional stress of fear-inducing situations -- they block the peripheral effects of fear. Duffy # Duffy | July 18, 2022 17:59 | Reply 31: Side note: I once had a born again Baptist tell me in all earnestness that Gandhi and other good people were burning in hell because they did no accept Jesus as their personal savior. This means all Hindus, Jews, Muslims, agnostics, etc., - as well as a good many Christians who are not fundies - burn in hell. OK, so Gandhi dies and wakes up in the agony of the infernal pit. After a few thousand years he starts to meditate and finds that he can think through the pain. After a few eons he can actually function as a creature with agency and ability to act. He starts teaching/preaching to the damned around him (the vast majority of which are good people like himself - there being comparatively few Nazis and mass murderers over the course of human history). They follow his path to a point where they can function in Hell and begin to create a "Society of the Unfairly Damned". Upon meeting Satan (who like his demons are also in agony in hell) Gandhi convinces him that the best way to revolt against God is to practice non-violent soul force and not cooperate with injustice. Soon everyone but the worst of hell (Hitler makes an obligatory appearance) has followed this new path, creating a new universe of their own making. God can do nothing to stop this since hell, by definition, is a place where God is absent. In a few million years, hell is transformed into a advanced universe of mathematics, poetry, songs, science and philosophy. Then its denizens, who have had millions of years to solve this problem, find a away out of hell and enter the universe of God.... Greg Tingey # Greg Tingey | July 18, 2022 18:09 | Reply 32: tomoyo & Duffy And ... C S Lewis was one of ( If not the ) first to warn us of this ... Of all tyrannies, a tyranny exercised for the good of its victims may be the most oppressive. It may be better to live under robber barons than under omnipotent moral busybodies. The robber baron's cruelty may sometimes sleep, his cupidity may at some point be satiated; but those who torment us for our own good will torment us without end, for they do so with the approval of their own conscience. They may be more likely to go to Heaven yet at the same time likelier to make a Hell of earth. Their very kindness stings with intolerable insult. To be 'cured' against one's will and cured of states which we may not regard as disease is to be put on a level of those who have not yet reached the age of reason or those who never will; to be classed with infants, imbeciles, and domestic animals. And Lewis was an almost-foaming-level christian, but even he could see the problem. Dave Lester # Dave Lester replied to this comment from LAvery | July 18, 2022 18:16 | Reply 33: Quite right LAvery. I have in mind that there was spoof article in the style of a neuroscience experiment, but looking at a transistor radio (or computer?). Can you help with a link? Ta. whitroth # whitroth | July 18, 2022 18:18 | Reply 34: Thank you so much, Charlie. A lot of this does matter in the future universe I'm writing, and now I have to reconsider a number of things in two whole bloody novels. (They're still in "considering revisions" stage, with one just having a first beta read). I also have a short dealing with downloading that I'm trying to sell, and yes, it's very disturbing. One thing, though: all of what I see in this thread assumes that the uploaded mind is always running, except for being shut down and restarted. What's not considered is, well, offline backups of your mind. whitroth # whitroth replied to this comment from tomoyo | July 18, 2022 18:22 | Reply 35: ARGH! That is a novel, and series, I HATE, and refer to it as That Hideous Trilogy. And I assure you that had it not been C.S. LEWIS, that the second and third would never have gotten out of the slush pile, except from a religious publisher. The one you mention, right, everyone who wants anything new (new, def: anything from after Lewis was born and grew up) is a direct agent of the Devil (tm), and in the end, he literally pulls the animals out of the city zoo to kill the leaders. This is so overwhelmingly Christian-biased I consider nothing in it worth it. JReynolds # JReynolds replied to this comment from Dave Lester | July 18, 2022 18:30 | Reply 36: Dave Lester: Katrin Amunts (look her up) has a basement in Julich where she slices up ... human brains in an attempt to construct a connectome. Why have I got the notion that a sinister version of her is right at home in the Laundryverse? (Ellipses indicate removal of "donor" from the original sentence.) Dave Lester # Dave Lester replied to this comment from SFReader | July 18, 2022 18:33 | Reply 37: SFReader, You ask some very interesting questions. On the matter of "things going wrong", there must be lots of mechanisms that try to keep the brain machinery on track. We know of relatively few. One of the crudest must be energy supply. If all our neurons fired at the same time (instead of ten times per second), we'd be having an epileptic attack, and will be exhausted afterwards. As I say, there must be hundreds of others. As you also say, a disconnected brain is not going to be doing anything very much -- that's why we had a part of the project that was devoted to providing a robotic front end. The leader of that effort (Alois Knoll) told me of his need to do "soft robotics", where instead of a rigid steel skeleton and a classical PID controller, we had a fibreglass/carbon fibre skeleton so that a wild movement wouldn't decapitate a human standing nearby! Plus, it could compensate for wear and increased friction, by adapting as it learns about its environment. A thought from the late Karlheinz Meier (one of the designers of the ATLAS-1 detector at CERN), who had a neuromorphic system that runs 10,000 times faster than real time, was to connect it up to financial data, and see if it did anything interesting. The usual mechanism for linking brains to the rest of the body for moods is supposed to be hormones -- I'm not sure whether it is that simple. Dave Lester # Dave Lester replied to this comment from JReynolds | July 18, 2022 18:38 | Reply 38: Katrin Amunts (look her up) has a basement in Julich where she slices up ... human brains in an attempt to construct a connectome. Why have I got the notion that a sinister version of her is right at home in the Laundryverse? (Ellipses indicate removal of "donor" from the original sentence.) There was a competition for literature inspired by the HBP. I had considered making an entry, along the lines of LaundryVerse FanFic, but I was a bit hectically busy at the time. "In a basement in Julich -- where the German government stores it's nuclear waste, and houses it's most powerful computers -- a lone female scientist works on late into the night. ..." Neil W # Neil W replied to this comment from Charlie Stross | July 18, 2022 18:45 | Reply 39: NB: this makes Roko's Basilisk a criminal Jail for Roko's Basilisk! Jail for one thousand years! Roko's Basilisk is wasting time (processing power) by torturing those who have knowingly impeded the once-and-future AI king, when they might simply create simulated versions of people who have already been tortured for an appropriate period. LAvery # LAvery | July 18, 2022 18:47 | Reply 40: I have in mind that there was spoof article in the style of a neuroscience experiment, but looking at a transistor radio (or computer?). Can you help with a link? Sorry, doesn't ring a bell. Or are you asking for a link to something else I said? I have been told that the problem can be solved for integrated circuits, i.e., for things that we engineer ourselves. I met an interesting guy once at Janelia Farm, which, as I'm sure you know, is a Howard Hughes Medical Research Institute campus devoted to the problem of understanding how nervous systems work. This man I met was not a (conventional) neuroscientist, but an electrical engineer. He told me that Intel was in the habit of making scans of their CPU chips available to academic researchers, without circuit diagrams or functional information. (This is the very rough equivalent of the C elegans EM reconstructions.) A group of academics decided to see if they could reconstruct the functional circuitry, and succeeded! (He claimed. I cannot evaluate his claim, but I was impressed with the guy. Definitely seemed to know what he was talking about.) This is, for a host of very obvious reasons, a much easier problem than reconstructing a working brain model. But even this much easier problem was not easy. LAvery # LAvery replied to this comment from Dave Lester | July 18, 2022 18:58 | Reply 41: The usual mechanism for linking brains to the rest of the body for moods is supposed to be hormones -- I'm not sure whether it is that simple. It certainly is not. At a minimum you need the autonomic nervous system. And remember, the autonomic nervous system is a NERVOUS SYSTEM, which means it has billions of specific connections. It is not the simple global fight-or-flight caricature that popular caricatures often depict. Troutwaxer # Troutwaxer replied to this comment from Charlie Stross | July 18, 2022 19:01 | Reply 42: I'm concerned about the simple question of what religions can do with nothing more than a smart phone. It's so horrific that I'm not even going to post what I'm thinking of. SFReader # SFReader | July 18, 2022 19:34 | Reply 43: Re: '... our identity has been linear: there is a start, there is a terminus, along the way we are indivisible,' Time as part of self-identity. Just remembered 'patient HM' who because of brain surgery: a) remembered himself/his life up to the moment that he had surgery b) nevertheless was able to 'learn' new materials such as how to solve a problem even though he was unable to recall when/how he learned to do this This is a must-read for neuroscience: 'The Legacy of Patient H.M. for Neuroscience' https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2649674/ Met someone who was doing a neurosci PhD at McGill that mentioned that Brenda Milner who interviewed HM for decades (basically helped establish that there is more than one memory system) was still doing lab work and seminars at the age of 96. And it looks like she's still going - amazing! (She's 104.) https://en.wikipedia.org/wiki/Brenda_Milner Michael Cain # Michael Cain | July 18, 2022 19:36 | Reply 44: As I was reading the description of the uploaded minds panicking, or suffering some other reaction, I wondered if there would be software/ data analogs of various mood-altering drugs? Something in the emulation environment that acts like one of the selective serotonin reuptake inhibitors? Or the "be happy, don't worry" synthetic opioids? AlanD2 # AlanD2 replied to this comment from Greg Tingey | July 18, 2022 19:48 | Reply 45: Of all tyrannies, a tyranny exercised for the good of its victims may be the most oppressive. Sort of like a parent-child relationship? Charlie Stross # Charlie Stross replied to this comment from AlanD2 | July 18, 2022 19:53 | Reply 46: Don't troll. Children are a special case insofar as they're demonstrably not competent, at least at first. (They mature at differential speeds, and I'm very far from convinced that a one-size-fits-all "you're 18 tomorrow, at which point you're an adult, but today you're my chattel and you will obey" approach is valid, but it's undeniable that a one year old needs continuous care and guidance to ensure they don't harm themselves.) Look, humans are squishy meatsacks with irregular boundaries. You can't generalize too much, beyond "prolonged exposure to vacuum may be detrimental to future metabolic viability of shaved apes". Otherwise, here's a blog essay I baked in 2016 about the limits of the simulation hypothesis and resurrection sims in general. kyb # kyb | July 18, 2022 19:59 | Reply 47: The horrors that will be available to those controlling the compute resource will be appalling, however it is likely to be something that gets sorted out within a generation or so - most probably, it'll be the elite that get to upload first and they're not going to do that unless they're pretty sure that the legal protections are reasonably solid. We probably need to enshrine in law (with some serious penalties) that no human simulation may be altered or run as a means to an end, but only as an end in itself. While I loved Charlie's version of the singularity where AIs were limited by needing to simulate human level minds, it seems pretty unlikely that we would achieve upload without being able to do fairly human equivalent things without simulating an actual human. Perhaps this would lead to two levels of AI - those that can claim that they ultimately the result of an upload, which will be fully protected by law, and those that cannot, the evolution of tools and 'watch-the-space-ship' functions. That second class could easily be as or more advanced than the first class, but it's hard to see how they would achieve rights without support from humans and the first class. whitroth # whitroth replied to this comment from kyb | July 18, 2022 20:05 | Reply 48: Um... and twenty or thirty years from now, exactly who will be controlling those compute resources? My Core-I7 desktop with 16G RAm and terabytes of storage, compared to my Really Cool 286 with half a meg of RAM and wow, my boss gave me for a holiday present a 30M hd, much bigger than my 20M hd.... (true story, holidays 1989). kyb # kyb replied to this comment from whitroth | July 18, 2022 20:38 | Reply 49: My Core-I7 desktop with 16G RAm and terabytes of storage And if you do much more with all that power than access services run on cloud compute then you're not the normal person. I'm finding that even dumb simple utilities that used to live on my harddrive are now SaaS websites with 9.99 per month basic plan. It would be great if the trend were to move things to human-person owned devices, but that's not the trend I'm seeing. Greg Tingey # Greg Tingey | July 18, 2022 21:08 | Reply 50: Comment deleted by moderator Charlie Stross # Charlie Stross replied to this comment from kyb | July 18, 2022 21:08 | Reply 51: it seems pretty unlikely that we would achieve upload without being able to do fairly human equivalent things without simulating an actual human. Yep, there's a lengthy passage--a no-shit "tell me, professor" lecture delivered by an actual Professor of AI in Rule 34, infodumping on the hapless detectives, about why nobody outside a few cognitive science academics wants an actual human-like artificial intelligence: it'd have its own goals and probably want to do the AI equivalent of sitting on the sofa watching reality TV shows and eating popcorn all day. (Hence ATHENA, which is very much not a human-like AI.) Charlie Stross # Charlie Stross replied to this comment from Greg Tingey | July 18, 2022 21:10 | Reply 52: Greg: I deleted your comment because you can use the previous discussion thread for random current events without derailing this one. whitroth # whitroth replied to this comment from Charlie Stross | July 18, 2022 21:27 | Reply 53: I agree. What purpose, other than study, would having an actual self-aware AI serve? You want a very good, general purpose system that does what you want. That's why I have problems with robot rebellions - who's going to revolt, the welding robots on the assembly line at a car plant? In my universe, I've put in Intelligent Agents. They are explicitly not self-aware, and so have a balanced, nuanced view of problems. AGMS # AGMS replied to this comment from Graham | July 18, 2022 21:31 | Reply 54: Another possible "Why" for uploading Humans is of course easier programming. Bunches of Humans and support data systems would be packaged up as "functionality libraries". For example, here's one that does income tax returns. Several accountants, a couple of lawyers, a copy of the tax code and a document workflow system are included. It's essentially a small tax processing company in a box. Lazy programmers like it because it can interface with their code automatically (it has the smarts to talk to other libraries to find out what documents they need). Who cares that it simulates the lives and equipment of a small company in these times of cheap cloud computing? It's just so easy to plug and play! So you thought code bloat was bad. Wait until some slob programmer includes a bureaucracy in a box to format international date strings. Sometimes the box sends a delegate to international date standards committee meetings. You now have excessive bloat AND long update times! There should be a contest for worst use of computonium. Using a small simulated country to find the length of a string would be one. Another contender would be that project to recreate historical people by repeatedly approximating their existence until their output matches historical values... Elderly Cynic # Elderly Cynic replied to this comment from LAvery | July 18, 2022 21:59 | Reply 55: This fits with my speculation that there are realistic models of computation that cannot be mapped onto a Turing machine, and that, if we want to understand human or even C. elegant mentation, we will probably have to use some of them. I have tried to get further but am not smart enough. whitroth # whitroth replied to this comment from AGMS | July 18, 2022 22:30 | Reply 56: That's absurd. Why would you want to waste all your CPU cycles simulating a human, or a bunch of them, when what you want is an expert system that interviews a bunch of accountants, and then can scan through the public documentation on taxes, and then produce your taxes? And it runs an update every year? Really and truly, folks - can you see any company you've worked for okaying a huge project to simulate an accountant, when you already have, Quickbooks or TurboTax? And maybe an expert system front end to them, if you're busy trying to avoid paying taxes? Charlie Stross # Charlie Stross replied to this comment from whitroth | July 18, 2022 22:44 | Reply 57: Depends on how creative the simulated humans are, compared to a pure AI system? (Expert systems are ... they're far less sophisticated than the 1980s hype suggests. What you're probably thinking of is a GAN.) Thing is, human accountants and tax lawyers can spontaneously game out what-if scenarios for creative tax avoidance (ahem: not the same thing as evasion, honest). Which a non-conscious/non-volitional system may be incapable of doing, insofar as such planning involves a bunch of self-directed adversarial thinking. Troutwaxer # Troutwaxer replied to this comment from Charlie Stross | July 18, 2022 23:02 | Reply 58: I'm not sure. What if the "theory of mind" of the accountant in a box is based on a hundred years of IRS court findings instead of figuring out how to survive a trip to the watering hole. Now you've got something that can understand the adversary and run simulations without human help. What was the line from TRON? "I don't know what to do. I'm an accounting program!" whitroth # whitroth replied to this comment from Charlie Stross | July 18, 2022 23:03 | Reply 59: I dunno - I think a correctly designed system could run simulations based on directives from the user, who can suggest possibilities... and also ask for a list of possibilities (as opposed, say, to text games from the 80's, where it frequently turned into a word-guessing game ("pick it up? put it on? get it?....) Greg Tingey # Greg Tingey | July 18, 2022 23:11 | Reply 60: Charlie - OK I understand. Will remember to post to "other" thread, tomorrow, after next round (!) Heteromeles # Heteromeles replied to this comment from Charlie Stross | July 18, 2022 23:19 | Reply 61: While I agree that Troutwaxer's accountant in a box could be trained to minimize taxes and chances of being audited, while being fully admissible in court...it looks like the simpler system is simply to put that money towards re-electing politicians who consistently vote to underfund the IRS. What's more interesting is that nonhuman entities are being granted the legal rights of humans. These aren't AIs, they're rivers and lakes:* in Bangladesh, Ecuador, Bolivia, New Zealand...: https://www.npr.org/ 2019/08/03/740604142/ should-rivers-have-same-legal-rights-as-humans-a-growing-number-of-voices-say-ye ...Colombia (the Amazon), US (Klamath, Lake Erie), Canada (Magpie River)...: https://www.nationalgeographic.com/travel/article/ these-rivers-are-now-considered-people-what-does-that-mean-for-travelers Jokes about paging Ben Aaronovitch aside, these are interesting cases about the legal potential, and hurdles, of enforcing the legal rights of extremely nonhuman entities, even when they don't apparently have the human equivalent of free will. *and of course, US corporations... DeMarquis # DeMarquis | July 18, 2022 23:32 | Reply 62: @LAvery at 23/ Elderly Cynic @ 25: This, exactly. Copying a human mind would be so freaking complex that I actually think it can't be done in a cost-effective manner--in that, by the time you have the computational technology to duplicate a human, you solved GAI long before. I can't see copying a human as being easier or cheaper than just raising a new one. Then there's the ethical issues involved in creating pilot or beta versions of a person--are we going to delete sapient beings just because they are imperfect copies? There's your first set of laws--no experimentation on newly sapient minds. (That doesn't necessarily shut down GAI research because you can do that in an evolutionary mode--ie, let primitive minds evolve into more complex ones with minimal suffering). Then again, it turns out that humans are amazingly good at a wide variety of things. The US has around one traffic fatality per 100 million miles driven (https://www.nhtsa.gov/press-releases/ nhtsa-releases-2019-crash-fatality-data), Tesla on their best day can't approach that. As of July 2022, there were around 40 million commercial flights per year, but only a handful of fatal crashes (https://en.wikipedia.org/wiki/ List_of_accidents_and_incidents_involving_commercial_aircraft). There are around 500 accidental gun deaths per year (https:// injuryfacts.nsc.org/home-and-community/safety-topics/guns/), there are similar statistics for deaths by fire, subway accidents, injuries while camping, etc. The point is that humans, for all the opportunities we expose ourselves to danger and potential mishap, are actually pretty reliable. Eventually, I expect AI will get there, but it won't be easy or cheap. So the idea of "replacing the human mind" may be more challenging than pop science makes it appear. Trying to upload a real human would be orders of magnitude more complex than that. Why would anyone want to do this? If the idea is lifespan extension (potentially to infinity) then the consumer market takes over, and I think that's all the horror you need. How do you make a very expensive service available to the majority, but keep "the wrong sort" from acquiring it? Price, typically. This is an idea for trillionaires, not normal people. Their rights will be respected, you and I are gunna' die. As the wise man said to the fish: "Sorry Charley." I suspect the hell will be entirely material, while the likes of Elon, Jeff and Mo are going to be playing doubles tennis for eternity. SFReader at 27: "Apart from identifying, pigeon-holing (mapping) individual neurons, I'm wondering how many potential combinations and permutations would then have to be mapped to result in a verifiably 'identical' mind?" And not only that--but the brain is not a static entity. If those weighted pathways change with every neural burst, then there is no "the mind". If the mind is a process, not a thing, then uploading it at one particular point in it's lifespan could be meaningless (as in, not a thing that could "run" again). At least not separately from the environmental stimuli (including physiological actions) that drive it. So you can't upload a mind, you can only upload a mind/ environment system. That means duplicating the world the mind grew up in. Whole 'nother order of complexity. Rocketpjs # Rocketpjs | July 18, 2022 23:42 | Reply 63: I can imagine an 'arms race' between tests to determine whether a program is a sentient being (and thus has rights' or a dumb program. Penalties for abusing sentient programs would have to be brutal, an echo of the Blood Code. Since it would be so very difficult to catch anyone abusing a sentient program, the rare occasions where it did happen would have to be comparable to a capital crime. Capital crime in a posthuman/transhuman world would probably be total deletion of all instances of anyone who committed such atrocities, their enablers, corporations. Complete disinheritance of any heirs. Perhaps worse would be for your identity to be deemed 'fair use' and put into the open source world for tweaking and enslavement by anyone, legally. Both examples are monstrous. Richard Morgan's books were what turned me off the idea of functional immortality. We're seeing enough of a mess with the gerontocracies of today that have been enabled by improvements to modern medicine. The only respite for the world from the likes of Mitch Mconnell, Trump and even Brenda is the fact that they will inevitably leave us. A 200 or 2000 year old lich-analogue still clinging to power is a terrifying notion. Leave a comment Here's the moderation policy. If this is your first time, please read it before you post. If you need to sign in and want to create a local account on this blog, select "Movable Type" from the "Sign in ..." menu. You will need a working email address. Name [ ] Email Address [ ] URL [ ] [ ] Remember personal info? [ ] Comments (This form requires JavaScript. You may use Markdown, HTML entities, and a limited subset of HTML formatting tags in comments.) [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [Preview] [Submit] Buy my Books * US English editions * UK English editions * Deutsche Ausgaben * Ri Ben Yu noBen Quick Stuff * FAQ: Who am I? * FAQ: Moderation Policy * FAQ: Why is there no tipjar? * FAQ: Copyright Notice * FAQ: Inviting Charles Stross to speak * FAQ: Fan Fiction * Bibliography and online fiction * FAQ: Laundry Files reading order * Talk to me * Non-blog writing (old) * Mastodon Specials * Common Misconceptions About Publishing--a series of essays about the industry I work in. * How I Got Here In The End --my non-writing autobiography, or what I did before becoming a full-time writer. * Unwirer--an experiment in weblog mediated collaborative fiction. * Shaping the Future--a talk I gave on the social implications of Moore's Law. * Japan: first impressions -- or, what I did on my holidays * Inside the MIT Media Lab--what it's like to spend a day wandering around the Media Lab. * The High Frontier, Redux -- space colonization: feasible or futile? * "Nothing like this will be built again"--inside a nuclear reactor complex. * Old blog--2003-2006 (RIP) Merchandise About this Entry This page contains a single entry by Charlie Stross published on July 18, 2022 1:29 PM. Books I will not write: BIGGLES!! was the previous entry in this blog. Find recent content on the main index or look in the archives to find all content. Search this blog [ ] [Search] Propaganda [223224] Categories * Administrative (5) * Computers (10) * Gadget Patrol (7) * Humour (12) * News (6) * Politics (15) * Publicity stuff (11) * Publishing (19) * Travel (7) * Writing (37) Monthly Archives * July 2022 (1) * June 2022 (1) * May 2022 (3) * April 2022 (3) * March 2022 (2) * February 2022 (2) * January 2022 (4) * December 2021 (2) * November 2021 (2) * October 2021 (1) * September 2021 (4) * August 2021 (1) Earlier, and other types of archive.