[HN Gopher] The Coming Technological Singularity (1993)
___________________________________________________________________
The Coming Technological Singularity (1993)
Author : jiekkkeow
Score : 64 points
Date : 2023-04-18 17:32 UTC (5 hours ago)
(HTM) web link (edoras.sdsu.edu)
(TXT) w3m dump (edoras.sdsu.edu)
| cs702 wrote:
| Please add "(1993)" to the title. [DONE]
|
| The OP was written by Vernor Vinge, the mathematician/computer-
| scientist-turned-science-fiction-writer who first popularized the
| term "singularity."[a]
|
| Abstract: "Within thirty years, we will have the technological
| means to create superhuman intelligence. Shortly after, human era
| will be ended. Is such progress avoidable? If not to be avoided,
| can events be guided so that we may survive? These questions are
| investigated. Some possible answers (and some further dangers)
| are presented."
|
| As you read the rest, keep in mind that Vinge wrote it 30 years
| ago, well before large AI models like GPT-4 were even remotely
| possible.
|
| Along the same vein, I'd also recommend reading Hans Moravec's
| books, "Mind Children" (1988) and "Robot: Mere Machine to
| Transcendent Mind" (1998). Looking at those books with the
| benefit of hindsight, Moravec was _right_ about a lot of things.
|
| [a] https://en.wikipedia.org/wiki/Vernor_Vinge
| rnk wrote:
| His book "a fire upon the deep" is one of the most brilliant
| things I've ever read. There's the singularity, a story about
| an AI that reboots itself from cold storage, a brilliant idea
| of life forms with a collective brain basically. Interplanetary
| Usenet! I command everyone to read it.
| l3mure wrote:
| Some of the concepts are interesting but his prose is rather
| turgid, large portions of the book are clearly filler written
| to showcase some idea. It's decent enough though.
| cs702 wrote:
| Thanks for the suggestion. I've added it to my reading list!
| edfletcher_t137 wrote:
| This whole series is one of the best of that era, I don't
| understand why it is lauded more often. Vinge clearly has a
| mind for the future like none other. His "computer
| programmer-archeologist" idea has lived rent-free in my head
| since I first read _A Fire Upon the Deep_ decades ago. And
| the zones of thought? Literally mind-bending. "Came here to
| say this". :)
| Teever wrote:
| I'm glad someone else mentioned Moravec.
|
| When I first encountered ideas like transhumanism and the
| singularity as a teen I started exploring these topics at my
| local library. Even in my youth I was mildly skeptical but
| still cautiously optimistic that the ideas in these books would
| come to pass.
|
| I became very jaded with the state of technology when FAANG
| companies took over everything and computing devices became
| these locked down and addicting little glow boxes that push
| socially division and advertisements for crypto scams into
| everyone's mind and I became doubtful that these wondrous ideas
| would ever come to pass.
|
| For the first time in a long while I'm less jaded about those
| prospects. While I'm still skeptical I'm once again cautiously
| optimistic at the potential with technologies like LLMs and
| machines like Starship.
|
| Maybe I actually will get to live to see those crazy fractal
| trees that Moravec described.
| icepat wrote:
| This is an important addendum, since the poster is also in here
| using the same language as the article. I mistakenly concluded
| they had written this at first.
| novaRom wrote:
| Quote: I'll be surprised if this event occurs before 2005 or
| after 2030
|
| Actually, my initial thought was this text generated by a LLM
| pretending it's from 1993
| olddustytrail wrote:
| Someone should ask ChatGPT to summerize it. It might give
| away its plans!
| FartyMcFarter wrote:
| This short essay by Moravec is great:
|
| https://frc.ri.cmu.edu/~hpm/project.archive/general.articles...
| cs702 wrote:
| Thank you. I'll take a look.
|
| BTW, your username made me chuckle. Inspired by Boaty
| McBoatface, I assume?
| FartyMcFarter wrote:
| Yep :)
| ingbue88 wrote:
| " The precipitating event will likely be unexpected -- perhaps
| even to the researchers involved."
| jiekkkeow wrote:
| Within thirty years, we will have the technological means to
| create superhuman intelligence. Shortly after, the human era will
| be ended
|
| Is such progress avoidable? If not to be avoided, can events be
| guided so that we may survive? These questions are investigated.
| Some possible answers (and some further dangers) are presented.
| gsatic wrote:
| Why? Did the ants die out cuz the chimps got smarter?
| neals wrote:
| The Mammoth and the Dodo did
| yamtaddle wrote:
| No, for obvious reasons.
|
| But a bunch of other species did, including some fairly
| intelligent ones.
| doctoboggan wrote:
| In fact, all the species with intelligence approaching or
| equal to Homo sapiens were wiped out.
| SanderNL wrote:
| Yes, but it takes a while and depends on if the "ants" are in
| the "chimps"'s way.
| goatlover wrote:
| I don't think we could make ants go extinct even if that
| was our sole purpose as humans. They're extremely adaptable
| and good generalists, plus they rival us in both total mass
| and ubiquity, at least outside of tundras. They survived
| past extinctions, they'll survive us.
| ceejayoz wrote:
| Humans generated the sixth mass extinction event of the last
| half billion years or so.
| https://en.wikipedia.org/wiki/Holocene_extinction
| harimau777 wrote:
| The smart chimps do routinely use horrific chemical weapons
| to kill the ants en masse.
| highwaylights wrote:
| Putting this in quotes might have helped with the downvotes
| mecsred wrote:
| You should make it clearer that you are quoting the abstract.
| anigbrowl wrote:
| As a general rule, don't post comments on your own submission,
| especially not quotes from the article. Explanatory comments on
| why you submitted it are sometimes appropriate if the headline
| is counterintuitive or the submission seems at first to fall
| outside HN guidelines. Add '(1993)' to the title.
| xilinx_guy wrote:
| It's totally hilarious, how organizations are training large
| language models and calling them "intelligent". These overgrown &
| overstuffed stochastic parrots are just toys to amuse and shock
| the ignorant. Once the novelty wears off, and everyone
| understands the serious limitations of these models, it will be
| back to business as usual. "That's not really AI" will return as
| a catchphrase.
| asdfman123 wrote:
| My wife's a journalist and she already uses ChatGPT for a good
| deal of gruntwork.
|
| Requires a lot of human oversight but is quite helpful.
| Freire_Herval wrote:
| [dead]
| sublinear wrote:
| I agree. We merely have new chisels with which to sculpt.
|
| Meanwhile, the layperson buys the hype and insists that the
| tools can work themselves. Maybe it's due to fear and
| insecurity.
| shswkna wrote:
| > Any intelligent machine of the sort he describes would not be
| humankind's "tool" -- any more than humans are the tools of
| rabbits or robins or chimpanzees
|
| Cats would disagree :)
|
| Maybe there would be an analogous strategy in the case of humans
| vs AI
| 1attice wrote:
| If your rosy scenario is that we all become bored, entrapped
| housecats.... that's still depressingly better than the other
| scenarios.
| drewda wrote:
| I just went to check the date and it was 23 years ago that Bill
| Joy (of Sun Microsystems) caused a tizzy with an essay on the
| cover of Wired magazine with the headline of "The Future Doesn't
| Need Us: Our most powerful 21st-century technologies--robotics,
| genetic engineering, and nanotech--are threatening to make humans
| an endangered species."
|
| For what it's worth, genetic engineering and nanotech are,
| broadly speaking, the tools that enabled our species to develop
| and roll-out an effective COVID-19 vaccine in under a year.
|
| And, for what it's worth, writing articles about how "THE
| SINGULARITY IS COMING!" continues to be a way to get some
| attention... whether or not one has anything useful or novel to
| say...
| smackmybishop wrote:
| Did you check the date of the article you're complaining about?
| drewda wrote:
| Ha, no, I didn't... but I just did :)
| jart wrote:
| Some biotech companies claimed to have developed their vaccines
| within hours of having the genetic code being emailed to them.
| kmeisthax wrote:
| My biggest singularity fear is not "superhuman intelligence kills
| us all". The closest biological parallels to that happened over
| many millenia of competition and interbreeding. Humans are still
| capable of learning from or making use of superintelligences, and
| superintelligence isn't immediately dangerous unless you give it
| some means to hurt you. If, say, ChatGPT is superintelligent, the
| only way it can hurt you is through insult or persuasion.
|
| Mass structural unemployment provides a more realistic and scary
| scenario. It can happen in a decade, perhaps faster, and a large
| pile of newly unemployed people are fertile ground for political
| extremists. When you take enough people's jobs away, they will
| demand a strongman to bring them back. Welfare won't placate
| them: even with the various proposals to have AI companies make
| binding covenants upon themselves to cap their profits and donate
| the excess to welfare, that won't be enough, because welfare is
| still economically disenfranchising and _socially_ problematic.
|
| Furthermore, there is no particular reason why an AI company
| that's replaced significant fractions of the labor force would
| need to remain benevolent. Your hypothetical welfare covenant or
| robot tax needs to be enforced by a government, and all
| governments are aligned to economic interests - not humanitarian
| or democratic ones[0]. At the limit, the world in which everyone
| is on singularity welfare is vulnerable not just from a robot
| uprising, but from the people who make the robots deciding to go
| full _Atlas Shrugged_. You will have a world populated by about
| 300 cybernetically-enhanced CEOs standing atop humanity 's
| starved corpse.
|
| Of course, that's also somewhat unlikely at least with current
| technology. But let's keep in mind _who_ is building the AI and
| what it does. It is being built by large capital-rich firms with
| the explicit goal of automating away the middle class, after they
| 've finished automating away the poor. The end goal is to build a
| machine that automates away all labor: a form of capitalist
| wankery. Note how nobody is talking about building AI that
| handles business management or strategy, even though that's
| arguably more automatable than anything we actually chuck AI at.
|
| [0] This is a pithy summary of one of selectorate theory's bigger
| predictions, which is summarized well in the book _The Dictator
| 's Handbook_ and a CGPGrey video that I promise I've actually
| watched.
| [deleted]
| pg314 wrote:
| > If, say, ChatGPT is superintelligent, the only way it can
| hurt you is through insult or persuasion.
|
| People are already hooking up ChatGPT to the command line of
| their computers and giving it access to the internet. There are
| plenty of ways a super-intelligent AI would be able to wreak
| havoc.
| DasCorCor wrote:
| I would say that's a rather Vinge Friew.
| TheDudeMan wrote:
| This holds up remarkably well.
| bediger4000 wrote:
| A lot of Vinge's fiction has mechanisms to prevent his futures
| from being past the singularity. Across Realtime features those
| left behind, A Fire Upon the Deep and a Deepness in the Sky have
| the Slow Zone and the Transcend. His short stories have various
| one off, not well explained things, wars, etc.
|
| Vinge's whole fiction career has been about dealing with the
| Singularity.
| ingbue88 wrote:
| Opening lines; "Within thirty years..." 1993 - 2023 (30 years),
| spot on.
| icepat wrote:
| This is a very bizarre article.
|
| The opening starts off stating that neural networks may "wake
| up", and never goes on to explain what that means. The concerns
| in here have been around for decades, and really don't sound like
| anything new: runaway development, the technology now is more
| advanced than we've ever seen, ect.
|
| Then the post leaps from these points, does not bother proving
| them, accepts them as fact, and launches into a series of
| progressively more alarmist "what ifs". I don't see anything of
| substance here, and the authors very confident predictions have
| been made countless times before. It's important if you're going
| to write something that ends in prescriptive action points, that
| you try and convince of of _why_ you believe this to be true.
| Just the author's confidence is insufficient for the leaps this
| article takes.
|
| The whole thing reads like a Cybernetic Cultural Research Unit
| story from the 90's.
|
| With the (later realised) fact that this _is_ from the 90's, I
| think we should classify this along with contemporary works (like
| the above CCRU stuff).
| bigdict wrote:
| Well, it _was_ written in 1993...
| hotpotamus wrote:
| From a quick look, this is _the_ paper that coined the idea
| of the technological singularity? I was aware of the concept
| but not the history.
| FemmeAndroid wrote:
| I'm not 100% sure this paper is the first time he used it,
| but the author did coin the term.
| jahsome wrote:
| Did you happen to spot the copyright date?
| icepat wrote:
| I did not, I noticed a comment from the poster of the article
| using the same sort of language in the article and mistakenly
| concluded they were the author (new account, and this is
| their only submission). I don't think that takes away from my
| analysis, just makes it stronger since it adds context. The
| 90's was rife with this sort of technoculture apocalypse fear
| mongering, so I'm even more willing to shelve this with the
| CCRU's work.
| randallsquared wrote:
| An interesting question: How much more seriously are you
| prepared to take this now that you realize that it was written
| 30 years ago, about the future of roughly 30 years from then?
| dang wrote:
| Related:
|
| _The coming technological singularity: How to survive in the
| post-human era [pdf]_ -
| https://news.ycombinator.com/item?id=35184764 - March 2023 (2
| comments)
|
| _The Coming Technological Singularity: How to Survive in the
| PostHuman Era (1993)_ -
| https://news.ycombinator.com/item?id=34456861 - Jan 2023 (1
| comment)
|
| _The Coming Technological Singularity (1993)_ -
| https://news.ycombinator.com/item?id=11278248 - March 2016 (8
| comments)
|
| _The Coming Technological Singularity_ -
| https://news.ycombinator.com/item?id=1273160 - April 2010 (1
| comment)
|
| _The Coming Technological Singularity (original essay on the
| Singularity, 1993)_ - https://news.ycombinator.com/item?id=823202
| - Sept 2009 (1 comment)
|
| _The original singularity paper_ -
| https://news.ycombinator.com/item?id=624573 - May 2009 (17
| comments)
| shams93 wrote:
| Once a machine takes your job basically you have 0 recourse.
| Going back to school costs 100k+, going back for an MS costs
| roughly 1 million with 6-8 years of school. Even without AI the
| way we charge our people for healthcare and education is upending
| everything. In the US your human rights are tied to employment,
| once you lose your job you have no more rights because without
| money you are basically paralyzed.
| Jtsummers wrote:
| What masters programs take 6-8 years and a million dollars? If
| you spent that much time on a masters you're finishing it part
| time. If you spent that much money, you got conned.
| isoprophlex wrote:
| Opportunity cost for a tech worker to learn something new
| from scratch?
| BobbyJo wrote:
| If your old job is gone, your opportunity cost is not your
| old salary, it's your potential new salary. Likely a lot
| less.
| hhhrmmm322 wrote:
| I think they meant if you didn't have a relevant bachelor's
| to go straight into the masters. Given we can't predict which
| bachelor's (or master's for that matter) is going to be
| rendered completely obsolete by technology, it's a valid
| concern. You wont necessarily be able to use your BS in CS to
| just immediately hop into an MBA program.
| Jtsummers wrote:
| Given that, it's still not going to cost $1 million for a
| BS + MS. Even taking 8 years for both, you're looking at
| around $800k at the most expensive programs and those are
| rarely worth entertaining when very good and affordable
| state programs exist for half or even a tenth the cost.
| [deleted]
| unsupp0rted wrote:
| Anyway it makes no sense to go back for a Masters. By the time
| you're done a semester or two a machine will have taken your
| next career.
| [deleted]
| olddustytrail wrote:
| The cost is irrelevant. What could you learn in 12 months that
| an AI couldn't learn in 12 hours (or 12 seconds a couple of
| generations later)?
| __MatrixMan__ wrote:
| Something that no other human knows. Isn't that what college
| is about, broadening existing knowledge?
| rcme wrote:
| Sure, but once you learn that new thing, an AI can learn it
| too and apply the knowledge more efficiently than you can.
| __MatrixMan__ wrote:
| Right, which is why we need more humans learning new
| things: so the AI's can be even more useful.
| JohnFen wrote:
| > so the AI's can be even more useful.
|
| That's an utterly depressing thought. It means we're
| slaves.
| pixl97 wrote:
| You're already a slave to many things, you just accept
| them as status quo.
| JohnFen wrote:
| Well, if you expand the definition of "slave" enough, I
| suppose so. But in reality, I have plenty of choice and
| freedom.
|
| If the only way I can live, though, is to feed my ideas
| to a master AI, then I really am a slave. I would argue
| that such a life is not worth the effort.
| __MatrixMan__ wrote:
| Does it? It seems to me like a milestone in the quest to
| eliminate scarcity (which is why we bother with having an
| economy in the first place, right?).
|
| We suck at collaborating. If an AI can mediate
| collaboration across time and space so that its as if
| every problem has precisely the right expertise at hand
| for its solving--even though the actual experts are on
| vacation--that's a win.
|
| If its a win that's incompatible with our habit of
| letting markets handle everything, then so much the worse
| for markets.
| JohnFen wrote:
| I honestly don't see the win there. If AI means that
| every problem has the right expertise even if the humans
| aren't there, that means the humans are dispensable.
| Which means they will be dispensed with. Why would any
| business employ people when they have a cheaper option?
|
| I can't see a field that this doesn't hold true with.
| What will happen when there's no work available at all
| for most people? How will they pay bills, eat, and so
| forth?
|
| If the enthusiasts are correct about the direction AI is
| going, I honestly can't see any way this ends up good for
| anybody but a small group of people.
|
| (To the downvoters: please, tell me how I'm wrong. I
| really do want to be wrong here!)
| __MatrixMan__ wrote:
| > How will they pay bills?
|
| If the problem that we're facing is a surplus of labor,
| then what do we need money for?
| JohnFen wrote:
| Because nothing comes for free, and it's certainly not
| going to start anytime soon.
| __MatrixMan__ wrote:
| Why not? Don't you think that axiom will have to be
| retired sooner or later?
| JohnFen wrote:
| It's not an axiom that can be arbitrarily retired, it's a
| description of reality. What is the path by which this
| could change?
|
| Cutting off people's air supply on a vague notion that
| air will get to people in some other way seems unwise.
| rcme wrote:
| I don't think you can really call "learning new things
| for the AI" recourse for an AI taking your job. Can you
| feed your family with new ideas?
| __MatrixMan__ wrote:
| If they're the only thing in short supply--given that the
| AIs are superior at everything else--then yes I suppose I
| can.
|
| And if it doesn't turn out that way, we can always eat
| the rich until it does.
| digging wrote:
| only in an abstract, idealistic sense. in reality, it is
| about developing job skills for certain classes of jobs.
| Arainach wrote:
| Doctoral-level, perhaps. Undergraduate work generally is
| not. It broadens an individual's knowledge and develops
| skills related to how to problem solve and learn, but the
| vast majority of undergraduate degrees will not involve
| broadening humanity's knowledge. I put non-thesis Masters'
| programs in the same classification.
| hhhrmmm322 wrote:
| Nope. It's mostly about maintaining the class structure,
| and at least at first this was openly the goal, we've just
| done a good job starting to pretend that isn't what it is
| about within the past 60 years or so. Research still
| happened before all that, it was just funded directly by a
| patron, which is actually kind of how trying to get a grant
| works today.
| [deleted]
| olddustytrail wrote:
| You had a pretty amazing college! Well, or...
| __MatrixMan__ wrote:
| I actually kind of detest my local college--as colleges
| go. But not enough to move, so I make do. What I lack in
| quality I make up for in quantity. Been going for 15
| years (mostly just a class or two at a time).
|
| It's fun.
| toxicFork wrote:
| - the machine will not take your job at every company - a human
| using a machine is better than only the machine - it doesn't
| take that much to be retrained these days, there's a lot of
| resources out there - maybe they will actually care about
| rights of people without tying things to being rich or
| employed? - can always cross the border to Mexico
| sxp wrote:
| > This article was for the VISION-21 Symposium sponsored by NASA
| Lewis Research Center and the Ohio Aerospace Institute, March
| 30-31, 1993.
|
| > Within thirty years, we will have the technological means to
| create superhuman intelligence. Shortly after, the human era will
| be ended.
|
| Impressive timing on that prediction assuming GPT-4's release on
| March 14th counts as "superhuman intelligence".
|
| I read Vinge's essay in college and it was a critical factor in
| affecting my career choices and transhumanist views. His sci-fi
| _Fire Upon the Deep_ / _Deepness in the Sky_ and _Fast Times at
| Fairmont High_ / _Rainbows End_ were also critical in affecting
| my career path into AR /VR. I highly recommend reading this essay
| and any of his other sci-fi. He work could be classified as "hard
| computer science fiction" because it is heavily influenced by his
| career in CS. When I was at Google X around 2011, I realized that
| almost all the stuff they were working on in secret at the time
| (AR glasses, Drone delivery, self-driving cars, balloon Internet
| with lasers, computers embedded in fabric, etc) were copied
| directly out of _Fast Times at Fairmont High_. _Fast Times_ and
| _Rainbows End_ aren 't available online but I suggest _Synthetic
| Serendipity_ which is set in the same universe and has concepts
| such as a gig economy (called "affiliate tasks"):
| https://spectrum.ieee.org/synthetic-serendipity
| AnimalMuppet wrote:
| > Impressive timing on that prediction assuming GPT-4's release
| on March 14th counts as "superhuman intelligence".
|
| How many legs does a dog have, if you assume a tail is a leg?
|
| Four. Assuming it doesn't make it so.
|
| In the same way, you can assume whatever you want, but GPT-4 is
| _not_ "superhuman intelligence". It just isn't, for any
| meaningful definition of those words.
| phkahler wrote:
| >> In the same way, you can assume whatever you want, but
| GPT-4 is not "superhuman intelligence". It just isn't, for
| any meaningful definition of those words.
|
| It's also not 2030 yet, so the prediction has 7 more years to
| be false in its full form. If it happens in 2040 will people
| be upset that it was 10 years later than predicted?
| Teever wrote:
| 1993 + 30 = 2023
| marcosdumay wrote:
| > assuming GPT-4's release on March 14th counts as "superhuman
| intelligence"
|
| If it did, nobody here would have a job and we would be already
| deep into the "reorient society or fight the Machine" phase.
___________________________________________________________________
(page generated 2023-04-18 23:02 UTC)