[HN Gopher] The Myth of a Superhuman AI
___________________________________________________________________
The Myth of a Superhuman AI
Author : conanxin
Score : 26 points
Date : 2021-08-24 15:23 UTC (1 days ago)
(HTM) web link (www.wired.com)
(TXT) w3m dump (www.wired.com)
| tech-historian wrote:
| Who is the author Kevin Kelly and why does he think he has a
| greater hold on this issue than luminaries like Ray Kurzweil?
| AnimalMuppet wrote:
| Why is Ray Kurzweil considered a luminary _in this context_?
|
| Sure, he talks a lot. He makes grand statements. Is there _any_
| reason to take him seriously in this context?
|
| He sure seems to believe, and to want to believe. But actual
| evidence that he's going to be right any time soon? That seems
| to be missing.
| enord wrote:
| Everybody wants to be the Fox Mulder of their story.
| tech-historian wrote:
| It's clear you're not familiar with Kurzweil's work in AI (or
| his background in general). He's written multiple books on
| the topic over multiple decades, including the seminal "The
| Singularity is Near". He's devoted his life to the study of
| this field.
|
| https://en.wikipedia.org/wiki/Ray_Kurzweil
|
| https://en.wikipedia.org/wiki/The_Singularity_Is_Near
| zaybqp wrote:
| Thank you for enlightening us with the new knowledge of
| this wonderful luminary. I have learned so much from these
| URLs. Do you have any more recommendations on what we
| should read next? I find myself deeply enthralled by his
| fascinating claims and I want to seek out more of his
| teachings.
| 6gvONxR4sf7o wrote:
| He's also written books on how much of your diet should be
| fat, and similar topics. I've never been able to tell if
| he's worth taking seriously. My instincts say no.
|
| https://en.wikipedia.org/wiki/The_10%25_Solution_for_a_Heal
| t...
|
| https://en.wikipedia.org/wiki/Fantastic_Voyage:_Live_Long_E
| n...
| GeorgeTirebiter wrote:
| Have you read Ray's books? I recommend 2005's The Singularity
| is Near.
| https://en.wikipedia.org/wiki/The_Singularity_Is_Near
| vkou wrote:
| I've read Ray's books. He's a big talker that writes
| speculative science fiction, but he doesn't _do_ anything.
|
| As the parent poster said, there is zero reason to take him
| seriously.
| AnimalMuppet wrote:
| It's been 16 years. Is the singularity any nearer? Is there
| any _evidence_ that the singularity is any nearer?
|
| So, how "near" was the singularity in 2005? How accurate
| was Kurzweil?
| Tenoke wrote:
| He has timeliness on the optimistic side but people were
| skeptical of his claims in the 80s that a computer will
| win at Chess by 2000 or that the internet will become
| ubiquitous. He did think a US company will reach 1
| trillion $ by 2010 and that speech recognition will be
| more widely used but even in cases like that he wasn't
| massively off.
|
| As far as I can tell for decades he's been consistently
| more right than his critics and often on issues that are
| a lot harder to get right than simple coin tosses.
| AnimalMuppet wrote:
| He's been right (approximately) on how fast the tech
| improves. That doesn't tell you anything about how likely
| he is to be right about the singularity, though, unless
| you think he actually has a handle on how much tech would
| have to improve to get us there, which means that he
| would have a handle on where "there" actually is, in
| technical terms. I don't see evidence that he has that.
| mindcrime wrote:
| To the first half of your question:
|
| https://en.wikipedia.org/wiki/Kevin_Kelly_(editor)
|
| To the second half:
|
| Dunno. Good question.
| cowpig wrote:
| Appeals to authority are not convincing
| mindcrime wrote:
| This isn't a formal debate, and authority _is_ useful
| especially as a "first pass" filter, or in terms of
| establishing Bayesian priors. The key, IMO, is not to assume
| that the mere existence of authority guarantees that someone
| is right. But in either case, it's certainly valid to ask the
| question "why should I take Kevin Kelly seriously on this
| topic?"
| cowpig wrote:
| I don't agree in this context. The author wrote an entire
| article about his position, and dismissing it entirely
| because some "luminary" disagrees is not an interesting
| response.
|
| OP didn't even bother to point out how or why Kurzweil
| purportedly disagrees.
| tech-historian wrote:
| I thought it was obvious. Clearly it was not. Ray
| Kurzweil is a futurist who has spent multiple decades
| researching this topic and has written multiple books on
| it. His bio speaks for itself.
|
| https://en.wikipedia.org/wiki/Ray_Kurzweil
| mindcrime wrote:
| I guess I interpreted tech-historian's post a little
| differently than you.
|
| Likewise, I don't support dismissing Kelly's article out-
| of-hand, based on Kurzweil's position. But I do think
| it's fair to raise the issue in terms of "why should one
| take Kevin Kelly seriously on this?"
| [deleted]
| habitue wrote:
| Other commenters have mentioned that these particular points have
| been refuted elsewhere, so I won't rehash that. But I do think
| it's interesting that the author frames the "orthodox" position
| as the one taking AI alignment seriously. That's pretty big
| progress and I think the folks who have been sounding the
| alarm[0] about AGI should give themselves a pat on the back that
| the zeitgeist has shifted so much.
|
| The arguments from the anti-alarmism camp have been so
| consistently weak, I think the best explanation for their
| motivation is still something like "All this AGI stuff sounds so
| sci-fi. It can't be a real concern". It just goes to show how
| strongly we bias our expectations of the future on our what we've
| personally experienced. You really have to try hard to escape
| that default mode of thinking.
|
| [0] By sounding the alarm I mean saying things like "This is at
| least a possible scenario with huge error bars on when it might
| happen, maybe at least _some_ people should be thinking hard
| about this, rather than _nobody_ "
| batavian wrote:
| It doesn't really even sound sci-fi, or rather, it would make
| for some bad sci-fi. What we are at risk of, is having
| deterministic model artifacts trigger complex unintended
| consequences from being fetishized as decision-making agents.
| Considering how few CS and AI people I've met who even know
| that e.g. pragmatics are a critical component of sentence
| meaning, I sense the most proximate AI dystopia is a lot closer
| to this latter scenario.
|
| Conceptually the possibility of an AGI _does_ present a real
| challenge ( & one worth grappling); in practice, we haven't
| made any progress really toward realizing that problem. We've
| become good at feeding "large" datasets into algorithms that
| are many orders of magnitude less advanced than a lizard brain.
| Then we get to the issue that our large datasets are only large
| when thinking of classical statistical sample sizes. They
| nonetheless are lacking in redundancy, interactivity, stakes or
| any centered perspective such that there is no chance of
| anything resembling intelligence, agency, or plain old
| intentions arising.
| dennisblue wrote:
| Why was this garbage even published?
| sxp wrote:
| tl;dr: Humans have souls which machines will never have. The core
| of the article is this quote "I will extend that further to claim
| that the only way to get a very human-like thought process is to
| run the computation on very human-like wet tissue. That also
| means that very big, complex artificial intelligences run on dry
| silicon will produce big, complex, unhuman-like minds." which is
| just the boring old argument against replicating human thinking.
|
| Similarly, there is an extreme amount of pro-human arrogance in
| the article "At the core of the notion of a superhuman
| intelligence -- particularly the view that this intelligence will
| keep improving itself -- is the essential belief that
| intelligence has an infinite scale. I find no evidence for
| this.... So the question is, where is the limit of intelligence?
| We tend to believe that the limit is way beyond us, way "above"
| us, as we are "above" an ant. Setting aside the recurring problem
| of a single dimension, what evidence do we have that the limit is
| not us? Why can't we be at the maximum?"
|
| Humans are just a bunch of atoms that randomly learned to think
| over a few billion years. Given how fast we can improve tech via
| directed means, we should be able to replicate what nature has
| done in thousands, hundreds, or tens of years.
| cowpig wrote:
| The word "soul" does not appear once in the article
| DuskStar wrote:
| > That also means that very big, complex artificial
| intelligences run on dry silicon will produce big, complex,
| unhuman-like minds.
|
| Incidentally, this is exactly what the AI superintelligence
| people are terrified of, and what the paperclip problem is
| meant to illustrate.
| https://www.decisionproblem.com/paperclips/index2.html
| bjornsing wrote:
| Only skimmed the bullet points, but one that's missing and seems
| important to me is:
|
| * The marginal utility of intelligence may not be so great. If
| you are already really smart, is there any point to being
| smarter?
|
| Most interesting problems are NP-complete. If you are really
| smart you can solve a big NP-complete problem. But how much
| smarter do you need to be to solve a slightly bigger (and
| therefore more interesting) problem? Well, twice as smart or
| something. Being just a little bit smarter gets you nothing. So
| (in this case) the marginal utility of intelligence approaches
| zero as your AI gets more intelligent.
| mindcrime wrote:
| Ugh. This is a pretty bad article. And don't get me wrong... I'm
| no "AI Alarmist." But to me, this article is equally as useless
| as the various pronouncements it seems to be trying to refute.
| There's a lot of verbiage here, but no cohesive, coherent
| argument to support the thesis that "Superhuman AI is just a
| myth" (which is to say, "Superhuman AI is not possible").
|
| The reality is, there are a lot of unknowns in this space, and to
| dismiss the possibility of an "evil super-intelligence" out-of-
| hand is, IMO, every bit as silly as buying into the "OMG, AI is
| unleashing the demon" fear-mongering. I just think it's too early
| to know for sure the answers to some of these questions, like:
|
| 1. Is AGI possible at all (for whatever definition of "AGI" you
| want to use)?
|
| 2. If the answer to (1) is "yes", then what about ASI (Artificial
| Super Intelligence) - is that possible?
|
| 3. If (1) and (2) are both "yes", then is it possible to create
| an ASI that will either be benevolent, or controllable, or both?
|
| And so on... we just don't know yet. And that in and of itself
| arguably suggests the exercise of a certain measure of caution.
| enord wrote:
| You're making an argument that is already in TFA. It's in
| addition -and only partially related to- the main point which
| is: The arguments that are commonly used to demonstrate the
| possibility or threat of "ASI" are made from claims presented
| _without evidence_.
|
| This will invalidate fantastical accelerationist woo until such
| evidence is presented, and -in the current state of the art
| research within relevant fields- it's not even certain we can
| know what would constitute such evidence _in principle_.
| mindcrime wrote:
| I assume you're referring to this part:
|
| _If the expectation of a superhuman AI takeover is built on
| five key assumptions that have no basis in evidence, then
| this idea is more akin to a religious belief -- a myth_
|
| My problem is, even this is wrong. I do not find the "five
| key assumptions" he cites to be the foundation of beliefs
| that a dangerous "Superhuman AI" could emerge. So to me, this
| article simply fails right from the jump. The thing he is
| arguing for and/or against just don't, collectively,
| constitute a well-developed argument for the overall thesis.
| Not in my opinion anyway.
|
| To go through them one by one:
|
| _Artificial intelligence is already getting smarter than us,
| at an exponential rate._
|
| I don't know anybody making this claim, nor is it apparent
| that belief in this is necessary to think that dangerous
| "Superhuman AI" could be a problem in the future. I guess it
| comes down to what you think Kelly meant by "soon" in the
| preceding sentence.
|
| _We'll make AIs into a general purpose intelligence, like
| our own._
|
| It's the "like our own" part of this that I have a problem
| with. Yes, I think many people believe that we will create
| _some_ manner of "general intelligence", but I also believe
| that a lot of researchers acknowledge that it may not be 100%
| "human like". But this may be my own bias showing, as I
| personally harp a lot on the importance of the distinction
| between "human level" intelligence and "human like"
| intelligence.
|
| _We can make human intelligence in silicon._
|
| This is probably the most sound one of all, but here's the
| thing... saying "there is no evidence for this" is simply
| wrong. To hold to this is to completely ignore all progress
| made building AI in silicon to date. And while we don't _yet_
| have "general intelligence", I don't see how one can just
| ignore incremental progress completely and say "there's
| nothing there". I would argue that the null hypothesis should
| probably be that general intelligence on silicon _is_
| possible that that the burden of proof is on the people
| arguing the opposite.
|
| Also, not everybody thinking about / working on AI is focused
| on the "in silicon" approach. The majority? Yes. But I'm sure
| there are a handful of people working on ideas related to DNA
| computing, chemical computing, growing synthetic brain
| matter, etc.
|
| _Intelligence can be expanded without limit._
|
| To me, this may or not not be true, but it's completely
| irrelevant to the thesis at hand. The question isn't "can
| machine intelligence be unbounded?", but rather "can machine
| intelligence exceed human intelligence?"
|
| _Once we have exploding superintelligence it can solve most
| of our problems._
|
| Again, this strikes me as mostly irrelevant to the question
| at hand.
|
| So yes, this article raises some interesting points, but I
| don't see it accomplishing what it seems to purport to do.
| enord wrote:
| "Som fanden leser bibelen" - Norwegian proverb, lit.: "Like
| the devil reads the bible"
|
| I sense that your arguments are in concordance with TFA,
| broadly speaking. Also, I suspect you read it rather
| uncharitably for reasons having to do with tone or stated
| thesis. Either that, or my reading comprehension is failing
| me. Maybe I want it to be more convincing that it really
| is, as i already laid bare my sentiments towards
| accelerationist woo.
| mindcrime wrote:
| _Norwegian proverb, lit.: "Like the devil reads the
| bible"_
|
| I'll take that as a compliment!
|
| All joking aside, I'm not saying every word of the
| article is wrong or anything. Maybe I'm just being too
| literal or pedantic or something. It just seems to me
| that the article sets out to make one very specific
| claim, and I just don't think it succeeds in making a
| strong case for that specific claim. Is there some
| interesting discussion there? Yes. Are there things that
| Kelly says that I agree with? Yes, absolutely.
|
| So maybe we're not too far apart on this after all.
| roywiggins wrote:
| Yeah. All you need is an intelligence that's human level
| but _faster_ , and you have something that is, to some
| extent, superhuman. If you _can_ get to human intelligence
| levels you can just crank up the clock speed a bit and give
| it a better working memory, and now you have something like
| a super-Einstein in a box.
|
| It could be that there's no way to do that _in silica_
| without outstripping plausible bounds on hardware size or
| power use, I don 't know...
| goatlover wrote:
| But Einstein was no ordinary human, he was a genius. So
| you would need the AGI to be more than just your average
| human. Would the average Joe sped up be some form of
| superintelligence? They would be able to perform tasks
| faster, but that doesn't make them smarter.
| idatum wrote:
| Maybe the article was AI generated? /s
| upearly4 wrote:
| >This is a pretty bad article.
|
| Agreed.
|
| Dunno why he thinks this: "At the core of the notion of a
| superhuman intelligence -- particularly the view that this
| intelligence will keep improving itself -- is the essential
| belief that intelligence has an infinite scale."
|
| The notion of self-improvement is both powerful and not assumed
| to be 'infinite'.
|
| I figure it goes like this:
|
| Q. Is AI potentially dangerous? A. Yes.
|
| Q. Will it be human-like? A. Probably not.
|
| People tend to couch these discussions as a series of
| anthropomorphisms.
| goatlover wrote:
| The article is a response to claims about the singularity and
| subsequent worries about concerning super AI.
| eutectic wrote:
| Humanity is an existence proof for human level AI, and it seems
| arrogant to assume that our brains are at the limit of what is
| possible.
|
| Reproductive fitness requires many attributes which obviously
| compete with intelligence, and evolution is a blind hill-
| climbing search which cannot easily escape deep local minima.
|
| To me it's obvious that the development of superhuman AI is
| inevitable assuming we don't wipe ourselves out before we have
| time to develop it.
| dragontamer wrote:
| But Google can read, injest, and search for information far
| faster than I can.
|
| Stockfish players chess far better than I can.
|
| AlphaGo plays Go far better than I can.
|
| Random Sudoku bot built on top of minizinc / z3 / whatever
| SAT solver can solve NP-complete problems better than any
| human.
|
| --------
|
| Here's the thing: what is "strong AI" ?? As far as I can
| tell, we're already at superhuman AI in most intelligence
| tasks (collecting data, collating it together, analyzing it,
| etc. etc.).
|
| Its called a computer. What if "superhuman AI" is just the
| stuff we have today at our fingertips? It wasnt very long ago
| when reading documents and organizing them was considered AI.
| But today, we call that Google.
| Isinlor wrote:
| There are no superhuman craftsman AI.
|
| Something that could in one package:
|
| - hold a conversation about what it is that needs to be
| done
|
| - clean a room
|
| - fix a sink
|
| - make a chair or table out of wood
|
| - dig a hole
|
| We don't know how to make AI that will learn to play even
| simple game like montezuma's revenge in some reasonable
| play time like 30 min.
|
| We are currently closer to months or years of gameplay
| required.
| dragontamer wrote:
| > We don't know how to make AI that will learn to play
| even simple game like montezuma's revenge in some
| reasonable play time like 30 min.
|
| Explain to me why a Tool-assisted Speedrun doesn't count
| as "AI". Its certainly not a human who is playing that
| (https://www.youtube.com/watch?v=pFB0kuPx5mQ).
|
| > Something that could in one package:
|
| I'm not sure if that sort of thing is worth the
| investment.
|
| But if we have "Fly the F35 airframe" (which is
| incredibly unstable), then we suddenly reach "only
| computers can fly this thing" levels. Just the assumption
| of super-human control systems is baked into modern
| products. https://www.youtube.com/watch?v=cyN-CRNrb3E
|
| Our cell phones are largely manufactured by machines
| these days. A few tidbits here and there remain in human
| hands for cheapness... but a lot of today's stuff is
| basically an AI.
| https://www.youtube.com/watch?v=SRu02F6AOmg
| qsort wrote:
| > Explain to me why a Tool-assisted Speedrun doesn't
| count as "AI". Its certainly not a human who is playing
| that (https://www.youtube.com/watch?v=pFB0kuPx5mQ).
|
| Because TAS are made by human speedrunners, who go
| through the painstaking process of specifying inputs for
| each frame. I'm not sure what you're trying to say here,
| TAS are basically the exact opposite of AI.
|
| > I'm not sure if that sort of thing...
|
| > But if we have "Fly the F35 airframe"...
|
| > Our cell phones are largely manufactured by machines...
|
| That's a red herring. There's plenty of evidence we can
| make very useful narrow AI in many fields, and nobody is
| arguing against that.
|
| The point is, they are, well, _narrow_. What your parent
| is saying is that there 's no evidence that the
| statistical techniques behind the current wave of AI are
| capable of anything else.
|
| To paraphrase Norvig, it could well be that we're trying
| to get to the moon by climbing a tree, reporting steady
| progress all the way to the top of the tree.
| tshaddox wrote:
| Stockfish and AlphaGo are also available for humans to use.
| A human using Stockfish will be just as good at chess as a
| strong AI using Stockfish, and likewise for any future
| improvements to chess engines that either humans or strong
| AIs develop.
| PaulHoule wrote:
| When I was involved in physics I thought that the range of
| problems that were solvable were like a continent (linear
| systems) surrounded by islands (integrable systems and a
| domain in which perturbation theory works around them.)
|
| I thought that the problems solved by Bethe, Feynman and
| their cohort were the easy problems and the current
| generation is struggling with problems that were much harder,
| possibly impossible to solve.
|
| One example is the problem of chaotic dynamics with six
| degrees of freedom as opposed to four degrees of freedom.
| With four degrees of freedom you have unbroken KAM tori that
| partition the phase space into areas that it will stay in
| forever. In 6 DOF or more, some tori are still unbroken but
| it's possible for the system to go around them, so even in a
| phase space region that has a lot of intact tori (e.g. the
| motion of the planets around the Sun) you can't prove
| anything for the long term (e.g. the Earth doesn't get
| ejected from the solar system.)
|
| Look also at examples such as the halting problem, Godel's
| theorems, etc. I'm not sure if a "higher intelligence" could
| actually solve that many problems we can't because the
| problems are intractable.
| tshaddox wrote:
| > Humanity is an existence proof for human level AI, and it
| seems arrogant to assume that our brains are at the limit of
| what is possible.
|
| Why? Is it arrogant to assume that Turing machines are at the
| limit of what is possible to compute? Perhaps, but we also
| have good reasons to believe that to be the case. If you're
| just talking about things like computational speed
| differences or being able to control powerful machinery and
| weaponry, then those are already available for humans to use,
| and can already be used by humans to do bad things much more
| effectively. It seems to me that any future improvements in
| computational speed, memory capacity, etc. will also continue
| to be available to humans as well as any strong AIs we might
| develop.
| arisAlexis wrote:
| Someone must have written at some point "the myth of a pandemic".
| Its incomprehensibly stupid to dismiss any kind of threat as a
| myth unless they prove that it's impossible by laws of physics.
| And still then there would be room for error. Sad thst a
| respectable magazine posts such naive articles. I dismiss it from
| the title before even reading it.
| blamestross wrote:
| The bottleneck for intelligence is experience. Running on silicon
| doesn't allow an intelligence to engage with reality
| significantly faster than a human can. There are some benefits
| due to access to data and processing power, but the data isn't
| useful without the context of experience and all processing power
| can do it overfit model without experience.
|
| Even if we put AI into an army of robots running around and
| experiencing things, there are still scaling limits to encoding
| and communicating knowledge and understanding. Human
| organizations are a great example of the scaling limits of
| intelligence.
| ryandvm wrote:
| That's a great point. However, whereas humans share their
| experience through the effective, but extremely lossy
| technology called language; machines can store and share their
| experience without loss. Sure there are scaling limits, but a
| brain dump back to the mothership is a lot quicker and more
| perfectly preserved than sitting around listening to grandpa's
| stories.
| namedlambda wrote:
| Everything in the above comment is wrong.
|
| In AlphaZero for example, there were 44 million training games
| total for 700,000 steps of training for the full 9 hours.
|
| Turning that to human-scale numbers, 44million games with on
| average 60 moves, at 1 second thinking time per move,
|
| > 44000000*60/60/60/24/365 = 83,7138508371 years of training
| experience in 9 hours
|
| The whole field of Reinforcement learning has agents training
| and playing games for many orders of magnitude more time than a
| human ever will. In-fact, we can scale this to over 100k of
| actions per second, in a single machine:
|
| https://github.com/alex-petrenko/sample-factory
|
| Then, there is also distributed Reinforcement Learning, where
| hundreds of agents can play at different machines and share
| experience, see AlphaZero, LeelaZero, R2D2 agent, R2D3 agent,
| Apex, Acer, Asynchronous PPO.
|
| > but the data isn't useful without the context of experience
|
| The experience is the data in Reinforcement Learning.
|
| > and all processing power can do it overfit model without
| experience.
|
| That is wrong, the agents perform what is called exploration to
| avoid getting stuck in simple strategies.
|
| > Even if we put AI into an army of robots running around and
| experiencing things, there are still scaling limits to encoding
| and communicating knowledge and understanding.
|
| True, but machines scale better because they speak the same
| language, or they can learn to tune their language to get their
| message across.
|
| > Human organizations are a great example of the scaling limits
| of intelligence.
|
| Human organization is a testament to how far we can get with
| something as limiting as the commonly used language. The
| language that we use to communicate is subject to
| misinterpretation due to our subjective experiences, this
| limitation is not shared by machines.
| blamestross wrote:
| If the universe is a game you are playing, then yes playing
| that game is "experience", but for an AI to engage with
| reality it has to have experience in reality, not a game. The
| ability to play go very well doesn't enable an AGI to better
| understand reality.
|
| > The experience is the data in Reinforcement Learning.
|
| This is very true, and the critical problem. Data about how
| reality responds to an AI's actions is very sparse right now.
|
| AIs do have a potential advantage in communications
| efficiency, but at some level of scale compression will
| happen, locally "irrelevant" data will be discarded and
| simplified approximations replace it. None of this will
| change the "big O" of the scalability of intelligence, just
| the constant factors. There is no exponential kickoff point.
| namedlambda wrote:
| What is the difference between experiencing reality and a
| game?
|
| The difference I can see is that there is no one explicit
| objective function, but this doesn't stop generally capable
| agents [1], and doesn't imply that inverse RL is not
| possible.
|
| > The ability to play go very well doesn't enable an AGI to
| better understand reality.
|
| I disagree, model based RL constructs a model of the
| agent's reality and can use it to plan ahead, train the
| agent, or do some form of monte-carlo tree search. The
| latter is something very similar to how we imagine and
| think about the future.
|
| [1] https://deepmind.com/blog/article/generally-capable-
| agents-e...
| habitue wrote:
| Really fascinating counter-argument to this view is here:
| https://www.lesswrong.com/posts/5wMcKNAwB6X4mp9og/that-alien...
|
| The basic idea is that, the smarter you are and the faster you
| can think, the more information you can extract from weak
| signals. It involves creativity, and building models, and some
| minimum amount of in-built bias, but the upshot is that the
| amount of information a super-intelligent AI could infer from
| very small amounts of experience might absolutely stagger the
| imagination.
| totally_not_ai wrote:
| Are you of the opinion that you can't get experience by reading
| books or watching youtube?
|
| Do you think everything taught in an oral form (as opposed to
| hands on) is completely useless?
|
| Are all military strategy books used by military academies a
| waste of time, since you can only learn by leading a real army?
| blamestross wrote:
| All of those things have to be integrated with experience to
| be useful. Often the learner already has a lot of that
| experience, so the data can be integrated easily.
|
| You can't understand those books or information without
| having a lifetime of experience to contextualize them. You
| can't convert them from rote knowledge into skills without
| trying them out.
| skybrian wrote:
| Actually no, there is almost no chance that it will be harder
| to transfer skills between machines than it is to raise and
| educate a child.
|
| Something very weird would have to happen for it to be that
| hard.
| shkkmo wrote:
| Is it?
|
| Humans have been optimizing for information transmission
| between individuals for a long time. Not just our hardware
| and neural nets have been optimized, but the structure of the
| information itself.
|
| It is quite possible that you need such a large set of compex
| interactions to develop intelligence that intelligence can
| only develop in reality. It is possible that the process for
| doing this inherently inolves a chaotic system that makes
| duplicating or paralleling that development impossible. I
| don't see any reason why these artificial intelligences with
| divergent chaotically emergent capabilities would have to be
| able to teach each other things as easily as we do.
|
| I've long been of the opinion culture / "social software" is
| vital for producing knowledge for humans and that this social
| software has been fine tuned and iterated upon for a very
| long time (and possibly co-developed with the evolution of
| our brain hardware so that both are co-specialized for each
| other). This software is likely very specialized to our brain
| structures and might not generalize well to other
| intelligence substrates. Thus even if we do develop neural
| network structres that can run on silicone and match human
| level intelligence, those entities would still need a lot of
| interaction with reality to develop their own culture that
| would allow them understand the world at the level we do.
| Edit: Is it even possible to efficiently run the necessary
| theories of human mind on non-human brains to operate
| effectively in our epistemic landscape? Communication
| requires huge amounts of inference using theories of mind to
| resolve ambiguity and add implicit information.
|
| It is possible that we can "recompile" human knowledge to run
| on other types of intelligences, but I suspect this will be
| far more difficult that commonly expected. Just look at how
| hard (nigh impossible) it is today to even boostrap a new
| general purpose computer without depending on the output
| other general purpose computers.
|
| We simply do not understand cognition or semantics well
| enough to know the answers to these questions adequately or
| with any degree of certainty.
| choeger wrote:
| > Intelligence is not a single dimension, so "smarter than
| humans" is a meaningless concept.
|
| That's not the point. We can map any multi-dimensional form of
| intelligence into a single dimension with any suitable norm. It
| just matters that for any dimension we can _quantify_
| intelligence in the sense that something more intelligent is
| imaginable.
|
| > Humans do not have general purpose minds, and neither will AIs.
|
| That does not matter at all. Then a superhuman intelligence is
| not "general", so what? It's still wastly more capable then any
| human.
|
| > Emulation of human thinking in other media will be constrained
| by cost.
|
| But what _is_ the cost constraints? Human brains are constrained
| by mechanical considerations. How efficient could a computerized
| AI become?
|
| > Dimensions of intelligence are not infinite.
|
| Again, what's the argument? The number of atoms in the universe
| is not infinite, either (probably).
|
| > Intelligences are only one factor in progress.
|
| That's not the problem of the super AI.
| jbay808 wrote:
| Reminds me of the brilliant "On the Impossibility of Supersized
| Machines" [1].
|
| _We show that it is not only implausible that machines will
| ever exceed human size, but in fact impossible._
|
| [...]
|
| _The term "supersized machine" implies a machine that has
| crossed some threshold, which is often denoted "human-level
| largeness." However, it is not clear what "human-level
| largeness" could refer to. Has a machine achieved human-level
| largeness if it has the same height as the average human? If it
| has the same volume? The same weight? [...]
|
| When one begins to consider these questions, one quickly
| concludes that there are an infinite number of metrics that
| could be used to measure largeness, and that people who speak
| of "supersized machines" do not have a particular metric in
| mind. Surely, then, any future machine will be larger than
| humans on some metrics and smaller than humans on others, just
| as they are today._
|
| [1] https://arxiv.org/abs/1703.10987
| [deleted]
| mindcrime wrote:
| FWIW, I agree with your sentiments pretty much 100%. I was
| going to say more or less the same thing(s) but didn't feel
| motivated enough to do more of a "point by point" response.
| Glad to see somebody chipping in with very similar viewpoints.
| a_square_peg wrote:
| Interesting discussions. Perhaps it may be easier to look at it
| from another angle.
|
| Current blend of ML/DL algorithms are not AI. I think the
| tendency of giving aspirational names to software algorithms have
| created a lot of confusion. Perhaps AI will be one day possible
| but if so, it's probably safe to say that it won't be anything
| based on the current ML/DL paradigm.
|
| Fun example, Google DeepMind is yet able to learn math:
|
| https://www.zdnet.com/article/ai-aint-no-a-student-deepmind-...
| lern_too_spel wrote:
| Old article. Previous discussion:
| https://news.ycombinator.com/item?id=14205042
| Analemma_ wrote:
| It's really frightening that "AI risk isn't real" is rapidly
| congealing into a tribe-signaling _political_ stance that
| generates reliable terrible articles like this one and the one in
| the Washington Post a few weeks ago. You'd think with Covid still
| killing thousands of people a day the media would more able to
| recognize that fault mode and not repeat the "Covid isn't a
| problem, because saying that it's a problem would be racist"
| error of February 2020, but apparently not.
| croes wrote:
| Bold claim. At the moment we don't fully understand the brain and
| intelligence. Even what makes the difference a living and a non
| living accumulation of atoms is unknown.
|
| But this doesn't not mean we couldn't create a superhuman AI in
| the future or by accident.
|
| I think we don't have the capabilities to do so but this isn't a
| 100% certaincy more like 99%
| arisAlexis wrote:
| "the problem with the world is that the intelligent are full of
| doubt and the idiots cocksure". Applies directly to the bold
| title. Doesn't matter if we are all annihilated of course
| nobody will remember.
| totally_not_ai wrote:
| > _Hyper-intelligent algorithms are not going to take over the
| world for these five reasons._
|
| I agree.
|
| Just like a nuclear power plant will never meltdown, because they
| are extremely well designed with large safety buffers.
|
| And just like a pandemic will never be started by lab
| experiments, because they use extreme safety measures when they
| work with viruses.
|
| We can always just unplug the AI or disconnect it from the
| Internet, or just not give it a survival instinct, right?
| gilch wrote:
| Pretty sure this is sarcasm, as in, the lies are so transparent
| they're humorous, and this was on purpose.
|
| For those who didn't notice, nuclear plants have, in fact,
| melted down, more than once. Biosafety level 4 labs have, in
| fact, had leaks of pathogens before, more than once. An agent
| AI will tend to develop a survival instinct by default, even if
| we don't give it one, because it is a convergent instrumental
| goal. Thus, it would be motivated to not simply let you yank
| it, would fight you if you tried, and may even take preemptive
| precautions.
| YetAnotherNick wrote:
| There is no reason we know that brain emulation is not possible.
| Even if one second takes 100 years in processing now, there is no
| reason to think that there is fundamental limit in how many
| processor you could connect. And for superhuman it just needs to
| do one thing well(which computers already does) than human and
| match humans in everything. I am not saying it would be done but
| I haven't seen any argument on why this exact argument is
| "impossible".
| otabdeveloper4 wrote:
| > There is no reason we know that brain emulation is not
| possible.
|
| Well, neither is there any reason we know that it's possible.
|
| You're just demonstrating your apriori biases ("the brain is
| just a computer"), nothing more.
| goatlover wrote:
| As the article points out, the nervous systems is larger than
| the brain. There are millions of neurons in your gut. Your
| sensory organs have neurons that prepare the signal sent to the
| brain from the sensory noise. No brain lives independent of a
| body. That's not how organisms evolved.
| qsort wrote:
| It's not impossible in principle, but:
|
| - we don't understand enough about the brain to emulate it.
|
| - we don't know how to model the brain.
|
| - even assuming we had the required scientific knowledge, there
| would likely be major engineering problems.
|
| So, in other words, it's not impossible in the same sense FTL
| travel is not impossible, our scientific understanding does not
| categorically exclude it, but making predictions as to when it
| will happen is kind of silly, because it will likely require
| several fundamentally unpredictable scientific breakthroughs.
___________________________________________________________________
(page generated 2021-08-25 23:02 UTC)