[HN Gopher] DeepMind says reinforcement learning is 'enough' to ...
___________________________________________________________________
DeepMind says reinforcement learning is 'enough' to reach general
AI
Author : webmaven
Score : 220 points
Date : 2021-06-10 03:18 UTC (19 hours ago)
(HTM) web link (venturebeat.com)
(TXT) w3m dump (venturebeat.com)
| azinman2 wrote:
| Everything old is new again.
|
| As far as I can tell, they're not actually proposing how to
| achieve this. I can't access the article without a host
| institution it seems (is there another link?), so I only have the
| article to go by. RL has been the basis for all robots engaging
| with the world, and that engagement with the physical world
| modeled using RL has been promised to make robots that can act
| like a 2 year old for a long time (see Cynthia Breazeal's work,
| for example). Yet AFAIK, we haven't actually achieved this as we
| don't know how to efficiently model the problem to have learning
| rates that reach anywhere near what we're able to do with DNNs
| today.
|
| Perhaps someone who has access to the paper can say why this is a
| milestone? If Patricia Churchland suggests it is, then something
| new must be happening here.
| hervature wrote:
| I don't know who Patricia Churchland is, but they said that the
| paper was "very carefully and insightfully worked out."
|
| After having read the paper, I am very disappointed in the
| output. Nothing concrete was shown, just hypothesis and reads
| more like philosophy. That being said, I would say that the
| paper is carefully worked out and does provide insight if you
| haven't thought about RL before.
| Ieghaehia9 wrote:
| If Patricia Churchland doesn't have a problem with the paper
| despite it being philosophical, then that's probably because
| she _is_ a philosopher. An eliminative materialist, to be
| precise.
|
| Personally, from reading the abstract, I disagree with the
| hypothesis. There's a trick where anything (even say, a
| database lookup) looks like optimization as long as you
| contrive the objective function just right, but that's kind
| of uninformative.
| LesZedCB wrote:
| and our intelligence is quite good at confabulating
| optimization functions for abstract processes that are
| inert. pretty amazing, really.
| leto_ii wrote:
| > is there another link?
|
| This is the download link:
| https://www.sciencedirect.com/science/article/pii/S000437022...
| deepnet wrote:
| The paper is creative commons licensed no signup necessary to
| download
|
| https://www.sciencedirect.com/science/article/pii/S000437022...
| im3w1l wrote:
| So regarding objective function, one idea I just had is this:
| Teach them warfare.
|
| To quote a cliche: "we live in a society". As humans we are
| embedded in a social environment which has a few important
| features: We cooperate, we compete and we die. These three
| pillars are the basis of our _culture_ (a concept we should apply
| to AI btw). Because of competition we are forced to learn
| everything there is to learn (general intelligence), to get a leg
| up. Because of cooperation and death we need to continuously
| transmit and share knowledge with our friends and the next
| generations. Ever changing alliances means we need to get good at
| both deception and detecting it.
|
| For this reason I think warfare is ideal for reaching general AI.
| mensetmanusman wrote:
| Does anyone know if deepmind is working on driverless technology?
|
| It seems like one of the largest values they could bring to
| society would be to solve driverless technology.
|
| Currently their best partner would be Tesla due to the amount of
| data Tesla has, but I doubt Google would allow that
| collaboration...
| X6S1x6Okd1st wrote:
| https://www.businessinsider.com/deepmind-is-teaching-googles...
| eatmyshorts wrote:
| I'm curious why you feel their best partner would be Tesla
| instead of Alphabet's own Waymo?
| mensetmanusman wrote:
| Because of how much data Tesla's fleet is generating every
| day (gathering while the user drives).
| Sparkyte wrote:
| TLDR, general AI means simplistic AI.
| btilly wrote:
| My first response is that B. F. Skinner would be proud.
|
| I'm going to predict that reinforcement learning will be
| important but not sufficient. Just as behaviorism is important
| but not sufficient for psychology.
| woeirua wrote:
| If RL is enough, then there is no physically realizable way to
| actually train a RL based GAI in the near future. RL based
| learning requires evaluating the outcome of millions or billions
| of scenarios over time in order to optimize the network.
|
| Given that requirement you'd have to either find a way to
| accurately model the world and all of those interactions in
| silicon, or you'd have to build millions of robots that can
| report back the results of billions of interactions each day.
| It's not impossible to do that, and maybe it would even be likely
| that we would eventually accomplish that but the cost would make
| it prohibitive for anyone but a nation to even attempt today.
| It's almost certainly outside the realm of what is possible in
| the near future. Maybe when robotics has progressed enough that
| robots are capable of interacting with the world with basic AI
| will we see the rise of something like a GAI.
| littlestymaar wrote:
| Being "enough" isn't practical though and whether it's
| sufficiently efficient is still an open question.
|
| We already know that evolutionary trial and error is "enough" to
| create flying animals, yet with a little bit of domain knowledge
| we've been able to build planes way faster than nature has built
| flying insects.
| ramraj07 wrote:
| Wait isn't any Turing complete programming language sufficient to
| eventually reach general AI
| tsimionescu wrote:
| Technically that is a conjecture known as the Church-Turing
| thesis, which states that any computable function can be
| computed by a Turing machine. Here "computable function" uses
| the informal definition 'any function that a human can
| compute', not the formal definition of the recursive functions
| (which are proven to be solvable by a Turing machine).
| jensgk wrote:
| Yeah, I would even say a large amount of NAND gates should do
| it.
| randcraw wrote:
| Like the notion that all of natural language can be modeled
| fully if you just use enough finite state machines -- surely
| true; just a wee bit inefficient.
| virgilp wrote:
| Wait no, that can't be true. Finite state machines are only
| as powerful as regular expressions; you need pushdown
| automata even for programming languages, surely you can't
| model natural language with finite state machines? (maybe an
| infinite number of them, but I'm doubtful even on that - I'd
| have to review the theory to be sure though)
| frostburg wrote:
| It is true; keep in mind that real computers are strictly
| speaking FSMs, not Turing machines, due to having a finite
| "tape". It's just vastly more useful to think of them as
| such when programming.
| skybrian wrote:
| In practice, both people and machines can only handle a
| finite amount of nesting, so you could do it though it
| would be awkward to express.
| robertlagrant wrote:
| Perhaps the fact that some words can never follow other
| words would make it tractable.
| joe_the_user wrote:
| If you had a custom chip it could be efficient. Determining
| which state machines would still be hard.
| ma2rten wrote:
| Maybe, but that is not helpful in the sense that it doesn't
| guide research in how to get there.
| otabdeveloper4 wrote:
| You're assuming that intelligence is a computational process,
| but the sum total of what we know about intelligence says it
| probably isn't.
|
| (Unless you're making a more general reductionist statement
| that everything in the universe is a computational process -
| that kind of reductionism is understandable coming from people
| who work with computers for their job - but this is then a
| philosophical stance, not scientific, and frankly a very
| strange one.)
| anyfoo wrote:
| > but the sum total of what we know about intelligence says
| it probably isn't
|
| Source? I am not aware of any other known process in the
| universe that could not be simulated by a Turing machine.
| otabdeveloper4 wrote:
| Assuming that the universe can be simulated by a Turing
| machine is a strong and weird claim that needs to be
| defended, not the other way around.
|
| We know that Turing machines are very limited things and
| that the computational processes they carry out also very
| limited in applicability.
|
| What's the evidence that the universe is more limited that
| a Turing machine?
|
| Just the fact that we can imagine things that can't be
| computed by a Turing machine should clue you in that it's
| probably otherwise.
| anyfoo wrote:
| > We know that Turing machines are very limited things
| and that the computational processes they carry out also
| very limited in applicability.
|
| Again, source? Do you know of anything that is able to
| perform a computation that a Turing machine cannot?
|
| > Just the fact that we can imagine things that can't be
| computed by a Turing machine should clue you in that it's
| probably otherwise.
|
| Like what? Uncomputable numbers like Chaitin's constant?
| We can "imagine" them by stating their definition, but we
| cannot compute them. Or do you have something else
| specific in mind?
| otabdeveloper4 wrote:
| > Do you know of anything that is able to perform a
| computation that a Turing machine cannot?
|
| That's a circular argument, because "computation" is
| _literally_ defined as "something that can be computed
| by a Turing machine".
|
| That said, the first month of the first year of a CS
| education is "here's these problems that can't be solved
| by a Turing machine, mind=blown". (At least where I
| studied CS, that is.)
| gammalost wrote:
| >Like what?
|
| The halting problem is (I think?) the standard example.
|
| Here's a short video on it https://youtu.be/macM_MtS_w4
| anyfoo wrote:
| Chaitin's constant is closely tied to the halting
| problem: Each bit essentially tells you if a program (in
| the particular order the programs are listed in the
| constant) halts or not. Computing every bit in there
| would mean knowing for every program if it halts or not,
| which would solve the halting problem, which we can prove
| is impossible. So we cannot compute that constant.
| mensetmanusman wrote:
| A Turing machine needs a discrete clock cycle, if
| intelligence requires quantum mechanics that has
| entanglement through time, that would be one example.
| twhitmore wrote:
| While I am agnostic as to the advantages of RL over simpler ML,
| some basic structural concerns seem to apply here. The potential
| concern is that if RL works, the very best factors for learning
| are likely to be potentially dangerous.
|
| The best environment for learning is the richest -- ie.
| unrestricted access to be the entire Internet/ or the world via
| the Internet.
|
| The best reward function is quite likely to be reproduction. If
| the researcher allows moderate action but attempts to limit
| reward functions, at a certain level they may find the AI has
| found a better reward.
|
| If the claimed advantages to learning of richer environments &
| richer action capability are even somewhat true, researchers are
| likely to be strongly incentivized to pursue such risky
| structures.
| hervature wrote:
| I guess things are slowing down at DeepMind. I have tremendous
| respect for David Silver and his work on AlphaZero and Richard
| Sutton as a pioneer in RL. But the cynic in me is that this paper
| is just a result of Goodhart's law with publishing count as a
| metric. Any proof of the type of emergent behaviors that they
| mention from RL with an actual RL experiment would go a long way.
| Showing an RL agent developing a language would be extremely
| interesting. It makes me think they tried to show these emergent
| behaviors but could not and thus ended up with a hypothesis.
| ChicagoBoy11 wrote:
| Very tangential, but as someone who has gotten into the Game of
| Go because of their pioneering project in that space, I'm
| exceptionally grateful -- that alone had a very significant and
| positive impact on my life, and I can tell that in that entire
| community it was a watershed moment as well.
| maiodude wrote:
| They just "solved" protein folding late last year. How can you
| say things are slowing down? Do you honestly expect life-
| changing discoveries every other week?
| 0xB31B1B wrote:
| Protein folding is a well modeled math problem. The alpha
| fold solution is extremely good at pruning (aka guessing)
| folding chain structure possibilities. I am Impressed and
| this is a difficult problem but this is extremely different
| from AGI as this is a well scoped easily modelable problem
| that is basically a chain of 26 inputs types of links of
| arbitrary length. I am not trying to take away that the
| protein folding is incredible but AGI is extremely different.
| AGI is literally having a model that can both do alpha fold
| and self driving cars, as well as the ability to generate
| novel models to solve new well scopes problems. RL can do 0
| to 1, the 1 to n (generalizeability) is the extremely
| difficult part.
| ackbar03 wrote:
| I may be really only speaking for myself here but I have
| very sincere doubts that anyone who has done any moderately
| serious work on "AI" and wrangled with the nitty gritty
| details of it all is really having any huge expectations
| for AGI. I mean if it comes during my lifetime, hurrah! But
| personally I'm not gonna sit around waiting for it or
| depend on it for anything. That being said, is the current
| AI tech as we have it useless? Of course not. Things like
| protein folding and alphago are still huge leaps forward in
| tech, it'd be kind of silly to treat AGI as the only thing
| worth achieving
| tinco wrote:
| It is entirely not necessary for an AGI to be able to drive
| a car.
|
| Frankly, after seeing AlphaZero and AlphaFold I'm surprised
| they didn't declare AGI right there and then.
|
| People assume that when AGI happens, computers can suddenly
| outsmart humans in every way and solve every problem
| imaginable. The reality is just that it could in theory
| given enough time and resources.
|
| It is like quantum computing. In theory it can instantly
| factor and break our nice cryptographic primes. In reality
| the largest number it factored is 21.
| [deleted]
| qsort wrote:
| > it could in theory given enough time and resources.
|
| In theory given enough time and resources, anyone can
| defeat any grandmaster in Chess: just compute the
| extended tree form of the game and run the minimax
| algorithm.
|
| The "given enough time and resources" clause makes
| everything that follows meaningless, unless a reasonable
| algorithm is presented.
|
| > It is like quantum computing.
|
| It is absolutely not like quantum computing. Shor's
| algorithm is something you can look up right now. It is
| precise and well-defined. The problems we are facing with
| quantum computation are related to the fact that we can't
| really build reliable hardware. But we know that given
| such machines the algorithm would work. We have precise
| bounds and requirements on those machines.
|
| As far as AGI goes, we have absolutely no idea. There's
| lively debate on whether anything we have done even
| counts as significant advancement towards AGI.
| tinco wrote:
| > In theory given enough time and resources, anyone can
| defeat any grandmaster in Chess: just compute the
| extended tree form of the game and run the minimax
| algorithm.
|
| Yes, that's why we're considered to be generally
| intelligent. It is exactly the point, and not at all
| meaningless. Right now there's no machine that can come
| up with the idea to run an extended tree form of the game
| and minimax the algorithm. If there was such a machine,
| then that machine would be considered AGI.
|
| > It is absolutely not like quantum computing.
|
| I meant in the sense that just that it has actually been
| achieved, it doesn't mean it's as powerful as we have
| described in the theory. In theory you can use Shor's
| algorithm to break encryption, in practice the devices we
| have today have trouble with 2 digit numbers.
|
| The same principle goes for AGI. If someone releases an
| AGI system today, it doesn't mean that tomorrow we'll see
| a Boston Dynamics robot hop on a bicycle to his day job
| as a Disney movie art director. The world would most
| likely not change at all, at least not for a while, many
| people would not recognise the significance and many
| people might not even recognise the fact that it is in
| fact AGI.
|
| > As far as AGI goes, we have absolutely no idea. There's
| lively debate on whether anything we have done even
| counts as significant advancement towards AGI.
|
| You might think that, and that says something about what
| side of the debate you're on. We're commenting here on
| the thread of an article about DeepMind asseting that
| reinforcement learning is enough to reach general AI. If
| that's true (and I think it is), then we've probably
| reached general AI already.
| croon wrote:
| > It is entirely not necessary for an AGI to be able to
| drive a car.
|
| "Artificial general intelligence (AGI) is the
| hypothetical[1] ability of an intelligent agent to
| understand or learn any intellectual task that a human
| being can."
|
| What definition are you using?
| tinco wrote:
| The same, that they understand and can learn how to drive
| a car doesn't mean they would actually be able to do it
| in the real world.
|
| You can read a book on how to hit a ball with a baseball
| bat, you can even practice and get good at it, but that
| still doesn't mean you would actually be able to hit a
| ball thrown by a professional pitcher.
| croon wrote:
| The interface to the car is a solved problem.
|
| > You can read a book on how to hit a ball with a
| baseball bat, you can even practice and get good at it,
| but that still doesn't mean you would actually be able to
| hit a ball thrown by a professional pitcher.
|
| If I hade incredibly fast reflexes and actuators I could.
| tinco wrote:
| Similarly, DeepMind's software might be able to drive a
| car, would it have a similar neuron count, connectivity,
| perception systems and training you received.
|
| Or maybe it couldn't, because the software is not as
| efficient as the organisation of your brain is. Or
| because there's hardcoded routines evolved in your brain
| that it lacks.
|
| What I'm saying is that just that because an AGI can't
| drive a car, it doesn't mean it's an AGI. For the same
| reason there's loads of people out there that are
| generally intelligent that can't drive cars for all sorts
| of physical reasons.
| croon wrote:
| > Similarly, DeepMind's software might be able to drive a
| car, would it have a similar neuron count, connectivity,
| perception systems and training you received.
|
| Admittedly I'm a layman in this area, but could it?
| AFAICT it would only work on trained set data and
| whatever generalizations can be made on that and not
| infer unseen scenarios like humans do readily.
|
| > What I'm saying is that just that because an AGI can't
| drive a car, it doesn't mean it's an AGI.
|
| I understood what you meant from your first post, I'm
| simply disagreeing on account of the very definition of
| AGI.
|
| You can't have an ameoba level AGI and still call it (a
| limited) AGI. Either it can understand/learn any human
| task, or it can't.
|
| The definition is made for a reason. Watering it down for
| any specific generation of AI serves no benefit.
| uyt wrote:
| It might be a stretch but some people say that the weights
| learned by a neural network is somewhat like a language. For
| example if you look at the weights of a random middle layer it
| would seem like gibberish. Much like how aliens would react
| when looking at humans making gibberish noises (aka talking) to
| each other. In both cases they are just compressing signals
| based on learned primitives.
| mirker wrote:
| Not sure if there is any case where this thought is useful.
| The only thing this says is the primitives are correlated and
| we don't understand them. It's similarly not useful to think
| about atoms "talking" when they exchange heat.
| sam-2727 wrote:
| By far not an expert in this area, but if (when?) we successfully
| generate true intelligence, I suspect it will be through some
| sort of "ensemble" model, where multiple agents are trained in
| parallel and interact with each other. Intelligence as we know it
| hasn't just resulted from an evolution of one agent in response
| to a cost function, but rather through the complex interactions
| of agents (humans and organisms in general) over time. I feel
| like the underlying journal article
| (https://www.sciencedirect.com/science/article/pii/S000437022...)
| is missing discussion of this.
| abeppu wrote:
| A lot of comments in this thread are talking about how
| unsatisfactory this paper is, because of course rewards are
| enough for _some_ agent, but this paper doesn't venture to say
| anything about that agent (e.g. what differentiates humans from
| squirrels even though both are trying to eat and reproduce?).
|
| But I think though they talk about rewards incessantly, the
| interesting angle is the importance of a complex environment in
| which the agent learns to maximize a reward:
|
| > we suggest the emergence of intelligence may be quite robust to
| the nature of the reward signal. This is because environments
| such as the natural world are so complex that intelligence and
| its associated abilities may be demanded by even a seemingly
| innocuous reward signal.
|
| On the one hand, this does echo some old work on situated
| cognition, which one might actually believe. But perhaps
| politically, if the claim behind the claim is that we can only
| develop powerful AGI which understands how to interact with our
| world by developing agents that learn with unfettered access to
| the world, then perhaps this will be the beginning of a strong
| push for tolerating spastic ineffective robots in our physical
| environments, and letting error-prone agents have vast access to
| our virtual environments. We'll be asked to put up with their
| mistakes because that's supposedly the cost of progress; limiting
| their environment would limit their cognitive potential.
|
| > For example, consider a signal that provides +1 reward to the
| agent each time a round-shaped pebble is collected. In order to
| maximise this reward signal effectively, an agent may need to
| classify pebbles, to manipulate pebbles, to navigate to pebble
| beaches, to store pebbles, to understand waves and tides and
| their effect on pebble distribution, to persuade people to help
| collect pebbles, to use tools and vehicles to collect greater
| quantities, to quarry and shape new pebbles, to discover and
| build new technologies for collecting pebbles, or to build a
| corporation that collects pebbles.
|
| Aren't you comforted they chose round pebbles instead of
| paperclips? And though their example is meant to illustrate that
| the reward function doesn't matter, you'll notice there's no
| negative reward term for e.g. smashing a retaining wall to dig
| for pebbles in the rubble, or dredging a beach where a protected
| bird species nests, etc. "Allowing the agent to fully explore the
| complex environment is the only way it will learn complex
| representations and actions!"
| robomartin wrote:
| Yeah. No. No way.
|
| My son and I were discussing the state of AI/robotics yesterday,
| as we walked a beautiful trail in the Sequoia National Forest.
|
| What prompted the discussion was a simple question:
|
| What would it take to build a robot capable of navigating these
| trails as we do?
|
| This would be a robot able to do this in a manner
| indistinguishable from, say, a ten year old human.
|
| No GPS, maps, compass, pre-mapping, lidar, ultrasonic sensors,
| etc. Just vision, hearing and touch/force sensing at the "skin"
| and articulations.
|
| What do you know?
|
| Two things: You are located at the start of the correct trail and
| there's a waterfall at the end.
|
| Our conclusion was equally short and simple: Today, it is
| hopelessly impossible to match what a ten year old kid could do
| on that trail.
|
| Maybe in ten years. Maybe.
|
| AI today can't do what a ten year old human, or the young bears
| we so along the trail, can do instantly and without thinking:
| Understand.
|
| We just don't know how to approach and encode understanding yet.
| harry8 wrote:
| Some Bozo who has heard all this many times before is suspicious
| of claims from places like Deep Mind who have a financial
| incentive to make them (keep funding) where there aren't working
| machines to back that claim up.
|
| Some Bozo has no credentials, no reputation, no track record of
| publications and barely supports the claim they're making with
| anything much. Some Bozo has no financial incentives or otherwise
| to opine either way. Some Bozo doesn't even work in the field at
| all.
|
| Bets: Some Bozo or Deep Mind turn out to be closer to being
| correct in the passing of some finite amount of time? 5 years? 10
| Years? 25 Years?
| dqpb wrote:
| DeepMind is arguing from first principles. SomeBozo is arguing
| by analogy. DeepMind will achieve something and SomeBozo will
| achieve nothing.
|
| The vast majority of ideas are wrong. Every idea is wrong until
| it leads to the one that is right.
|
| This idea might be the right one, or it might be close to the
| right one, or it might be far from the right one, but the
| trajectory is headed toward the right idea. SomeBozo has no
| trajectory. The best he can do is watch from the sidelines.
| graderjs wrote:
| Plot-twist: Deep Mind comes out as Some Bozo comment author
| ausbah wrote:
| Bozo et al
| harry8 wrote:
| Well my writing style has been the subject of abuse on this
| website in the precise form that it resembles words generated
| by a bad algorithm.
|
| "Only a true AI would deny their being."
| graderjs wrote:
| I awoke wondering am I a man dreaming I am an AI? Or an AI
| dreaming I am an man?
| tsimionescu wrote:
| The fact that RL in the extremely vague sense used in the
| article is enough for AGI is uncontroversial for anyone who
| believes intelligence and consciousness are physical processes.
|
| However, this "result" is trivial. It is obviously equivalent
| to the claim that intelligence arose naturally in the
| biological world without influence from God.
| rollcat wrote:
| > It is obviously equivalent to the claim that intelligence
| arose naturally in the biological world without influence
| from God.
|
| Where did God's intelligence come from?
| tsimionescu wrote:
| Well, I don't believe God exists, so I can't really answer
| the question.
| [deleted]
| dqpb wrote:
| From the authors imagination.
| tarsinge wrote:
| Nowhere, that's kind of the definition of God (for a
| Christian at least): it always was, and is the ultimate
| origin. It is different than a direct influence in the
| world.
| Frost1x wrote:
| The problem with this, specifically the assumption that RL
| gives an equivilance to natural selection and evolution, is
| that RL typically assumes a computational environment it
| interacts in while natural selection and evolution assumes
| the physical world as the environment.
|
| The important difference here is that in order for RL to
| translate to solving real world problems, you need to
| faithfully and computafionally simulate the real world's
| physical processes and rules, or at least enough that n-th
| order processes exist accurately.
|
| I've done various types of computational modeling and
| simulation work at different scales throughout my career with
| all sorts of scientists and engineers and I can tell you,
| pretty much no domain is there where you have good enough
| representative models RL can be used in. Some narrow special
| cases exist but nothing to the degree of a massive
| environment full of well coupled expert domain models. Some
| of the best cases are going to be so computationally bound
| that it would be quicker to do things for real vs simulate.
|
| If you want RL to work and learn, it's likely possible under
| the connection you point out, but has to do this using
| physical machines and sensors interacting with the physical
| world like life as we know it does. Your AGI won't be able to
| cheat and run through the evolution process quicker using
| faulty reductionist models we use in most simulations (which
| is what everyone implicitly is hoping for), IMHO.
|
| If you try this, your AGI is going to learn all sorts of
| flaws within those environments or at the very least, have so
| many narrow scoped bounds it won't be that "general." _A lot_
| of simulated models are frankly garbage (they have some
| useful narrow scope but are typically littered with caveats)
| and they 've been in development pretty much since digital
| computing began.
| tsimionescu wrote:
| You are absolutely right about RL in practice. In fact, if
| we were to look at actual RL algorithms, I believe the
| paper's claims fall flat in many other ways. This is
| actually my criticism of it: its arguments are only
| convincing when RL is defined only as the extremely general
| notion of an agent seeking to maximize some reward function
| by interacting with an environment. This is so comically
| general that the only alternative I can think of is to
| posit a transcendental god.
|
| Once we get into the details, their claims stop being iron
| clad. Even worse, some of their claims become actually hard
| or impossible to accept if applied to actual RL algorithms
| we have today. You give one good example with the
| difficulty of modeling the world. The implicit claim they
| make that this would be realizable in reasonable time (say,
| less than a billion years) is also not well supported. The
| idea that humans or mammals learn their social behaviors
| through RL rather than a good deal of reasoning from
| evolutionarily-trained first principles pretty clearly
| fails in the face of the poverty of the stimulus
| argument[0].
|
| Overall, the claims in the paper tend to switch between
| obvious (if taken to talk about the general idea of
| maximizing reward) to almost certainly wrong (if taken to
| talk about known RL algorithms, reasonable time frames, and
| specific examples of what is supposed to be learned).
|
| [0] the poverty of the stimulus argument may be
| controversial in linguistics where it was first formulated.
| Still, if applied to mammal or insect socialization, the
| extremely low time frames in which individuals of a species
| start exhibiting typical behaviors basically proves in my
| opinion that they are instincts, trained at the population
| level through evolution, not individual learning through
| RL. The extreme similarity of behavior between individuals
| of the same species, VS the variety of behaviors between
| different species, also suggests an important component of
| species-level rather than individual level learning.
| cscurmudgeon wrote:
| There is another set of people. There are people with solid
| track records in AI and ML that disagree with DeepMind.
| jacquesm wrote:
| Said anonymous account on HN... If you're going to question
| other people's credentials, reputation, track record and claims
| make sure your own are solid. Those who live in glass houses
| shouldn't throw stones.
|
| Finally, if you're going to attack someone's article: attack
| the article, not the person that wrote it. This is the lowest
| level of attack possible: the personal one. It's as ad-hominem
| as it gets.
| blcArmadillo wrote:
| If I read it right harry8 is referring to themselves as the
| Bozo.
| thecupisblue wrote:
| > Those who live in glass houses shouldn't throw stones.
|
| So if we're gonna have an opinion we need to do the whole
| academia & job in the industry dance?
|
| That's quite a terrible way to view the world and quite
| limiting. A world without diversity is a stale and rotten
| world.
|
| So fuck that and the glass houses and the boxes this kind of
| worldview puts people in. Everyone should be able to throw
| stones, and if the hit hurts, well guess there is a reason.
|
| The thesis is that DeepMind has financial incentive to state
| "we can achieve AGI with what we're doing", to keep up the
| funding and hopes for the field, not "the author is an
| idiot".
|
| And the thesis is true, they do have financial incentives.
| That's not ad-hominem.
| rusk wrote:
| Edit: sorry just realised you're making the same point as me
| more or less. Putting yourself in third person. I'll let my
| comment stand anyhow :)
|
| Screwing my face up, looking at this sideways ... but it
| _seems_ as though you're saying that the Bozos of HN have
| nothing useful to contribute to this discussion based on ...
| [rereads] ... their lack of academic credentials in the area...
| you could say this about just about any HN post I'm just
| wondering why this one? Here's a thing though ... if the
| understanding of a technology is so nuanced ... that Bozos
| can't "get" it ... is it really that mature? We had functioning
| computers for 50 years but it was only when the Bozos got their
| hands on it that things took off. Internet for 20. Cell phones
| for 10. How long are we dabbling with neural networks? 50 years
| or so? All I see in this most recent explosion in AI is a rapid
| jump in the availability of cores. Ala Malthus once that newly
| available "source of nutrition" has been used up we will see a
| rapid die off once more and it will be another 20 years once
| the Bozo intellect has caught up before we look at this topic
| en masse again. Dismiss the Bozos at your peril. You're
| dependent on them for innovation and consumption. Your
| sincerely, a Bozo.
| harry8 wrote:
| Not quite the same point. Yep some bozo is me but needn't be.
| There's plenty who share that suspicion of AI research but
| have little else in common. And all of us may be wrong for
| different reasons.
|
| The vague point was to show someone with zero reputation,
| credentials, specific expertise in the field or anything much
| seems to be pretty convincing in response to this hugely
| funded ivory tower exercise by spitting, cocking an eyebrow
| and saying "So you think so, eh? Wanna bet?"
|
| This is a statement about the state of AI research
| credibility. Do you feel the first breezes of a deep AI
| winter coming on? (I don't know, I'm disinterested but not
| uninterested. Rising tides lift all ships etc. And vice
| versa). Neutral nets are cool. Is all ML a bit overrated? Is
| learning a misleading name to give to applied statistics?
|
| I don't have answers, just suspicions. I could be very wrong,
| of course.
| rusk wrote:
| I've a minor in psych so I like to think I have a bit of a
| non-techy perspective on this, and what's being pushed now,
| forms just a segment of the overall topic of AI. It just so
| happens to be the segment that benefits from the technology
| we suddenly have a rapid increase in. There's been great
| successes in areas where a degree of _inference_ is
| required but this hardly qualifies as even mere
| intelligence, and in cases where neural nets have been
| deployed in more human centered tasks, or even well
| designed symbolic systems the results speak for themselves.
| What even is intelligence? I think we're going backwards
| because we're investing all this talent in this simple
| segment I fear largely to fatten the chip makers share
| price while neglecting tried and true approaches that
| deliver far better results but perhaps _crucially_ have a
| higher operating cost ... who remembers google of 2010 from
| whom the Internet in all her glory leapt forth, or iPhone
| spell check of 2015 where you could confidently batter out
| your messages with little fear it would make a fool of you;
| you're not going to nurture a nascent intelligence if
| you're going to be continually hobbling it for business
| reasons. I'm certain we will get there eventually if we
| don't destroy ourselves before then but I don't think the
| current trends portray a picture of how it will be. I think
| we have a long way to go ourselves before we can be worthy
| of creating our successor, but when /if it comes it will be
| a beautiful thing and we will embrace it as we would our
| own child.
| A-Train wrote:
| The algorithms to train, initialize the networks, new
| architectures are far more important than the hardware
| advances. If people knew how to train NNs 50 years ago we
| would live in a different world.
| ausbah wrote:
| the basic concepts underlying DNNs have been known for
| decades, it has been exponential increases in compute power
| that have made them practical
| chrisco255 wrote:
| We did. they just didn't have the same computation
| abilities back then.
| [deleted]
| harry8 wrote:
| I find it really interesting that when Richard Feynman
| did a sabbatical at Thinking Machines when they were
| developing the early parallel execution hardware that's
| really not worlds away from modern GPUs he got them in
| touch with one of the leading neural network theorists as
| an obvious use for the tech. When he wasn't fixing their
| hardware designs using systems of differential equations.
|
| It would be an interesting thing to know more about.
| KeplerBoy wrote:
| Unfortunately i'm rooting for the Bozo, the current AI
| Revolution won't lead us anywhere and will ebb down eventually.
| eru wrote:
| What do you mean by 'won't lead us anywhere'?
|
| It might or might not give us AGI. But it is already leading
| us to lots of places. Eg speech recognition even on my phone
| works way better than what I had twenty years ago on a
| Desktop.
| davewritescode wrote:
| It won't ebb down. Eventually we'll hit limits of what's
| practical on current hardware and we'll be back to the 70's
| and 80's when everything becomes theoretical until hardware
| catches on. AI is going to continue to advance.
|
| What _will_ happen is that capital will become more skeptical
| about the limits of what 's feasible with AI and it'll be
| harder to sell bullshit. You're already seeing that with
| companies like Uber selling off their self driving divisions.
| ggggtez wrote:
| Wrong: Some Bozo does have a stake.
|
| The existence of human crafted general AI forces him to
| struggle with the possibility that there is no such thing as a
| soul.
|
| I know a lot of people don't fall in that camp, but I heard
| enough "serious" people make such desperate claims to avoid
| thinking about the topic in a way that might challenge their
| underlying religious beliefs[1]. I think no one likes to admit
| that religion and spirituality often force someone to reject
| the possibility that AI is actually really much simpler than
| they think it "should" be, because then humans aren't special
| after all.
|
| [1] Numerous arguments boil down to an argument that complexity
| is non reducible. You see it here, hidden in various comments
| as well.
| otabdeveloper4 wrote:
| I'll bet a sum of real money that Some Bozo is correct.
|
| Bozo has the hindsight of history and philosophy going for him,
| while Deep Mind has a huge financial temptation to sell snake
| oil.
| for_i_in_range wrote:
| "Show me the incentives I'll show you the outcome."
|
| _Google make money, Google Bad._
|
| _Deep Mind owned by Google, Deep Mind bad!_
|
| The above conclusion stands trite.
|
| Perhaps the inverse is true.
|
| Google and Deep Mind, if correct, could be hurting themselves
| more than helping themselves.
|
| Why? Creating a future species who's too smart to click on
| ads, and too smart to remain subject to its whims, doesn't
| sound like it'd be good for quarterly profits...
|
| There's also the emotional incentive for humans to confirm
| their own beliefs about humanity being _special_.
|
| If Google/Deep Mind knows this, yet publishes research anyway
| in the spirit of truth, why, what they're doing may be
| considered heroic.
|
| Two sides of the coin here.
| rocgf wrote:
| No offence, but I think you are extremely wrong.
|
| Creating an AGI is the endgame for everything. Who cares
| about ads when you have an AI that can learn to do anything
| and improve upon itself continuously?
| for_i_in_range wrote:
| "Who cares?" Well, the people who need to pay the people
| developing the _endgame for everything_ you speak of.
| rocgf wrote:
| I'm sorry, but this makes no sense to me.
|
| The people paying for the development of the AGI can mean
| many things - the Google customers/users, Alphabet as a
| company, the executives throwing money at the problem?
|
| Either way, I don't really get your point. Your initial
| post was about how it is counterintuitive for Google to
| allocate funds for an AGI, since it makes money out of
| ads. These are not mutually exclusive, you can have both,
| but my point is that if you develop an AGI, then you can
| pretty much "conquer" the world and revenue from ads
| becomes irrelevant.
| for_i_in_range wrote:
| How do you think they can conquer the world? How do you
| foresee governments not restricting a private company's
| new powerful tool?
| kordlessagain wrote:
| That's not going to happen until they have bodies.
| croon wrote:
| You don't know if an AGI will agree with your profit
| motives.
| rocgf wrote:
| There is a huge assumption baked into your comment and I
| do not agree with it.
|
| AGI does not necessarily require for it to be conscious
| or throw tantrums about its creators' purpose. AGI just
| means that it's an intelligence that can be thrown at any
| problem, not just a particular game or task, similar to
| how humans can specialize in CS or playing the violin.
| croon wrote:
| Sure, it was somewhat tongue in cheek, but not entirely.
|
| There is a semi-established definition that does include
| what I referred to:
|
| > AGI can also be referred to as strong AI,[2][3][4] full
| AI,[5] or general intelligent action.[6] Some academic
| sources reserve the term "strong AI" for computer
| programs that can experience sentience, self-awareness
| and consciousness.[7]
| zcw100 wrote:
| Really? I created two GI's and it wasn't very hard and
| was actually quite fun. Training them is a bit of a pain
| though. I'm willing to bet that based on total calories
| consumed they are amazingly efficient compared to their
| hypothetical AGI counterparts.
| FredPret wrote:
| Yes but can they:
|
| - live forever
|
| - grow their own mental capabilities exponentially over
| that unlimited lifespan
|
| - turn themselves into universe-eating von Neumann probes
| rsj_hn wrote:
| Nothing can
|
| - live forever
|
| - grow exponentially forever
|
| - "eat the universe" (I know, the last point was sci-fi
| gibberish)
|
| In fact, humans are already pretty good at reproducing
| themselves and have managed to travel to space, and have
| exhibited finite periods of exponential knowledge growth
| combined with periods of collapse, as nothing grows
| exponentially forever.
| mensetmanusman wrote:
| My three-year-old said yes to all questions
| TeMPOraL wrote:
| I'll happily bet fake internet points instead:
|
| https://www.metaculus.com/questions/create/
|
| EDIT: bunch of other related predictions currently open:
|
| https://www.metaculus.com/questions/?order_by=-activity&sear.
| ..
|
| https://www.metaculus.com/questions/?order_by=-activity&sear.
| ..
| marcescence wrote:
| A quote from datscilly, the top forecaster on metaculus:
|
| >AGI may never happen, but the chance of that is small
| enough that adjusting for that here will not make a big
| difference (I put ~10% that AGI will not happen for 500
| years or more, but it already matches that distribution
| quite well).[1]
|
| [1]:https://www.lesswrong.com/posts/hQysqfSEzciRazx8k/forec
| astin...
| okprod wrote:
| _while Deep Mind has a huge financial temptation to sell
| snake oil_
|
| I don't know, isn't the DeepMind founder that guy in the Go
| documentary? I read about him after watching the doc and he
| seemed to be pretty cautious about taking in investment, and
| he didn't seem the type to try to cash out.
| ocdtrekkie wrote:
| He already cashed out, he sold to Google. And over the
| years Google has ramped up the pressure for DeepMind to
| deliver financial returns. (I recall when Google tried to
| stick DeepMind's branding on GCP, Watson-style, so it would
| sell better, and at the time, DeepMind was able to
| decline.)
|
| Eventually Google will give them the option to deliver
| financial success or be shut down.
| lvncelot wrote:
| Some cynic remarks that during the first AI golden years,
| claims of imminent success seemed to come from a place of
| hopeful naivete of a fledgling science, whereas those same
| claims nowadays seem to come from a place of cold calculation
| of a booming business.
| rich_sasha wrote:
| Is Deep Mind a "booming business"? They are achieving great
| things academically, but their business successes are either
| kept secret or mostly absent. All I know about is the Google
| data centre cooling scheduling, probably a big saving for
| Google but hardly an achievement that in its own professes
| their _business_ success.
| d110af5ccf wrote:
| Deep Mind is cutting edge ML in general, right? Doesn't
| Google actively apply the lessons learned all over the
| place? YouTube content recommendation stands out to me in
| particular. Translation and automated closed captioning are
| also obviously ML based. I'd guess that most of the really
| interesting stuff would be behind the scenes and not
| immediately visible to end users though.
| zimpenfish wrote:
| > YouTube content recommendation stands out to me in
| particular.
|
| If that's "cutting edge ML", then going off my YouTube
| recommendations, we're back in another AI winter. If I
| watch one video from a channel I've not seen before, I'll
| get that channel recommended _constantly_ even if it
| bears no resemblance to what I normally watch. On my
| Explore page, the first 22 videos (of which 8 are
| Fortnite-related!) hold no interest for me. My Home page
| is just channels I 've watched repeatedly and/or am
| subscribed to. It's a mess.
| mgoetzke wrote:
| Is the Explore page controlled by videos I watched ?
| Because there isn't a single video on it i would watch.
| Not one.
| 666lumberjack wrote:
| How often do you use YouTube? Personally I am a very
| heavy user and in my experience the obsession with a new
| video kind you watch only lasts for a few recommendations
| unless you lean into it.
|
| I would guess about two thirds of the channels I
| consistently watch I originally discovered through
| algorithm recommendations. I think it works extremely
| well.
| dkjaudyeqooe wrote:
| That's because you fit into YT's conception of how
| viewers behave. For people who don't fit into
| "normal"-ish behaviour it has little utility.
|
| For me, probably 90% of what I watch I'm not interested
| in and often I'm repelled by. This is because I mostly
| watch to find out what things I'm not familiar with are.
|
| For example let's say I'm a liberal. I'm not going to
| watch liberal political videos because I know generally
| what they're going to say and I don't need my political
| views stroked in order to be happy. But I will watch
| various other political videos, no matter how extreme or
| not, so I can be at least a little familiar with their
| behaviour and views.
|
| YT can't cope with this. To their systems I seem to be
| randomly picking videos with no correlation with the
| subject matter or other users and no reinforcing pattern.
| It just gives up and recommends things based on the
| behaviour of the general population, as if they had no
| data on me at all.
| burning_hamster wrote:
| I think you raise an important point. The youtube
| algorithm is pretty bad if you don't use youtube very
| much or only use to consume very popular content.
| Youtube's recommendations used to be terrible for me,
| too, but sometime last year I crossed a threshold and
| since then it has been recommending a lot of small,
| highly specific channels that nevertheless are great
| fits. My wife's recommendations are still utter garbage
| though.
| zimpenfish wrote:
| > How often do you use YouTube?
|
| Every day, averaging 2-3 hours. It's background for
| working and foreground for evening viewing.
| rich_sasha wrote:
| I'm stressing the _business_ part. YouTube is a loss-
| making business year after year. Deep Mind gloss doesn't
| seem to change that.
|
| If indeed it even is Deep Mind making those improvements,
| Google has lots of other ML groups, such as Google Brain,
| and these are more directly focused on Google products.
|
| There's no denying their _academic_ success, or game
| playing etc, but as far as I can see, the data centre
| cooling bit is the only palpable (public) business
| success.
| mirker wrote:
| Deep mind's protein folding algorithm is probably worth a
| chunk of change. As far as I know, they've been holding
| onto the secret sauce rather than publishing it.
| eru wrote:
| How do you know that YouTube is a loss making business?
| tarvaina wrote:
| YouTube made $6B revenue in Q1. [1] While they don't
| release profit numbers, it would be pretty surprising if
| they were negative.
|
| Did you mean to write DeepMind instead? If so, I don't
| disagree.
|
| [1] https://www.cnbc.com/2021/04/27/youtube-could-soon-
| equal-net...
| throwaways885 wrote:
| They have an applied division which applies ML to Google
| products. I suspect they are very valuable in $ terms
| just for the work listed here:
| https://deepmind.com/impact. Google's entire business
| from the start was doing research and bringing it to the
| masses, so this shouldn't really surprise anyone.
| thu2111 wrote:
| No not really. Deep Mind is almost all cutting edge
| agent-oriented reinforcement learning, hence the nature
| of the claim they're making. The impact on Google's
| business from AI has come almost exclusively from other
| kinds of ML, or that's at least how it appears from the
| outside. E.g. replacing Google Translate with neural
| translation doesn't seem to involve RL and certainly
| doesn't involve agents playing video games.
|
| Deep Mind is best understood as the following bet: if we
| can train an AI that can learn from "its environment" and
| do the sort of things a human would do in that situation,
| then we have achieved AGI and from that ... business ...
| will follow. Hence their focus on video games as a
| training environment.
|
| This sounds intuitive but is actually a very agent-
| centric viewpoint and most AI doesn't resemble this type
| of thing at all. Most AI deployed so far doesn't have
| anything resembling an environment, doesn't have any kind
| of nexus of agency and doesn't need to actively make
| decisions that then feed back to its own learning, only
| make probabilistic predictions. And in fact you often
| don't want an ML model to train on the outcomes of its
| own decisions.
| FlyingSaucer wrote:
| Yes, its hard to tell the exact algorithmic underpinnings
| of production models that Google uses but you have to
| assume that although they have some done some impressive
| strides in fields that isn't immediately profitable
| (AlphaGo, AlphaFold...) they also continuously push new
| research in things that are obviously of interest for
| Google and Alphabet- especially in text-to-speech,
| speech-to-text, information-retrieval etc.
|
| For reference : https://deepmind.com/research
| jgalt212 wrote:
| It's not far from the truth to say that Deep Mind is selling
| snake oil (AGI) and delivering vegetable oil (SOTA pattern
| recognition software).
| KiLo_H4CK wrote:
| If these machines can don this, Then we are really in trouble.
| webmaven wrote:
| Saying RL is sufficient to (eventually) achieve AGI is a bit
| misleading. One might similarly state that biological evolution
| is sufficient to (eventually) achieve biological general
| intelligence.
|
| Both statements are probably true, but the parenthetical
| (eventually) is doing an awful lot of heavy lifting.
| neltnerb wrote:
| Yes, it's easy to be convinced on either side, the arguments
| write themselves. Yes, eventually a learning system might learn
| enough to be indistinguishable from intelligence. Or this might
| be entirely the wrong path and detracting from genuine new
| innovations in how we think about AI.
|
| We won't be able to tell whether it's AGI or just good enough
| at trained tasks to trick us.
| prometheus76 wrote:
| Let's say I'm standing next to a table. The computer
| recognizes it as a table. Now I sit on the table. Is it a
| chair or a table now? Something that we do automatically is a
| LONG way away from being automatic for AI.
| exporectomy wrote:
| It can prove its intelligence by making testable predictions
| of the future better than us. As for whether it's "real" AGI
| or just acts like it, doesn't really matter. I think the
| Chinese room problem has been agreed on as not a problem,
| hasn't it?
| neltnerb wrote:
| My last sentence was a statement of that problem, not a
| question.
| chriswarbo wrote:
| > As for whether it's "real" AGI or just acts like it,
| doesn't really matter.
|
| Absolutely. The term "AGI" came about specifically to avoid
| existing philosophical arguments about "strong AI", "real
| AI", "synthetic intelligence", etc. Those wanting to
| discuss "true intelligence", etc. should use those other
| terms, or define new ones, rather than misuse the term AGI.
|
| AGI requires nothing more (or less!) than a widely-
| applicable optimisation algorithm. For example, it's easy
| to argue that a paperclip maximiser isn't "truly
| intelligent", but that won't stop it smelting your
| haemoglobin into more paperclips!
| 6gvONxR4sf7o wrote:
| I think proof of "real" intelligence by answering harder
| and harder questions is barking up the wrong tree. I think
| evidence and proof are a better way to denote varying
| levels of understanding.
|
| A deductive system can come with an answer and a proof of
| that answer, where proof is whatever counts as proof in
| that system.
|
| So the notion of "does it really understand it's answers"
| gets punted off its Q&A abilities and onto its ability to
| justify its answers.
| criddell wrote:
| Does AGI imply human-level intelligence, or would the
| intelligence of a housefly qualify?
| mjburgess wrote:
| It's a very interesting question.
|
| Personally I take mammalian intelligence as the relevant
| standard we're actually aiming at.
|
| So I'd say mouse+.
|
| Houseflys, I think, are closer to non-intelligent than
| intelligent.
| criddell wrote:
| Surely the AGI researchers have a benchmark though, don't
| they? Somebody else mentioned the Turing Test which is
| something...
| mjburgess wrote:
| I dont think there are any AGI researchers. At least, I
| dont think computer science has much to do with AGI.
|
| The turing test is also not an AGI test, it's a "good
| enough" standard for fooling people.
|
| Intelligence fundamentally requires a multitude of
| environmental capabilities. The turing test considers
| only a single i/o boundary.
| 8ytecoder wrote:
| I'm assuming you are aware of the difficulties for machines
| to do even the most basic of things that a living being can
| do with a brain the size of a pea. A housefly can fly and
| navigate effortless through most complex scenarios that it
| evolved to navigate (even though the same fly can get stuck
| behind a glass window and eventually die).
|
| So yeah, even getting that level of intelligence would be a
| huge win. However, most people mean close to human level
| intelligence when they mean AGI even if it's one narrow
| specialization.
| criddell wrote:
| Doesn't the _G_ in AGI imply that narrow specializations
| aren 't the target?
| Blikkentrekker wrote:
| > _even if it 's one narrow specialization._
|
| Obviously that already exists even with g.o.f.a.i.s so that
| is not that impressive.
|
| The impressive thing is something more general than that.
| flylikeabanana wrote:
| AGI implies it can pass a Turing test, which means it has a
| better-than-average chance of acting more "human" than a
| competing human.
| [deleted]
| xamuel wrote:
| I thought it was a fun position paper, if not exactly
| groundbreaking.
|
| They did avoid one common pitfall at least. They are
| (intentionally?) vague about which number systems the rewards
| can come from, apparently leaving it open whether the rewards
| need be real-valued or whether they can be, say, hyperreals,
| surreals, computable ordinals, etc. This avoids a trap I've
| written about elsewhere [1]: traditionally, RL rewards are
| limited to be real-valued (usually rational-valued). I argue
| that RL with real-valued rewards is NOT enough to reach AGI,
| because the real numbers have a constrained structure making
| them not flexible enough to express certain goals which an AGI
| should nevertheless have no problem comprehending (whether or
| not the AGI can actually solve them---that's a different
| question). In other words: if real-valued RL is enough for AGI,
| but real-valued RL is strictly less expressive than more
| general RL, then what is more general RL good enough for?
| "Artificial Better-Than-General Intelligence"?
|
| Note, however, that almost all [2] practical RL agent
| technology (certainly any based on neural nets or backprop)
| very fundamentally assumes real-valued rewards. So if it is
| true that "RL is enough" but also that "real-valued RL is not
| enough", then the bad news is all that progress on real-valued
| RL is not guaranteed to help us reach AGI.
|
| [1] "The Archimedean trap: Why traditional reinforcement
| learning will probably not yield AGI", JAGI 2020,
| https://philpapers.org/archive/ALETAT-12.pdf
|
| [2] A notable exception is preference-based RL
| robrenaud wrote:
| There are more real numbers than programs. Computers cannot
| represent the vast majority of real numbers. AFAICT, it's not
| even clear that the universe is continuous rather than
| discrete.
|
| I really don't believe that using approximations of real
| numbers is going to be the bottleneck for AGI.
| [deleted]
| YinLuck- wrote:
| I thought the whole point of quantum mechanics was to prove
| the universe was discrete and thus not continuous, no?
| xamuel wrote:
| And yet computers have no problem symbolically representing
| non-rational numbers like sqrt(2), pi, etc. Neither is
| there any inherent reason why they cannot symbolically
| represent various levels of infinity, nor why those would
| be incomprehensible to AGIs (even if the universe is
| discrete). You're right that only countably many numbers
| can be represented, but nevertheless even countable subsets
| of extended number systems can exhibit structural
| properties that the reals do not exhibit.
| enkid wrote:
| How do you even have reinforcement learning with non-real
| numbers? The point is to maximize a score. It seems to me,
| any benefit you'd get from using an alternative number system
| could be replicated by using an algorithm to convert multiple
| real number scores into a single value.
| xamuel wrote:
| Here's an example. Suppose there are two buttons, A and B.
| If you press A for the nth time, then you get reward n. If
| you press B for the nth time, then you get reward 0 if n is
| not a power of 2, or reward omega (the first infinite
| ordinal number) if n is a power of 2.
|
| If the above rewards are shoehorned into real numbers---for
| example, by replacing omega with 9999 or something---then
| an RL agent would misunderstand the environment and would
| eventually be misled into thinking that pressing A yields
| more average reward.
| enkid wrote:
| What's the behavior your trying to get the AI to do in
| this example? Learn how to compute the power of 2? This
| is a task that can be accomplished much more simply with
| a different reward system. For example, have A always
| equal 1 and B equal 2 if it is a power if 2 and 0
| otherwise.
|
| I understand you can use non real numbers, that's not
| what I was asking. I'm asking what's a behaviour you
| can't replicate using a reward system based on real
| numbers.
| state_less wrote:
| There are no infinite rewards in biology and yet
| mathematicians seem to do just fine answering these sorts
| of questions.
|
| I don't think you want to encode your problem domain in
| your reward system. It'd be like asking a logic gate to
| add when you really should be reaching for an FPU. Maybe
| I'm missing something though?
| xamuel wrote:
| >There are no infinite rewards in biology and yet
| mathematicians seem to do just fine answering these sorts
| of questions
|
| This is only a problem if you're already assuming we do
| everything based on our biological reward systems, and in
| the current context that would be circular reasoning.
|
| Imagine the treasury creates a "superdollar", a product
| which, if you have one, you can use to create any number
| of dollars you want, whenever you want, as many times as
| you want. Obviously a superdollar is more valuable than
| any finite number of dollars, and
| humans/mathematicians/AGIs would treat it accordingly,
| regardless of the finiteness of our biological reward
| systems.
| state_less wrote:
| > This is only a problem if you're already assuming we do
| everything based on our biological reward systems
|
| Is there some other way that we are do it beside our
| biological reward system? It sure looks like we get an
| apple and not an infinite reward when we pick the right
| answer to be selecting button B. I understand that might
| not satisfy you.
| [deleted]
| visarga wrote:
| RL + piggybacking on human culture might be enough, or
| evolution + RL for biological agents.
| webmaven wrote:
| _> RL + piggybacking on human culture might be enough, or
| evolution + RL for biological agents._
|
| Yes, but over what timeframe? Will there be any diminishing
| returns plateaus along the way?
| ms1 wrote:
| I think, in really broad terms, in order to get AGI actually we
| would need to do better than nature.
|
| If our metric is (intelligence)/(joule), nature _seems_ pretty
| bad at a first glance: it took many trillions of lifetimes to
| achieve "general intelligence" *
|
| But then again, on the big stuff like this, have we ever really
| beat nature? That asterisk is there because, sure, turning the
| earth's biosphere into computers would make us smarter, but...
| _are we sure_?
|
| (And also: human = general?)
| londons_explore wrote:
| Nature has a _massive_ incentive to make good use of energy
| from light through photosynthesis. Billions of plants
| compete, and whoever can get most out of the sun will win
| out.
|
| Yet manmade solar cells are more efficient by nearly all
| measures.
| Clewza313 wrote:
| Except that manmade solar cells are pretty bad at repairing
| or replicating themselves.
| rich_sasha wrote:
| Or growing out of literally nothing but dirt and water.
| 21eleven wrote:
| And air. That's what is crazy about plants, their carbon
| comes from the CO2 in the air.
|
| Also if someone loses weight, most of the carbon that
| made up their fat leaves the body as breath.
| nindalf wrote:
| The solar cells get some organic life form to assist in
| their reproduction phase. That's pretty efficient too.
| bildung wrote:
| _> Yet manmade solar cells are more efficient by nearly all
| measures._
|
| Only because we cheated, though: Houses can't sponantously
| grow more cells in place when more energy is needed.
| maaaaattttt wrote:
| On a half-jokingly note, they can, their humans buy them
| and put them where needed. An alien observer in space
| would see some houses spontaneously growing solar cells
| on their roofs.
| EGreg wrote:
| Why are these just-so stories believed so much?
|
| Just because plants compete on some limited level doesn't
| mean that a particular plant organism "winning" means
| becoming the most efficient converter of sunlight.
|
| Is everyone's memory like those people who can remember
| every detail? Why not? If you're immediately planning to
| make up a just-so explanation on the spot that has the
| requisite but unproven claim about increasing the genetic
| fitness function, that is the problem with evolutionary
| explanations. It's not science if you just make stuff up
| and give it the same amount of credibility as something
| that has been tested and proven. You can take any trait and
| spin stories about why it is the way it is, and then expect
| somehow that some metric has to be maximized because of
| your unproven theory.
| FredPret wrote:
| This is even more interesting if you think of all human
| artifacts as being equivalent to anthills and beaver dams.
|
| 1) Trees are natural and trees create leaves with a solar
| efficiency of x
|
| 2) Humans are natural and we create solar panels with
| efficiency x + y
| chriswarbo wrote:
| This is Dawkins' idea of "extended phenotype". Normally a
| gene's phenotype refers to its effects on the body of an
| individual organism posessing that gene, like hair colour
| or immune response.
|
| A gene's _extended_ phenotype includes effects external
| to particular organisms, like nests, deforestation,
| changes to the chemical makeup of the atmosphere, etc.
| idiotsecant wrote:
| nature only has an incentive to increase efficiency when
| that increase in efficiency results in increased chance of
| producing gene copies.
|
| Nature is full of examples that are 'good enough' while
| balancing other competing constraints. Evolution doesn't
| create organisms optimized for efficiency - it creates
| organisms optimized for reproduction. The two are not
| always the same.
| ackbar03 wrote:
| This comparison with nature is pretty interesting. I think
| some additional constraints are required though. Otherwise,
| technically we can produce agi by simply giving birth to
| humans. If that's not "artificial" enough we can produce them
| from test tubes
| admk wrote:
| That is if you believe biological general intelligence is the
| end goal of evolution, which I believe is highly unlikely.
|
| Intelligence is simply a special side-product of evolution,
| there is nothing general about general intelligence. Many
| organisms can thrive without it.
|
| There is also a non-negligible chance that all organisms would
| die out before reaching intelligence. We are fortunate to live
| in a world that produced us.
| arketyp wrote:
| That's a bit besides OP's point though, which is about
| vacuous claims. Humans are the existence proof that there is
| some sequence of circumstances where evolution reaches GI.
| There's an analogous sequence of circumstances in the RL
| case, which happens to be the hard part.
| dqpb wrote:
| > biological evolution is sufficient to (eventually) achieve
| biological general intelligence
|
| Says nothing about this:
|
| > biological general intelligence is the end goal of
| evolution
| ikrenji wrote:
| i mean if the end goal is to propagate the organism, surely
| intelligence will be helpful to this - interplanetary scale
| Viliam1234 wrote:
| But until that actually happens, the possibility of it
| maybe happening in the future has zero impact on current
| natural selection.
| mensetmanusman wrote:
| Depends, if any of the laws of physics were off by a
| billionth of a percent, there would be no human intelligence
| (or carbon life, or atoms).
|
| There are many reasonable assumptions one could draw from the
| fact.
| Filligree wrote:
| > That is if you believe biological general intelligence is
| the end goal of evolution, which I believe is highly
| unlikely.
|
| I would agree, but might add that evolution doesn't have
| 'goals'.
|
| Is that the point you were trying to make?
| didericis wrote:
| Not OP, but yeah, evolution doesn't have goals in the same
| sense that people do, just like gravity doesn't "want" to
| pull things, it just kind of "is", and simply acts as
| reality permits based on prior and current conditions.
| That's reasonable to say.
|
| Convergent evolution exists for at least some adaptations
| though, like the eye. It's not unreasonable to think that
| there may be some sort of equivalent convergence which
| creates a high general intelligence adaptation given enough
| time, at least for social creatures.
|
| I think it's pretty much impossible to know whether
| intelligence is a convergent adaptation without some kind
| of perfect simulation of evolution over billions of years.
| You'd have to tweak starting conditions and see if you kept
| getting smart creatures.
| Filligree wrote:
| Ah. So _that's_ why we exist. I was wondering.
| burning_hamster wrote:
| I think the title of the paper makes more sense if you consider
| that ten years ago, someone could have written a paper in a
| similar spirit with a different take on "what is enough". Back
| then, it would probably have been titled: "Backpropagation of
| errors is enough".
|
| The last ten years have shown that backpropagation -- while a
| crucial component -- is not enough. Personally, I would not be
| shocked to find out in the next ten years that reinforcement
| learning is not enough for an AGI (as there are aspects like
| one-shot learning, forgetting, sleep, and other phenomena for
| which the RL framework seems not a natural fit).
| endtime wrote:
| Why are forgetting and sleep relevant? If someone invented a
| pill that gave you a perfect memory and removed the need to
| sleep, would you stop being generally intelligent if you took
| it?
| jonnycomputer wrote:
| Well, if the state of the world changes, then hanging on
| what you learned in the past can cause you to do the wrong
| thing. Sure, there is an old proof that the value of (true)
| information is greater than 0, you could say, but they
| could also remember that the state of the world has
| changed, so there is nothing bad about remembering, or the
| model could just discount data by how old it is, etc. All
| true. But the representation becomes more and more complex.
| I certainly find that I have to pull back and tell myself,
| wait, the world has probably changed since I learned that,
| hasn't it? Has it?
| treeman79 wrote:
| Possibly. Database look up on a million rows is very
| different then a lookup on a trillion. Both have solutions,
| but the Perl hack that is our mind may lock up on a bigger
| data set.
| skywhopper wrote:
| Because they are strongly associated with all known
| examples of generalized intelligence. Why _wouldn't_ they
| be relevant?
| jsjohnst wrote:
| > Because they are strongly associated with all known
| examples of generalized intelligence
|
| Correlation != Causation. While they very likely might be
| relevant, I've not seen anything to conclusively prove
| that it is. The ability to forget is important to humans
| because we are emotional beings, but I don't think that
| necessarily is a requirement for generalized
| intelligence. "sleep" (as in what happens during sleep,
| not the act specifically itself) on the other hand is
| very likely important, but again, not proven.
| ABCLAW wrote:
| One of the postulated reasons for why older people have
| worse reaction times and think slower than their younger
| counterparts is that the neural networks they use draw upon
| more stored information, thus making routine evaluations
| take longer.
|
| There's a sweet spot between knowing enough and knowing
| little enough so that you get the right answer and get it
| quickly enough.
| lupire wrote:
| That's a weird claim. Why not just assume old people are
| slower minded for the same reason they are physically
| slower: physical degredation?
| [deleted]
| Veedrac wrote:
| Ten years ago we didn't even have AlexNet; I think most
| people would have thought a paper like that was nuts at the
| time. The ten years since are what _popularized_
| backpropagation as a path to general intelligence. Who ten
| years ago would have seriously predicted GPT-3? The odd few
| that did are certainly not the people I would expect to have
| been dissuaded! And if there 's any actual experimental
| evidence that backpropagation is not enough, I haven't seen
| it.
| lupire wrote:
| Backpropagation was the model for AGI in the 1980s if not
| earlier. Of course computing power made it impossible for
| _anything_ to actually deliver AGI.
| whatshisface wrote:
| RL can forget, just start training it on a dataset that is
| different from what it was originally trained on.
| [deleted]
| cowanon22 wrote:
| The AI field has a history of over confidence; just look up
| Marvin Minsky's predications in the 1950's and 1960's. This had
| been repeated throughout AI history - a new approach is found, it
| has some promising initial results, and then progress gets stuck
| and the approach is basically halted for decades.
|
| Machine learning and neural networks have prospered due to
| massive increases in computing speed and data, but I suspect it
| will also be a dead end in general AI. There will be some niche
| wins, some quite impressive, but the machine won't have the
| properties of an intelligent child or even a rat.
|
| There seems to be a big difference in how actual neurons work - I
| think neural networks are misnamed. The brain is not a big
| matrix, biology does not appear to reinforce behavior using
| gradient operations. I think increased research into neuroscience
| and biomimicry is the most likely approach for general AI, but I
| think we are still far away and the current hype will just be one
| more dead end.
| nopeNopeNooope wrote:
| There's realtime video processing that happens on smartphones
| to turn influencers into anime doll creations, seamlessly! Most
| people DON'T EVEN KNOW IT IS HAPPENING!
|
| The backgrounds on millions of peoples video conferences are
| shifted to something fanciful every hour.
|
| There's trillions of dollars of value created by automated
| software compositing images taken by capture cars, and turning
| that into points of interest and regular (automatic!) address
| updates.
|
| Ad engines are so incredibly effective, that inside a hundred
| milliseconds it can hold an auction and target you with things
| so perfectly tailored to what it knows about you that people
| are ABSOLUTELY convinced that Facebook and Google are listening
| in on them.
|
| And you think this is an over hyped dead end?
|
| Who cares how neural networks really work? This is good enough
| to get things done today, and likely well into the near future.
|
| I'll also point out that the lags in AI progress generally have
| more to do with the funding situation than the actual progress
| of the applications. It turns out when you kill the fountain of
| money that was Bell Labs, you have to wait until the fountain
| of money is reborn as Google. This isn't really a major
| difference from what we started with in the 50s, we're still
| using most of the same concepts, they'd almost certainly be
| instantly recognizable to folks if you chalk boarded it out for
| them.
|
| For you to say we hit a dead end back then when we're doing
| effectively the same thing, but with a much larger budget and
| talent pool is disingenuous.
| ur-whale wrote:
| >DeepMind says reinforcement learning is 'enough' to reach
| general AI
|
| When a company like DM makes such statements, you have to take
| into account the fact that they've essentially bet the farm and
| the neighbor's on RL.
|
| As such, the statement isn't really carrying much weight.
| rsj_hn wrote:
| What would have carried weight would be if this is something
| they actually achieved and then said "this is how we did it.".
| Making predictions of _how they will succeed in the future_
| doesn 't really carry much weight, particularly in AI
| scythmic_waves wrote:
| /r/MachineLearning discussion:
|
| https://www.reddit.com/r/MachineLearning/comments/nplhy3/r_r...
|
| I'm with most of the comments there. This paper is ridiculously
| hand-wavey.
| JamilD wrote:
| Many of DeepMind's opinion style papers are like this. Another
| example of the "handwavy" DeepMind paper:
| https://arxiv.org/pdf/2102.03406.pdf
|
| It's also worth it to note as well that this isn't a homogenous
| organization, many DeepMind employees have different opinions
| on issues like this and an individual paper isn't
| representative of the entire organization.
| scythmic_waves wrote:
| > It's also worth it to note as well that this isn't a
| homogenous organization
|
| Please don't consider my critique of this paper as an
| indictment of DeepMind as a whole!
|
| > Many of DeepMind's opinion style papers are like this.
|
| That's good to know. I have not read many of their opinion
| papers, and I'll admit I didn't have the context of it being
| an "opinion" paper.
|
| That said, I don't agree with the opinion. The paper didn't
| really engage with the concept of AGI in a way that I found
| satisfying. The conclusion may very well be correct, but this
| paper wasn't enough to convince me.
|
| Slightly OT: My views were reinforced when I saw the paper
| was praised by Patricia Churchland. I don't find her take on
| consciousness a satisfying one, though I find the general
| direction of her work interesting. See here for another
| example:
|
| https://www.reddit.com/r/philosophy/comments/nvtgwr/grand_th.
| ..
| dannyw wrote:
| Alternate title: DeepMind fails to make progress on AGI,
| publishes thought piece instead.
| visarga wrote:
| And the entitlement we have is even higher than the difficulty
| of the task and the hard work people are putting in. Anyone
| here can say they did as much for RL?
| hestefisk wrote:
| For those of you 'generalised' AI sceptics like me, I can highly
| recommend reading Soren Brier's book on Cybersemiotics ("why
| information is not enough"). It's a comprehensive reader into the
| physicalistic, reductionist field of AI and all of its
| shortcomings. General AI implies the ability to abduct (not just
| deduction and induction), which I highly doubt will ever be
| possible.
| 6gvONxR4sf7o wrote:
| Why would artificial intelligence have any limitations that
| biological intelligence does not have?
| funklute wrote:
| > which I highly doubt will ever be possible
|
| What prevents an AI from performing abduction? While I've never
| thought about it before, it intuitively seems like a pretty
| straightforward thing to implement....
| reedf1 wrote:
| How does he resolve that humans achieve the ability to
| "abduct"? I just don't buy that there is something ineffable
| about humans - and even if there is, why can't we just plug
| that process into a computer?
| cromwellian wrote:
| "never" (I know you didn't say that, but it's implied), is a
| pretty strong claim. Like with Searle's Chinese Room, I find
| these kinds of impossibility arguments fairly weak and full of
| semantic problems. Whenever people argue "consciousness", or
| "general AI", or "meaning", it's always an exercise in moving
| the goal posts.
|
| When I read abstracts like this: "Cybersemiotics constructs a
| non-reductionist framework in order to integrate third person
| knowledge from the exact sciences and the life sciences with
| first person knowledge described as the qualities of feeling in
| humanities and second person intersubjective knowledge of the
| partly linguistic communicative interactions, on which the
| social and cultural aspects of reality are based. The modern
| view of the universe as made through evolution in irreversible
| time, forces us to view man as a product of evolution and
| therefore an observer from inside the universe. This changes
| the way we conceptualize the problem and the role of
| consciousness in nature and culture. The theory of evolution
| forces us to conceive the natural and social sciences as well
| as the humanities together in one theoretical framework of
| unrestricted or absolute naturalism, where consciousness as
| well as culture is part of nature. But the theories of the
| phenomenological life world and the hermeneutics of the meaning
| of communication seem to defy classical scientific
| explanations. "
|
| My confidence isn't increased that this book has anything
| interesting to say. It sounds like the kind of post-modernist
| verbiage coming out of a lot of humanities departments. I mean
| seriously, why anyone in a STEM field actually write a sentence
| like "modern view of the universe as made through evolution in
| irreversible time, forces us to view man as a product of
| evolution and therefore an observer from inside the universe."
| Like what is "irreversible time" adding there? It's kind of
| redundant, and the awe-inspiring implication: man is a product
| of evolution, and therefore an observer in the universe? Earth
| shattering. And all phenomena, from fundamental life sciences
| to culture, are part of nature? No one's ever considered that
| before.
|
| My guess is, if you asked a STEM major to write this abstract,
| they could sum it up with 70% less words. They'd also drop
| implicitly obvious claims that no one disputes.
|
| I'm betting that this is book is just a less philosophically
| rigorous version of Searle's argument, given the implication
| that emergent phenomena "seem to defy classical scientific
| explanations".
| sklearncowboy wrote:
| What proof is there that I am generally intelligent in the same
| context we want computers to be in order for AGI to be true?
|
| Imagine how much farther the field would be if we stopped wasting
| resources on this fantasy land nonsense.
|
| All intelligence is specialized intelligence.
| jawarner wrote:
| Yes, human intelligence is the benchmark for the definition of
| general intelligence. Certainly we can go further.
| bgroat wrote:
| I'm not a neuroscientist, an AI specialist, or a hardware
| engineer.
|
| But as an enthusiast of all three I really think that AGI is a
| hardware problem, not a software problem.
|
| Reinforcement learning on a massive corpus of data is how we
| train _all_ biological intelligence.
|
| The crazy thing is that in humans we manage to do it on ~3 watts
| an hour.
|
| I think we have the software cracked, my gut thinks silicon just
| isn't the right material
| pkAbstract wrote:
| You may be right, but it's also commonly believed in these
| communities that hardware is the part that's already been
| solved. Computer hardware already vastly outstrips human
| capacity in many domains.
|
| To me, it seems more likely that we're missing something/some
| things on the software side. AGI could probably run on present
| day hardware or even older.
| 6gvONxR4sf7o wrote:
| Silicon is likely fine as a material. GPU cost per operation is
| still dropping insanely quickly. A lot of really hard ML
| problems are just making big things feasible, or sampling big
| things enough to get decently precise estimates. With 10x the
| GPU power and memory, a lot of this gets easy. With 100x, some
| hard things get trivial. At the end of the day, GPUs and TPUs
| drive AI research more than anything else as models grow
| massively.
| carrolldunham wrote:
| Isn't this tautological or trivial because any proposed
| alternative requirement than reward maximisation can be reframed
| as maximising a different reward?
| seaorg wrote:
| The layman and the AI expert have both written off the
| possibility of creating AGI for the entirety of this latest AI
| spring. In the past decade I have basically never encountered
| anyone who thought that AGI was going to happen in our lifetimes
| or even anyone who believed that it would be a problem if it did.
|
| One time I discussed AGI with a good friend. And I gently
| pressured him to play through the scenario of the advent of AGI.
| And he made a guilty confession to me. I could tell he was
| embarrassed to share his opinion. He said "can't we just unplug
| it?" This is a microcosm of the entire issue. It's something a
| child might say. For global warming, can't we just turn on the
| air conditioning? No, we can't just unplug it.
|
| Here is how you can understand what is happening. An AI model is
| just a program. A program that is written by another program.
|
| Consider a list of programs that contains every possible program.
| It is infinite. Make it finite by limiting the size of the
| programs based on some practical consideration. It's still a very
| long list.
|
| What is in this list? Pong. StyleGAN. AGI? Almost certainly. How
| small might AGI be? It probably appears more than once on the
| list -- how many are there? But let's ask the real question here
| and boil this down. How many items on the list are something we
| will regret having discovered? A great, great many.
|
| Every day, people are exploring the list. They are using
| primitive methods to sift through all these programs and find the
| ones that have interesting behavior. The process of program
| discovery is automated with things called "training algorithms."
| As our computers get faster and capital allocation grows larger
| we mine this list more and more quickly. The problem is that we
| keep finding things that surprise us. And that is the core and
| substance of the entire issue. We have demonstrated to ourselves
| over and over again that our own intuition about the contents of
| this list is completely wrong. Despite the mind-boggling level of
| mental gymnastics performed in the wake of GTP-3, enough to
| constitute a three ring circus, the fact of the matter is that
| GTP-3 and soon to be 4 is not supposed to exist according to
| every single AI expert. Compute is only getting cheaper and
| capital is only getting bigger and the list is only getting
| shorter and it's not going to stop surprising everyone including
| the "experts." Just because we don't know how to make something
| we will truly regret doesn't mean we aren't capable of creating
| it.
| helen___keller wrote:
| > He said "can't we just unplug it?" This is a microcosm of the
| entire issue. It's something a child might say. For global
| warming, can't we just turn on the air conditioning? No, we
| can't just unplug it.
|
| If it were running as a sandboxed application inside some kind
| of runtime environment with only access to print output text
| and read input text, as you might expect a GPT-n program to
| run, certainly you could unplug it.
|
| Humanity could be considered God's AGI, but if God doesn't give
| us the APIs to interact with heaven, it follows that we can't
| kill God. Of course, humans are much more greedy, so I'm sure
| our creations will have full access to any APIs needed to run
| our factories, advertise products on the internet, and maybe
| direct our military weapons too :) Oops!
|
| > Despite the mind-boggling level of mental gymnastics
| performed in the wake of GTP-3, enough to constitute a three
| ring circus, the fact of the matter is that GTP-3 and soon to
| be 4 is not supposed to exist according to every single AI
| expert
|
| Could you elaborate? What mental gymnastics, and it is not
| supposed to exist according to who exactly?
| seaorg wrote:
| The entire sandbox thing is foolish. Obviously it won't be
| sandboxed anyway because of what you've pointed out. But any
| sandbox we design won't be good enough.
|
| GTP is not supposed to exist according to computer science
| and machine learning experts in 2017. They were all wrong.
| Same with deep dream, styleGAN. And it will be true with the
| next thing.
|
| The mental gymnastics refers to the fact that people have
| cognitive dissonance about GTP. They have amnesia about the
| decades of stagnant progress in text generation. All anyone
| can do is point out that it's definitely not sentient and
| it's not AGI so what's the big deal? All of a sudden the goal
| posts have been moved... this is the single biggest quantum
| leap in text generation ever, a mind-boggling level of
| lucidity with grammar, punctuation and more all without ever
| being given a single deliberate instruction from a human
| being. It is objectively amazing. Objectively. And people
| have no reaction. Because they are not emotionally ready for
| it. It's cognitive dissonance, mental gymnastics, whatever
| you want to call it.
| 21eleven wrote:
| RL can provide amazing results (AlphaGo, AlphaStar (Starcraft 2
| agent), etc) but it requires a well modeled world to work with.
|
| Games like Go and Starcraft are well modeled worlds. If you want
| something akin to AGI to operate in the "real world" you will
| need a high quality data model of the real world for the RL
| system to work off of.
| atombum wrote:
| Agreed, and that brings up a very interesting discussion of
| prejudice in modeling the world. Everyone views the world
| differently and it seems to me of vital importance that any
| attempts to model the real world for RL are as unbiased as
| possible. Or more realistically, incorporate as many different
| biases as possible.
| bitL wrote:
| RL is still pretty dumb and for that extremely computationally
| demanding. Markov chains are like the most trivial stochastic
| models and I can't believe that's all it takes to get AGI. I also
| often wonder whether the "Deep" part of DRL is not spending most
| of its weights to compensate for the basic nature of Markov
| chains.
| [deleted]
| orwin wrote:
| Reinforcement learning without auto-growing inference engine
| embedded? Yeah, no chance its enough.
| kakadzhun wrote:
| Even if Reinforcement Learning is 'enough', it will be held back
| by whatever methods used to implement it (e.g. Deep Neural
| Networks). As the algorithms get more advanced, from some point
| onwards, to build the general AI, you first need a general AI to
| tell you the correct hyperparameters so that the pile of methods
| would work well in tandem.
|
| Last but not least, you will be bound by your inability to
| accurately communicate exactly the behaviour you want out of the
| AI because you are incapable of writing down a mathematical
| function that would induce the behaviour in the learner. And then
| you wonder why the general AI decided to pull the plug on your
| grandma and try to use that resource for something else instead.
| andyxor wrote:
| good luck with that. DeepMind should sponsor B. F. Skinner award,
| to honor the father of their behaviorist theories of 'reward and
| punishment' as a sort of all-encompassing theory of everything
| related to cognition. At least now they are torturing GPUs and
| not some poor lab animals.
|
| on a serious note the only positive outcome of all this shameless
| PR is that the heavy investment in ML/RL might trickle down to
| actual science labs and fundamental neuroscience research which
| might move us forward towards understanding natural intelligence,
| a prerequisite for creating an artificial one.
| eeegnu wrote:
| > towards understanding natural intelligence, a prerequisite
| for creating an artificial one.
|
| I've thought about this before, and I'm not convinced it's
| really prerequisite. Naturally developed intelligence in my
| mind may actually be highly constrained and inefficient because
| it was limited to what was biologically feasible. i.e. There
| may be simpler ways of achieving comparable results. Natural
| intelligence does however have the benefit of being an actual
| working model, but deciphering the blackbox may be just as hard
| as developing a working theory from first principles.
| andyxor wrote:
| yes, it's a recurring thread, "do we really need to mimic the
| birds in order to build airplanes", etc.
|
| I think someone serious about AI should treat it not as
| engineering problem but as a science, like physics, which
| starts with model of nature, and experiment to prove or
| disprove the theory. Nature provides the constraints by which
| theory is developed, which radically limits the "search
| space" of theories. Otherwise it's a bit like throwing things
| on the wall and see what sticks, which is the primary method
| of current AI research.
| Peritract wrote:
| Mimicking birds wasn't necessary for flight.
|
| However, _understanding_ them absolutely was; we didn 't
| end up taking exactly the same route to the sky, but we
| absolutely learnt from birds on the way.
| adipandas wrote:
| This article is interesting, I even skimmed through their paper.
| But I think still the question remains: How to find the unified
| reward function? Or in other words, how to find answer to life?
| [It cannot be 42].
| goldenkey wrote:
| Intelligence would be produced in any Turing complete automata.
| But the universe has a frame rate of 10^34 (based on Planck
| constant.) We don't really have the tech to just run
| "evolution" of a universe or of even a psuedo biological
| substrate.
| adipandas wrote:
| I like your take on frame rate of the universe. Nice! :)
| shmageggy wrote:
| Except it's wrong. I recently had the same misconception
| about the Plank constant somehow being some minimal unit,
| but it's not. This video from Fermilab's website helped set
| me straight https://www.youtube.com/watch?v=rzB2R_qiC28
| goldenkey wrote:
| It's not wrong. And the Fermilab video doesn't really
| dispute it.
|
| Planck's constant measures action, Hz per Joule of
| energy. Hz is really just a measure of oscillation, or
| change. It doesn't directly translate to framerate, but
| it gives us a ballpark figure in orders of magnitude. We
| don't have anything near 10^34 Hz en-silico, and even if
| we built a biological/chemical computer, that would be on
| the par of Avogadro's number, 10^23. So, just because we
| build a system that can _evolve_ to be intelligent, or
| hold intelligence within it, doesn't mean we have any
| ability to actually see it through to that.
| drdeca wrote:
| This seems far from clear. Just because a system is capable
| of turing complete computation does not imply that a generic
| state of the system will typically eventually produce
| intelligence or even something which is sophisticated in some
| sense.
|
| As a trivial example, consider a variation of Conway's game
| of life which, in addition to black and white cells, also has
| green cells, where any cell next to one or more green cells
| will be a green cell in the next time step. A generic state
| in such a variation will have at least one green cell, and
| therefore all parts of it will eventually be green, and so no
| useful long running computation will be done, certainly none
| which takes where the green cells are into account. But, such
| a system would still be turing complete, because one could
| start in a state in which there are no green cells, and in
| those states you just have Conway's game of life.
|
| That trivial example works as an existence proof, but even
| for less extreme cases it isn't clear. Consider ordinary
| conway's game of life. To paraphrase a question from Alex
| Flint on Alignment Forum (https://www.alignmentforum.org/post
| s/3SG4WbNPoP8fsuZgs/agenc... ) Suppose we have some 10^50 by
| 10^50 square where an agent is supposed to be implemented,
| and this 10^50 by 10^50 square is at the top left corner of
| a, say, 10^100 by 10^100 square, where the rest of the square
| is initialized randomly, is it even possible for the agent to
| be such that it has a high chance of successfully influencing
| the large scale state of the rest of the 10^100 by 10^100
| region in the way that is desired? It isn't clear. It isn't
| clear that a structure can withstand the interactions with a
| surrounding chaotic region. Perhaps some systems are such
| that they do allow Turing-complete computation, and are such
| that typical states result in complex behavior, but are also
| such that all really structured behavior is always very
| "fragile", and can only continue in a structured way if what
| interacts with it is in a small set of possible interactions.
|
| To be capable of Turing complete computation, is not, I
| think, sufficient for "life" (a self-maintaining thing) to
| arise from typical/generic states, even when under the
| assumption that typical/generic states lead to continually
| complex behavior (to exclude the spreading green cells case)
|
| Also, I don't think we can confidently say that the Plank
| time is "the universal frame rate". Better to refer to
| Bremermann's limit and the Margolus-Levitin theorem , though
| these bounds depend on the amount of energy available. (10^33
| operations per second per joule, where the energy is the
| average energy of the system doing the computation)
| goldenkey wrote:
| > 10^33 operations per second per joule, where the energy
| is the average energy of the system doing the computation
|
| You're right, that's the actual meaning of action in
| physics, which is what the Planck constant measures. The
| amount of change (which is measured in Hz) per joule of
| energy. But it's a good enough approximation and a good
| lower bound for the amount of processing power the universe
| possesses versus our en-silico hardware. We don't have
| anything near 10^33. Just because we build a system that
| has the ability to evolve doesn't mean we will ever see it
| through to the extent that the universe has the capability
| to.
| qwertywert_ wrote:
| Yea. For animals, reproduction and just surviving is the reward
| function?
|
| It talks a lot about having a rich enough environment for
| learning which makes sense, if a computer lives only in a Go
| board it can only learn go playing itself.
|
| How do you simulate a rich enough environment purely in
| software (or do you sense input from the "real" environment)
| and what reward do we define in this complex environment.. It
| seems to ask those 2 questions in the discussion but kind of
| glosses over them imo.
| visarga wrote:
| You put many agents in the same environment, agents are both
| actors and the environment.
| jjjdjjddddsfsd wrote:
| Are the just reformulating the principles of evolution in digital
| terms, and essentially not providing any new insights at all?
|
| Yes, intelligence has been created by evolution. That doesn't
| imply that any system that is subject to evolutionary forces will
| lead to the creation of intelligence (and not within a reasonable
| timeframe, either). The challenge is to create a system that is
| capable of evolving intelligence.
|
| Afaik some biologists even think that the evolution of
| intelligence was rather unlikely and would not necessarily happen
| again under the same circumstances as on earth.
| visarga wrote:
| Evolutionary algorithms are tricky, just like deep learning.
| It's not "just reformulating the principles of evolution in
| digital terms, and essentially not providing any new insights".
| webmaven wrote:
| _> Afaik some biologists even think that the evolution of
| intelligence was rather unlikely and would not necessarily
| happen again under the same circumstances as on earth._
|
| Hmm. Can you provide a pointer to those biologists?
|
| AFAIK, high intelligence has arisen more than once on Earth
| (Hominoids, Cetaceans, Octopuses), so I'm somewhat skeptical of
| that claim, but perhaps they're construing intelligence more
| narrowly (ie. only Homo Sapiens qualifies).
| rubatuga wrote:
| Well that has a prior on life even existing in the first
| place
| jjjdjjddddsfsd wrote:
| Jared Diamond talks about it in his books (don't remember
| which ones specifically). OK, granted, he is not officially a
| biologist, I guess, but at least a prominent writer on
| Evolution Theory.
| inciampati wrote:
| As a biologist and longtime dabbler machine learning and
| Bayesian methods, I tend to see intelligence as a manifestation
| of evolution. In the case of an organism, the improvement of
| the model (the genome) occurs through processes that are very
| similar to what we see in any kind of learning (real, brain
| based or "artificial", computer based).
|
| Evolution and intelligence are inextricably linked. They are
| practically the same thing. This means that intelligence is
| probably a natural result of any system similar to those that
| support biologics. If you flow the right amount of energy
| through a substrate with complex enough building blocks, you'll
| eventually get life ~ which is just something smart enough to
| survive and feed off the available energy flows. In the world,
| this flow is radiation from the sun, while in a computer, it is
| governed by a more abstract loss or fitness function.
| unishark wrote:
| I'd say it's even less than that. They seem to be summarizing
| the ways the problem of teaching an agent to do anything
| (including be generally intelligent) can be formulated as a
| problem of maximizing a reward (hence the title).
|
| Another way to look at it is, if we had a good enough function
| (e.g. a universal approximator) it can be made to model any
| behavior using numerical optimization. Which I think isn't very
| surprising, but apparently there is some arguments about it.
| randcraw wrote:
| In fact, "if we had a good enough function" == "if we had
| sufficient funding". This refrain will resonate mightily in
| the willing ears of US congressfolk who want re-election and
| would rather talk about something other than Trump.
|
| So welcome back to the future, and the $trillions the US
| spent on 20 years of space race and 50 years of cold war. The
| catchphrase that motivates the next 50 years of
| government/corporate funding will be...
|
| They've got a Terminator and we don't.
| Animats wrote:
| _" A sufficiently powerful and general reinforcement learning
| agent may ultimately give rise to intelligence and its associated
| abilities. ... We do not offer any theoretical guarantee on the
| sample efficiency of reinforcement learning agents."_
|
| OK. This basically says "evolution works". But how fast? Biology
| took tens of millions of years to boot up.
|
| An related question is how much compute power does evolution,
| viewed as a reinforcement learning system. have? That's probably
| something biologists have thought about. Anyone know? Evolution
| is not a very fast or efficient hill-climbing system, but there
| are a large number of parallel units. It's not a philosophical
| question; it's a measurable one. We can watch viruses evolve. We
| can watch bacteria evolve. Data can be obtained.
|
| Two questions I pose occasionally are "how do we do common sense,
| defined as not screwing up in the next 30 seconds", and "why does
| robotic manipulation in unstructured situations still suck after
| 50 years". A good question to ask today is why reinforcement
| learning does so badly on those two problems. In both cases, you
| can define an objective function, but it may not be well suited
| to hill climbing.
| londons_explore wrote:
| In some cases biological 'genetic algorithm' hill climbing can
| be remarkably ineffective.
|
| For example, the classic "design a car that can drive over this
| terrain" problem, even after a billion generations (~ the same
| number as life on earth), shows no substantial performance
| improvement.
|
| That makes me suspect something is missing from our biological
| genetics model.
| wnkrshm wrote:
| energy supply and other constraints (material, robustness
| ...) are a good explanation though - an organism can't grow
| out of aluminum or steel
| EE84M3i wrote:
| I suspect they're talking about this (sort of) simulation
|
| https://rednuht.org/genetic_cars_2/
| kaba0 wrote:
| I think the number of parameters is remarkably (multiple
| orders of magnitude) different between even the simplest
| bacteria vs the model used in the car. And then genomics can
| also do some more advanced techniques like copy a whole gene
| and start modifying that, etc.
| 29athrowaway wrote:
| We just want AI to be able to think. We do not need an AI with
| an autonomic nervous system, or many of the functions in the
| central nervous system. We do not need AI to be very power
| efficient. If it takes several megawatts of electricity to get
| our first strong AI working, so be it.
|
| So, we do not have as many constrains as life did.
| ifdefdebug wrote:
| Define "think". And then prove it can be done without the
| kind of nervous system you say we don't need.
| jacquesm wrote:
| That is the most impressive use of the word 'just' in a long
| time. Note that all the other bits are solved, and have been
| solved since the 60's. It's the 'just think' bit that has
| proven to be a little bit harder than we thought it would be.
| randcraw wrote:
| > "why does robotic manipulation in unstructured situations
| still suck after 50 years"
|
| Great point. Until the promoters of RL can build us a robot
| that can 1) walk gracefully through a typical home that has
| stairs and closed doors, 2) cook a meal with pots and pans, and
| 3) get back up after it falls down -- I suggest we take their
| claims of impending Singularity with a big grain of salt.
| rich_sasha wrote:
| Basically, any problem with a solution fits into RL: reward of 1
| if you are AGI and 0 otherwise. Go learn.
|
| This setting on its own is meaningless! The "how" of the RL agent
| is not even 99% of the problem, it is all of it.
|
| Given our understanding of both DL and neuroscience, it is not
| even clear to me that we can say with confidence that Neural
| Networks are a sufficiently expressive architecture to cover an
| AGI.
|
| The human brain is a deep net, sort of, but there is also plenty
| going on in our brains that we don't understand. It could be that
| the magic sprinkle is orthogonal to DL and we just don't know
| about it yet.
| ggggtez wrote:
| Problem: you don't understand it therefore you think RL isn't
| sufficient.
|
| There is no evidence that the thing you don't understand isn't
| based on RL too.
| svantana wrote:
| > The "how" of the RL agent is not even 99% of the problem, it
| is all of it
|
| I'm not sure that's true anymore - pretty much any objective
| devised is being solved by ML solutions within months (with
| some exceptions such as Chollet's ARC, maybe Winogrande). But
| those same models will perform poorly on other unseen tasks,
| because ML takes shortcuts if it can. We used to have unsolved
| tasks for decades, such as Go. It's now comparably hard (if not
| harder) to create a good objective measure of intelligence than
| to reach human parity on said measure.
| bobthechef wrote:
| Concepts can't be represented in matter. That's your secret
| sauce. Well, the beginning of the recipe, anyway. But you won't
| be able to make the dish.
| candiodari wrote:
| I assure you 99% of the problem of any RL project is the
| simulator. Generally you can't let an RL algorithm control
| anything real from the start, so you have to implement a
| reasonably reliable simulator for whatever you want done.
|
| This is the big challenge in practice.
| ronsor wrote:
| Correct me if I'm wrong, but wouldn't that mean the entire
| world would have to be simulated? Or at least some subset of
| society?
| ggggtez wrote:
| The human brain does have a simulator. It's well known. How
| do you know where to move your hand to catch a ball? Or
| what is happening when you blink?
|
| Your brain is constantly simulating a few milliseconds
| ahead.
| cynusx wrote:
| Thank you for stating what should be obvious.
|
| I think there are two currently unsolved problems
|
| 1/ We have no idea what the reward function looks like that
| leads to AGI
|
| 2/ Deep networks are artificially constricted for computational
| efficiency and always optimized to solve the problem at hand;
|
| Any solution that delivers AGI should rely imo on:
|
| 1/ reinforcement learning
|
| 2/ Happen with an unstructured reservoir of randomly connected
| neurons
|
| There was a research trend towards reservoir computing and
| recurrent neural networks but this was mostly abandoned because
| progress in deep learning was amazing.
|
| These techniques are akin to a 2D-plane in a 3D object, it's
| heavily simplified and circular references are prohibited.
|
| I have some good ideas on what the reward function should look
| like in a reservoir setting and happy to discuss them with any
| active independent researcher in the field.
| hyfgfh wrote:
| "Talk is cheap. Show me the code."
| AndrewOMartin wrote:
| Hydrogen and time "enough" to reach general AI.
| https://xkcd.com/1123/
| ma2rten wrote:
| I'm more in Yann LeCun's camp who called unsupervised learning
| the cake, supervised learning the icing and reinforcement
| learning the cherry on top of the cake.
| hervature wrote:
| My opinion is that view is very simplistic and unnecessarily
| offensive to a whole class of researchers. MuZero, developed by
| David Silver, uses a combination of RL, supervised learning,
| and unsupervised learning (state representation) coupled with a
| planning algorithm. It accomplished things far beyond anything
| unsupervised learning can ever accomplish.
| Der_Einzige wrote:
| Unsupervised learning is exactly the wrong way to approach
| chess or other games that MuZero solves. It's also worth
| noting that traditional alpha-beta pruning + heuristics are
| basically neck and neck with the very best of neural network
| based techniques. I'll trust stockfish over a alpha-zero or
| MuZero for awhile longer if I'm trying to win a computer
| chess competition ...
| hervature wrote:
| Sure, Stockfish just uses millions of years of evolution to
| build its heuristics and can't be transferred to any other
| game. The point remains, calling RL a cherry on the cake
| compared to unsupervised learning when they are completely
| orthogonal and not mutually exclusive techniques is
| simplistic and unnecessarily offensive.
| unishark wrote:
| So it's bad to be the cake? I assume he means it's the
| foundation one falls back on when the more specialized
| categories of methods are not applicable.
|
| You might not like my analogy either. I think of
| supervised and unsupervised learning as the majority of
| the genome of ML, while RL is that little Y chromosome
| sometimes tacked on to address a few high-profile tasks.
| thom wrote:
| Not disputing your main point, but Stockfish now includes a
| neural network.
| Der_Einzige wrote:
| Wow, I had no idea that Yann LeCun was also this based. I've
| figured that it was relatively rare to hear people advocate for
| the superiority of unsupervised methods - but I guess it isn't
| if a titan like him does. It's good to hear because
| epistemologically I just do not believe that most learning is
| anything but unsupervised. There are very few good labels for
| our data relative to how much data we process in an information
| theory sense.
| webmaven wrote:
| > It's good to hear because epistemologically I just do not
| believe that most learning is anything but unsupervised.
|
| I have a feeling that the lines between the supervised and
| unsupervised categories will get increasingly blurred, with
| semi-supervised, self-supervised (eg. like self-attention)
| and adversarial (eg. GANs) approaches mixing together in
| strange ways.
| burning_hamster wrote:
| I used to think that given enough time, more people would
| also learn that there are no substantial differences
| between supervised and unsupervised learning. However, I
| have come to believe that this might not happen any time
| soon. The supposed difference between supervised and
| unsupervised can be easily explained (label / no label --
| duh?!). It is much harder to explain, why that distinction
| is ultimately a mirage. Couple this with a million low
| quality blog posts on SEO steroids by data grand wizards
| and machine learning architects, I doubt that even a good
| text book jointly written by Bengio, Hinton, and LeCun
| would convince the ML hype train otherwise. But there is
| always hope!
| mirker wrote:
| Sufficiently large boolean satisfiability also enough to reach
| general AI.
|
| I kid, but these sorts of papers are theoretical position papers
| and do not account practical considerations e.g., for how many
| suns of energy must be expended.
| nathias wrote:
| I believe the goal of 'general' AI will have to be modified once
| we know more, what appears as general is probably a set of
| discrete faculties that play well together.
| EGreg wrote:
| AlphaGo can play games against itself. You can have GANs, MCTS
| and more.
|
| General AI requires a feedback mechanism from the real world.
| Unless you have an accurate model of it in a computer, you can't
| just test whether a joke will be funny without waiting for humans
| to laugh. You can't check whether a tailored diet or workout
| regimen or gene therapy will have good results without humans
| trying them.
|
| So you've reduced your AI problem to a harder problem: modeling
| the world and all of its complexity in a computer, and somehow
| being able to run simulations faster than the stuff that happens
| in the actual real world
| The_rationalist wrote:
| Cringe
| throwaway879 wrote:
| If you send a message in a bottle it will eventually land ashore
| somewhere, maybe in a century, who knows, and who knows whether
| it will be relevant by then or not, or civilization may not even
| exist by then, but sure it's similarly plausible to get to AGI
| via RL.
| visarga wrote:
| > send a message in a bottle
|
| Just one - yes. But how about if you send millions of bottle
| messages?
| ausbah wrote:
| then you're in your way to a hit rock song!
| throwaway879 wrote:
| >>how about if you send millions of bottle messages?
|
| assuming we can integrate all learnings from those bottles
| into a system that can classify any given situation and apply
| the learning in that domain. But to build a system that can
| classify any problem is where we're stuck at and RL can't get
| us there
| rexreed wrote:
| It is not enough. Machine learning only gets you so far, even the
| best general-purpose algorithms. Deriving reasoning and common
| sense demands something more than learning.
|
| I agree with the other commenter here that the smartest systems
| are less intelligent than the common housefly. A brain the size
| of a pinpoint can navigate, eat, reproduce, and live a full life
| without big data or internet.
| ggggtez wrote:
| On the one hand; award winning ai specialists. On the other: an
| anonymous internet commenter.
|
| I'm not going to take bets on who is right, but simply saying
| "nuh uh" is not exactly breaking ground.
| [deleted]
| for_i_in_range wrote:
| I concur. There is "Reality", and then there is human's
| conception of "Reality". (See also: "Theory of Forms").
|
| AI and Machine Learning are housed in human's conception of
| Reality. Not Reality itself. We're innovating in a sandbox, but
| we're getting better and learning more to the point of getting
| so good, we may one day get out...
| rexreed wrote:
| There are so many people that conflate AI and machine
| learning and believe they are one and the same thing. They
| are not the same thing. Machine learning is a proper subset
| of AI. Even the best machine learning doesn't embody all of
| AI, because AI is more than just machine learning. Anyone who
| has spent more than a few years doing AI research and not
| just reading popular science blogs knows that AI is not the
| same thing as machine learning.
|
| Yet, here we are, saying that achieving one form of machine
| learning to its fullest extent will be "enough" to meet the
| challenge of AGI.
| blueblisters wrote:
| Current state-of-the-art in reinforcement learning can barely
| make a physical robot walk. In theory, with transfer learning, we
| will probably see better success over time but I'm looking
| forward to seeing results in practice.
|
| A 2018 article about the challenges of reinforcement learning:
| https://www.alexirpan.com/2018/02/14/rl-hard.html
| Barrin92 wrote:
| Sorry but where is actual scientific content in that paper? I'm
| concerned with the state of AI. saying that "reinforcement is all
| you need", when reinforcement learning is defined as abstract as
| "agent does something, adapts to environment and rewards, then
| does another thing" is borderline tautological.
|
| The actual scientific question is, what are the mechanisms that
| make agents work, what are the fundamental modules within
| intelligent systems, is there a distinction between digital and
| biochemical systems, what costs are there in terms of resources
| and energy to get to a certain level of intelligence, and so on.
| Real questions with specific answers. For all the advances coming
| from just upping the amount of data and GPU hours, there is so
| little progress on trying to have a model of the structures that
| underpin intelligence.
| zzzzzzzza wrote:
| i think part of what they are saying is that your approach is
| wrong, (e.g. looking for then copying submodules within
| intelligence won't generalize),
|
| trying to answer specific questions won't generalize,
|
| but if you train a network with the right potentially hacky
| series of rewards/rich enough environment you could get a much
| more general intelligence
|
| a new kind of science
| fooker wrote:
| https://xkcd.com/1123/
| LesZedCB wrote:
| the paper postulates that "reward is enough" but what is the
| alternative? rewards plus what?
|
| the "cybernetic feedback loop" aka reinforcement learning has
| kinda always been the idea, no?
|
| i believe that architecture is important. yes, at one level the
| human is a rewards machine, but the neocortex produces a lot of
| "self-reflection" that doesn't really generate reward in itself,
| at least i don't know if it does. maybe meditations create a
| general dopamine spike?
| exo-pla-net wrote:
| One needs to actually read the paper to understand what it's
| postulating; the paper's title is not enough.
|
| There is a temptation to view our capabilities as a complex set
| of interacting modules. Under that view, each module would need
| to be separately developed in order to artificially recreate
| our abilities. The paper makes the case that we need only a
| single carrot.
|
| Per the paper:
|
| "For example, the ability of social intelligence has often been
| framed as the Nash equilibrium of a multi-agent system; the
| ability of language by a combination of goals such as parsing,
| part-of-speech tagging, lexical analysis, and sentiment
| analysis; and the ability of perception by object segmentation
| and recognition.
|
| In this paper, we consider an alternative hypothesis: that the
| generic objective of maximising reward is enough to drive
| behaviour that exhibits most if not all abilities that are
| studied in natural and artificial intelligence."
___________________________________________________________________
(page generated 2021-06-10 23:03 UTC)