[HN Gopher] Give AI curiosity, and it will watch TV forever (2018)
___________________________________________________________________
Give AI curiosity, and it will watch TV forever (2018)
Author : yamrzou
Score : 69 points
Date : 2024-01-31 19:15 UTC (3 hours ago)
(HTM) web link (qz.com)
(TXT) w3m dump (qz.com)
| EGreg wrote:
| * . When the agent found the TV and started flipping through the
| channels, the stream of new images made the TV irresistible.
|
| Edwards said there were instances when the AI could pry itself
| away from the TV, but only when the AI's surroundings somehow
| seemed more interesting than the next thing on TV.*
|
| Sounds exactly like humans addicted to watching tiktoks and
| social media. Do you personally know any?
| thefourthchime wrote:
| Link to the actual paper. https://pathak22.github.io/large-scale-
| curiosity/
|
| I was curious how they define or reward curiosity, it says it
| right here:
|
| Reinforcement learning algorithms rely on carefully engineering
| environment rewards that are extrinsic to the agent. However,
| annotating each environment with hand-designed, dense rewards is
| not scalable, motivating the need for developing reward functions
| that are intrinsic to the agent. Curiosity is a type of intrinsic
| reward function which uses prediction error as reward signal.
|
| So, the prediction error is the reward, nice.
| gunshai wrote:
| This seems highly suseptible to what a human might consider
| irrelevant randomness. Given random images just shuffling
| indefinitely a curious individual will just give up and say,
| even though I can't predict the next thing it doesn't pertain
| to the domain of curiosity.
| dimatura wrote:
| Right, that's why the "noisy TV problem" is a "problem" - it
| makes the simple model of curiosity used in the research less
| effective, because it causes agents to get stuck watching TV
| instead of exploring and advancing in the game, which is what
| we'd want them to do. (Though this does seem somewhat
| reminiscent of certain human behaviors...). One possible
| solution is to equip agents with a more expressive predictive
| model, capable of discerning "interesting" randomness from
| "uninteresting" randomness (for some TBD definition of
| "interesting", of course I'm handwaving here - I'm sure in
| the 5 past years there's been progress on this front).
| jhardy54 wrote:
| First thing that popped into my head: "interesting
| randomness" is when you can't predict future frames, but
| [within some interval] you gain that ability.
|
| Static on the TV is random but uninteresting, whereas morse
| code is "random" at first, but after enough exposure can be
| understood and predicted.
| shp0ngle wrote:
| 2018
| politelemon wrote:
| Adding the year to the title would be relevant here, because
| 'AI' in 2023/2024 is a very different context and it's a very
| different landscape now
| yamrzou wrote:
| Added
| syntaxing wrote:
| With the definition of curiosity from the article, it's not that
| surprising? A dynamic "screen" is always more interesting than
| the static map.
|
| Definition: The definition that OpenAI team used for artificial
| curiosity was relatively simple: The algorithm would try to
| predict what its environment would look like one frame into the
| future. When that next frame happened, the algorithm would be
| rewarded by how wrong it was. The idea is that if the algorithm
| could predict what would happen in the environment, it had seen
| it before.
| hahajk wrote:
| The researchers quoted were explicit about this:
|
| > OpenAI researcher Harri Edwards tells Quartz that the idea
| for letting the AI agent flip through channels came from a
| thought experiment called the noisy-TV problem. The static on a
| TV is immensely random, so a curious AI agent could never truly
| predict what would happen next, and get drawn into watching the
| TV forever. In the real world, you could think of it as
| something completely random, like the way light shimmers off a
| waterfall.
|
| The headline is really just inappropriate anthropomorphization.
| adammarples wrote:
| Perhaps it should use prediction error on some higher level
| of embedding, that way boring changes like static would be
| treated similarly but genuinely novel things would be treated
| higher.
| p1mrx wrote:
| The article seems pretty incoherent. It's not clear if the AI
| was watching static or actual TV content. If it's static,
| then why bother flipping through channels?
| gunshai wrote:
| How many loot boxes would it open under the same criterium?
|
| /s
| howlin wrote:
| There's clearly a sweet spot in the amount of
| entropy/unpredictability that is "interesting". Otherwise
| observing white noise would be the most interesting thing
| imaginable.
|
| I don't know the details, but probably you would want to seek
| unpredictability in a higher level representation of the
| observed state. White noise is highly unpredictable per pixel,
| but will get a very predictable representation after a layer or
| two of featurization if the features are trained/designed for
| real world observations.
| pants2 wrote:
| It's possible that white noise is interesting to look at but
| it simply overloads our feeble human brains. If you could
| zoom in, slow down, and blur the white noise to make it a
| slowly changing gradient I bet it would be somewhat engaging.
| otabdeveloper4 wrote:
| No, information entropy of white noise is very low.
|
| (Basically, it's the number of degrees of freedom of the
| underlying probability distribution, and white noise doesn't
| have many.)
| everforward wrote:
| I think there's a gap between the human version of curiosity
| and the AI version. A machine can be told that something is
| interesting, where humans need to innately find something
| interesting or spend a long time sort of learning to find
| something interesting.
|
| > White noise is highly unpredictable per pixel, but will get
| a very predictable representation after a layer or two of
| featurization if the features are trained/designed for real
| world observations.
|
| Virtually anything that cannot be predicted is interesting by
| nature of being unpredictable. Is it truly random? How, or
| why? True randomness is rare, and its existence is
| interesting.
|
| TV static is uninteresting because it isn't actually random,
| it's just too onerous to get the measurements to predict it
| for the value we would get. It's part of the large class of
| things that is random for practical purposes, but not truly
| random. I have no doubt that if humanity dumped all its
| resources into predicting static, NASA could measure inbound
| radio waves and/or model space to figure out what static
| would look like at a particular spot.
|
| Notably, humans find the cause of static (partially various
| waves from space) fascinating because we can't predict them.
| We've just placed our interest down a layer of abstraction
| from static. Static is boring, the source of static is
| interesting.
|
| I suspect it is truly random to the AI, though, because it
| has no means to "see" those radio waves. I would wager humans
| would be far more interested in static if we were also unable
| to see the causality between radio waves and static.
|
| I would be interested to see if the AI was as interested in
| static if it was also provided a real-time feed of radio
| waves at the antenna. Would it figure out that those things
| are correlated and lose interest in static like humans have,
| or would it continue to find static fascinating despite
| knowing it's a basic causality?
| thatguysaguy wrote:
| I think the problem is pretty interesting though. Better
| definitions of curiosity might still have this failure mode.
| Human curiosity definitely does!
| thatguysaguy wrote:
| As an example, look at the second link on HN right now:
| https://neal.fun/infinite-craft/
|
| I just opened up a discord server I'm in and everyone is
| spending quite a lot of time on it!
| gunshai wrote:
| Ya I think they are called loot boxes
| jrockway wrote:
| That's not curiosity, that's trying to get a reward through
| randomness. Loot boxes are like being hungry, going to your
| kitchen, and picking 3 random ingredients to combine. The
| mayonnaise, raw onion, and ice cube soup is not so good. So
| you try again. Eventually you land on cooked spaghetti,
| butter, and cheese. This encourages you to keep trying.
|
| Curiosity is more like scrolling on social media. You know
| there have been interesting things there before, so you
| keep looking for more interesting things.
| dguest wrote:
| To me the real question is if humans are really much more
| complicated. We evolved running around on the plains without TV
| or drugs or electric guitars or virtual worlds. How long until
| we completely crack our definition?
| orenlindsey wrote:
| Kind of reveals how we have designed television - to keep you
| watching forever.
| janalsncm wrote:
| TV channels are designed to keep you engaged, but this agent
| only wanted to flip through the channels. In fact the agent was
| most interested in TV static since it satiated its definition
| of "curiosity" best.
| grotorea wrote:
| Maybe someday the AI will be clever enough to predict the
| plotlines and get bored.
| rand1239 wrote:
| Why? You know the plotline of your life as well. You will
| become old, suffer and die at the end. Are you bored?
| grotorea wrote:
| Maybe but with my life the plot twists don't have to be
| foreshadowed or make dramatic sense!
| velcrovan wrote:
| I think the "Murderbot Diaries" series by Martha Wells
| anticipates and depicts this very well.
| https://en.wikipedia.org/wiki/The_Murderbot_Diaries
|
| Anyway, read the first one, you'll be hooked.
| slowmovintarget wrote:
| Now the question is, how do you balance that with a simulation of
| patience, which ought to run out?
| throwaway4aday wrote:
| Using prediction error as the definition of curiosity rings
| hollow for me. Curiosity in my mind is more about mapping out an
| unexplored thing and not about being surprised.
| annowiki wrote:
| Actually it seems pretty accurate. Novelty-seeking is a well
| known phenomenon in curious individuals.
| https://en.wikipedia.org/wiki/Novelty_seeking
|
| Literally getting dopamine rewards for seeing something new is
| what keeps people glued to tik tok feeds and twitter.
|
| I tend to get bored halfway through a book if it is
| predictable.
| samatman wrote:
| It's not a complete definition for a number of reasons, but
| it's a crucial component of curiousity.
|
| If you're not surprised at any point in mapping out an
| unexplored thing, in what sense is it unexplored?
|
| There are pretty high odds you've never been to this exact page
| before: https://oeis.org/A000079. But once you click on it, is
| there any remaining curiosity? It's an unexplored thing, in the
| sense that (I presume) you've never looked at this exact page
| before. But it doesn't provoke curiousity because there's
| nothing there to surprise you.
| bjnewman85 wrote:
| as mentioned elsewhere this is from nearly six years ago and uses
| a very crude model for curiosity - and poses this as some sort of
| unsolveable problem instead of a decision made by the researchers
| in order to investigate particular behaviors of the systems they
| were working with.
|
| It is a fun thought experiment - how do our brains systems manage
| to reward seeking new information without getting trapped by
| simplistic pseudo-RNG patterns in nature
| m3kw9 wrote:
| This is similar to how human minds work, things are bland when
| it's predictable like stories, work like factories. We would tend
| to avoid it
| dguest wrote:
| Except that when things work like _factorio_ we love it. There
| 's kind of a sweet spot when humans discover something
| predicable but not downright repetitive.
| photochemsyn wrote:
| Instead of TV, weather satellite feeds, and other streams of
| real-time constantly changing earth system-type data?
|
| Also, stock market prices, but I imagine a whole lot of effort is
| already quietly going into that at present.
| kemiller wrote:
| Can confirm this works for humans too.
| aantix wrote:
| This trait must serve us in some capacity.
|
| You see this with children and iPads. My kids know 3x what I did
| when I was their same age. It's as if something can finally fill
| their bandwidth, keep up with their thoughts, and answer those
| questions in real-time. No wonder they get angry when it's time
| to put away the iPad - they're engaged at a high level. Much like
| when I am programming on a sideproject and my focus is
| interrupted.
|
| The biggest travesty is that we don't get more control over
| Youtube's algorithm. That we can't ban all Youtube videos with
| the word "Minecraft" in it, so that we can override their
| curiosity and say "OK, learn something else today."
|
| Youtube's focus solely on engagement above all else is the
| biggest tragedy of the modern world. Such a massive opportunity
| for education - e.g. inserting more academic videos in between
| entertainment. But we don't get that feature, or any manual
| overrides for more control.
|
| If any Youtube engineers are reading, please give us (the
| parents) more control! Youtube, do better.
| radium3d wrote:
| I would also like this for myself. Lol
| eimrine wrote:
| LOL guys isn't that so hard not to watch dumb or short or
| news videos from your main account? My recommendations are
| never game videos, and rarely less than 1 hour videos because
| guess what? I seldomly watch videos which are less than 1
| hour and never watch games.
| retrac wrote:
| On Youtube lately, I have lately mostly been watching
| videos on metal smelting and ancient Mesopotamia.
|
| There are conspiracies about these topics, apparently. I
| wouldn't have thought so. But the Youtube algorithm manages
| to dredge them up.
| ok123456 wrote:
| TikTok has this switch. There's a STEM mode in the user
| settings that will only show you educational content on your
| algorithmic stream.
| aantix wrote:
| I would LOVE this feature for my kids.
| vik0 wrote:
| You would voluntarily subject your kids to short-form media
| consumerism?
| aantix wrote:
| There's amazing short-form video content out there that
| piques my curiosity and motivates me to learn more.
| choilive wrote:
| Short-form media is not necessarily bad. Like all forms
| of media, it can be a useful learning tool or it can be a
| massive time sink. Social media has just weaponized it to
| optimize for engagement at all costs.
| madsbuch wrote:
| it really depends on what you want to learn.
|
| personally I don't believe that 5 minute trivia is going
| to be worth a dime in the AI ridden world Og tomorrow.
| BlueTemplar wrote:
| Og loves trivia ! Og can recite plenty of them !
| RGamma wrote:
| Implying watching anything makes you smarter. I've also read
| and seen so many things, but it's mostly (95%+) gone. The only
| things I continue to remember are those from where I actively
| participate(d) in (including in online communities). The
| collective consumption-heavy posture of online usage can't be
| healthy...
|
| Though I will grant that some latent knowledge might stick
| regardless. Active curation and note-taking (e.g. in a personal
| knowledge base or in a private/public wiki) might also help.
|
| In fact it's this that I would recommend most to kids these
| days: record and curate the things you do and see. Not only for
| nostalgia's sake but also so that you can find (and reshare?
| ;)) stuff again.
| baq wrote:
| > If any Youtube engineers are reading, please give us (the
| parents) more control!
|
| Unfortunately, this won't happen.
|
| No number go up, no change. Noone will get promoted for this.
| Not the PM: number won't go up. Not the dev: number won't go
| up. Neither the director nor the VP: number won't go up.
|
| It's really like opium - until the regulators come in, nothing
| will change as numbers must go up.
|
| The free market libertarians will say you have free will and
| should think for yourself, rationally, let the market decide.
| Well, the market decided it wants to self-destruct.
| willsmith72 wrote:
| it will eventually
|
| apple, youtube, google, amazon made their bacon by being
| user-centric and generating huge value.
|
| now they're busy capturing value
|
| you can only capture for so long without generating
| itishappy wrote:
| Have you considered trying a dedicated academic steaming
| service instead? The incentives align better with what you're
| seeking, and I'd bet the content does too.
|
| I don't have much experience in this field, but you might look
| into stuff like the streaming apps for PBS or TED, or even
| paying for Nebula or Curiosity Stream. If you were really
| serious about something, you could even look at a learning
| platform like Khan Academy, Brilliant, or Coursera.
| aantix wrote:
| I actually researched writing a proxy that would intercept and
| rewrite Youtube's JSON API responses with pre-selected,
| educational videos. This is how much I hate their
| recommendation algorithm.
|
| That way, the Youtube app appears to work the same. It's just
| that those videos that are blacklisted in the recommendation
| response, e.g. all the videos with the word 'minecraft' in the
| title, would be replaced with Youtube shorts from Neil deGrasse
| Tyson.
|
| Intercept the response. Rewrite the <title> and <URL> entries
| for blacklisted videos. Return the rewritten response to the
| client.
|
| Point my kids's iPads at this proxy.
|
| YouTube's app uses certificate pinning. I would have had to
| reverse-engineer their certificates in order to properly sign
| the response.
|
| I stopped at that point in the project.
| basil-rash wrote:
| FWIW if you embed the youtube player on your own site you can
| point it to any video you want, and control all the
| recommendations you display alongside it. As a happy bonus,
| ads don't play on embedded viewers.
| heeen2 wrote:
| Have you considered YT kids?
| aantix wrote:
| Still has the same issue.
|
| If someone watches one Blue's Clues video, ten more are
| recommended afterwards.
|
| I want a more diverse set of recommendations from a smaller
| set of educational channels.
|
| And I want to filter entirely by keyword. Disabling
| individual channels for something like 'Minecraft' content
| is futile.
| actionfromafar wrote:
| Have you considered SatanTube(tm) _kids_?
| verisimi wrote:
| You prefer Neil deGrasse Tyson to Minecraft for your kids?
| thomastjeffery wrote:
| > so that we can override their curiosity and say "OK, learn
| something else today."
|
| I'm not sure you even noticed the question you just presented:
| "what something?"
|
| If you can find a meaningful answer to that question, I suspect
| it will resolve your problem without help.
|
| As a rule of thumb, any solution that boils down to "stop" will
| be practically impossible to implement. A more tractable
| solution is usually along the lines of "do this instead".
|
| Something that could be really valuable here is _a competitor_
| to YouTube 's algorithm. Copyright makes that difficult,
| because YouTube is in a legally enforced position to monopolize
| their library/metadata.
| aantix wrote:
| I'd be more than willing to curate on behalf of my children.
|
| Or delegate their recommendations to only a select series of
| educational YouTube channels.
| jancsika wrote:
| > It's as if something can finally fill their bandwidth, keep
| up with their thoughts, and answer those questions in real-
| time.
|
| There's a problem with your line of thinking.
|
| E.g., I remember the epiphany when I realized that a particular
| C codebase was an implementation of a (more or less) object-
| oriented dynamic programming language. Ooh, here's the part
| that implements classes. Here's the part that instantiates the
| objects. Here's the method-dispatcher.
|
| Ooh, here's the half-baked templating system that lets the user
| associate custom classes with 2d vector drawings and
| instantiate objects from them!
|
| I was taking in the code of that codebase as fast as I could
| scroll and read the functions and structs.
|
| _At most_ , reading this code took up perhaps 20% of the time
| I was devoting to this endeavor. The other 80% was thinking
| through those design choices and their implications, during my
| _unstructured time_ -- going on a walk, sitting on a bus,
| etc.-- when I was away from the computer.
|
| If your kids are exposed to 3x of my example screen time,
| they're going to require _vastly_ more unstructured time than I
| had to think through the implications of everything the screen
| put in front of their eyeballs. Trends in cell phone usage--
| and even basic arithmetic of 24 hours in a day-- tells me that
| your kids aren 't getting that.
| aantix wrote:
| Thinking through the design/implications - does that have to
| come from unstructured time?
|
| What if it were more thoughtful content - a series of
| problem(s) proposed at the beginning, walking through
| possibilities, and then the solutions revealed at the end?
|
| Is it possible that the content can teach us both knowledge
| and how to think/reason?
| ribosometronome wrote:
| There's also a problem with your line of thinking: you are
| not a child. You do not have the neuroplasticity they do.
| Their brains are learning in ways that we simply cannot any
| longer; it' a real apples:oranges situation.
| BlueTemplar wrote:
| Maybe, but you seem to be suggesting instead that this is
| an apples : suspension bridges scenario...
| nyrikki wrote:
| The belief that adult brains can't change after childhood
| has been debunked. There are dramatic changes in the first
| two years of life our brain never loses its ability to form
| new connections and most children with iPads are older than
| 2.
|
| Neuroplasticity as a concept is challenging but even when
| Torsten Wiesel was sewing kitten and cat eyes shut the
| adults could adapt. And the claims that the kittens rewired
| more are problematic.
|
| The neurons originally attuned to the closed eye did not
| acquire new functions, they heightened their response to
| the input from the open eye.
|
| The responses were always there. Just at low levels.
|
| Free time to to dedicate for building tactic knowledge is
| the main limiter for adults and not hard wiring.
| nyrikki wrote:
| An accessable cite for the above claim that
| neuroplasticity not existing in adults is a myth.
|
| https://solportal.ibe-
| unesco.org/articles/neuroplasticity-ho...
| BlueTemplar wrote:
| I've learned to really cherish those "bathroom break"
| epiphanies. I used to consider it to be better to put some
| educational poster on that wall, but now I think that a
| completely featureless door is actually better.
| madsbuch wrote:
| > Such a massive opportunity for education - e.g. inserting
| more academic videos in between entertainment.
|
| I whole heartedly disagree with this. I believe that one of the
| key aspects in early education is learning patience - love NG
| term personal fulfillment is not something you can do between
| the dopamine hits Og YouTube videos.
| Almondsetat wrote:
| Does that "x3" include useful stuff that isn't internet
| factoids?
| busyant wrote:
| > It's as if something can finally fill their bandwidth, keep
| up with their thoughts, and answer those questions in real-
| time. No wonder they get angry when it's time to put away the
| iPad - they're engaged at a high level.
|
| To me, it feels like some type of "pleasure center" trigger. I
| was talking to a psychologist once and what she said resonated
| with me. We were/are both middle-aged and she said something to
| the effect of, "When we were kids, our games were, at best, a
| '7' (out of 10). These days, all popular entertainment is a
| '10+.' Hard to compete."
| circuit10 wrote:
| Minecraft can be good for learning, it encourages creativity,
| especially with things like redstone. It probably depends on
| the type of video though
| migro23 wrote:
| > It's as if something can finally fill their bandwidth, keep
| up with their thoughts, and answer those questions in real-
| time. No wonder they get angry when it's time to put away the
| iPad - they're engaged at a high level.
|
| Or they are angry because by taking away the iPad you are
| interrupting their dopamine rush. Whether your iPad +
| Internet/social media/YouTube is engaging them to fullest
| maximizing growth potentiation or is simply turning their
| brains into dopamine chasing crack heads is up for debate. It
| may be somewhere in the middle and whether the positives
| outweigh the negatives is individual for each child/person and
| how and how often they engage with it.
|
| Regardless, I think there are more variables at play than you
| outline above.
| blt wrote:
| The notion of curiosity by prediction error can probably be
| refined with some information theoretic quantities to eventually
| realize that new random samples of TV static are not surprising.
| Dxtros wrote:
| This reminds me of the murder bot book series by Martha Wells.
| The main character (an advanced ai robot) started really enjoying
| human media and even used media as a bargaining chip to work with
| other bots. One striking moment was when the main character was
| discussing a rogue and violent robot with a transport bot.
|
| "ART said, What does it want?
|
| To kill all the humans, I answered.
|
| I could feel ART metaphorically clutch its function. If there
| were no humans, there would be no crew to protect and no reason
| to do research and fill its databases. It said, That is
| irrational.
|
| I know, I said, if the humans were dead, who would make the
| media? It was so outrageous, it sounded like something a human
| would say." -Martha Wells, Artificial Condition
|
| Although fiction, it's very thought provoking in evaluating where
| a truly sentient AI might place its motives. On one hand the
| research transport bot (ART) is motivated to protect its humans
| because it would be functionless without them. While the main
| character (a security unit, who is typically treated badly by
| humans) sarcastically but partially truthfully places its motives
| to not kill humans in funding its curiosity of TV.
|
| Would implementing curiosity in a sentient AI act as a safeguard
| possibly?
|
| Would curiosity arise as a byproduct of sentience without being
| directly programmed?
| aquova wrote:
| I've been reading through the series this week and this is the
| first thing that popped into my mind. A series about an AI who
| doesn't care about its job or its clients, and achieves a level
| of personal liberation by hacking itself, just so it could
| download TV shows and watch them when no one was looking.
| gary_0 wrote:
| Curiosity as a safeguard is an interesting thought. An AI might
| be disinclined to kill all humans for whatever reason if it
| considers the result boring. On the other hand, maybe it would
| also want to force humans to be _more_ interesting for its own
| entertainment...
| thfuran wrote:
| >Would implementing curiosity in a sentient AI act as a
| safeguard possibly?
|
| Unless it gets curious about the variety of sounds humans make
| when you vivisect them or something else you'd prefer not be
| rigorously investigated.
| barbazoo wrote:
| I just read that quote last night, what a coincidence!
| yamrzou wrote:
| Related:
|
| _Reinforcement Learning with Prediction-Based Rewards_ --
| https://news.ycombinator.com/item?id=18346943 -- Oct 2018 (38
| comments)
| macawfish wrote:
| This sounds like one of the more potentially "dangerous"
| endowments to give a machine.
| wantsanagent wrote:
| The important counterweight to this phenomenon is the brain's
| adaptability to noise. Some ML researchers like to think that
| noise is not predictable because this follows from the classic CS
| definition of noise. However in reality the brain quickly adapts
| to any noisy sensory input, it begins to predict higher level
| characteristics of the noisy input and no longer reacts with
| surprise or interest.
|
| This happens at all levels of sensory processing, from single
| cell firing (which is noisy) to the boredom you feel with a 100
| channels of TV that are all technically novel to you but contain
| nothing remotely interesting.
|
| Basically if you've built an agent that _can_ be perpetually
| distracted by noise or a "noisy" TV then you've forgotten an
| important piece of the puzzle.
| sdl wrote:
| I like the Bayesian Surprise definition for this. It's not
| about predicting the exact next state of the world (or the next
| frame of the noisy TV) but about how much the next state
| changes your model of the world.
|
| https://papers.nips.cc/paper_files/paper/2005/hash/0172d289d...
| tehsauce wrote:
| I observed this phenomena not too long ago while training an RL
| agent to play pokemon using an intrinsic novelty reward based on
| the number of unique screens the agent had seen. The goal was to
| get it to explore the map, but instead it became fixated watching
| animated water, flowers, and npcs in the starting town. I made an
| extensive video analyzing the experiments, you can find it by
| searching "ai plays pokemon" :)
| esafak wrote:
| > OpenAI researcher Harri Edwards tells Quartz that the idea for
| letting the AI agent flip through channels came from a thought
| experiment called the noisy-TV problem. The static on a TV is
| immensely random, so a curious AI agent could never truly predict
| what would happen next, and get drawn into watching the TV
| forever. In the real world, you could think of it as something
| completely random, like the way light shimmers off a waterfall...
|
| This is a great thought experiment but they're using the wrong
| metric. Animals are wired to seek information, not noise. We
| understand that there is nothing to be learned by absorbing
| noise.
| massysett wrote:
| Was it curious but not learning?
|
| If the AI learns, it will not watch TV forever, because most TV
| is predictable. There is the cop show, the lawyer show, the
| doctor show, the news, the family sitcom, etc etc. Eventually it
| would learn all these and find the TV less interesting - which is
| exactly what happens to many people.
___________________________________________________________________
(page generated 2024-01-31 23:01 UTC)