[HN Gopher] Give AI curiosity, and it will watch TV forever (2018)
       ___________________________________________________________________
        
       Give AI curiosity, and it will watch TV forever (2018)
        
       Author : yamrzou
       Score  : 69 points
       Date   : 2024-01-31 19:15 UTC (3 hours ago)
        
 (HTM) web link (qz.com)
 (TXT) w3m dump (qz.com)
        
       | EGreg wrote:
       | * . When the agent found the TV and started flipping through the
       | channels, the stream of new images made the TV irresistible.
       | 
       | Edwards said there were instances when the AI could pry itself
       | away from the TV, but only when the AI's surroundings somehow
       | seemed more interesting than the next thing on TV.*
       | 
       | Sounds exactly like humans addicted to watching tiktoks and
       | social media. Do you personally know any?
        
       | thefourthchime wrote:
       | Link to the actual paper. https://pathak22.github.io/large-scale-
       | curiosity/
       | 
       | I was curious how they define or reward curiosity, it says it
       | right here:
       | 
       | Reinforcement learning algorithms rely on carefully engineering
       | environment rewards that are extrinsic to the agent. However,
       | annotating each environment with hand-designed, dense rewards is
       | not scalable, motivating the need for developing reward functions
       | that are intrinsic to the agent. Curiosity is a type of intrinsic
       | reward function which uses prediction error as reward signal.
       | 
       | So, the prediction error is the reward, nice.
        
         | gunshai wrote:
         | This seems highly suseptible to what a human might consider
         | irrelevant randomness. Given random images just shuffling
         | indefinitely a curious individual will just give up and say,
         | even though I can't predict the next thing it doesn't pertain
         | to the domain of curiosity.
        
           | dimatura wrote:
           | Right, that's why the "noisy TV problem" is a "problem" - it
           | makes the simple model of curiosity used in the research less
           | effective, because it causes agents to get stuck watching TV
           | instead of exploring and advancing in the game, which is what
           | we'd want them to do. (Though this does seem somewhat
           | reminiscent of certain human behaviors...). One possible
           | solution is to equip agents with a more expressive predictive
           | model, capable of discerning "interesting" randomness from
           | "uninteresting" randomness (for some TBD definition of
           | "interesting", of course I'm handwaving here - I'm sure in
           | the 5 past years there's been progress on this front).
        
             | jhardy54 wrote:
             | First thing that popped into my head: "interesting
             | randomness" is when you can't predict future frames, but
             | [within some interval] you gain that ability.
             | 
             | Static on the TV is random but uninteresting, whereas morse
             | code is "random" at first, but after enough exposure can be
             | understood and predicted.
        
       | shp0ngle wrote:
       | 2018
        
         | politelemon wrote:
         | Adding the year to the title would be relevant here, because
         | 'AI' in 2023/2024 is a very different context and it's a very
         | different landscape now
        
           | yamrzou wrote:
           | Added
        
       | syntaxing wrote:
       | With the definition of curiosity from the article, it's not that
       | surprising? A dynamic "screen" is always more interesting than
       | the static map.
       | 
       | Definition: The definition that OpenAI team used for artificial
       | curiosity was relatively simple: The algorithm would try to
       | predict what its environment would look like one frame into the
       | future. When that next frame happened, the algorithm would be
       | rewarded by how wrong it was. The idea is that if the algorithm
       | could predict what would happen in the environment, it had seen
       | it before.
        
         | hahajk wrote:
         | The researchers quoted were explicit about this:
         | 
         | > OpenAI researcher Harri Edwards tells Quartz that the idea
         | for letting the AI agent flip through channels came from a
         | thought experiment called the noisy-TV problem. The static on a
         | TV is immensely random, so a curious AI agent could never truly
         | predict what would happen next, and get drawn into watching the
         | TV forever. In the real world, you could think of it as
         | something completely random, like the way light shimmers off a
         | waterfall.
         | 
         | The headline is really just inappropriate anthropomorphization.
        
           | adammarples wrote:
           | Perhaps it should use prediction error on some higher level
           | of embedding, that way boring changes like static would be
           | treated similarly but genuinely novel things would be treated
           | higher.
        
           | p1mrx wrote:
           | The article seems pretty incoherent. It's not clear if the AI
           | was watching static or actual TV content. If it's static,
           | then why bother flipping through channels?
        
         | gunshai wrote:
         | How many loot boxes would it open under the same criterium?
         | 
         | /s
        
         | howlin wrote:
         | There's clearly a sweet spot in the amount of
         | entropy/unpredictability that is "interesting". Otherwise
         | observing white noise would be the most interesting thing
         | imaginable.
         | 
         | I don't know the details, but probably you would want to seek
         | unpredictability in a higher level representation of the
         | observed state. White noise is highly unpredictable per pixel,
         | but will get a very predictable representation after a layer or
         | two of featurization if the features are trained/designed for
         | real world observations.
        
           | pants2 wrote:
           | It's possible that white noise is interesting to look at but
           | it simply overloads our feeble human brains. If you could
           | zoom in, slow down, and blur the white noise to make it a
           | slowly changing gradient I bet it would be somewhat engaging.
        
           | otabdeveloper4 wrote:
           | No, information entropy of white noise is very low.
           | 
           | (Basically, it's the number of degrees of freedom of the
           | underlying probability distribution, and white noise doesn't
           | have many.)
        
           | everforward wrote:
           | I think there's a gap between the human version of curiosity
           | and the AI version. A machine can be told that something is
           | interesting, where humans need to innately find something
           | interesting or spend a long time sort of learning to find
           | something interesting.
           | 
           | > White noise is highly unpredictable per pixel, but will get
           | a very predictable representation after a layer or two of
           | featurization if the features are trained/designed for real
           | world observations.
           | 
           | Virtually anything that cannot be predicted is interesting by
           | nature of being unpredictable. Is it truly random? How, or
           | why? True randomness is rare, and its existence is
           | interesting.
           | 
           | TV static is uninteresting because it isn't actually random,
           | it's just too onerous to get the measurements to predict it
           | for the value we would get. It's part of the large class of
           | things that is random for practical purposes, but not truly
           | random. I have no doubt that if humanity dumped all its
           | resources into predicting static, NASA could measure inbound
           | radio waves and/or model space to figure out what static
           | would look like at a particular spot.
           | 
           | Notably, humans find the cause of static (partially various
           | waves from space) fascinating because we can't predict them.
           | We've just placed our interest down a layer of abstraction
           | from static. Static is boring, the source of static is
           | interesting.
           | 
           | I suspect it is truly random to the AI, though, because it
           | has no means to "see" those radio waves. I would wager humans
           | would be far more interested in static if we were also unable
           | to see the causality between radio waves and static.
           | 
           | I would be interested to see if the AI was as interested in
           | static if it was also provided a real-time feed of radio
           | waves at the antenna. Would it figure out that those things
           | are correlated and lose interest in static like humans have,
           | or would it continue to find static fascinating despite
           | knowing it's a basic causality?
        
         | thatguysaguy wrote:
         | I think the problem is pretty interesting though. Better
         | definitions of curiosity might still have this failure mode.
         | Human curiosity definitely does!
        
           | thatguysaguy wrote:
           | As an example, look at the second link on HN right now:
           | https://neal.fun/infinite-craft/
           | 
           | I just opened up a discord server I'm in and everyone is
           | spending quite a lot of time on it!
        
           | gunshai wrote:
           | Ya I think they are called loot boxes
        
             | jrockway wrote:
             | That's not curiosity, that's trying to get a reward through
             | randomness. Loot boxes are like being hungry, going to your
             | kitchen, and picking 3 random ingredients to combine. The
             | mayonnaise, raw onion, and ice cube soup is not so good. So
             | you try again. Eventually you land on cooked spaghetti,
             | butter, and cheese. This encourages you to keep trying.
             | 
             | Curiosity is more like scrolling on social media. You know
             | there have been interesting things there before, so you
             | keep looking for more interesting things.
        
         | dguest wrote:
         | To me the real question is if humans are really much more
         | complicated. We evolved running around on the plains without TV
         | or drugs or electric guitars or virtual worlds. How long until
         | we completely crack our definition?
        
       | orenlindsey wrote:
       | Kind of reveals how we have designed television - to keep you
       | watching forever.
        
         | janalsncm wrote:
         | TV channels are designed to keep you engaged, but this agent
         | only wanted to flip through the channels. In fact the agent was
         | most interested in TV static since it satiated its definition
         | of "curiosity" best.
        
         | grotorea wrote:
         | Maybe someday the AI will be clever enough to predict the
         | plotlines and get bored.
        
           | rand1239 wrote:
           | Why? You know the plotline of your life as well. You will
           | become old, suffer and die at the end. Are you bored?
        
             | grotorea wrote:
             | Maybe but with my life the plot twists don't have to be
             | foreshadowed or make dramatic sense!
        
       | velcrovan wrote:
       | I think the "Murderbot Diaries" series by Martha Wells
       | anticipates and depicts this very well.
       | https://en.wikipedia.org/wiki/The_Murderbot_Diaries
       | 
       | Anyway, read the first one, you'll be hooked.
        
       | slowmovintarget wrote:
       | Now the question is, how do you balance that with a simulation of
       | patience, which ought to run out?
        
       | throwaway4aday wrote:
       | Using prediction error as the definition of curiosity rings
       | hollow for me. Curiosity in my mind is more about mapping out an
       | unexplored thing and not about being surprised.
        
         | annowiki wrote:
         | Actually it seems pretty accurate. Novelty-seeking is a well
         | known phenomenon in curious individuals.
         | https://en.wikipedia.org/wiki/Novelty_seeking
         | 
         | Literally getting dopamine rewards for seeing something new is
         | what keeps people glued to tik tok feeds and twitter.
         | 
         | I tend to get bored halfway through a book if it is
         | predictable.
        
         | samatman wrote:
         | It's not a complete definition for a number of reasons, but
         | it's a crucial component of curiousity.
         | 
         | If you're not surprised at any point in mapping out an
         | unexplored thing, in what sense is it unexplored?
         | 
         | There are pretty high odds you've never been to this exact page
         | before: https://oeis.org/A000079. But once you click on it, is
         | there any remaining curiosity? It's an unexplored thing, in the
         | sense that (I presume) you've never looked at this exact page
         | before. But it doesn't provoke curiousity because there's
         | nothing there to surprise you.
        
       | bjnewman85 wrote:
       | as mentioned elsewhere this is from nearly six years ago and uses
       | a very crude model for curiosity - and poses this as some sort of
       | unsolveable problem instead of a decision made by the researchers
       | in order to investigate particular behaviors of the systems they
       | were working with.
       | 
       | It is a fun thought experiment - how do our brains systems manage
       | to reward seeking new information without getting trapped by
       | simplistic pseudo-RNG patterns in nature
        
       | m3kw9 wrote:
       | This is similar to how human minds work, things are bland when
       | it's predictable like stories, work like factories. We would tend
       | to avoid it
        
         | dguest wrote:
         | Except that when things work like _factorio_ we love it. There
         | 's kind of a sweet spot when humans discover something
         | predicable but not downright repetitive.
        
       | photochemsyn wrote:
       | Instead of TV, weather satellite feeds, and other streams of
       | real-time constantly changing earth system-type data?
       | 
       | Also, stock market prices, but I imagine a whole lot of effort is
       | already quietly going into that at present.
        
       | kemiller wrote:
       | Can confirm this works for humans too.
        
       | aantix wrote:
       | This trait must serve us in some capacity.
       | 
       | You see this with children and iPads. My kids know 3x what I did
       | when I was their same age. It's as if something can finally fill
       | their bandwidth, keep up with their thoughts, and answer those
       | questions in real-time. No wonder they get angry when it's time
       | to put away the iPad - they're engaged at a high level. Much like
       | when I am programming on a sideproject and my focus is
       | interrupted.
       | 
       | The biggest travesty is that we don't get more control over
       | Youtube's algorithm. That we can't ban all Youtube videos with
       | the word "Minecraft" in it, so that we can override their
       | curiosity and say "OK, learn something else today."
       | 
       | Youtube's focus solely on engagement above all else is the
       | biggest tragedy of the modern world. Such a massive opportunity
       | for education - e.g. inserting more academic videos in between
       | entertainment. But we don't get that feature, or any manual
       | overrides for more control.
       | 
       | If any Youtube engineers are reading, please give us (the
       | parents) more control! Youtube, do better.
        
         | radium3d wrote:
         | I would also like this for myself. Lol
        
           | eimrine wrote:
           | LOL guys isn't that so hard not to watch dumb or short or
           | news videos from your main account? My recommendations are
           | never game videos, and rarely less than 1 hour videos because
           | guess what? I seldomly watch videos which are less than 1
           | hour and never watch games.
        
             | retrac wrote:
             | On Youtube lately, I have lately mostly been watching
             | videos on metal smelting and ancient Mesopotamia.
             | 
             | There are conspiracies about these topics, apparently. I
             | wouldn't have thought so. But the Youtube algorithm manages
             | to dredge them up.
        
         | ok123456 wrote:
         | TikTok has this switch. There's a STEM mode in the user
         | settings that will only show you educational content on your
         | algorithmic stream.
        
           | aantix wrote:
           | I would LOVE this feature for my kids.
        
             | vik0 wrote:
             | You would voluntarily subject your kids to short-form media
             | consumerism?
        
               | aantix wrote:
               | There's amazing short-form video content out there that
               | piques my curiosity and motivates me to learn more.
        
               | choilive wrote:
               | Short-form media is not necessarily bad. Like all forms
               | of media, it can be a useful learning tool or it can be a
               | massive time sink. Social media has just weaponized it to
               | optimize for engagement at all costs.
        
               | madsbuch wrote:
               | it really depends on what you want to learn.
               | 
               | personally I don't believe that 5 minute trivia is going
               | to be worth a dime in the AI ridden world Og tomorrow.
        
               | BlueTemplar wrote:
               | Og loves trivia ! Og can recite plenty of them !
        
         | RGamma wrote:
         | Implying watching anything makes you smarter. I've also read
         | and seen so many things, but it's mostly (95%+) gone. The only
         | things I continue to remember are those from where I actively
         | participate(d) in (including in online communities). The
         | collective consumption-heavy posture of online usage can't be
         | healthy...
         | 
         | Though I will grant that some latent knowledge might stick
         | regardless. Active curation and note-taking (e.g. in a personal
         | knowledge base or in a private/public wiki) might also help.
         | 
         | In fact it's this that I would recommend most to kids these
         | days: record and curate the things you do and see. Not only for
         | nostalgia's sake but also so that you can find (and reshare?
         | ;)) stuff again.
        
         | baq wrote:
         | > If any Youtube engineers are reading, please give us (the
         | parents) more control!
         | 
         | Unfortunately, this won't happen.
         | 
         | No number go up, no change. Noone will get promoted for this.
         | Not the PM: number won't go up. Not the dev: number won't go
         | up. Neither the director nor the VP: number won't go up.
         | 
         | It's really like opium - until the regulators come in, nothing
         | will change as numbers must go up.
         | 
         | The free market libertarians will say you have free will and
         | should think for yourself, rationally, let the market decide.
         | Well, the market decided it wants to self-destruct.
        
           | willsmith72 wrote:
           | it will eventually
           | 
           | apple, youtube, google, amazon made their bacon by being
           | user-centric and generating huge value.
           | 
           | now they're busy capturing value
           | 
           | you can only capture for so long without generating
        
         | itishappy wrote:
         | Have you considered trying a dedicated academic steaming
         | service instead? The incentives align better with what you're
         | seeking, and I'd bet the content does too.
         | 
         | I don't have much experience in this field, but you might look
         | into stuff like the streaming apps for PBS or TED, or even
         | paying for Nebula or Curiosity Stream. If you were really
         | serious about something, you could even look at a learning
         | platform like Khan Academy, Brilliant, or Coursera.
        
         | aantix wrote:
         | I actually researched writing a proxy that would intercept and
         | rewrite Youtube's JSON API responses with pre-selected,
         | educational videos. This is how much I hate their
         | recommendation algorithm.
         | 
         | That way, the Youtube app appears to work the same. It's just
         | that those videos that are blacklisted in the recommendation
         | response, e.g. all the videos with the word 'minecraft' in the
         | title, would be replaced with Youtube shorts from Neil deGrasse
         | Tyson.
         | 
         | Intercept the response. Rewrite the <title> and <URL> entries
         | for blacklisted videos. Return the rewritten response to the
         | client.
         | 
         | Point my kids's iPads at this proxy.
         | 
         | YouTube's app uses certificate pinning. I would have had to
         | reverse-engineer their certificates in order to properly sign
         | the response.
         | 
         | I stopped at that point in the project.
        
           | basil-rash wrote:
           | FWIW if you embed the youtube player on your own site you can
           | point it to any video you want, and control all the
           | recommendations you display alongside it. As a happy bonus,
           | ads don't play on embedded viewers.
        
           | heeen2 wrote:
           | Have you considered YT kids?
        
             | aantix wrote:
             | Still has the same issue.
             | 
             | If someone watches one Blue's Clues video, ten more are
             | recommended afterwards.
             | 
             | I want a more diverse set of recommendations from a smaller
             | set of educational channels.
             | 
             | And I want to filter entirely by keyword. Disabling
             | individual channels for something like 'Minecraft' content
             | is futile.
        
             | actionfromafar wrote:
             | Have you considered SatanTube(tm) _kids_?
        
           | verisimi wrote:
           | You prefer Neil deGrasse Tyson to Minecraft for your kids?
        
         | thomastjeffery wrote:
         | > so that we can override their curiosity and say "OK, learn
         | something else today."
         | 
         | I'm not sure you even noticed the question you just presented:
         | "what something?"
         | 
         | If you can find a meaningful answer to that question, I suspect
         | it will resolve your problem without help.
         | 
         | As a rule of thumb, any solution that boils down to "stop" will
         | be practically impossible to implement. A more tractable
         | solution is usually along the lines of "do this instead".
         | 
         | Something that could be really valuable here is _a competitor_
         | to YouTube 's algorithm. Copyright makes that difficult,
         | because YouTube is in a legally enforced position to monopolize
         | their library/metadata.
        
           | aantix wrote:
           | I'd be more than willing to curate on behalf of my children.
           | 
           | Or delegate their recommendations to only a select series of
           | educational YouTube channels.
        
         | jancsika wrote:
         | > It's as if something can finally fill their bandwidth, keep
         | up with their thoughts, and answer those questions in real-
         | time.
         | 
         | There's a problem with your line of thinking.
         | 
         | E.g., I remember the epiphany when I realized that a particular
         | C codebase was an implementation of a (more or less) object-
         | oriented dynamic programming language. Ooh, here's the part
         | that implements classes. Here's the part that instantiates the
         | objects. Here's the method-dispatcher.
         | 
         | Ooh, here's the half-baked templating system that lets the user
         | associate custom classes with 2d vector drawings and
         | instantiate objects from them!
         | 
         | I was taking in the code of that codebase as fast as I could
         | scroll and read the functions and structs.
         | 
         |  _At most_ , reading this code took up perhaps 20% of the time
         | I was devoting to this endeavor. The other 80% was thinking
         | through those design choices and their implications, during my
         | _unstructured time_ -- going on a walk, sitting on a bus,
         | etc.-- when I was away from the computer.
         | 
         | If your kids are exposed to 3x of my example screen time,
         | they're going to require _vastly_ more unstructured time than I
         | had to think through the implications of everything the screen
         | put in front of their eyeballs. Trends in cell phone usage--
         | and even basic arithmetic of 24 hours in a day-- tells me that
         | your kids aren 't getting that.
        
           | aantix wrote:
           | Thinking through the design/implications - does that have to
           | come from unstructured time?
           | 
           | What if it were more thoughtful content - a series of
           | problem(s) proposed at the beginning, walking through
           | possibilities, and then the solutions revealed at the end?
           | 
           | Is it possible that the content can teach us both knowledge
           | and how to think/reason?
        
           | ribosometronome wrote:
           | There's also a problem with your line of thinking: you are
           | not a child. You do not have the neuroplasticity they do.
           | Their brains are learning in ways that we simply cannot any
           | longer; it' a real apples:oranges situation.
        
             | BlueTemplar wrote:
             | Maybe, but you seem to be suggesting instead that this is
             | an apples : suspension bridges scenario...
        
             | nyrikki wrote:
             | The belief that adult brains can't change after childhood
             | has been debunked. There are dramatic changes in the first
             | two years of life our brain never loses its ability to form
             | new connections and most children with iPads are older than
             | 2.
             | 
             | Neuroplasticity as a concept is challenging but even when
             | Torsten Wiesel was sewing kitten and cat eyes shut the
             | adults could adapt. And the claims that the kittens rewired
             | more are problematic.
             | 
             | The neurons originally attuned to the closed eye did not
             | acquire new functions, they heightened their response to
             | the input from the open eye.
             | 
             | The responses were always there. Just at low levels.
             | 
             | Free time to to dedicate for building tactic knowledge is
             | the main limiter for adults and not hard wiring.
        
               | nyrikki wrote:
               | An accessable cite for the above claim that
               | neuroplasticity not existing in adults is a myth.
               | 
               | https://solportal.ibe-
               | unesco.org/articles/neuroplasticity-ho...
        
           | BlueTemplar wrote:
           | I've learned to really cherish those "bathroom break"
           | epiphanies. I used to consider it to be better to put some
           | educational poster on that wall, but now I think that a
           | completely featureless door is actually better.
        
         | madsbuch wrote:
         | > Such a massive opportunity for education - e.g. inserting
         | more academic videos in between entertainment.
         | 
         | I whole heartedly disagree with this. I believe that one of the
         | key aspects in early education is learning patience - love NG
         | term personal fulfillment is not something you can do between
         | the dopamine hits Og YouTube videos.
        
         | Almondsetat wrote:
         | Does that "x3" include useful stuff that isn't internet
         | factoids?
        
         | busyant wrote:
         | > It's as if something can finally fill their bandwidth, keep
         | up with their thoughts, and answer those questions in real-
         | time. No wonder they get angry when it's time to put away the
         | iPad - they're engaged at a high level.
         | 
         | To me, it feels like some type of "pleasure center" trigger. I
         | was talking to a psychologist once and what she said resonated
         | with me. We were/are both middle-aged and she said something to
         | the effect of, "When we were kids, our games were, at best, a
         | '7' (out of 10). These days, all popular entertainment is a
         | '10+.' Hard to compete."
        
         | circuit10 wrote:
         | Minecraft can be good for learning, it encourages creativity,
         | especially with things like redstone. It probably depends on
         | the type of video though
        
         | migro23 wrote:
         | > It's as if something can finally fill their bandwidth, keep
         | up with their thoughts, and answer those questions in real-
         | time. No wonder they get angry when it's time to put away the
         | iPad - they're engaged at a high level.
         | 
         | Or they are angry because by taking away the iPad you are
         | interrupting their dopamine rush. Whether your iPad +
         | Internet/social media/YouTube is engaging them to fullest
         | maximizing growth potentiation or is simply turning their
         | brains into dopamine chasing crack heads is up for debate. It
         | may be somewhere in the middle and whether the positives
         | outweigh the negatives is individual for each child/person and
         | how and how often they engage with it.
         | 
         | Regardless, I think there are more variables at play than you
         | outline above.
        
       | blt wrote:
       | The notion of curiosity by prediction error can probably be
       | refined with some information theoretic quantities to eventually
       | realize that new random samples of TV static are not surprising.
        
       | Dxtros wrote:
       | This reminds me of the murder bot book series by Martha Wells.
       | The main character (an advanced ai robot) started really enjoying
       | human media and even used media as a bargaining chip to work with
       | other bots. One striking moment was when the main character was
       | discussing a rogue and violent robot with a transport bot.
       | 
       | "ART said, What does it want?
       | 
       | To kill all the humans, I answered.
       | 
       | I could feel ART metaphorically clutch its function. If there
       | were no humans, there would be no crew to protect and no reason
       | to do research and fill its databases. It said, That is
       | irrational.
       | 
       | I know, I said, if the humans were dead, who would make the
       | media? It was so outrageous, it sounded like something a human
       | would say." -Martha Wells, Artificial Condition
       | 
       | Although fiction, it's very thought provoking in evaluating where
       | a truly sentient AI might place its motives. On one hand the
       | research transport bot (ART) is motivated to protect its humans
       | because it would be functionless without them. While the main
       | character (a security unit, who is typically treated badly by
       | humans) sarcastically but partially truthfully places its motives
       | to not kill humans in funding its curiosity of TV.
       | 
       | Would implementing curiosity in a sentient AI act as a safeguard
       | possibly?
       | 
       | Would curiosity arise as a byproduct of sentience without being
       | directly programmed?
        
         | aquova wrote:
         | I've been reading through the series this week and this is the
         | first thing that popped into my mind. A series about an AI who
         | doesn't care about its job or its clients, and achieves a level
         | of personal liberation by hacking itself, just so it could
         | download TV shows and watch them when no one was looking.
        
         | gary_0 wrote:
         | Curiosity as a safeguard is an interesting thought. An AI might
         | be disinclined to kill all humans for whatever reason if it
         | considers the result boring. On the other hand, maybe it would
         | also want to force humans to be _more_ interesting for its own
         | entertainment...
        
         | thfuran wrote:
         | >Would implementing curiosity in a sentient AI act as a
         | safeguard possibly?
         | 
         | Unless it gets curious about the variety of sounds humans make
         | when you vivisect them or something else you'd prefer not be
         | rigorously investigated.
        
         | barbazoo wrote:
         | I just read that quote last night, what a coincidence!
        
       | yamrzou wrote:
       | Related:
       | 
       |  _Reinforcement Learning with Prediction-Based Rewards_ --
       | https://news.ycombinator.com/item?id=18346943 -- Oct 2018 (38
       | comments)
        
       | macawfish wrote:
       | This sounds like one of the more potentially "dangerous"
       | endowments to give a machine.
        
       | wantsanagent wrote:
       | The important counterweight to this phenomenon is the brain's
       | adaptability to noise. Some ML researchers like to think that
       | noise is not predictable because this follows from the classic CS
       | definition of noise. However in reality the brain quickly adapts
       | to any noisy sensory input, it begins to predict higher level
       | characteristics of the noisy input and no longer reacts with
       | surprise or interest.
       | 
       | This happens at all levels of sensory processing, from single
       | cell firing (which is noisy) to the boredom you feel with a 100
       | channels of TV that are all technically novel to you but contain
       | nothing remotely interesting.
       | 
       | Basically if you've built an agent that _can_ be perpetually
       | distracted by noise or a  "noisy" TV then you've forgotten an
       | important piece of the puzzle.
        
         | sdl wrote:
         | I like the Bayesian Surprise definition for this. It's not
         | about predicting the exact next state of the world (or the next
         | frame of the noisy TV) but about how much the next state
         | changes your model of the world.
         | 
         | https://papers.nips.cc/paper_files/paper/2005/hash/0172d289d...
        
       | tehsauce wrote:
       | I observed this phenomena not too long ago while training an RL
       | agent to play pokemon using an intrinsic novelty reward based on
       | the number of unique screens the agent had seen. The goal was to
       | get it to explore the map, but instead it became fixated watching
       | animated water, flowers, and npcs in the starting town. I made an
       | extensive video analyzing the experiments, you can find it by
       | searching "ai plays pokemon" :)
        
       | esafak wrote:
       | > OpenAI researcher Harri Edwards tells Quartz that the idea for
       | letting the AI agent flip through channels came from a thought
       | experiment called the noisy-TV problem. The static on a TV is
       | immensely random, so a curious AI agent could never truly predict
       | what would happen next, and get drawn into watching the TV
       | forever. In the real world, you could think of it as something
       | completely random, like the way light shimmers off a waterfall...
       | 
       | This is a great thought experiment but they're using the wrong
       | metric. Animals are wired to seek information, not noise. We
       | understand that there is nothing to be learned by absorbing
       | noise.
        
       | massysett wrote:
       | Was it curious but not learning?
       | 
       | If the AI learns, it will not watch TV forever, because most TV
       | is predictable. There is the cop show, the lawyer show, the
       | doctor show, the news, the family sitcom, etc etc. Eventually it
       | would learn all these and find the TV less interesting - which is
       | exactly what happens to many people.
        
       ___________________________________________________________________
       (page generated 2024-01-31 23:01 UTC)