[HN Gopher] MuZero's first step from research into the real world
___________________________________________________________________
MuZero's first step from research into the real world
Author : tokyopanda
Score : 52 points
Date : 2022-02-25 15:51 UTC (7 hours ago)
(HTM) web link (deepmind.com)
(TXT) w3m dump (deepmind.com)
| syntaxing wrote:
| Kind of confused how this work, doesn't my browser have to
| support this compression as well? Like watching it on deepmind's
| website, doesn't there have to be another layer of compression
| for me to watch it?
| jeffbee wrote:
| The decoder in your player can be considered a programmable
| machine, and the job of the encoder is to emit an optimal
| program that achieves or approximates the desired results. Just
| like any program compiler, there will be more than one way to
| do it, depending on how much space and time you are willing to
| dedicate to the job.
| espadrine wrote:
| Video uses codecs which encode information in frames of various
| types; in particular, the I-frame is essentially a self-
| contained image, a bit like JPEG, and the P-frame only encodes
| the changes from the previous frame.
|
| The choice of when to have an I-frame or a P-frame is
| arbitrary, but the rendered video will look the same. However,
| too many I-frames can bloat the filesize, and too few can
| degrade the appearance significantly as errors add up.
|
| They act on a codec parameter related to the I-frame, to pick
| better rules for good compression without visible errors in the
| P-frame.
| TaylorAlexander wrote:
| They are using MuZero to tweak encoder settings on a codec your
| computer already supports. It is decoded with existing
| decoders.
| cyrux004 wrote:
| They talk about compression but leave the most important metric
| out. What's your Weissman score ?
| [deleted]
| ajnin wrote:
| Can someone enlighten me how this paves the way to AGI? At the
| surface this looks like a rather mundane optimization problem.
| eigenvalue wrote:
| Seems like a sensible project for them since shaving 4% of
| YouTube's traffic translates to millions of dollars in savings.
| But I'm more excited about the possibility of using deep learning
| image models to get incredibly higher compression rates. Some of
| the work I've seen on de-noising and super-resolution suggests
| that we are barely scratching the surface on what might be
| possible in terms of high-def video compression. Of course there
| is something of a time vs. space tradeoff, since these techniques
| would require way more compute for both encoding and decoding.
| But compute is pretty cheap and underutilized on the client side
| now, and Google probably has a huge amount of excess compute
| power to handle usage spikes that could be used for background
| processing.
| smaddox wrote:
| Deep-learning-based image compression is particularly
| interesting because it turns the problem into a massively
| parallel one. As GPUs and TPUs become more powerful and more
| common in commodity hardware, there will be a drive to move
| many existing algorithms/applications to that hardware.
| zitterbewegung wrote:
| In procedurally generated Anime there is Waifu 2x
| https://github.com/nagadomi/waifu2x . After procedural
| generation it is recommended to do denoiseing on the resulting
| image to improve quality.
| _hark wrote:
| While an interesting use of applied RL, in some sense isn't this
| just another way to cast the compression/compute tradeoff? I.e.
| can't we just achieve the same effects by using another
| compression scheme which trades off local compute for better
| compression?
|
| Running MuZero online sounds like a fairly computationally
| expensive prospect...
| jeffbee wrote:
| I doubt this is intended as a real efficiency effort for
| YouTube, since anything that doesn't fit into their hardware-
| accelerated video compression framework isn't going to be
| economical for them.
|
| https://dl.acm.org/doi/abs/10.1145/3445814.3446723
| Veedrac wrote:
| YouTube has different needs for different videos. The average
| barely-viewed video can use quick and dirty hardware
| compression, but a large fraction of views are on a small
| fraction of videos, which is the sort of thing that more
| intense AI-assisted optimization would help with.
| jeffbee wrote:
| Id bet the median YouTube view count is zero, in which case
| the real gains would be from accurate inference of which
| videos fell into that class.
| eigenvalue wrote:
| In that case, they could probably just trigger a re-
| encoding of videos when they cross into 100+ views from
| over 50 unique IP addresses or something like that.
| ulber wrote:
| I think you misunderstood the approach. MuZero is being used to
| optimize the choices made in the VP9 compression. In modern
| video encodings there's many ways to encode the same content.
| As a very simple example, you can vary how often you provide a
| full encoding of a frame and how often you encode differences
| between frames. Once this off-line optimization is done, the
| result is still a valid VP9 encoding, just a smaller one.
| MuZero is not needed for decompression at all.
| gwern wrote:
| The way to think of this is as part of Jeff Dean's "deep-
| learn all the cloud things!" thesis:
| https://www.gwern.net/Tool-AI#dean-2017
|
| A cloud stack, from OS kernel settings to TCP/IP to database
| query optimizers to video codec settings to compiler
| settings, is made of thousands upon thousands of toggleable
| options, each of which is usually left at the default because
| no one on earth understands more than a small fraction of
| them, much less how to set them all appropriately for each
| task end-to-end. It's blackboxes on top of blackboxes all the
| way down. Collectively, inferior options could be giving up
| an incredible amount of performance. As has been demonstrated
| by experts in performance tuning, depending on how pessimal
| the defaults are, you could easily gain orders of magnitude
| performance by setting them to saner settings, much less
| truly optimal settings - these sorts of posts turn up
| routinely on HN, and even in very well-tuned cloud stacks,
| you have to figure that gains like >10% should be possible.
|
| MuZero here shows that it can work for one piece of the
| stack. And MuZero is, by design, an insanely general
| architecture: handles two-player games like chess/Go &
| handles one-player like ALE, handles continuous action spaces
| (Sampled-MuZero), reasonably sample-efficient (because it
| learns an environment model, so using that more is MuZero-
| Reanalyzed), handles hidden information games against
| adversaries (Player of Games), and now OP shows self-play in
| a weird setting. (It still requires problem-specific input
| layers but even that can be lifted if you're willing to pay
| for Perceiver inputs which do arbitrary input modalities.)
|
| So you can see the potential here for doing much more of
| cloud operations (beyond current applications like datacenter
| cooling control) with DRL agents. Plunk down a MuZero on your
| entire stack and assign it the goal of optimizing end-to-end
| for each specific task - DRL is expensive, but cloud-scale is
| even more so. Needless to say, don't expect any released
| checkpoints on Github...
___________________________________________________________________
(page generated 2022-02-25 23:01 UTC)