[HN Gopher] Speeding Up Reinforcement Learning with a New Physic...
       ___________________________________________________________________
        
       Speeding Up Reinforcement Learning with a New Physics Simulation
       Engine
        
       Author : apsec112
       Score  : 63 points
       Date   : 2021-07-16 18:29 UTC (1 days ago)
        
 (HTM) web link (ai.googleblog.com)
 (TXT) w3m dump (ai.googleblog.com)
        
       | anon_tor_12345 wrote:
       | if you're wondering why lots of these differentiable pipelines
       | are tasked with learning physics (and what that has to do with
       | google) the answer is that this is "compute oriented
       | development". by "compute oriented development" i mean that since
       | google has access to unlimited compute they can use this compute
       | to run physics kinematics solvers (ie pde solvers) that are then
       | used to generate training data for RL models. what's the point of
       | the RL model if the physics model already exists and gives you
       | high fidelity simulations? well it's clearly an easy paper to
       | write... but other than that, some people claim the RL models are
       | faster than the physics solver. i guess that's true if you don't
       | take into account the millions of hours of compute spent on the
       | solvers themselves.
        
         | nmca wrote:
         | (disclaimer: work on RL, have trained models for simulated
         | tasks)
         | 
         | I'm fairly sure that people work on control because general
         | algorithms for control would be very useful (e.g., robot that
         | can skin a cat and drive a car by holding the steering wheel).
         | Such a robot would exist in our 3d physical world, so
         | simulations of of our 3d world are used for training. If this
         | could be done with radically less compute, it would be.
        
           | anon_tor_12345 wrote:
           | sure but it doesn't hurt that you have infinite data too
           | (i.e. the thing most other ML research is bound by). like you
           | can't argue that it's not a very comfortable corner to be in
           | wrt being able to publish.
        
             | kadoban wrote:
             | Sounds quite a bit like you're complaining that they
             | chose/engineered a fruitful field of study. I think I'm
             | missing what the problem with that is.
        
         | velcroscientist wrote:
         | Good point. This is why robotics researchers do not take deep
         | RL papers seriously unless they have some real world robotics
         | results. I'm looking at you, people who only show mujoco
         | results and claim their algorithm is useful for robotics.
         | 
         | Simulators are useful though for real world robotics. You can
         | prototype your environment and algorithm, and also attempt
         | sim2real transfer. For example, use the simulator to generate a
         | lot of image data, and train image based controllers. Add
         | enough domain randomization and _maybe_ your controller trained
         | on the simulator can transfer to real images.
        
       ___________________________________________________________________
       (page generated 2021-07-17 23:01 UTC)