[HN Gopher] Laser: Learning a Latent Action Space for Efficient ...
       ___________________________________________________________________
        
       Laser: Learning a Latent Action Space for Efficient Reinforcement
       Learning
        
       Author : tosh
       Score  : 55 points
       Date   : 2021-04-04 11:55 UTC (11 hours ago)
        
 (HTM) web link (arxiv.org)
 (TXT) w3m dump (arxiv.org)
        
       | thisisauserid wrote:
       | Department of Redundancy Department (please knock twice, please)
        
       | elasticventures wrote:
       | So they manually restricted the search space for possible answers
       | and it went faster, like a LASER? Is this April Fools?
        
         | hntrader wrote:
         | Looks like they used a variational encoder-decoder net to
         | reduce the dimensionality of the action space.
         | 
         | Not manual and it sounds like a good idea.
        
           | networdtwo wrote:
           | Author here, was pretty surprised to see this on HN when
           | browsing over my coffee this morning. Your interpretation is
           | correct, you use an encoder-decoder model to figure out what
           | the dimensions of the task best for learning are.
           | 
           | The drawback is you can only learn tasks which are relatively
           | similar (any time you restrict what motions are possible to
           | improve learning, you obviously restrict what tasks are
           | possible). The benefit is that you can learn tasks which do
           | fall within the learned motion ranges a lot more quickly.
           | 
           | The best analogy within 'classical' control is task space
           | control, where you do control in cartesian dimensions rather
           | than the joint positions. But this has its own drawbacks in
           | that you have to define these controllers manually, and
           | Cartesian space is not sufficiently expressive / appropriate
           | for many tasks.
        
             | [deleted]
        
         | haffi112 wrote:
         | They used a variational autoencoder where the latent space
         | representation is disentangled.
         | 
         | That approach is a promising way to make it easier to navigate
         | the latent space as changes in one dimension will have a
         | reduced or no influence on other aspects of the data encoded.
         | 
         | Here is a nice overview on disentanglement with further
         | references: https://paperswithcode.com/method/beta-vae
        
         | lloeki wrote:
         | Ludicrous acronyms shirk efficient reasoning.
        
       | MasterScrat wrote:
       | There was already a LASER method in reinforcement learning :-/
       | 
       | LArge Scale Experience Replay - https://arxiv.org/abs/1909.11583
       | 
       | I am missing a link between the two?
        
       | abrichr wrote:
       | More info (including a video):
       | 
       | https://www.pair.toronto.edu/laser/
        
       ___________________________________________________________________
       (page generated 2021-04-04 23:01 UTC)