[HN Gopher] Predictive Coding Approximates Backprop Along Arbitr...
       ___________________________________________________________________
        
       Predictive Coding Approximates Backprop Along Arbitrary Computation
       Graphs (2020)
        
       Author : DanielBMarkham
       Score  : 60 points
       Date   : 2021-05-02 14:06 UTC (8 hours ago)
        
 (HTM) web link (arxiv.org)
 (TXT) w3m dump (arxiv.org)
        
       | yewenjie wrote:
       | AstralCodexTen (formerly SlateStarCodex) has discussed this here
       | - https://astralcodexten.substack.com/p/link-unifying-predicti...
       | 
       | He mostly points to this post in LessWrong -
       | https://www.lesswrong.com/posts/JZZENevaLzLLeC3zn/predictive...
        
       | coolness wrote:
       | Needs [2020] in the title. Interesting work nevertheless.
        
       | [deleted]
        
       | amelius wrote:
       | If backprop is not needed, would this finding make automatic-
       | differentiation functionality obsolete in DL frameworks, allowing
       | these frameworks to become much simpler? Or is there still some
       | constant factor that makes backprop favorable?
        
         | lumost wrote:
         | Reading the openreview link, the current understanding is that
         | this approach is dramatically more computationally intensive
         | than standard backprop - limiting its utility.
        
       | l33tman wrote:
       | This (and its follow up papers) have already been discussed
       | multiple times here. Don't have the links handy though..
        
       | pishpash wrote:
       | Also, most recent state of progress: Predictive Coding Can Do
       | Exact Backpropagation on Any Neural Network (2021)
       | 
       | https://arxiv.org/abs/2103.04689
        
       | obstbraende wrote:
       | here's a link with reviewer comments
       | https://openreview.net/forum?id=PdauS7wZBfC (praise to
       | openreview!)
        
         | pishpash wrote:
         | The decision reasoning is super helpful to put things in
         | context. The arbitrary binary decision to "accept" vs "reject"
         | especially for the snooty "high bar for acceptance at ICLR" is
         | laughable in a world of free information access.
        
       ___________________________________________________________________
       (page generated 2021-05-02 23:01 UTC)