[HN Gopher] U-Net CNN in APL: Exploring Zero-Framework, Zero-Lib...
       ___________________________________________________________________
        
       U-Net CNN in APL: Exploring Zero-Framework, Zero-Library Machine
       Learning
        
       Author : tosh
       Score  : 55 points
       Date   : 2023-06-09 12:31 UTC (10 hours ago)
        
 (HTM) web link (dl.acm.org)
 (TXT) w3m dump (dl.acm.org)
        
       | lopuhin wrote:
       | Impressively concise implementation, really interesting paper!
       | Benchmark looks quite questionable though -- e.g. they use fp64
       | (while any sane person would use at least f32, if not f16), batch
       | 1 (while normally one would try to get the max batch size which
       | fits into memory, and it would reach 1 only for much bigger
       | models or inputs), and also measure the time including transfer
       | to/from GPU (while it would normally be interleaved with GPU
       | operations). Not sure what results would look like in a more
       | realistic setup, but still getting within 2x of PyTorch even in
       | such a setting looks impressive!
        
       | mlajtos wrote:
       | Nobody should write backward pass by hand.
        
         | constantcrying wrote:
         | Why? As a learning experience implementing backpropagation is
         | extremely helpful, implementing an entire FNN/CNN from scratch
         | is, to be honest.
         | 
         | Also Implementing some basic automatic differentiation is
         | something you should probably have done once in your life you
         | are interested in Machine learning or numerical mathematics.
        
           | mkaic wrote:
           | Agreed. Implementing backprop myself--even if it _was_ a
           | crappy, slow version in MatLab--is what finally got me to
           | understand it. I 've worked as an ML researcher for 2 years
           | since then and I'm still routinely happy that I have that
           | deeper understanding of what's going on under the hood of the
           | models I'm training.
        
         | mcbuilder wrote:
         | Tell that to any ML Compiler Engineer
        
           | dybber wrote:
           | Standard ML? I'm getting confused
        
         | dpflan wrote:
         | Maybe grad students...
        
       | natas wrote:
       | I wonder what it would look like on kdb+?
        
       | sctb wrote:
       | It's neat to see ongoing Co-dfns work from Aaron and others!
       | There are a number of YouTube videos online if anyone is
       | interested in very cool and esoteric yet serious programming:
       | https://www.youtube.com/playlist?list=PLDU0iEj6f8duXzmgnlGX4....
        
       ___________________________________________________________________
       (page generated 2023-06-09 23:01 UTC)