[HN Gopher] Minimal auto-differentiation engine in Rust
       ___________________________________________________________________
        
       Minimal auto-differentiation engine in Rust
        
       Author : lschneider
       Score  : 51 points
       Date   : 2025-06-20 13:53 UTC (9 hours ago)
        
 (HTM) web link (github.com)
 (TXT) w3m dump (github.com)
        
       | tnlogy wrote:
       | Nice! I made a small toy version myself to learn Rust and freshen
       | up on ML. https://github.com/tnlogy/telegrad
       | 
       | I wanted to store the graph in a heap to be able to send it to
       | the gpu later on, but then I got lazy and abandoned it. But you
       | always learn something. :)
        
         | kragen wrote:
         | That sounds interesting; what do you mean by "in a heap"? Is
         | the stack they're currently linearized into not GPU-friendly? I
         | don't know much about GPU programming, so this might be a dumb
         | question.
        
           | tnlogy wrote:
           | My idea was to make a Vec of nodes with pointers to indexes
           | in the vec, so it would be easier to send this array into the
           | gpu. I wanted to make a minimal version example of making a
           | micrograd network run on the gpu, with wgpu or macroquad, but
           | I didn't complete it, so would be nice if someone else did
           | it. :)
        
             | kragen wrote:
             | I see! I thought that was the stack.
        
       | kragen wrote:
       | Probably it would be good to put "backward-mode" in the title.
        
       | amelius wrote:
       | Looks like this uses mutation of global/shared state. For the
       | example:
       | 
       | z=x*y+3,
       | 
       | what if there is another function that does:
       | 
       | w=x+2*y
       | 
       | and then both functions do a backward pass (simultaneously,
       | perhaps in different threads or otherwise); then it seems
       | dangerous to collect the results of the backward pass (partial
       | derivatives) in the shared variables x and y and make them
       | accessible through x.get_grad() and y.get_grad(). Imho, in a
       | better design, you'd say z.get_grad(x) and z.get_grad(y), and
       | w.get_grad(x) and w.get_grad(y) to get the partial derivatives.
        
       ___________________________________________________________________
       (page generated 2025-06-20 23:00 UTC)