[HN Gopher] GSoC'24: Differentiable Logic for Interactive System...
       ___________________________________________________________________
        
       GSoC'24: Differentiable Logic for Interactive Systems and
       Generative Music
        
       Author : jarmitage
       Score  : 54 points
       Date   : 2024-09-24 17:02 UTC (5 hours ago)
        
 (HTM) web link (ijc8.me)
 (TXT) w3m dump (ijc8.me)
        
       | benob wrote:
       | Difflogic should lead to very efficient hardware implementations
       | of inference for a wide range of problems. But is it going to
       | allow to generate the logical chains that elude current LLMs?
       | 
       | --- edit ---
       | 
       | After reading the paper more thoroughly, I find the way they
       | implement differentiable logic clever. They use continuous
       | relaxations of 16 logic operators, run them in parallel and apply
       | a softmax to select the most useful operator. At inference time,
       | everything is binarized.
        
         | gwern wrote:
         | The original cited paper https://arxiv.org/abs/2210.08277 "Deep
         | Differentiable Logic Gate Networks" struck me then as very
         | clever, ultra-efficient/small, hardly any inductive bias or
         | prior... But you have to wonder if it's able to scale
         | reasonably well. Differentiating continuous versions of 16
         | discrete operations in parallel sounds expensive, especially
         | since you presumably need a bunch of them chained in order to
         | approximate a single neural primitive. Even distilling an
         | existing LLM down might be too hard.
        
           | pizza wrote:
           | It's definitely slept on. I do think it ought to be very
           | powerful given enough compute to throw at it, hopefully. I
           | think short description length algorithms such as simple
           | compression algorithms or instant-ngp could be interesting to
           | play with through that paradigm.
        
       | Archit3ch wrote:
       | Most interesting! :D
       | 
       | Logic gates implemented with non-ideal transistors have non-zero
       | rise times. Therefore, they are smooth and differentiable.
        
       ___________________________________________________________________
       (page generated 2024-09-24 23:00 UTC)