[HN Gopher] Distributed Inference and Fine-Tuning of Large Langu...
       ___________________________________________________________________
        
       Distributed Inference and Fine-Tuning of Large Language Models over
       the Internet
        
       Author : cyanf
       Score  : 57 points
       Date   : 2024-01-02 16:06 UTC (6 hours ago)
        
 (HTM) web link (browse.arxiv.org)
 (TXT) w3m dump (browse.arxiv.org)
        
       | amrb wrote:
       | Can check out their project at https://github.com/bigscience-
       | workshop/petals
        
       | knightoffaith wrote:
       | Great that folks are starting to think about this, I think it's
       | important to keep the power of AI decentralized. Thought it seems
       | like for this to be practical, they need to flesh out the
       | Security section in the appendix to deal with bad actors. I like
       | the probabilistic approach in lieu of full byzantine fault
       | tolerance. Would be interesting to see what kind of guarantees we
       | could still have about correctness/convergence even with bad
       | actors.
        
       | pama wrote:
       | Here is the main arxiv page (html is a mess on lockdown mode):
       | 
       | https://arxiv.org/abs/2312.08361
        
       | bugglebeetle wrote:
       | A distributed network of GPUs for something other than funny-
       | money sudoku puzzle-solving is a nice development.
        
         | lifty wrote:
         | Would be nice to have some kind of proof of useful AI work.
        
           | monkeydust wrote:
           | Made me think of Gridcoin and BOINC
           | https://boinc.berkeley.edu/
        
           | sdenton4 wrote:
           | If you have to pay $X for AI inference, then it proves that
           | the work is worth at least $X to you... Which suggests the
           | obvious marketplace scheme: Compute providers are paid (or
           | credited) $X/k to contribute their nodes to the network, and
           | users pay $X for inference.
           | 
           | (inevitably, someone will implement this with o.g. bitcoin
           | and we can all cry.)
        
       | sjwhevvvvvsj wrote:
       | I've long assumed something like this existed in a corporate lab
       | but hadn't been made public (or, ahem, "open").
       | 
       | It's an obvious move as open source models get bigger, really
       | happy to see this out in the world - especially with a HF author
       | attached.
        
       ___________________________________________________________________
       (page generated 2024-01-02 23:01 UTC)