[HN Gopher] DiLoCo: Distributed Low-Communication Training of La...
       ___________________________________________________________________
        
       DiLoCo: Distributed Low-Communication Training of Language Models
        
       Author : Anon84
       Score  : 14 points
       Date   : 2023-12-06 20:55 UTC (2 hours ago)
        
 (HTM) web link (arxiv.org)
 (TXT) w3m dump (arxiv.org)
        
       | lucubratory wrote:
       | So we can SETI@Home or Folding@Home for large language models,
       | now? Not sure how small the minimum size of the compute cluster
       | can be. If it's still out of consumer reach then this would
       | either be just an intermediate research step, or a way for small-
       | er (but still professional/well-resourced) labs to collaborate
       | together. I'm not sure the latter would be helpful, as if they
       | wanted to collaborate together they could probably already do
       | that by pooling resources for a large cloud compute run.
        
       ___________________________________________________________________
       (page generated 2023-12-06 23:00 UTC)