[HN Gopher] If you are using LLM RAG - you should be doing RAFT
       ___________________________________________________________________
        
       If you are using LLM RAG - you should be doing RAFT
        
       Author : shishirpatil
       Score  : 24 points
       Date   : 2024-03-19 18:31 UTC (4 hours ago)
        
 (HTM) web link (techcommunity.microsoft.com)
 (TXT) w3m dump (techcommunity.microsoft.com)
        
       | tianjunz wrote:
       | This is a three-way collaboration between Berkeley AI, Microsoft
       | Azure, and Meta AI! RAFT involves the concept of domain-specific
       | RAG, which represents a more focused and growingly favored area
       | compared to the broader concept of the general open-book exam. In
       | such exams, the domain in which the LLM will be evaluated is
       | known in advance and used for inference. The LLM is capable of
       | addressing prompts by leveraging any and all information from
       | this particular domain, on which it has been specifically fine-
       | tuned.
       | 
       | Blogs: https://gorilla.cs.berkeley.edu/blogs/9_raft.html
        
       | petervandijck wrote:
       | It's "Retrieval Augmented Fine Tuning". The related blog post is
       | interesting: https://gorilla.cs.berkeley.edu/blogs/9_raft.html
        
       | jondwillis wrote:
       | Page me when this process is at least partially automated and
       | continuous.
        
       | jnwatson wrote:
       | Does any of this matter when you have 5 million token input and
       | can just shove everything into the input?
        
         | jscheel wrote:
         | Yes, because 5 million token input is slow, expensive, and
         | error-prone.
        
       | catchnear4321 wrote:
       | > They hypothesized that a student who studies the textbooks
       | before the open-book exam was likely to perform better than a
       | student who studies the textbook.
       | 
       | interesting hypothesis, but probably not what was meant.
       | 
       | "editor" is shaping up to be the hot new job for fleshy
       | consciousnesses by the early thirties.
        
       | matt3D wrote:
       | Am I missing the point or is this not how everyone was doing RAG
       | before?
       | 
       | I've had my much greater success doing the RAG process on a fine
       | tuned model.
        
         | dragonwriter wrote:
         | > Am I missing the point or is this not how everyone was doing
         | RAG before?
         | 
         | No, especially people wrapping RAG frameworks around models
         | they don't have access to fine tune (e.g., GPT-4 if you aren't
         | OpenAI and/or Microsoft.)
         | 
         | Edit: Of course, further evidence that this method is useful
         | doesn't help people in that condition, though.
        
       ___________________________________________________________________
       (page generated 2024-03-19 23:01 UTC)