[HN Gopher] Winner of the SF Mistral AI Hackathon: Automated Tes...
       ___________________________________________________________________
        
       Winner of the SF Mistral AI Hackathon: Automated Test Driven
       Prompting
        
       Author : carlcortright
       Score  : 84 points
       Date   : 2024-03-27 17:31 UTC (5 hours ago)
        
 (HTM) web link (prompting.flyflow.dev)
 (TXT) w3m dump (prompting.flyflow.dev)
        
       | carlcortright wrote:
       | "Learning" by prompting, calculating the loss against evals, and
       | updating the prompt
        
       | paradite wrote:
       | Yeah I've been thinking about this lately.
       | 
       | LLMs come and go.
       | 
       | Prompt engineering techniques come and go.
       | 
       | But eval / labelled dataset is always useful once you built it.
        
         | nestorD wrote:
         | In my experience, architecture stays. A nice modular codebase
         | (where you can switch-out the various models and have control
         | on the minutia of the pipeline from data to model) brings a lot
         | of value and can fairly easily be kept up-to-date with best
         | practices.
        
       | bugglebeetle wrote:
       | Isn't this just a very naive implementation of what DsPY does?
       | 
       | https://github.com/stanfordnlp/dspy
       | 
       | I don't understand what is exceptional here.
        
         | carlcortright wrote:
         | (author) It's very similar. I think the extension of this is
         | that we could build a frontend gui for something like this for
         | prompt management. Love what they're doing
        
       | carlcortright wrote:
       | Post Author: getting a lot of requests so scaling the backend.
       | Standby.
        
         | quadrature wrote:
         | You should add a very short description of what this is and how
         | it works. I kind of get that it is a way to tune/generate
         | prompts for specific input output test cases, But its not clear
         | exactly what it does.
        
       | imranq wrote:
       | Shouldn't this be LoRA training?
        
       | hitchstory wrote:
       | I built something like this too:
       | 
       | https://github.com/hitchdev/hitchstory/blob/master/examples%...
        
       | toisanji wrote:
       | can you please add more info on the page to show why it is
       | important and how its helpful
        
         | carlcortright wrote:
         | Yeah I'll do that now!
        
           | imdsm wrote:
           | It'd be great to get a little write up of the technique
        
             | carlcortright wrote:
             | Added it!
        
       | k__ wrote:
       | Just yesterday, I wrote an article about FT and learned about
       | services like Entry Point AI.
       | 
       | Seems like an awesome idea. I'm curious how long it will take to
       | get a model on a reasonable level.
       | 
       | Phind is pretty good and also the fastest model I used recently,
       | so I'd assume it's quite small, no?
        
       ___________________________________________________________________
       (page generated 2024-03-27 23:02 UTC)