[HN Gopher] AIConfig - source control format for gen AI prompts,...
       ___________________________________________________________________
        
       AIConfig - source control format for gen AI prompts, models and
       settings
        
       Author : saqadri
       Score  : 71 points
       Date   : 2023-11-17 17:06 UTC (5 hours ago)
        
 (HTM) web link (github.com)
 (TXT) w3m dump (github.com)
        
       | saqadri wrote:
       | Hi HN! OP here - this is our first open source project and we are
       | really looking for your feedback about this direction, because we
       | haven't seen something like aiconfig in generative AI developer
       | tools yet.
       | 
       | Our basic premise is that AI application development should be
       | config-based, so you can track the prompts, models and model
       | parameters being used more rigorously. Having this AI artifact
       | then lets you iterate on it separately from your application
       | code, and also set up evals that provide "test coverage" for the
       | gen AI parts of your application.
       | 
       | We were also inspired by the ipynb format for Jupyter notebooks,
       | and you'll see parallels to that in the aiconfig format.
       | 
       | Please ask any questions, and share your thoughts on config vs.
       | code.
        
       | smy20011 wrote:
       | Why not store them in code? Easier to read and source controlled.
        
         | saqadri wrote:
         | We have a bit of context about this in the readme:
         | https://github.com/lastmile-ai/aiconfig#what-problem-it-
         | solv.... The main issue with keeping it in code is that it
         | tangles application code with prompts and model-specific logic.
         | 
         | That makes it hard to evaluate the genAI parts of the
         | application, and also iterating on the prompts is not as
         | straightforward as opening up a playground.
         | 
         | Having the config be the source of truth let's you connect it
         | to your application code (and still source controlled), lets
         | you evaluate the config as the AI artifact, and also lets you
         | open the config in a playground to edit and iterate.
         | 
         | For example, compare how much simpler openai function calling
         | becomes with storing the stuff as a config:
         | https://github.com/lastmile-ai/aiconfig/blob/main/cookbooks/...
         | vs using vanilla openai directly
         | (https://github.com/openai/openai-
         | node/blob/v4/examples/funct...)
        
           | throwanem wrote:
           | Yeah, looking at this just briefly it might be the wheel I've
           | (pre)-invented like 1/3 of for one of my couch projects.
           | Definitely can see the appeal of a conventional format here,
           | I'll check it out in more detail when time permits!
        
             | saqadri wrote:
             | Please definitely let me know when you get a chance to try
             | it out! The readme has a link to our discord too if you
             | want to get in touch (and you can email me directly too)
        
               | throwanem wrote:
               | I mean don't get your hopes up too far, it's gonna be a
               | minute :D But I'll pass along what I find when I can!
        
       | activescott wrote:
       | Well done. Now that I've seen your getting started video
       | (https://aiconfig.lastmileai.dev/docs/getting-started) this seems
       | obvious. I feel really silly that I copy & paste from the browser
       | back into a file just so I can save different prompts and results
       | to see what I liked best. Thanks for sharing and for open
       | sourcing!
        
         | saqadri wrote:
         | Thanks! Would love your feedback both on the UX editor as well
         | as the APIs for interacting with aiconfig. We are also working
         | on a local editor so you can have a locally hosted "AI
         | workbook" playground
        
       | jdwyah wrote:
       | I published https://www.prefab.cloud/blog/dynamic-config-for-
       | openai-pyth... a few weeks ago as a way to do config of AI params
       | like temperature, etc. But I've had a few people start using it
       | for prompts as well and I really curious to see where it goes.
       | 
       | In particular for 1. teams that have complex slow deploys, but
       | want to change prompt now 2. when there are data analyst types
       | doing the prompts and people don't want them to be able to "break
       | things". 3. being able to alpha test / rollout / target new
       | prompts easily.
       | 
       | Definitely an interesting question whether prompts is code or
       | configuration.
        
       | thatxliner wrote:
       | How does this compare to Ollama
        
       | kordlessagain wrote:
       | I love this! Been working on something similar that uses
       | containers and Jinja2 templates:
       | https://github.com/kordless/SlothAI. A demo is running at
       | https://mitta.ai. No docs yet and the auth system is getting an
       | upgrade.
        
       ___________________________________________________________________
       (page generated 2023-11-17 23:01 UTC)