https://github.com/normal-computing/outlines Skip to content Toggle navigation Sign up * Product + Actions Automate any workflow + Packages Host and manage packages + Security Find and fix vulnerabilities + Codespaces Instant dev environments + Copilot Write better code with AI + Code review Manage code changes + Issues Plan and track work + Discussions Collaborate outside of code Explore + All features + Documentation + GitHub Skills + Blog * Solutions For + Enterprise + Teams + Startups + Education By Solution + CI/CD & Automation + DevOps + DevSecOps Resources + Customer Stories + White papers, Ebooks, Webinars + Partners * Open Source + GitHub Sponsors Fund open source developers + The ReadME Project GitHub community articles Repositories + Topics + Trending + Collections * Pricing Search or jump to... Search code, repositories, users, issues, pull requests... Search [ ] Clear Search syntax tips Provide feedback We read every piece of feedback, and take your input very seriously. [ ] [ ] Include my email address so I can be contacted Cancel Submit feedback Saved searches Use saved searches to filter your results more quickly Name [ ] Query [ ] To see all available qualifiers, see our documentation. Cancel Create saved search Sign in Sign up You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. {{ message }} normal-computing / outlines Public * Notifications * Fork 40 * Star 1.9k Generative Model Programming normal-computing.github.io/outlines/ License Apache-2.0 license 1.9k stars 40 forks Activity Star Notifications * Code * Issues 55 * Pull requests 10 * Discussions * Actions * Projects 0 * Security * Insights More * Code * Issues * Pull requests * Discussions * Actions * Projects * Security * Insights normal-computing/outlines This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. main Switch branches/tags [ ] Branches Tags Could not load branches Nothing to show {{ refName }} default View all branches Could not load tags Nothing to show {{ refName }} default View all tags Name already in use A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch? Cancel Create 10 branches 8 tags Code * Local * Codespaces * Clone HTTPS GitHub CLI [https://github.com/n] Use Git or checkout with SVN using the web URL. [gh repo clone normal] Work fast with our official CLI. Learn more about the CLI. * Open with GitHub Desktop * Download ZIP Sign In Required Please sign in to use Codespaces. Launching GitHub Desktop If nothing happens, download GitHub Desktop and try again. Launching GitHub Desktop If nothing happens, download GitHub Desktop and try again. Launching Xcode If nothing happens, download Xcode and try again. Launching Visual Studio Code Your codespace will open once ready. There was a problem preparing your codespace, please try again. Latest commit @lukestanley @rlouf lukestanley and rlouf Fix ReAct example link path ... 8cdd72c Aug 15, 2023 Fix ReAct example link path 8cdd72c Git stats * 194 commits Files Permalink Failed to load latest commit information. Type Name Latest commit message Commit time .github/workflows Parse JSON schema into a generation schedule August 14, 2023 12:39 docs Fix 'Hello world' example in README July 13, 2023 15:46 examples Clone and patch initial parse state in the parser example July 19, 2023 20:14 outlines Generate from JSON schema with JSON class August 14, 2023 12:39 tests Generate from JSON schema with JSON class August 14, 2023 12:39 .gitignore Add documentation May 4, 2023 14:26 .pre-commit-config.yaml Update pre-commit versions April 19, 2023 11:05 .readthedocs.yaml Add ReadTheDocs configuration May 4, 2023 14:26 LICENSE Add LICENSE May 18, 2023 17:32 README.md Fix ReAct example link path August 14, 2023 19:23 environment.yml Parse JSON schema into a generation schedule August 14, 2023 12:39 pyproject.toml Parse JSON schema into a generation schedule August 14, 2023 12:39 requirements-doc.txt Add ReadTheDocs configuration May 4, 2023 14:26 setup.cfg Move pytest configuration to pyproject.toml March 29, 2023 13:49 View code [ ] Outlines ~[?] Features Stay tuned for Installation Guided generation Early stopping Multiple choices Type constraint Efficient regex-guided generation Efficient JSON generation following a Pydantic model Prompting Tools Response models Contributing What contributions? How to contribute? Examples Cite Outlines License README.md Outlines Logo Outlines ~[?] Fast and reliable neural text generation. Install * Guided generation * Prompting primitives * Examples * Stay tuned Outlines ~ is a library for neural text generation. You can think of it as a more flexible replacement for the generate method in the transformers library. Outlines ~ helps developers guide text generation to build robust interfaces with external systems. Provides generation methods that guarantee that the output will match a regular expressions, or follow a JSON schema. Outlines ~ provides robust prompting primitives that separate the prompting from the execution logic and lead to simple implementations of few-shot generations, ReAct, meta-prompting, agents, etc. Outlines ~ is designed as a library that is meant to be compatible the broader ecosystem, not to replace it. We use as few abstractions as possible, and generation can be interleaved with control flow, conditionals, custom Python functions and calls to other libraries. Outlines ~ is compatible with all models. It only interfaces with models via the next-token logits. It can be used with API-based models as well. Features * [*] [?]Simple and powerful prompting primitives based on the Jinja templating engine * [*] Guided generation, including multiple choice, type constraints and dynamic stopping * [*] [?] Fast regex-guided generation * [*] Fast JSON generation following a JSON schema or a Pydantic model * [*] Interleave completions with loops, conditionals, and custom Python functions * [*] Caching of generations * [*] Integration with HuggingFace's transformers models Outlines ~ has new releases and features coming every week! Make sure to star and watch this repository to stay up to date. Stay tuned for * Context-Free Grammar guided generation (#178); * Prompt-token alignment so you don't have to think about tokenization details (#201) * An infilling DSL (#182) You can follow @NormalComputing, @remilouf or @BrandonTWillard for regular updates! Installation Outlines is available on PyPi: pip install outlines Guided generation The first step towards reliability of systems that include large language models is to ensure that there is a well-defined interface between their output and user-defined code. Outlines provides ways to control the generation of language models to make their output more predictable. Early stopping You can stop the generation after a given sequence has been found: import outlines.text.generate as generate import outlines.models as models model = models.transformers("gpt2") answer = generate.continuation(model, stop=["."])("Tell me a one-sentence joke.") Multiple choices You can reduce the completion to a choice between multiple possibilities: import outlines.text.generate as generate import outlines.models as models model = models.transformers("gpt2") prompt = labelling("Just awesome", examples) answer = generate.choice(model, ["Positive", "Negative"])(prompt) Type constraint You can instruct the model to only return integers or floats: import outlines.text.generate as generate import outlines.models as models model = models.transformers("gpt2") prompt = "1+1=" answer = generate.integer(model)(prompt) prompt = "sqrt(2)=" answer = generate.float(model)(prompt) Efficient regex-guided generation Outlines also comes with fast regex-guided generation. In fact, the choice, integer and float functions above all use regex-guided generation under the hood: import outlines.models as models import outlines.text.generate as generate model = models.transformers("gpt2-medium") prompt = "Is 1+1=2? " unguided = generate.continuation(model, max_tokens=30)(prompt) guided = generate.regex(model, r"\s*([Yy]es|[Nn]o|[Nn]ever|[Aa]lways)", max_tokens=30)( prompt ) print(unguided) # Is 1+1=2? # # This is probably the most perplexing question. # As I said in one of my articles describing how # I call 2 and 1, there isn't print(guided) # Is 1+1=2? Always import outlines.models as models import outlines.text.generate as generate model = models.transformers("gpt2-medium") prompt = "What is the IP address of the Google DNS servers? " unguided = generate.continuation(model, max_tokens=30)(prompt) guided = generate.regex( model, r"((25[0-5]|2[0-4]\d|[01]?\d\d?)\.){3}(25[0-5]|2[0-4]\d|[01]?\d\d?)", max_tokens=30, )(prompt) print(unguided) # What is the IP address of the Google DNS servers? # # Passive DNS servers are at DNS servers that are private. # In other words, both IP servers are private. The database # does not contain Chelsea Manning print(guided) # What is the IP address of the Google DNS servers? # 2.2.6.1 Unlike other libraries, regex-guided generation in Outlines is almost as fast as non-guided generation. Efficient JSON generation following a Pydantic model Outlines ~ allows to guide the generation process so the output is guaranteed to follow a JSON schema or Pydantic model: from typing import List from enum import Enum from pydantic import BaseModel, constr import outlines.models as models import outlines.text.generate as generate class Weapon(str, Enum): sword = "sword" axe = "axe" mace = "mace" spear = "spear" bow = "bow" crossbow = "crossbow" class Armor(str, Enum): leather = "leather" chainmail = "chainmail" plate = "plate" class Character(BaseModel): name: constr(max_length=10) age: int armor: Armor weapon: Weapon strength: int model = models.transformers("gpt2") sequence = generate.json(model, Character)("Give me a character description") print(sequence) # { # "name": "ranbelt", # "age": 26, # "armor": "chainmail", # "weapon": "bow", # "strength": 5 # } parsed = Character.model_validate_json(sequence) print(parsed) # name='ranbelt' age=26 armor= weapon= strength=5 The method works with union types, optional types, arrays, nested schemas, etc. Some field constraints are not supported yet, but everything else should work. Prompting Writing prompts by concatenating strings in pure Python quickly becomes cumbersome: the prompt building logic gets entangled with the rest of the program, and the structure of the rendered prompt is obfuscated.Outlines makes it easier to write and manage prompts by encapsulating templates inside "template functions". These functions make it possible to neatly separate the prompt logic from the general program logic; they can be imported from other modules and libraries. Template functions require no superfluous abstraction, they use the Jinja2 templating engine to help build complex prompts in a concise manner: import outlines.text as text import outlines.models as models examples = [ ("The food was digusting", "Negative"), ("We had a fantastic night", "Positive"), ("Recommended", "Positive"), ("The waiter was rude", "Negative") ] @text.prompt def labelling(to_label, examples): """You are a sentiment-labelling assistant. {% for example in examples %} {{ example[0] }} // {{ example[1] }} {% endfor %} {{ to_label }} // """ model = models.transformers("gpt2") prompt = labelling("Just awesome", examples) answer = text.generate.continuation(model, max_tokens=100)(prompt) Tools We can teach language models to call external functions to get additional informations or perform tasks, by encoding the functions' description in the prompt. To avoid duplicating information between the function definition and the description passed to the prompt, we define custom Jinja filters that can extract the function's name, description, signature and source: from typing import Callable, List import outlines.text as text def google_search(query: str): """Google Search""" pass def wikipedia_search(query: str): """Wikipedia Search""" pass @text.prompt def agent(tools: List[Callable]): """AVAILABLE COMMANDS: {% for tool in tools %} TOOL {{ tool | name }}, {{ tool | description }}, args: {{ tool | signature }} {{ tool | source }} {% endfor %} """ prompt = my_commands([google_search, wikipedia_search]) Response models We can instruct models to return their output in a pre-defined format, often JSON. To avoid duplicating information between the function definition and the description passed to the prompt we define a custom Jinja filter that can extract the expected response's schema: from pydantic import BaseModel import outlines.text as text class Joke(BaseModel): joke: str explanation: str @text.prompt def joke_ppt(response_model): """Tell a joke and explain why the joke is funny. RESPONSE FORMAT: {{ response_model | schema }} """ joke_ppt(Joke) # Tell a joke and explain why the joke is funny. # # RESPONSE FORMAT: # { # "joke": "The joke" # "explanation": "The explanation of why the joke is funny" # } With these prompting primitives Outlines makes building agents like AutoGPT, BabyAGI, ViperGPT or Transformers Agent easier by removing boilerplate prompting code. Contributing What contributions? We curently only accept bug fixes and documentation contributions. If you have a feature request, please start a new discussion. The issue tracker is only intended for actionable items. How to contribute? Run pip install -e .[test] or conda env create -f environment.yml. To build the documentation you will also need to run pip install -r requirements-doc.txt. Before pushing your code to repository please run pre-commit run --all-files and pytest to make sure that the code is formatted correctly and that the tests pass. Do not hesitate to open a draft PR before your contribution is ready, especially if you have questions and/or need feedback. Examples * Pick the odd one out * Meta prompting * ReAct * Generate code to solve math problems * BabyAGI * Uncertainty * Simulation-based inference Cite Outlines @article{willard2023efficient, title={Efficient Guided Generation for LLMs}, author={Willard, Brandon T and Louf, R{\'e}mi}, journal={arXiv preprint arXiv:2307.09702}, year={2023} } License Outlines is open-source and licensed under the Apache License 2.0. About Generative Model Programming normal-computing.github.io/outlines/ Resources Readme License Apache-2.0 license Activity Stars 1.9k stars Watchers 17 watching Forks 40 forks Report repository Releases 8 Outlines v0.0.8 Latest Aug 14, 2023 + 7 releases Packages 0 No packages published Used by 3 * @ddrscott @ddrscott / ai-dump * @ericmjl @ericmjl / llamabot Contributors 10 * @rlouf * @brandonwillard * @brosand * @harsh-sprinklr * @tiendung * @lukestanley * @dgerlanc * @mondaychen * @arunpatro * @mlelarge Languages * Python 100.0% Footer (c) 2023 GitHub, Inc. Footer navigation * Terms * Privacy * Security * Status * Docs * Contact GitHub * Pricing * API * Training * Blog * About You can't perform that action at this time.