https://github.com/pinecone-io/canopy Skip to content Toggle navigation Sign up * Product + Actions Automate any workflow + Packages Host and manage packages + Security Find and fix vulnerabilities + Codespaces Instant dev environments + Copilot Write better code with AI + Code review Manage code changes + Issues Plan and track work + Discussions Collaborate outside of code Explore + All features + Documentation + GitHub Skills + Blog * Solutions For + Enterprise + Teams + Startups + Education By Solution + CI/CD & Automation + DevOps + DevSecOps Resources + Learning Pathways + White papers, Ebooks, Webinars + Customer Stories + Partners * Open Source + GitHub Sponsors Fund open source developers + The ReadME Project GitHub community articles Repositories + Topics + Trending + Collections * Pricing Search or jump to... Search code, repositories, users, issues, pull requests... Search [ ] Clear Search syntax tips Provide feedback We read every piece of feedback, and take your input very seriously. [ ] [ ] Include my email address so I can be contacted Cancel Submit feedback Saved searches Use saved searches to filter your results more quickly Name [ ] Query [ ] To see all available qualifiers, see our documentation. Cancel Create saved search Sign in Sign up You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert {{ message }} pinecone-io / canopy Public * Notifications * Fork 51 * Star 450 Retrieval Augmented Generation (RAG) framework and context engine powered by Pinecone www.pinecone.io/ License Apache-2.0 license 450 stars 51 forks Activity Star Notifications * Code * Issues 11 * Pull requests 10 * Discussions * Actions * Projects 0 * Security * Insights Additional navigation options * Code * Issues * Pull requests * Discussions * Actions * Projects * Security * Insights pinecone-io/canopy This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. main Switch branches/tags [ ] Branches Tags Could not load branches Nothing to show {{ refName }} default View all branches Could not load tags Nothing to show {{ refName }} default View all tags Name already in use A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch? Cancel Create 3 branches 6 tags Code * Local * Codespaces * Clone HTTPS GitHub CLI [https://github.com/p] Use Git or checkout with SVN using the web URL. [gh repo clone pineco] Work fast with our official CLI. Learn more about the CLI. * Open with GitHub Desktop * Download ZIP Sign In Required Please sign in to use Codespaces. Launching GitHub Desktop If nothing happens, download GitHub Desktop and try again. Launching GitHub Desktop If nothing happens, download GitHub Desktop and try again. Launching Xcode If nothing happens, download Xcode and try again. Launching Visual Studio Code Your codespace will open once ready. There was a problem preparing your codespace, please try again. Latest commit @eburnette eburnette and Ed Burnette Fix some typos, add dev container, faux streaming (#200) ... f372033 Dec 7, 2023 Fix some typos, add dev container, faux streaming (#200) * Fix some typos, add dev container, faux streaming * Go back to Python 3.9, oldest supported version * Missing whitespace after colon --------- Co-authored-by: Ed Burnette f372033 Git stats * 1,063 commits Files Permalink Failed to load latest commit information. Type Name Latest commit message Commit time .devcontainer Fix some typos, add dev container, faux streaming (#200) December 7, 2023 18:25 .github Add Anyscale Endpoint support and Llama Tokenizer (#173) November 27, 2023 11:48 .readme-content replace image for RAG flow November 6, 2023 11:28 config Add Anyscale Endpoint support and Llama Tokenizer (#173) November 27, 2023 11:48 docs Changed library name to canopy-sdk November 2, 2023 15:47 examples Upgrade openai sdk to v.1.2.3 (#171) November 15, 2023 16:06 src Fix some typos, add dev container, faux streaming (#200) December 7, 2023 18:25 tests Add Anyscale Endpoint support and Llama Tokenizer (#173) November 27, 2023 11:48 .env.example Fix some typos, add dev container, faux streaming (#200) December 7, 2023 18:25 .gitignore Fix some typos, add dev container, faux streaming (#200) December 7, 2023 18:25 CHANGELOG.md Added release 0.2.0 to CHANGELOG (#180) November 16, 2023 14:45 LICENSE Add LICENSE file October 24, 2023 23:01 README.md Update README.md and change href (#202) December 7, 2023 18:20 pyproject.toml Add Anyscale Endpoint support and Llama Tokenizer (#173) November 27, 2023 11:48 View code [ ] Canopy RAG with Canopy What's inside the box? Setup Quickstart 1. Create a new Canopy Index 2. Uploading data 3. Start the Canopy server Stopping the server Evaluation chat tool Considerations Advanced usage Migrating an existing OpenAI application to Canopy Running Canopy server in production README.md Canopy Supported Python versions Package version Canopy is an open-source Retrieval Augmented Generation (RAG) framework and context engine built on top of the Pinecone vector database. Canopy enables you to quickly and easily experiment with and build applications using RAG. Start chatting with your documents or text data with a few simple commands. Canopy takes on the heavy lifting for building RAG applications: from chunking and embedding your text data to chat history management, query optimization, context retrieval (including prompt engineering), and augmented generation. Canopy provides a configurable built-in server so you can effortlessly deploy a RAG-powered chat application to your existing chat UI or interface. Or you can build your own, custom RAG application using the Canopy library. Canopy lets you evaluate your RAG workflow with a CLI based chat tool. With a simple command in the Canopy CLI you can interactively chat with your text data and compare RAG vs. non-RAG workflows side-by-side. Check out our blog post to learn more, or see a quick tutorial here. RAG with Canopy [rag_flow] Canopy implements the full RAG workflow to prevent hallucinations and augment your LLM with your own text data. Canopy has two flows: knowledge base creation and chat. In the knowledge base creation flow, users upload their documents and transform them into meaningful representations stored in Pinecone's Vector Database. In the chat flow, incoming queries and chat history are optimized to retrieve the most relevant documents, the knowledge base is queried, and a meaningful context is generated for the LLM to answer. What's inside the box? 1. Canopy Core Library - The library has 3 main classes that are responsible for different parts of the RAG workflow: + ChatEngine - Exposes a chat interface to interact with your data. Given the history of chat messages, the ChatEngine formulates relevant queries to the ContextEngine, then uses the LLM to generate a knowledgeable response. + ContextEngine - Performs the "retrieval" part of RAG. The ContextEngine utilizes the underlying KnowledgeBase to retrieve the most relevant documents, then formulates a coherent textual context to be used as a prompt for the LLM. + KnowledgeBase - Manages your data for the RAG workflow. It automatically chunks and transforms your text data into text embeddings, storing them in a Pinecone vector database. Given a text query - the KnowledgeBase will retrieve the most relevant document chunks from the database. More information about the Core Library usage can be found in the Library Documentation 2. Canopy Server - This is a webservice that wraps the Canopy Core library and exposes it as a REST API. The server is built on top of FastAPI, Uvicorn and Gunicorn and can be easily deployed in production. The server also comes with a built-in Swagger UI for easy testing and documentation. After you start the server, you can access the Swagger UI at http://host:port/docs (default: http://localhost:8000/docs) 3. Canopy CLI - A built-in development tool that allows users to swiftly set up their own Canopy server and test its configuration. With just three CLI commands, you can create a new Canopy server, upload your documents to it, and then interact with the Chatbot using a built-in chat application directly from the terminal. The built-in chatbot also enables comparison of RAG-infused responses against a native LLM chatbot. Setup 0. set up a virtual environment (optional) python3 -m venv canopy-env source canopy-env/bin/activate More information about virtual environments can be found here 1. install the package pip install canopy-sdk 2. Set up the environment variables export PINECONE_API_KEY="" export PINECONE_ENVIRONMENT="" export OPENAI_API_KEY="" export INDEX_NAME="" CLICK HERE for more information about the environment variables Name Description How to get it? The API key for Register or log into your Pinecone. Used to Pinecone account in the authenticate to console. You can access PINECONE_API_KEY Pinecone services to your API key from the create indexes and to "API Keys" section in the insert, delete and sidebar of your dashboard search data Determines the Pinecone service cloud You can find the Pinecone PINECONE_ENVIRONMENT environment of your environment next to the index e.g west1-gcp, API key in console us-east-1-aws, etc API key for OpenAI. You can find your OpenAI Used to authenticate API key here. You might OPENAI_API_KEY to OpenAI's services need to login or register for embedding and chat to OpenAI services API API key for Anyscale. You can register Anyscale ANYSCALE_API_KEY Used to authenticate Endpoints and find your to Anyscale Endpoints API key here for open source LLMs Name of the Pinecone You can choose any name INDEX_NAME index Canopy will as long as it follows underlying work with Pinecone's restrictions The path of a Optional - if not CANOPY_CONFIG_FILE configuration yaml provided, default file to be used by the configuration would be Canopy server. used 3. Check that installation is successful and environment is set, run: canopy Output should be similar to this: Canopy: Ready Usage: canopy [OPTIONS] COMMAND [ARGS]... # rest of the help message Quickstart In this quickstart, we will show you how to use the Canopy to build a simple question answering system using RAG (retrieval augmented generation). 1. Create a new Canopy Index As a one-time setup, Canopy needs to create a new Pinecone index that is configured to work with Canopy, just run: canopy new And follow the CLI instructions. The index that will be created will have a prefix canopy--. You only have to do this process once for every Canopy index you want to create. To learn more about Pinecone indexes and how to manage them, please refer to the following guide: Understanding indexes 2. Uploading data You can load data into your Canopy index using the command: canopy upsert /path/to/data_directory # or canopy upsert /path/to/data_directory/file.parquet # or canopy upsert /path/to/data_directory/file.jsonl # or canopy upsert /path/to/directory_of_txt_files/ # ... Canopy supports files in jsonl, parquet and csv formats. Additionally, you can load plaintext data files in .txt format. In this case, each file will be treated as a single document. The document id will be the filename, and the source will be the full path of the file. Note: Document fields are used in the RAG flow and should comply with the following schema: +----------+--------------+--------------+---------------+ | id(str) | text(str) | source | metadata | | | | Optional[str]| Optional[dict]| |----------+--------------+--------------+---------------| | "id1" | "some text" | "some source"| {"key": "val"}| +----------+--------------+--------------+---------------+ # id - unique identifier for the document # # text - the text of the document, in utf-8 encoding. # # source - the source of the document, can be any string, or null. # ** this will be used as a reference in the generated context. ** # # metadata - optional metadata for the document, for filtering or additional context. # Dict[str, Union[str, int, float, List[str]]] This notebook shows how you create a dataset in this format, Follow the instructions in the CLI when you upload your data. 3. Start the Canopy server The Canopy server exposes Canopy's functionality via a REST API. Namely, it allows you to upload documents, retrieve relevant docs for a given query, and chat with your data. The server exposes a / chat.completion endpoint that can be easily integrated with any chat application. To start the server, run: canopy start Now, you should be prompted with the following standard Uvicorn message: ... INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit) That's it! you can now start using the Canopy server with any chat application that supports a /chat.completion endpoint. The canopy start command will keep the terminal occupied (recommended use). If you want to run the server in the background, you can use the following command - nohup canopy start & Stopping the server To stop the server, simply press CTRL+C in the terminal where you started it. If you have started the server in the background, you can stop it by running: canopy stop Evaluation chat tool Canopy's CLI comes with a built-in chat app that allows you to interactively chat with your text data and compare RAG vs. non-RAG workflows side-by-side to evaluate the results In a new terminal window, set the required environment variables then run: canopy chat This will open a chat interface in your terminal. You can ask questions and the RAG-infused chatbot will try to answer them using the data you uploaded. To compare the chat response with and without RAG use the --no-rag flag canopy chat --no-rag This will open a similar chat interface window, but will show both the RAG and non-RAG responses side-by-side. Considerations * Canopy currently only supports OpenAI as the backend for both the embedding model and the LLM. Rate limits and pricing set by OpenAI will apply. * More integrations will be supported in the near future. Advanced usage Migrating an existing OpenAI application to Canopy If you already have an application that uses the OpenAI API, you can migrate it to Canopy by simply changing the API endpoint to http:// host:port/v1, for example with the default configuration: import openai openai.api_base = "http://localhost:8000/v1" # now you can use the OpenAI API as usual or without global state change: import openai openai_response = openai.Completion.create(..., api_base="http://localhost:8000/v1") Running Canopy server in production Canopy is using FastAPI as the web framework and Uvicorn as the ASGI server. It is recommended to use Gunicorn as the production server, mainly because it supports multiple worker processes and can handle multiple requests in parallel, more details can be found here. To run the canopy server for production, please run: gunicorn canopy_cli.app:app --worker-class uvicorn.workers.UvicornWorker --bind 0.0.0.0:8000 --workers The server interacts with services like Pinecone and OpenAI using your own authentication credentials. When deploying the server on a public web hosting provider, it is recommended to enable an authentication mechanism, so that your server would only take requests from authenticated users. About Retrieval Augmented Generation (RAG) framework and context engine powered by Pinecone www.pinecone.io/ Topics rag vector-database llm generative-ai Resources Readme License Apache-2.0 license Activity Stars 450 stars Watchers 14 watching Forks 51 forks Report repository Releases 5 0.2.0 Latest Nov 16, 2023 + 4 releases Packages 0 No packages published Contributors 12 * * * * * * * * * * * * Languages * Python 100.0% Footer (c) 2023 GitHub, Inc. Footer navigation * Terms * Privacy * Security * Status * Docs * Contact * You can't perform that action at this time.