https://github.com/imartinez/privateGPT Skip to content Toggle navigation Sign up * Product + Actions Automate any workflow + Packages Host and manage packages + Security Find and fix vulnerabilities + Codespaces Instant dev environments + Copilot Write better code with AI + Code review Manage code changes + Issues Plan and track work + Discussions Collaborate outside of code Explore + All features + Documentation + GitHub Skills + Blog * Solutions For + Enterprise + Teams + Startups + Education By Solution + CI/CD & Automation + DevOps + DevSecOps Case Studies + Customer Stories + Resources * Open Source + GitHub Sponsors Fund open source developers + The ReadME Project GitHub community articles Repositories + Topics + Trending + Collections * Pricing [ ] * # In this repository All GitHub | Jump to | * No suggested jump to results * # In this repository All GitHub | Jump to | * # In this user All GitHub | Jump to | * # In this repository All GitHub | Jump to | Sign in Sign up {{ message }} imartinez / privateGPT Public * Notifications * Fork 2.2k * Star 21.7k Interact privately with your documents using the power of GPT, 100% privately, no data leaks License Apache-2.0 license 21.7k stars 2.2k forks Star Notifications * Code * Issues 148 * Pull requests 29 * Discussions * Actions * Projects 0 * Security * Insights More * Code * Issues * Pull requests * Discussions * Actions * Projects * Security * Insights imartinez/privateGPT This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. main Switch branches/tags [ ] Branches Tags Could not load branches Nothing to show {{ refName }} default View all branches Could not load tags Nothing to show {{ refName }} default View all tags Name already in use A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch? Cancel Create 1 branch 0 tags Code * Local * Codespaces * Clone HTTPS GitHub CLI [https://github.com/i] Use Git or checkout with SVN using the web URL. [gh repo clone imarti] Work fast with our official CLI. Learn more about the CLI. * Open with GitHub Desktop * Download ZIP Sign In Required Please sign in to use Codespaces. Launching GitHub Desktop If nothing happens, download GitHub Desktop and try again. Launching GitHub Desktop If nothing happens, download GitHub Desktop and try again. Launching Xcode If nothing happens, download Xcode and try again. Launching Visual Studio Code Your codespace will open once ready. There was a problem preparing your codespace, please try again. Latest commit @imartinez imartinez Merge pull request #387 from maozdemir/patch-1 ... fb94b9d May 22, 2023 Merge pull request #387 from maozdemir/patch-1 typo: Change `pip` to `pip3` in README.md fb94b9d Git stats * 91 commits Files Permalink Failed to load latest commit information. Type Name Latest commit message Commit time .github/ISSUE_TEMPLATE Update issue templates May 16, 2023 20:44 source_documents End-to-end working version May 2, 2023 20:32 .gitignore End-to-end working version May 2, 2023 20:32 .pre-commit-config.yaml Make scripts executeable, add basic pre-commit setup May 19, 2023 23:21 LICENSE Initial commit May 2, 2023 11:15 README.md typo: Change pip to pip3 May 22, 2023 19:04 constants.py Formatting fixes May 20, 2023 12:29 example.env Update code to use sentence-transformers through huggingfaceembeddings May 17, 2023 00:32 ingest.py Better logs during ingestion May 20, 2023 12:11 privateGPT.py Make scripts executeable, add basic pre-commit setup May 20, 2023 11:15 requirements.txt Formatting fixes May 20, 2023 12:29 View code [ ] privateGPT Environment Setup Test dataset Instructions for ingesting your own dataset Ask questions to your documents, locally! CLI How does it work? System Requirements Python Version C++ Compiler For Windows 10/11 Mac Running Intel Disclaimer README.md privateGPT Ask questions to your documents without an internet connection, using the power of LLMs. 100% private, no data leaves your execution environment at any point. You can ingest documents and ask questions without an internet connection! Built with LangChain, GPT4All, LlamaCpp, Chroma and SentenceTransformers. demo Environment Setup In order to set your environment up to run the code here, first install all requirements: pip3 install -r requirements.txt Then, download the LLM model and place it in a directory of your choice: * LLM: default to ggml-gpt4all-j-v1.3-groovy.bin. If you prefer a different GPT4All-J compatible model, just download it and reference it in your .env file. Rename example.env to .env and edit the variables appropriately. MODEL_TYPE: supports LlamaCpp or GPT4All PERSIST_DIRECTORY: is the folder you want your vectorstore in MODEL_PATH: Path to your GPT4All or LlamaCpp supported LLM MODEL_N_CTX: Maximum token limit for the LLM model EMBEDDINGS_MODEL_NAME: SentenceTransformers embeddings model name (see https://www.sbert.net/docs/pretrained_models.html) Note: because of the way langchain loads the SentenceTransformers embeddings, the first time you run the script it will require internet connection to download the embeddings model itself. Test dataset This repo uses a state of the union transcript as an example. Instructions for ingesting your own dataset Put any and all your files into the source_documents directory The supported extensions are: * .csv: CSV, * .docx: Word Document, * .doc: Word Document, * .enex: EverNote, * .eml: Email, * .epub: EPub, * .html: HTML File, * .md: Markdown, * .msg: Outlook Message, * .odt: Open Document Text, * .pdf: Portable Document Format (PDF), * .pptx : PowerPoint Document, * .ppt : PowerPoint Document, * .txt: Text file (UTF-8), Run the following command to ingest all the data. python ingest.py Output should look like this: Creating new vectorstore Loading documents from source_documents Loading new documents: 100%|######################| 1/1 [00:01<00:00, 1.73s/it] Loaded 1 new documents from source_documents Split into 90 chunks of text (max. 500 tokens each) Creating embeddings. May take some minutes... Using embedded DuckDB with persistence: data will be stored in: db Ingestion complete! You can now run privateGPT.py to query your documents It will create a db folder containing the local vectorstore. Will take 20-30 seconds per document, depending on the size of the document. You can ingest as many documents as you want, and all will be accumulated in the local embeddings database. If you want to start from an empty database, delete the db folder. Note: during the ingest process no data leaves your local environment. You could ingest without an internet connection, except for the first time you run the ingest script, when the embeddings model is downloaded. Ask questions to your documents, locally! In order to ask a question, run a command like: python privateGPT.py And wait for the script to require your input. > Enter a query: Hit enter. You'll need to wait 20-30 seconds (depending on your machine) while the LLM model consumes the prompt and prepares the answer. Once done, it will print the answer and the 4 sources it used as context from your documents; you can then ask another question without re-running the script, just wait for the prompt again. Note: you could turn off your internet connection, and the script inference would still work. No data gets out of your local environment. Type exit to finish the script. CLI The script also supports optional command-line arguments to modify its behavior. You can see a full list of these arguments by running the command python privateGPT.py --help in your terminal. How does it work? Selecting the right local models and the power of LangChain you can run the entire pipeline locally, without any data leaving your environment, and with reasonable performance. * ingest.py uses LangChain tools to parse the document and create embeddings locally using HuggingFaceEmbeddings (SentenceTransformers). It then stores the result in a local vector database using Chroma vector store. * privateGPT.py uses a local LLM based on GPT4All-J or LlamaCpp to understand questions and create answers. The context for the answers is extracted from the local vector store using a similarity search to locate the right piece of context from the docs. * GPT4All-J wrapper was introduced in LangChain 0.0.162. System Requirements Python Version To use this software, you must have Python 3.10 or later installed. Earlier versions of Python will not compile. C++ Compiler If you encounter an error while building a wheel during the pip install process, you may need to install a C++ compiler on your computer. For Windows 10/11 To install a C++ compiler on Windows 10/11, follow these steps: 1. Install Visual Studio 2022. 2. Make sure the following components are selected: + Universal Windows Platform development + C++ CMake tools for Windows 3. Download the MinGW installer from the MinGW website. 4. Run the installer and select the gcc component. Mac Running Intel When running a Mac with Intel hardware (not M1), you may run into clang: error: the clang compiler does not support '-march=native' during pip install. If so set your archflags during pip install. eg: ARCHFLAGS="-arch x86_64" pip3 install -r requirements.txt Disclaimer This is a test project to validate the feasibility of a fully private solution for question answering using LLMs and Vector embeddings. It is not production ready, and it is not meant to be used in production. The models selection is not optimized for performance, but for privacy; but it is possible to use different models and vectorstores to improve performance. About Interact privately with your documents using the power of GPT, 100% privately, no data leaks Resources Readme License Apache-2.0 license Stars 21.7k stars Watchers 220 watching Forks 2.2k forks Report repository Releases No releases published Packages 0 No packages published Contributors 19 * @imartinez * @alxspiker * @R-Y-M-R * @jiangzhuo * @mdeweerd * @ivan-ontruck * @andreakiro * @abhiruka * @Fabio3rs * @sorin * @maozdemir + 8 contributors Languages * Python 100.0% Footer (c) 2023 GitHub, Inc. Footer navigation * Terms * Privacy * Security * Status * Docs * Contact GitHub * Pricing * API * Training * Blog * About You can't perform that action at this time. You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.