https://github.com/explodinggradients/ragas Skip to content Toggle navigation Sign in * Product + Actions Automate any workflow + Packages Host and manage packages + Security Find and fix vulnerabilities + Codespaces Instant dev environments + Copilot Write better code with AI + Code review Manage code changes + Issues Plan and track work + Discussions Collaborate outside of code Explore + All features + Documentation + GitHub Skills + Blog * Solutions For + Enterprise + Teams + Startups + Education By Solution + CI/CD & Automation + DevOps + DevSecOps Resources + Learning Pathways + White papers, Ebooks, Webinars + Customer Stories + Partners * Open Source + GitHub Sponsors Fund open source developers + The ReadME Project GitHub community articles Repositories + Topics + Trending + Collections * Pricing Search or jump to... Search code, repositories, users, issues, pull requests... Search [ ] Clear Search syntax tips Provide feedback We read every piece of feedback, and take your input very seriously. [ ] [ ] Include my email address so I can be contacted Cancel Submit feedback Saved searches Use saved searches to filter your results more quickly Name [ ] Query [ ] To see all available qualifiers, see our documentation. Cancel Create saved search Sign in Sign up You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert {{ message }} explodinggradients / ragas Public * Notifications * Fork 330 * Star 3.9k * Evaluation framework for your Retrieval Augmented Generation (RAG) pipelines docs.ragas.io License Apache-2.0 license 3.9k stars 330 forks Branches Tags Activity Star Notifications * Code * Issues 178 * Pull requests 17 * Actions * Projects 0 * Security * Insights Additional navigation options * Code * Issues * Pull requests * Actions * Projects * Security * Insights explodinggradients/ragas This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. main BranchesTags Go to file Code Folders and files Name Name Last commit Last commit message date Latest commit History 355 Commits .github .github docs docs requirements requirements src/ragas src/ragas tests tests .dockerignore .dockerignore .gitattributes .gitattributes .gitignore .gitignore .readthedocs.yml .readthedocs.yml LICENSE LICENSE Makefile Makefile README.md README.md pyproject.toml pyproject.toml references.md references.md View all files Repository files navigation * README * Apache-2.0 license [logo] Evaluation framework for your Retrieval Augmented Generation (RAG) pipelines GitHub release Build License Open In Colab discord-invite Downloads Documentation | Installation | Quickstart | Community | Open Analytics | Hugging Face Dedicated solutions to evaluate, monitor and improve performance of LLM & RAG application in production including custom models for production quality monitoring.Talk to founders Ragas is a framework that helps you evaluate your Retrieval Augmented Generation (RAG) pipelines. RAG denotes a class of LLM applications that use external data to augment the LLM's context. There are existing tools and frameworks that help you build these pipelines but evaluating it and quantifying your pipeline performance can be hard. This is where Ragas (RAG Assessment) comes in. Ragas provides you with the tools based on the latest research for evaluating LLM-generated text to give you insights about your RAG pipeline. Ragas can be integrated with your CI/CD to provide continuous checks to ensure performance. [?] Installation pip install ragas if you want to install from source git clone https://github.com/explodinggradients/ragas && cd ragas pip install -e . Quickstart This is a small example program you can run to see ragas in action! from datasets import Dataset import os from ragas import evaluate from ragas.metrics import faithfulness, answer_correctness os.environ["OPENAI_API_KEY"] = "your-openai-key" data_samples = { 'question': ['When was the first super bowl?', 'Who won the most super bowls?'], 'answer': ['The first superbowl was held on Jan 15, 1967', 'The most super bowls have been won by The New England Patriots'], 'contexts' : [['The First AFL-NFL World Championship Game was an American football game played on January 15, 1967, at the Los Angeles Memorial Coliseum in Los Angeles,'], ['The Green Bay Packers...Green Bay, Wisconsin.','The Packers compete...Football Conference']], 'ground_truth': ['The first superbowl was held on January 15, 1967', 'The New England Patriots have won the Super Bowl a record six times'] } dataset = Dataset.from_dict(data_samples) score = evaluate(dataset,metrics=[faithfulness,answer_correctness]) score.to_pandas() Refer to our documentation to learn more. Community If you want to get more involved with Ragas, check out our discord server. It's a fun community where we geek out about LLM, Retrieval, Production issues, and more. Open Analytics We track very basic usage metrics to guide us to figure out what our users want, what is working, and what's not. As a young startup, we have to be brutally honest about this which is why we are tracking these metrics. But as an Open Startup, we open-source all the data we collect. You can read more about this here. Ragas does not track any information that can be used to identify you or your company. You can take a look at exactly what we track in the code To disable usage-tracking you set the RAGAS_DO_NOT_TRACK flag to true. About Evaluation framework for your Retrieval Augmented Generation (RAG) pipelines docs.ragas.io Topics llm llmops Resources Readme License Apache-2.0 license Activity Custom properties Stars 3.9k stars Watchers 20 watching Forks 330 forks Report repository Releases 34 v0.1.5 Latest Mar 20, 2024 + 33 releases Used by 224 * @IshikaGopie * @misbahsy * @TestRAG * @uray-lu * @SRH-Heidelberg-University * @deboramachadoandrade * @FranciscoAlves00 * @tsravan + 216 Contributors 81 * @jjmachan * @shahules786 * @tinomaxthayil * @Yongtae723 * @yuukidach * @jokokojote * @joy13975 * @lucasiscovici * @arm-diaz * @pmbaumgartner * @starrywheat * @TruscaPetre * @mspronesti * @MANISH007700 + 67 contributors Languages * Python 98.8% * Makefile 1.1% * Dockerfile 0.1% Footer (c) 2024 GitHub, Inc. Footer navigation * Terms * Privacy * Security * Status * Docs * Contact * Manage cookies * Do not share my personal information You can't perform that action at this time.