https://github.com/OpenSPG/KAG Skip to content Navigation Menu Toggle navigation Sign in * Product + GitHub Copilot Write better code with AI + Security Find and fix vulnerabilities + Actions Automate any workflow + Codespaces Instant dev environments + Issues Plan and track work + Code Review Manage code changes + Discussions Collaborate outside of code + Code Search Find more, search less Explore + All features + Documentation + GitHub Skills + Blog * Solutions By company size + Enterprises + Small and medium teams + Startups By use case + DevSecOps + DevOps + CI/CD + View all use cases By industry + Healthcare + Financial services + Manufacturing + Government + View all industries View all solutions * Resources Topics + AI + DevOps + Security + Software Development + View all Explore + Learning Pathways + White papers, Ebooks, Webinars + Customer Stories + Partners + Executive Insights * Open Source + GitHub Sponsors Fund open source developers + The ReadME Project GitHub community articles Repositories + Topics + Trending + Collections * Enterprise + Enterprise platform AI-powered developer platform Available add-ons + Advanced Security Enterprise-grade security features + GitHub Copilot Enterprise-grade AI features + Premium Support Enterprise-grade 24/7 support * Pricing Search or jump to... Search code, repositories, users, issues, pull requests... Search [ ] Clear Search syntax tips Provide feedback We read every piece of feedback, and take your input very seriously. [ ] [ ] Include my email address so I can be contacted Cancel Submit feedback Saved searches Use saved searches to filter your results more quickly Name [ ] Query [ ] To see all available qualifiers, see our documentation. Cancel Create saved search Sign in Sign up Reseting focus You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert {{ message }} OpenSPG / KAG Public * Notifications You must be signed in to change notification settings * Fork 142 * Star 2.4k KAG is a logical form-guided reasoning and retrieval framework based on OpenSPG engine and LLMs. It is used to build logical reasoning and factual Q&A solutions for professional domain knowledge bases. It can effectively overcome the shortcomings of the traditional RAG vector similarity calculation model. spg.openkg.cn/en-us License Apache-2.0 license 2.4k stars 142 forks Branches Tags Activity Star Notifications You must be signed in to change notification settings * Code * Issues 25 * Pull requests 7 * Discussions * Actions * Projects 0 * Security * Insights Additional navigation options * Code * Issues * Pull requests * Discussions * Actions * Projects * Security * Insights OpenSPG/KAG master BranchesTags [ ] Go to file Code Folders and files Name Name Last commit Last commit message date Latest commit History 99 Commits .github/ .github/ ISSUE_TEMPLATE ISSUE_TEMPLATE _static/images _static/images docs docs kag kag tests tests .gitignore .gitignore CITATION.cff CITATION.cff KAG_VERSION KAG_VERSION LEGAL.md LEGAL.md LICENSE LICENSE MANIFEST.in MANIFEST.in README.md README.md README_cn.md README_cn.md README_ja.md README_ja.md requirements.txt requirements.txt setup.py setup.py upload_dev.sh upload_dev.sh View all files Repository files navigation * README * Apache-2.0 license KAG: Knowledge Augmented Generation openspg logo English | Jian Ti Zhong Wen | Ri Ben Yu Ban dokiyumento [6874747073] Latest Release license 1. What is KAG? KAG is a logical reasoning and Q&A framework based on the OpenSPG engine and large language models, which is used to build logical reasoning and Q&A solutions for vertical domain knowledge bases. KAG can effectively overcome the ambiguity of traditional RAG vector similarity calculation and the noise problem of GraphRAG introduced by OpenIE. KAG supports logical reasoning and multi-hop fact Q&A, etc., and is significantly better than the current SOTA method. The goal of KAG is to build a knowledge-enhanced LLM service framework in professional domains, supporting logical reasoning, factual Q&A, etc. KAG fully integrates the logical and factual characteristics of the KGs. Its core features include: * Knowledge and Chunk Mutual Indexing structure to integrate more complete contextual text information * Knowledge alignment using conceptual semantic reasoning to alleviate the noise problem caused by OpenIE * Schema-constrained knowledge construction to support the representation and construction of domain expert knowledge * Logical form-guided hybrid reasoning and retrieval to support logical reasoning and multi-hop reasoning Q&A [?] Star our repository to stay up-to-date with exciting new features and improvements! Get instant notifications for new releases! Star KAG 2. Core Features 2.1 Knowledge Representation In the context of private knowledge bases, unstructured data, structured information, and business expert experience often coexist. KAG references the DIKW hierarchy to upgrade SPG to a version that is friendly to LLMs. For unstructured data such as news, events, logs, and books, as well as structured data like transactions, statistics, and approvals, along with business experience and domain knowledge rules, KAG employs techniques such as layout analysis, knowledge extraction, property normalization, and semantic alignment to integrate raw business data and expert rules into a unified business knowledge graph. KAG Diagram This makes it compatible with schema-free information extraction and schema-constrained expertise construction on the same knowledge type (e. G., entity type, event type), and supports the cross-index representation between the graph structure and the original text block. This mutual index representation is helpful to the construction of inverted index based on graph structure, and promotes the unified representation and reasoning of logical forms. 2.2 Mixed Reasoning Guided by Logic Forms Logical Form Solver KAG proposes a logically formal guided hybrid solution and inference engine. The engine includes three types of operators: planning, reasoning, and retrieval, which transform natural language problems into problem solving processes that combine language and notation. In this process, each step can use different operators, such as exact match retrieval, text retrieval, numerical calculation or semantic reasoning, so as to realize the integration of four different problem solving processes: Retrieval, Knowledge Graph reasoning, language reasoning and numerical calculation. 3. Release Notes 3.1 Latest Updates * 2024.11.21 : Support Word docs upload, model invoke concurrency setting, User experience optimization, etc. * 2024.10.25 : KAG initial release 3.2 Future Plans * domain knowledge injection, domain schema customization, QFS tasks support, Visual query analysis, etc. * Logical reasoning optimization, conversational tasks support * kag-model release, kag solution for event reasoning knowledge graph and medical knowledge graph * kag front-end open source, distributed build support, mathematical reasoning optimization 4. Quick Start 4.1 product-based (for ordinary users) 4.1.1 Engine & Dependent Image Installation * Recommend System Version: macOS User:macOS Monterey 12.6 or later Linux User:CentOS 7 / Ubuntu 20.04 or later Windows User:Windows 10 LTSC 2021 or later * Software Requirements: macOS / Linux User:Docker,Docker Compose Windows User:WSL 2 / Hyper-V,Docker,Docker Compose Use the following commands to download the docker-compose.yml file and launch the services with Docker Compose. # set the HOME environment variable (only Windows users need to execute this command) # set HOME=%USERPROFILE% curl -sSL https://raw.githubusercontent.com/OpenSPG/openspg/refs/heads/master/dev/release/docker-compose-west.yml -o docker-compose-west.yml docker compose -f docker-compose-west.yml up -d 4.1.2 Use the product Navigate to the default url of the KAG product with your browser: http://127.0.0.1:8887 See the Quick Start for Product Mode for detailed introduction. 4.2 toolkit-based (for developers) 4.2.1 Engine & Dependent Image Installation Refer to the 3.1 section to complete the installation of the engine & dependent image. 4.2.2 Installation of KAG macOS / Linux developers # Create conda env: conda create -n kag-demo python=3.10 && conda activate kag-demo # Clone code: git clone https://github.com/OpenSPG/KAG.git # Install KAG: cd KAG && pip install -e . Windows developers # Install the official Python 3.8.10 or later, install Git. # Create and activate Python venv: py -m venv kag-demo && kag-demo\Scripts\activate # Clone code: git clone https://github.com/OpenSPG/KAG.git # Install KAG: cd KAG && pip install -e . 4.2.3 Use the toolkit Please refer to the Quick Start for Developer Mode guide for detailed introduction of the toolkit. Then you can use the built-in components to reproduce the performance results of the built-in datasets, and apply those components to new busineness scenarios. 5. Technical Architecture KAG technical architecture The KAG framework includes three parts: kg-builder, kg-solver, and kag-model. This release only involves the first two parts, kag-model will be gradually open source release in the future. kg-builder implements a knowledge representation that is friendly to large-scale language models (LLM). Based on the hierarchical structure of DIKW (data, information, knowledge and wisdom), IT upgrades SPG knowledge representation ability, and is compatible with information extraction without schema constraints and professional knowledge construction with schema constraints on the same knowledge type (such as entity type and event type), it also supports the mutual index representation between the graph structure and the original text block, which supports the efficient retrieval of the reasoning question and answer stage. kg-solver uses a logical symbol-guided hybrid solving and reasoning engine that includes three types of operators: planning, reasoning, and retrieval, to transform natural language problems into a problem-solving process that combines language and symbols. In this process, each step can use different operators, such as exact match retrieval, text retrieval, numerical calculation or semantic reasoning, so as to realize the integration of four different problem solving processes: Retrieval, Knowledge Graph reasoning, language reasoning and numerical calculation. 6. Contact us GitHub: https://github.com/OpenSPG/KAG OpenSPG: https://spg.openkg.cn/ Contact Us: OpenSPG QR-code 7. Differences between KAG, RAG, and GraphRAG KAG introduction and applications: https://github.com/orgs/OpenSPG/ discussions/52 8. Citation If you use this software, please cite it as below: * KAG: Boosting LLMs in Professional Domains via Knowledge Augmented Generation * KGFabric: A Scalable Knowledge Graph Warehouse for Enterprise Data Interconnection @article{liang2024kag, title={KAG: Boosting LLMs in Professional Domains via Knowledge Augmented Generation}, author={Liang, Lei and Sun, Mengshu and Gui, Zhengke and Zhu, Zhongshu and Jiang, Zhouyu and Zhong, Ling and Qu, Yuan and Zhao, Peilong and Bo, Zhongpu and Yang, Jin and others}, journal={arXiv preprint arXiv:2409.13731}, year={2024} } @article{yikgfabric, title={KGFabric: A Scalable Knowledge Graph Warehouse for Enterprise Data Interconnection}, author={Yi, Peng and Liang, Lei and Da Zhang, Yong Chen and Zhu, Jinye and Liu, Xiangyu and Tang, Kun and Chen, Jialin and Lin, Hao and Qiu, Leijie and Zhou, Jun} } License Apache License 2.0 About KAG is a logical form-guided reasoning and retrieval framework based on OpenSPG engine and LLMs. It is used to build logical reasoning and factual Q&A solutions for professional domain knowledge bases. It can effectively overcome the shortcomings of the traditional RAG vector similarity calculation model. spg.openkg.cn/en-US Topics knowledge-graph logical-reasoning large-language-model multi-hop-question-answering trustfulness Resources Readme License Apache-2.0 license Activity Custom properties Stars 2.4k stars Watchers 32 watching Forks 142 forks Report repository Releases 2 Version 0.5.1 Latest Nov 21, 2024 + 1 release Packages 0 No packages published Contributors 11 * @northmachine * @caszkgui * @xionghuaidong * @andylau-55 * @royzhao * @zhuzhongshu123 * @leywar * @tpoisonooo * @eltociear * @thundax-lyp * @zzzcccxx Languages * Python 99.9% * Other 0.1% Footer (c) 2024 GitHub, Inc. Footer navigation * Terms * Privacy * Security * Status * Docs * Contact * Manage cookies * Do not share my personal information You can't perform that action at this time.