{"id":18167945,"url":"https://github.com/OpenSPG/KAG","last_synced_at":"2025-04-01T11:31:51.454Z","repository":{"id":259362572,"uuid":"860912552","full_name":"OpenSPG/KAG","owner":"OpenSPG","description":"KAG is a logical form-guided reasoning and retrieval framework based on OpenSPG engine and LLMs.  It is used to build logical reasoning and factual Q\u0026A solutions for professional domain knowledge bases. It can effectively overcome the shortcomings of the traditional RAG vector similarity calculation model.","archived":false,"fork":false,"pushed_at":"2025-03-25T09:29:06.000Z","size":171006,"stargazers_count":6155,"open_issues_count":135,"forks_count":412,"subscribers_count":57,"default_branch":"master","last_synced_at":"2025-03-25T09:36:19.885Z","etag":null,"topics":["knowledge-graph","large-language-model","logical-reasoning","multi-hop-question-answering","trustfulness"],"latest_commit_sha":null,"homepage":"https://spg.openkg.cn/en-US","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/OpenSPG.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-09-21T13:56:44.000Z","updated_at":"2025-03-25T07:53:54.000Z","dependencies_parsed_at":"2025-03-10T09:29:17.045Z","dependency_job_id":"c13841c2-0d21-4b1b-9736-eaffcace4eec","html_url":"https://github.com/OpenSPG/KAG","commit_stats":null,"previous_names":["openspg/kag"],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenSPG%2FKAG","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenSPG%2FKAG/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenSPG%2FKAG/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenSPG%2FKAG/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/OpenSPG","download_url":"https://codeload.github.com/OpenSPG/KAG/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246632229,"owners_count":20808833,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["knowledge-graph","large-language-model","logical-reasoning","multi-hop-question-answering","trustfulness"],"created_at":"2024-11-02T14:00:45.197Z","updated_at":"2025-04-01T11:31:51.448Z","avatar_url":"https://github.com/OpenSPG.png","language":"Python","readme":"# KAG: Knowledge Augmented Generation\n\n\u003cdiv align=\"center\"\u003e\n\u003ca href=\"https://spg.openkg.cn/en-US\"\u003e\n\u003cimg src=\"./_static/images/OpenSPG-1.png\" width=\"520\" alt=\"openspg logo\"\u003e\n\u003c/a\u003e\n\u003c/div\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"./README.md\"\u003eEnglish\u003c/a\u003e |\n  \u003ca href=\"./README_cn.md\"\u003e简体中文\u003c/a\u003e |\n  \u003ca href=\"./README_ja.md\"\u003e日本語版ドキュメント\u003c/a\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n    \u003ca href='https://arxiv.org/pdf/2409.13731'\u003e\u003cimg src='https://img.shields.io/badge/arXiv-2409.13731-b31b1b'\u003e\u003c/a\u003e\n    \u003ca href=\"https://github.com/OpenSPG/KAG/releases/latest\"\u003e\n        \u003cimg src=\"https://img.shields.io/github/v/release/OpenSPG/KAG?color=blue\u0026label=Latest%20Release\" alt=\"Latest Release\"\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://openspg.yuque.com/ndx6g9/docs_en\"\u003e\n        \u003cimg src=\"https://img.shields.io/badge/User%20Guide-1e8b93?logo=readthedocs\u0026logoColor=f5f5f5\" alt=\"User Guide\"\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://github.com/OpenSPG/KAG/blob/main/LICENSE\"\u003e\n        \u003cimg height=\"21\" src=\"https://img.shields.io/badge/License-Apache--2.0-ffffff?labelColor=d4eaf7\u0026color=2e6cc4\" alt=\"license\"\u003e\n    \u003c/a\u003e\n\u003c/p\u003e\n\u003cp align=\"center\"\u003e\n   \u003ca href=\"https://discord.gg/PURG77zhQ7\"\u003e\n        \u003cimg src=\"https://img.shields.io/discord/1329648479709958236?style=for-the-badge\u0026logo=discord\u0026label=Discord\" alt=\"Discord\"\u003e\n   \u003c/a\u003e\n\u003c/p\u003e\n\n# 1. What is KAG?\n\nKAG is a logical reasoning and Q\u0026A framework based on the [OpenSPG](https://github.com/OpenSPG/openspg) engine and large language models, which is used to build logical reasoning and Q\u0026A solutions for vertical domain knowledge bases.  KAG can effectively overcome the ambiguity of traditional RAG vector similarity calculation and the noise problem of GraphRAG introduced by OpenIE.  KAG supports logical reasoning and multi-hop fact Q\u0026A, etc., and is significantly better than the current SOTA method.\n\nThe goal of KAG is to build a knowledge-enhanced LLM service framework in professional domains, supporting logical reasoning, factual Q\u0026A, etc. KAG fully integrates the logical and factual characteristics of the KGs. Its core features include:\n\n- Knowledge and Chunk Mutual Indexing structure to integrate more complete contextual text information\n- Knowledge alignment using conceptual semantic reasoning to alleviate the noise problem caused by OpenIE\n- Schema-constrained knowledge construction to support the representation and construction of domain expert knowledge\n- Logical form-guided hybrid reasoning and retrieval to support logical reasoning and multi-hop reasoning Q\u0026A\n\n⭐️ Star our repository to stay up-to-date with exciting new features and improvements! Get instant notifications for new releases! 🌟\n\n![Star KAG](./_static/images/star-kag.gif)\n\n# 2. Core Features\n\n## 2.1 Knowledge Representation\n\nIn the context of private knowledge bases, unstructured data, structured information, and business expert experience often coexist. KAG references the DIKW hierarchy to upgrade SPG to a version that is friendly to LLMs. \n\nFor unstructured data such as news, events, logs, and books, as well as structured data like transactions, statistics, and approvals, along with business experience and domain knowledge rules, KAG employs techniques such as layout analysis, knowledge extraction, property normalization, and semantic alignment to integrate raw business data and expert rules into a unified business knowledge graph.\n\n![KAG Diagram](./_static/images/kag-diag.jpg)\n\nThis makes it compatible with schema-free information extraction and schema-constrained expertise construction on the same knowledge type (e. G., entity type, event type), and supports the cross-index representation between the graph structure and the original text block. \n\nThis mutual index representation is helpful to the construction of inverted index based on graph structure, and promotes the unified representation and reasoning of logical forms.\n\n## 2.2 Mixed Reasoning Guided by Logic Forms\n\n![Logical Form Solver](./_static/images/kag-lf-solver.png)\n\nKAG proposes a logically formal guided hybrid solution and inference engine. \n\nThe engine includes three types of operators: planning, reasoning, and retrieval, which transform natural language problems into problem solving processes that combine language and notation. \n\nIn this process, each step can use different operators, such as exact match retrieval, text retrieval, numerical calculation or semantic reasoning, so as to realize the integration of four different problem solving processes: Retrieval, Knowledge Graph reasoning, language reasoning and numerical calculation.\n\n# 3. Release Notes\n\n## 3.1 Latest Updates\n\n* 2025.01.07 : Support domain knowledge injection, domain schema customization, QFS tasks support, Visual query analysis, enables schema-constraint mode for extraction, etc.\n* 2024.11.21 : Support Word docs upload, model invoke concurrency setting, User experience optimization, etc.\n* 2024.10.25 : KAG initial release\n\n## 3.2 Future Plans\n\n* Logical reasoning optimization, conversational tasks support\n* kag-model release, kag solution for event reasoning knowledge graph and medical knowledge graph\n* kag front-end open source, distributed build support, mathematical reasoning optimization\n\n# 4. Quick Start\n\n## 4.1 product-based (for ordinary users)\n\n### 4.1.1 Engine \u0026 Dependent Image Installation\n\n* **Recommend System Version:**\n\n  ```text\n  macOS User：macOS Monterey 12.6 or later\n  Linux User：CentOS 7 / Ubuntu 20.04 or later\n  Windows User：Windows 10 LTSC 2021 or later\n  ```\n\n* **Software Requirements:**\n\n  ```text\n  macOS / Linux User：Docker，Docker Compose\n  Windows User：WSL 2 / Hyper-V，Docker，Docker Compose\n  ```\n\nUse the following commands to download the docker-compose.yml file and launch the services with Docker Compose.\n\n```bash\n# set the HOME environment variable (only Windows users need to execute this command)\n# set HOME=%USERPROFILE%\n\ncurl -sSL https://raw.githubusercontent.com/OpenSPG/openspg/refs/heads/master/dev/release/docker-compose-west.yml -o docker-compose-west.yml\ndocker compose -f docker-compose-west.yml up -d\n```\n\n### 4.1.2 Use the product\n\nNavigate to the default url of the KAG product with your browser: \u003chttp://127.0.0.1:8887\u003e\n```text\nDefault Username: openspg\nDefault password: openspg@kag\n```\nSee [KAG usage (product mode)](https://openspg.yuque.com/ndx6g9/cwh47i/rs7gr8g4s538b1n7#rtOlA) for detailed introduction.\n\n## 4.2 toolkit-based (for developers)\n\n### 4.2.1 Engine \u0026 Dependent Image Installation\n\nRefer to the 3.1 section to complete the installation of the engine \u0026 dependent image.\n\n### 4.2.2 Installation of KAG\n\n\n**macOS / Linux developers**\n\n```text\n# Create conda env: conda create -n kag-demo python=3.10 \u0026\u0026 conda activate kag-demo\n\n# Clone code: git clone https://github.com/OpenSPG/KAG.git\n\n# Install KAG: cd KAG \u0026\u0026 pip install -e .\n```\n\n**Windows developers**\n\n```text\n# Install the official Python 3.8.10 or later, install Git.\n\n# Create and activate Python venv: py -m venv kag-demo \u0026\u0026 kag-demo\\Scripts\\activate\n\n# Clone code: git clone https://github.com/OpenSPG/KAG.git\n\n# Install KAG: cd KAG \u0026\u0026 pip install -e .\n```\n\n### 4.2.3 Use the toolkit\n\nPlease refer to [KAG usage (developer mode)](https://openspg.yuque.com/ndx6g9/cwh47i/rs7gr8g4s538b1n7#cikso) guide for detailed introduction of the toolkit. Then you can use the built-in components to reproduce the performance results of the built-in datasets, and apply those components to new busineness scenarios.\n\n# 5. Technical Architecture\n\n![KAG technical architecture](./_static/images/kag-arch.png)\n\nThe KAG framework includes three parts: kg-builder, kg-solver, and kag-model. This release only involves the first two parts, kag-model will be gradually open source release in the future.\n\nkg-builder implements a knowledge representation that is friendly to large-scale language models (LLM). Based on the hierarchical structure of DIKW (data, information, knowledge and wisdom), IT upgrades SPG knowledge representation ability, and is compatible with information extraction without schema constraints and professional knowledge construction with schema constraints on the same knowledge type (such as entity type and event type), it also supports the mutual index representation between the graph structure and the original text block, which supports the efficient retrieval of the reasoning question and answer stage.\n\nkg-solver uses a logical symbol-guided hybrid solving and reasoning engine that includes three types of operators: planning, reasoning, and retrieval, to transform natural language problems into a problem-solving process that combines language and symbols. In this process, each step can use different operators, such as exact match retrieval, text retrieval, numerical calculation or semantic reasoning, so as to realize the integration of four different problem solving processes: Retrieval, Knowledge Graph reasoning, language reasoning and numerical calculation.\n\n# 6. Community \u0026 Support\n\n**GitHub**: \u003chttps://github.com/OpenSPG/KAG\u003e\n\n**Website**: \u003chttps://spg.openkg.cn/\u003e\n\n## Discord \u003ca href=\"https://discord.gg/PURG77zhQ7\"\u003e \u003cimg src=\"https://img.shields.io/discord/1329648479709958236?style=for-the-badge\u0026logo=discord\u0026label=Discord\" alt=\"Discord\"\u003e\u003c/a\u003e\n\nJoin our [Discord](https://discord.gg/PURG77zhQ7) community.\n\n## WeChat\n\nFollow OpenSPG Official Account to get technical articles and product updates about OpenSPG and KAG.\n\n\u003cimg src=\"./_static/images/openspg-qr.png\" alt=\"Contact Us: OpenSPG QR-code\" width=\"200\"\u003e\n\nScan the QR code below to join our WeChat group. \n\n\u003cimg src=\"./_static/images/robot-qr.JPG\" alt=\"Join WeChat group\" width=\"200\"\u003e\n\n\n# 7. Differences between KAG, RAG, and GraphRAG\n\n**KAG introduction and applications**: \u003chttps://github.com/orgs/OpenSPG/discussions/52\u003e\n\n# 8. Citation\n\nIf you use this software, please cite it as below:\n\n* [KAG: Boosting LLMs in Professional Domains via Knowledge Augmented Generation](https://arxiv.org/abs/2409.13731)\n\n* KGFabric: A Scalable Knowledge Graph Warehouse for Enterprise Data Interconnection\n\n```bibtex\n@article{liang2024kag,\n  title={KAG: Boosting LLMs in Professional Domains via Knowledge Augmented Generation},\n  author={Liang, Lei and Sun, Mengshu and Gui, Zhengke and Zhu, Zhongshu and Jiang, Zhouyu and Zhong, Ling and Zhao, Peilong and Bo, Zhongpu and Yang, Jin and others},\n  journal={arXiv preprint arXiv:2409.13731},\n  year={2024}\n}\n\n@article{yikgfabric,\n  title={KGFabric: A Scalable Knowledge Graph Warehouse for Enterprise Data Interconnection},\n  author={Yi, Peng and Liang, Lei and Da Zhang, Yong Chen and Zhu, Jinye and Liu, Xiangyu and Tang, Kun and Chen, Jialin and Lin, Hao and Qiu, Leijie and Zhou, Jun}\n}\n```\n\n# License\n\n[Apache License 2.0](LICENSE)\n\n# KAG Core Team\nLei Liang, Mengshu Sun, Zhengke Gui, Zhongshu Zhu, Zhouyu Jiang, Ling Zhong, Peilong Zhao, Zhongpu Bo, Jin Yang, Huaidong Xiong, Lin Yuan, Jun Xu, Zaoyang Wang, Zhiqiang Zhang, Wen Zhang, Huajun Chen, Wenguang Chen, Jun Zhou, Haofen Wang\n","funding_links":[],"categories":["Python","2024.11","Repos","知识库 RAG"],"sub_categories":["KAG【逻辑大师】"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FOpenSPG%2FKAG","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FOpenSPG%2FKAG","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FOpenSPG%2FKAG/lists"}