{"id":50718191,"url":"https://github.com/BUAADreamer/EasyRAG","last_synced_at":"2026-06-26T22:00:42.781Z","repository":{"id":295827969,"uuid":"810288328","full_name":"BUAADreamer/EasyRAG","owner":"BUAADreamer","description":"Easy-to-Use RAG Framework; CCF AIOps International Challenge 2024 Top3 Solution; CCF AIOps 国际挑战赛 2024 季军方案","archived":false,"fork":false,"pushed_at":"2024-11-17T07:45:07.000Z","size":31814,"stargazers_count":629,"open_issues_count":1,"forks_count":76,"subscribers_count":4,"default_branch":"master","last_synced_at":"2026-05-21T08:44:02.114Z","etag":null,"topics":["aiops","bge","bm25","ccf","glm4","gte","hyde","llm","minicpm","network-operations","qa","rag","retrieval","retrieval-augmented-generation","rrf"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2410.10315","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/BUAADreamer.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2024-06-04T12:04:28.000Z","updated_at":"2026-05-15T14:25:01.000Z","dependencies_parsed_at":null,"dependency_job_id":"8ce1364d-4ede-4d79-9cec-00d6539ecaa5","html_url":"https://github.com/BUAADreamer/EasyRAG","commit_stats":null,"previous_names":["buaadreamer/easyrag"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/BUAADreamer/EasyRAG","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BUAADreamer%2FEasyRAG","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BUAADreamer%2FEasyRAG/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BUAADreamer%2FEasyRAG/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BUAADreamer%2FEasyRAG/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/BUAADreamer","download_url":"https://codeload.github.com/BUAADreamer/EasyRAG/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BUAADreamer%2FEasyRAG/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34834415,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-26T02:00:06.560Z","response_time":106,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aiops","bge","bm25","ccf","glm4","gte","hyde","llm","minicpm","network-operations","qa","rag","retrieval","retrieval-augmented-generation","rrf"],"created_at":"2026-06-09T21:00:25.963Z","updated_at":"2026-06-26T22:00:42.776Z","avatar_url":"https://github.com/BUAADreamer.png","language":"Python","funding_links":[],"categories":["*Ops for AI","Automation \u0026 Self-Healing"],"sub_categories":["Model Serving \u0026 Inference","ChatOps \u0026 AI Assistants for Ops"],"readme":"# EasyRAG: Efficient Retrieval-Augmented Generation Framework for Automated Network Operations\n\n[![license](https://img.shields.io/github/license/mashape/apistatus.svg?maxAge=2592000)](https://github.com/BUAADreamer/EasyRAG/blob/main/licence)\n[![arxiv badge](https://img.shields.io/badge/arxiv-2410.10315-red)](https://arxiv.org/abs/2410.10315)\n[![GitHub Repo stars](https://img.shields.io/github/stars/BUAADreamer/EasyRAG?style=social)](https://github.com/BUAADreamer/EasyRAG/stargazers)\n[![zhihu blog](https://img.shields.io/badge/zhihu-Blog-informational)](https://zhuanlan.zhihu.com/p/7272025344)\n## Table of Contents\n\n- [Overview](#Overview)\n- [Requirements](#Requirements)\n- [Reproduce](#Reproduce)\n- [Usage](#Usage)\n- [Project Structure](#Project-Structure)\n- [Citation](#Citation)\n- [Acknowledgement](#Acknowledgement)\n\n## Overview\n\nThis paper presents EasyRAG, a simple, lightweight, and efficient retrieval-augmented generation framework for automated network operations. Our framework has three advantages. The first is accurate question answering. We designed a straightforward RAG scheme based on (1) a specific data processing workflow (2) dual-route sparse retrieval for coarse ranking (3) LLM Reranker for reranking (4) LLM answer generation and optimization. This approach achieved first place in the GLM4 track in the preliminary round and second place in the GLM4 track in the semifinals. The second is simple deployment. Our method primarily consists of BM25 retrieval and BGE-reranker reranking, requiring no fine-tuning of any models, occupying minimal VRAM, easy to deploy, and highly scalable; we provide a flexible code library with various search and generation strategies, facilitating custom process implementation. The last one is efficient inference. We designed an efficient inference acceleration scheme for the entire coarse ranking, reranking, and generation process that significantly reduces the inference latency of RAG while maintaining a good level of accuracy; each acceleration scheme can be plug-and-play into any component of the RAG process, consistently enhancing the efficiency of the RAG system.\n\n\u003cdiv align=\"center\"\u003e\n    \u003cimg src=\"assets/overview.png\" width=\"95%\" height=\"auto\" /\u003e\n\u003c/div\u003e\n\n## Requirements\n\nEasyRAG needs Python3.10.14 and at least 1 GPU with 16GB. \n\nYou need to change `llm_keys` in `src/easyrag.yaml` to your GLM keys.\n\n```shell\npip install -r requirements.txt\ngit lfs install\nbash scripts/download.sh # download models\nbash scripts/process.sh # process zedx data\n\n```\n\n## Reproduce\n\n### 1. Run Directly\n\n```bash\ncd src\n# run challenge questions\npython3 main.py \n# copy answer file\ncp submit_result.jsonl ../answer.jsonl\n```\n\n### 2. Run with Docker\n\n```bash\nchmod +x scripts/run.sh\n./scripts/run.sh\n```\n\n## Usage\n\n### 1. API\n\n```bash\ncd src\nuvicorn api:app --host 0.0.0.0 --port 8000 --workers 1\n```\n\n### 2.WebUI\n\nYou need to run the API first\n\n```bash\ncd src\nstreamlit run webui.py\n```\n\n## Project Structure\n\nOnly the code that may be used in the semi-final is explained.\n\n```yaml\n- src\n    - custom\n        - splitter.py # Custom Chunk Splitter\n        - hierarchical.py # hierarchical Chunk Splitter\n        - transformation.py # File path and title extraction\n        - embeddings # Implement a separate embedding class for the GTE\n            - ...\n        - retrievers.py # Implementation of a qdrant-based dense retriever, Chinese BM25 retriever, implementation of a fusion retriever with rrf and simple merge\n        - rerankers.py # Implement some classes separately for the bge series of rerankers for easy custom use\n        - template.py # QA Prompt Template\n    - pipeline\n        - ingestion.py # Data processing flow: data reader, metadata extraction, document chunking, document encoding, metadata filters, vector database creation\n        - pipeline.py # EasyRAG Pipeline class, containing initialisation of various data and models, custom RAG Pipeline definitions\n        - rag.py # Some tool functions for rag\n        - qa.py # Read the question file and save the answer\n    - utils # The hf-adapted custom llm for the usage in China and is copied directly from the code in the hf link of the corresponding model.\n        - ...\n    - configs\n        - easyrag.yaml # configuration file\n    - data\n        - nltk_data # stop word lists and tokenizer data in nltk\n        - hit_stopwords.txt # HIT Chinese Stop Word List\n        - imgmap_filtered.json # Processed by get_ocr_data.py\n        - question.jsonl # Semi-Final Test Set\n    - main.py # Main Functions, Entry Files\n    - api.py # FastAPI Service\n    - preprocess_zedx.py # zedx data preprocessing\n    - get_ocr_data.py # paddleocr+glm4v extracts image content\n    - submit.py # Submit Results to Challenge\n- requirements.txt # python requirements\n- run.sh # docker run script\n- Dockerfile # docker configuration file\n```\n\n## Citation\n\n```latex\n@article{feng2024easyrag,\n  title={EasyRAG: Efficient Retrieval-Augmented Generation Framework for Automated Network Operations},\n  author={Feng, Zhangchi, Kuang Dongdong, Wang Zhongyuan, Nie Zhijie, Zheng Yaowei and Zhang, Richong},\n  journal={arXiv preprint arXiv:2410.10315},\n  year={2024}\n}\n```\n\n## Acknowledgement\n\nThanks to the [CCF AIOps 2024 Challenge Organising Committee](https://competition.aiops-challenge.com/home/competition/1780211530478944282) , they provide high quality data and a good atmosphere.\n\n## Star History\n\n![Star History Chart](https://api.star-history.com/svg?repos=BUAADreamer/EasyRAG\u0026type=Date)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FBUAADreamer%2FEasyRAG","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FBUAADreamer%2FEasyRAG","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FBUAADreamer%2FEasyRAG/lists"}