{"id":29821166,"url":"https://github.com/thunlp-mt/extagents","last_synced_at":"2025-07-28T23:41:01.991Z","repository":{"id":303961921,"uuid":"991054499","full_name":"THUNLP-MT/ExtAgents","owner":"THUNLP-MT","description":"Public code repository for \"Scaling External Knowledge Input Beyond the Context Length of LLMs via Multi-Agent Collaboration\"","archived":false,"fork":false,"pushed_at":"2025-07-10T10:59:49.000Z","size":335,"stargazers_count":3,"open_issues_count":0,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-07-10T18:21:57.618Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/THUNLP-MT.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-05-27T04:07:34.000Z","updated_at":"2025-07-10T10:59:52.000Z","dependencies_parsed_at":"2025-07-10T18:31:50.669Z","dependency_job_id":"c036b435-fb99-410f-9813-c46dc76ec9c3","html_url":"https://github.com/THUNLP-MT/ExtAgents","commit_stats":null,"previous_names":["thunlp-mt/extagents"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/THUNLP-MT/ExtAgents","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THUNLP-MT%2FExtAgents","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THUNLP-MT%2FExtAgents/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THUNLP-MT%2FExtAgents/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THUNLP-MT%2FExtAgents/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/THUNLP-MT","download_url":"https://codeload.github.com/THUNLP-MT/ExtAgents/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THUNLP-MT%2FExtAgents/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":267604846,"owners_count":24114561,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-28T02:00:09.689Z","response_time":68,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-07-28T23:40:54.971Z","updated_at":"2025-07-28T23:41:01.981Z","avatar_url":"https://github.com/THUNLP-MT.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ExtAgents\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://github.com/THUNLP-MT/ExtAgents\"\u003e🌐 Github\u003c/a\u003e |\n  \u003ca href=\"https://arxiv.org/abs/2505.21471\"\u003e📖 Paper\u003c/a\u003e |\n  \u003ca href=\"https://huggingface.co/datasets/zhennan1/ExtAgents\"\u003e🤗 Data\u003c/a\u003e\n\u003c/p\u003e\n\n## Introduction\n\nExtAgents is a framework for scaling external knowledge input beyond the context length of LLMs via multi-agent collaboration.\n\n![overview](./figs/overview.png)\n\n## Setup\n\n```bash\nconda create -n extagents python=3.10 -y\nconda activate extagents\npip install -r requirements.txt\n```\n\n## Data\n\nYou can download the data with the script:\n\n```bash\nbash scripts/download_data.sh\n```\n\nOr you can download the data manually from one of the following links:\n\n- [Google Drive](https://drive.google.com/drive/folders/1FQSojqgF1VdumXxSh1UbIoE6lQ2E_xJn?usp=sharing)\n- [Tsinghua Cloud](https://cloud.tsinghua.edu.cn/d/b8aab568cf5c4785b457/)\n\nThe data should be organized as follows:\n\n```bash\n./\n└── data/\n    ├── sampled_hotpot_questions.json\n    ├── rag_1000k.jsonl\n    ├── longbook_qa_eng.jsonl\n    └── longbook_qa_chn.jsonl\n```\n\n**Update**: We have uploaded the long context Q\u0026A datasets both before and after enhancement to [Hugging Face](https://huggingface.co/datasets/zhennan1/ExtAgents). `original` refers to the original dataset, `full-enhanced` indicates the fully enhanced dataset, and `partial-enhanced` signifies the partially enhanced dataset, where only samples with a length not exceeding 128k tokens have been augmented. Welcome to download and use!\n\n## Usage\n\n### Generation\n\nWe currently support three tasks: RAG, En.QA, Zh.QA.\n\nThe RAG task is a question answering task, where the input is a question and a context. The question and answer are sampled from the [HotpotQA](https://github.com/hotpotqa/hotpot). The context is a long text, which is the concatenation of documents retrieved from Wikipedia using BM25 embedding. We use [KILT knowledge source](http://dl.fbaipublicfiles.com/KILT/kilt_knowledgesource.json) as our knowledge source. It is based on the [2019/08/01 Wikipedia dump](http://dl.fbaipublicfiles.com/BLINK/enwiki-pages-articles.xml.bz2). We have provided the context in the `data/rag_1000k.jsonl` file.\n\nThe En.QA and Zh.QA tasks are question answering tasks, where the input is a question and a context. The question, answer and context are from the [InfiniteBench](https://github.com/OpenBMB/InfiniteBench).\n\nHere is an example command to generate predictions for RAG task:\n\n```bash\npython main.py \\\n    --task rag \\\n    --output_dir results_rag \\\n    --chunk_length 8000 \\\n    --input_length 128000 \\\n    --api_url \"YOUR_API_URL\" \\\n    --api_key \"YOUR_API_KEY\" \\\n    --model \"gpt-4o-mini-2024-07-18\" \\\n    --num_workers 8 \\\n    \u003e rag.log\n```\n\nThe generated predictions will be saved in the `results_rag` directory.\n\n- `--task`: Task, can be `rag`, `en`, `zh`.\n- `--output_dir`: Directory to save the generated predictions.\n- `--chunk_length`: Chunk length.\n- `--input_length`: Input length.\n- `--model`: Model to use, default is `gpt-4o-mini-2024-07-18`.\n- `--api_url`: Your API URL, default is os.getenv(\"OPENAI_BASE_URL\").\n- `--api_key`: Your API Key, default is os.getenv(\"OPENAI_API_KEY\").\n- `--num_workers`: Number of workers, each worker will process one example.\n\nYou can also set the environment variables `OPENAI_BASE_URL` and `OPENAI_API_KEY` to avoid typing them in the command line.\n\n```bash\nexport OPENAI_BASE_URL=\"YOUR_API_URL\"\nexport OPENAI_API_KEY=\"YOUR_API_KEY\"\n```\n\n### Evaluation\n\nWe provide a script to evaluate the generated predictions. For RAG task, the evaluation is based on the [HotpotQA](https://github.com/hotpotqa/hotpot). For En.QA and Zh.QA task, the evaluation is based on the [InfiniteBench](https://github.com/OpenBMB/InfiniteBench).\n\nFor RAG task:\n\n```bash\nbash scripts/eval_rag.sh /path/to/your/output_dir\n```\n\nFor En.QA task: \n\n```bash\nbash scripts/eval_en.sh /path/to/your/output_dir\n```\n\nFor Zh.QA task:\n\n```bash\nbash scripts/eval_zh.sh /path/to/your/output_dir\n```\n\n## Citation\n\nIf you find this project helpful, please cite it as follows:\n\n```bibtex\n@article{liu2025extagents,\n  title={Scaling External Knowledge Input Beyond The Context Length of LLMs via Multi-Agent Collaboration},\n  author={Zijun Liu and Zhennan Wan and Peng Li and Ming Yan and Ji Zhang and Fei Huang and Yang Liu},\n  year={2025}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthunlp-mt%2Fextagents","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthunlp-mt%2Fextagents","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthunlp-mt%2Fextagents/lists"}