{"id":23932586,"url":"https://github.com/allenai/lumos","last_synced_at":"2025-10-13T15:56:31.358Z","repository":{"id":204756113,"uuid":"698799276","full_name":"allenai/lumos","owner":"allenai","description":"Code and data for \"Lumos: Learning Agents with Unified Data, Modular Design, and Open-Source LLMs\"","archived":false,"fork":false,"pushed_at":"2024-03-19T07:56:29.000Z","size":14782,"stargazers_count":470,"open_issues_count":3,"forks_count":29,"subscribers_count":10,"default_branch":"main","last_synced_at":"2025-09-11T16:28:35.253Z","etag":null,"topics":["decision-making","grounding","language-agent","maths","planning","question-answering","reasoning","web-agent"],"latest_commit_sha":null,"homepage":"https://allenai.github.io/lumos","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/allenai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-10-01T02:20:31.000Z","updated_at":"2025-09-03T09:15:48.000Z","dependencies_parsed_at":"2024-01-12T20:52:34.671Z","dependency_job_id":"f5f0bc6b-b6fc-4b02-af99-09bda0531f3d","html_url":"https://github.com/allenai/lumos","commit_stats":{"total_commits":107,"total_committers":4,"mean_commits":26.75,"dds":0.06542056074766356,"last_synced_commit":"b1d48c62eeab82bfaa398e841d2c67d1c5ecda77"},"previous_names":["allenai/lumos"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/allenai/lumos","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allenai%2Flumos","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allenai%2Flumos/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allenai%2Flumos/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allenai%2Flumos/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/allenai","download_url":"https://codeload.github.com/allenai/lumos/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allenai%2Flumos/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279015939,"owners_count":26085777,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-13T02:00:06.723Z","response_time":61,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["decision-making","grounding","language-agent","maths","planning","question-answering","reasoning","web-agent"],"created_at":"2025-01-06T00:29:09.087Z","updated_at":"2025-10-13T15:56:31.342Z","avatar_url":"https://github.com/allenai.png","language":"Python","funding_links":[],"categories":["A01_文本生成_文本对话","Learning","Agentic Systems"],"sub_categories":["大语言对话模型及数据","Repositories"],"readme":"# 🪄 Agent Lumos: Unified and Modular Training for Open-Source Language Agents\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://allenai.github.io/lumos/\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/🌐-Website-red\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://arxiv.org/abs/2311.05657\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/📝-Paper-blue\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://huggingface.co/datasets?sort=trending\u0026search=ai2lumos\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/🤗-Data-orange\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://huggingface.co/models?sort=trending\u0026search=ai2lumos\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/🤗-Model-green\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://huggingface.co/spaces/ai2lumos/lumos_data_demo\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/🤗-Demo-yellow\"\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\n🖋 **Authors:** [Da Yin](https://wadeyin9712.github.io/), [Faeze Brahman](https://fabrahman.github.io/), [Abhilasha Ravichander](https://www.cs.cmu.edu/~aravicha/), [Khyathi Chandu](https://www.cs.cmu.edu/~kchandu/), [Kai-Wei Chang](http://web.cs.ucla.edu/~kwchang/), [Yejin Choi](https://homes.cs.washington.edu/~yejin/), [Bill Yuchen Lin](https://yuchenlin.xyz/)\n\nWe introduce 🪄**Lumos**, Language Agents with **Unified** Data Formats, **Modular** Design, and **Open-Source** LLMs. **Lumos** unifies a suite of complex interactive tasks and achieves competitive performance with GPT-4/3.5-based and larger open-source agents. \n\n‼️ **Lumos** has following features:\n* 🧩 **Modular Architecture**:\n  - 🧩 **Lumos** consists of planning, grounding, and execution modules built based on LLAMA-2-7B/13B and off-the-shelf APIs.\n  - 🤗 **Lumos** utilizes a unified data format that encompasses multiple task types, thereby enabling the developed agent framework to conveniently support a range of interactive tasks.\n* 🌍 **Diverse Training Data**:\n  - 🌍 **Lumos** is trained with ~56K diverse high-quality subgoal/action annotations from ground-truth reasoning steps in existing benchmarks with GPT-4.\n  - ⚒️ **Lumos** data can be instrumental for future research in developing open-source agents for complex interactive tasks.\n* 🚀 **Competitive Performance**:\n  - 🚀 **Lumos** is comparable or even beats **GPT-series** agents on web/complex QA tasks Mind2Web and HotpotQA, and **larger open agents** on math and multimodal tasks.\n  - 🚀 **Lumos** exceeds contemporaneous agents that have been **fine-tuned** with in-domain HotpotQA, Mind2Web and ScienceQA annotations, such as **FiReAct**, **AgentLM**, and **AutoAct**.\n  - 🚀 **Lumos** performs better than open agent baseline formulations including **chain-of-thoughts** and **integrated** training.\n  - 🚀 **Lumos** surpasses larger open LLM agents and domain-specific agents on unseen tasks, WebShop and InterCode_SQL.\n \n## 🤩 Citation\n\nIf you find this work is relevant with your research, please feel free to cite our work!\n```\n@article{yin2023lumos,\n  title={{Agent Lumos: Unified and Modular Training for Open-Source Language Agents}},\n  author={Yin, Da and Brahman, Faeze and Ravichander, Abhilasha and Chandu, Khyathi and Chang, Kai-Wei and Choi, Yejin and Lin, Bill Yuchen},\n  journal={arXiv preprint arXiv:2311.05657},\n  year={2023}\n}\n```\n\n## 🔥 News\n- **[2024, Mar 18]** We release the latest **Lumos** version:\n  - 📑 **Lumos** paper that covers new **multimodal** tasks and 13B-scale model experiments\n  - 🤗 **Lumos** demo that illustrates **Lumos** planning and grounding processes\n- **[2023, Nov 8]** We release the important items for training and evaluating **Lumos**:\n  - 💻 **Lumos** code for annotation generation, training and evaluation\n  - 🤗 **Lumos** checkpoints with 7B model size\n  - 🤗 **Lumos** training annotations and their raw data\n \n## 🧩 Architecture\n\u003cp align=\"center\"\u003e\n\u003cimg src=assets/lumos.png width=850/\u003e\n\u003c/p\u003e\n\n## 🛠️ Setup\n```\n./setup.sh\n```\nPlease make sure that the cudatoolkit version in `setup.sh` aligns with your local cuda version.\n\n## Training\n### 📈 Training Data Download\nWe collect all the training annotations, raw data and prompt converted annotations in a single [Google Drive folder](https://drive.google.com/drive/folders/1ASFhOkhezgewVxR01dQg-8KUVR8IdBlY?usp=sharing). It can be downloaded by\n```\ncd data\npython -c \"import gdown; gdown.download_folder('https://drive.google.com/drive/folders/1ASFhOkhezgewVxR01dQg-8KUVR8IdBlY?usp=sharing', quiet=True)\" \n```\n\nWe also provide generated annotations for planning and grounding modules in 🤗 Huggingface Datasets. \n| Dataset Names | 🤗 Huggingface Links  |\n|----------------|----------------|\n| lumos_complex_qa_iterative    |  [Planning](https://huggingface.co/datasets/ai2lumos/lumos_complex_qa_plan_iterative), [Grounding](https://huggingface.co/datasets/ai2lumos/lumos_complex_qa_ground_iterative)        |\n| lumos_complex_qa_onetime         |  [Planning](https://huggingface.co/datasets/ai2lumos/lumos_complex_qa_plan_onetime), [Grounding](https://huggingface.co/datasets/ai2lumos/lumos_complex_qa_ground_onetime)         |\n| lumos_web_agent_iterative  |  [Planning](https://huggingface.co/datasets/ai2lumos/lumos_web_agent_plan_iterative), [Grounding](https://huggingface.co/datasets/ai2lumos/lumos_web_agent_ground_iterative)     |\n| lumos_multimodal_iterative  |  [Planning](https://huggingface.co/datasets/ai2lumos/lumos_multimodal_plan_iterative), [Grounding](https://huggingface.co/datasets/ai2lumos/lumos_multimodal_ground_iterative)     |\n| lumos_maths_iterative         |  [Planning](https://huggingface.co/datasets/ai2lumos/lumos_maths_plan_iterative), [Grounding](https://huggingface.co/datasets/ai2lumos/lumos_maths_ground_iterative)       |\n| lumos_maths_onetime         |  [Planning](https://huggingface.co/datasets/ai2lumos/lumos_maths_plan_onetime), [Grounding](https://huggingface.co/datasets/ai2lumos/lumos_maths_ground_onetime)    |\n| lumos_unified_iterative     | [Planning](https://huggingface.co/datasets/ai2lumos/lumos_unified_plan_iterative), [Grounding](https://huggingface.co/datasets/ai2lumos/lumos_unified_ground_iterative)    |\n\n### 🧑‍🎓️ Train Modules with Generated Annotation\n```\n./train.sh [MODULE] [FORMULATION]\n```\n`[MODULE]` can be either `plan` or `ground`. `[FORMULATION]` can be either `iterative` or `onetime`.\n\nYou can adjust the fine-tuning hyperparameters and specific task you want to fine-tune in the training scripts such as `finetune_llama2_plan_iterative.sh` in [`scripts/train`](./scripts/train).\n\nWe also provide the fine-tuned planning and grounding module checkpoints in 🤗 Huggingface.\n| Model Names | 🤗 Huggingface Links  |\n|----------------|----------------|\n| lumos_complex_qa_iterative    |  [Planning](https://huggingface.co/ai2lumos/lumos_complex_qa_plan_iterative), [Grounding](https://huggingface.co/ai2lumos/lumos_complex_qa_ground_iterative)        |\n| lumos_complex_qa_iterative-13B    |  [Planning](https://huggingface.co/ai2lumos/lumos_complex_qa_plan_iterative-13B), [Grounding](https://huggingface.co/ai2lumos/lumos_complex_qa_ground_iterative-13B)        |\n| lumos_complex_qa_onetime         |  [Planning](https://huggingface.co/ai2lumos/lumos_complex_qa_plan_onetime), [Grounding](https://huggingface.co/ai2lumos/lumos_complex_qa_ground_onetime)         |\n| lumos_web_agent_iterative  |  [Planning](https://huggingface.co/ai2lumos/lumos_web_agent_plan_iterative), [Grounding](https://huggingface.co/ai2lumos/lumos_web_agent_ground_iterative)     |\n| lumos_web_agent_iterative-13B  |  [Planning](https://huggingface.co/ai2lumos/lumos_web_agent_plan_iterative-13B), [Grounding](https://huggingface.co/ai2lumos/lumos_web_agent_ground_iterative-13B)     |\n| lumos_maths_iterative         |  [Planning](https://huggingface.co/ai2lumos/lumos_maths_plan_iterative), [Grounding](https://huggingface.co/ai2lumos/lumos_maths_ground_iterative)       |\n| lumos_maths_onetime         |  [Planning](https://huggingface.co/ai2lumos/lumos_maths_plan_onetime), [Grounding](https://huggingface.co/ai2lumos/lumos_maths_ground_onetime)    |\n| lumos_maths_onetime-13B         |  [Planning](https://huggingface.co/ai2lumos/lumos_maths_plan_onetime-13B), [Grounding](https://huggingface.co/ai2lumos/lumos_maths_ground_onetime-13B)    |\n| lumos_unified_iterative     |  [Planning](https://huggingface.co/ai2lumos/lumos_unified_plan_iterative), [Grounding](https://huggingface.co/ai2lumos/lumos_unified_ground_iterative)    |\n| lumos_unified_iterative-13B     |  [Planning](https://huggingface.co/ai2lumos/lumos_unified_plan_iterative-13B), [Grounding](https://huggingface.co/ai2lumos/lumos_unified_ground_iterative-13B)    |\n\n## ✅ Evaluation\nEvaluation scripts for different datasets are under [`scripts/eval`](./scripts/eval). For example, you can evaluate Lumos on HotpotQA by running:\n```\n./scripts/eval/hotpotqa.sh\n```\n\n## Others\n### 📈 Data Annotation Generation\nWe provide the code for generating training annotations based on raw existing benchmarks from scratch. \n\nBefore generating annotations, we first need to download the existing benchmarks providing ground-truth intermediate reasoning steps. \nThe raw data are can be downloaded via this [Google Drive folder](https://drive.google.com/drive/folders/1ASFhOkhezgewVxR01dQg-8KUVR8IdBlY?usp=sharing). \n```\npython -m data.prompt_convertion \\\n  --domain DOMAIN \\\n  --data_fn DATA_FN \\\n  --convert_all\n```\n`domain` covers maths, complex QA, web agent, multimodal. `data_fn` is the path where raw benchmarks are stored.\n\nFor multimodal task annotation generation, please download [COCO 2017 train images](http://images.cocodataset.org/zips/train2017.zip) in `data/train/multimodal/raw_data` and unzip it.\n\n## ❤️ Acknowledgement\nWe greatly thank Tulu team for providing awesome [code](https://github.com/allenai/open-instruct) to finetune LLAMA-2. We also sincerely appreciate the contributors of [zeno-build](https://github.com/zeno-ml/zeno-build), [Mind2Web](https://github.com/OSU-NLP-Group/Mind2Web), and [WebShop](https://github.com/princeton-nlp/WebShop) for providing fast GPT prompting, HTML preprocessing and evaluation docker environment.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fallenai%2Flumos","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fallenai%2Flumos","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fallenai%2Flumos/lists"}