{"id":13572121,"url":"https://github.com/microsoft/ToRA","last_synced_at":"2025-04-04T09:31:46.201Z","repository":{"id":196894240,"uuid":"697405312","full_name":"microsoft/ToRA","owner":"microsoft","description":"ToRA is a series of Tool-integrated Reasoning LLM Agents designed to solve challenging mathematical reasoning problems by interacting with tools [ICLR'24].","archived":false,"fork":false,"pushed_at":"2024-02-22T09:46:48.000Z","size":16953,"stargazers_count":1051,"open_issues_count":6,"forks_count":73,"subscribers_count":18,"default_branch":"main","last_synced_at":"2025-04-02T06:32:16.406Z","etag":null,"topics":["autonomous-agents","language-model","llm","mathematical-reasoning","tool-learning"],"latest_commit_sha":null,"homepage":"https://microsoft.github.io/ToRA/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/microsoft.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-09-27T16:58:12.000Z","updated_at":"2025-03-26T10:09:15.000Z","dependencies_parsed_at":"2023-10-01T19:57:21.199Z","dependency_job_id":"c22f49b8-0889-495f-860b-9c32f6287ff8","html_url":"https://github.com/microsoft/ToRA","commit_stats":null,"previous_names":["microsoft/roolra","microsoft/tora"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/microsoft%2FToRA","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/microsoft%2FToRA/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/microsoft%2FToRA/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/microsoft%2FToRA/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/microsoft","download_url":"https://codeload.github.com/microsoft/ToRA/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247153544,"owners_count":20892688,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["autonomous-agents","language-model","llm","mathematical-reasoning","tool-learning"],"created_at":"2024-08-01T14:01:13.954Z","updated_at":"2025-04-04T09:31:46.192Z","avatar_url":"https://github.com/microsoft.png","language":"Python","readme":"\n\u003ch1 align=\"center\"\u003e\n\u003cimg src=\"./docs/static/images/tora_logo.png\" width=\"100\" alt=\"ToRA\" /\u003e\n\u003cbr\u003e\nToRA: A Tool-Integrated Reasoning Agent\n\u003c/h1\u003e\n\n\u003cdiv align=\"center\"\u003e\n\n![](https://img.shields.io/badge/Task-Mathematical%20Reasoning-orange)\n![](https://img.shields.io/badge/Model-Released-blue)\n![](https://img.shields.io/badge/Code%20License-MIT-green)\n\u003cbr\u003e\n[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/tora-a-tool-integrated-reasoning-agent-for/math-word-problem-solving-on-math)](https://paperswithcode.com/sota/math-word-problem-solving-on-math?p=tora-a-tool-integrated-reasoning-agent-for)\n\n\u003c/div\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://microsoft.github.io/ToRA/\"\u003e\u003cb\u003e[🌐 Website]\u003c/b\u003e\u003c/a\u003e •\n  \u003ca href=\"https://arxiv.org/abs/2309.17452\"\u003e\u003cb\u003e[📜 Paper]\u003c/b\u003e\u003c/a\u003e •\n  \u003ca href=\"https://huggingface.co/llm-agents\"\u003e\u003cb\u003e[🤗 HF Models]\u003c/b\u003e\u003c/a\u003e •\n  \u003ca href=\"https://github.com/microsoft/ToRA\"\u003e\u003cb\u003e[🐱 GitHub]\u003c/b\u003e\u003c/a\u003e\n  \u003c!-- \u003ca href=\"https://9557c5365a6f44dc84.gradio.live\"\u003e\u003cb\u003e[🐯 Gradio Demo]\u003c/b\u003e\u003c/a\u003e --\u003e\n  \u003cbr\u003e\n  \u003ca href=\"https://twitter.com/zhs05232838/status/1708860992631763092\"\u003e\u003cb\u003e[🐦 Twitter]\u003c/b\u003e\u003c/a\u003e •\n  \u003ca href=\"https://www.reddit.com/r/LocalLLaMA/comments/1703k6d/tora_a_toolintegrated_reasoning_agent_for/\"\u003e\u003cb\u003e[💬 Reddit]\u003c/b\u003e\u003c/a\u003e •\n  \u003ca href=\"https://notes.aimodels.fyi/researchers-announce-tora-training-language-models-to-better-understand-math-using-external-tools/\"\u003e[🍀 Unofficial Blog]\u003c/a\u003e\n  \u003c!-- \u003ca href=\"#-quick-start\"\u003eQuick Start\u003c/a\u003e • --\u003e\n  \u003c!-- \u003ca href=\"#%EF%B8%8F-citation\"\u003eCitation\u003c/a\u003e --\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\nRepo for \"\u003ca href=\"https://arxiv.org/abs/2309.17452\" target=\"_blank\"\u003eToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving\u003c/a\u003e\" [ICLR'2024]\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n    \u003cimg src=\"./docs/static/images/math_gsm_hist.png\" width=\"1000\"\u003e\n        \u003cbr\u003e\n    \u003cem\u003eFigure 1: Comparing ToRA with baselines on LLaMA-2 base models from 7B to 70B.\u003c/em\u003e\n\u003c/p\u003e\n\n\n## 🔥 News\n\n\u003c!-- - [2023/10/13] 🔥🔥🔥 We release a demo for ToRA at [🐯 Gradio](https://9557c5365a6f44dc84.gradio.live), try it out!!! --\u003e\n- [2023/10/08] 🔥🔥🔥 All ToRA models released at [🤗 HuggingFace](https://huggingface.co/llm-agents)!\n- [2023/09/29] ToRA paper, repo, and website released.\n\n## 💡 Introduction\n\nToRA is a series of Tool-integrated Reasoning Agents designed to solve challenging mathematical reasoning problems by interacting with tools, e.g., computation libraries and symbolic solvers. ToRA series seamlessly integrate natural language reasoning with the utilization of external tools, thereby amalgamating the analytical prowess of language and the computational efficiency of external tools.\n\n| Model | Size | GSM8k | MATH | AVG@10 math tasks\u003csup\u003e\u0026dagger;\u003c/sup\u003e |\n|---|---|---|---|---|\n| GPT-4 | - | 92.0 | 42.5 | 78.3 |\n| GPT-4 (PAL) | - | 94.2 | 51.8 | 86.4 |\n| [\u003cimg src=\"./docs/static/images/tora_logo.png\" width=\"16\" alt=\"ToRA\" /\u003e ToRA-7B](https://huggingface.co/llm-agents/tora-7b-v1.0) | 7B | 68.8 | 40.1 | 62.4|\n| [\u003cimg src=\"./docs/static/images/tora_logo.png\" width=\"16\" alt=\"ToRA\" /\u003e ToRA-Code-7B](https://huggingface.co/llm-agents/tora-code-7b-v1.0) | 7B | 72.6 | 44.6 | 66.5| \n| ToRA-Code-7B + self-consistency (k=50) | 7B | 76.8 | 52.5 | - |\n| [\u003cimg src=\"./docs/static/images/tora_logo.png\" width=\"16\" alt=\"ToRA\" /\u003e ToRA-13B](https://huggingface.co/llm-agents/tora-13b-v1.0) | 13B |  72.7 | 43.0 | 65.9|\n| [\u003cimg src=\"./docs/static/images/tora_logo.png\" width=\"16\" alt=\"ToRA\" /\u003e ToRA-Code-13B](https://huggingface.co/llm-agents/tora-code-13b-v1.0) | 13B | 75.8 | 48.1 | 71.3 |\n| ToRA-Code-13B + self-consistency (k=50) | 13B | 80.4 | 55.1 | - |\n| [\u003cimg src=\"./docs/static/images/tora_logo.png\" width=\"16\" alt=\"ToRA\" /\u003e ToRA-Code-34B\u003csup\u003e*\u003c/sup\u003e](https://huggingface.co/llm-agents/tora-code-34b-v1.0) | 34B | 80.7 | **51.0** | 74.8 |\n| ToRA-Code-34B + self-consistency (k=50) | 34B | 85.1 | \u003cu\u003e**60.0**\u003c/u\u003e | - |\n| [\u003cimg src=\"./docs/static/images/tora_logo.png\" width=\"16\" alt=\"ToRA\" /\u003e ToRA-70B](https://huggingface.co/llm-agents/tora-70b-v1.0) | 70B | **84.3** | 49.7 | **76.9** |\n| ToRA-70B + self-consistency (k=50) | 70B | \u003cu\u003e**88.3**\u003c/u\u003e | 56.9 | - |\n\n- \u003csup\u003e*\u003c/sup\u003eToRA-Code-34B is currently the first and only open-source model to achieve over 50% accuracy (pass@1) on the MATH dataset, which significantly outperforms GPT-4’s CoT result (51.0 vs. 42.5), and is competitive with GPT-4 solving problems with programs. By open-sourcing our codes and models, we hope more breakthroughs will come!\n\n- \u003csup\u003e\u0026dagger;\u003c/sup\u003e10 math tasks include GSM8k, MATH, GSM-Hard, SVAMP, TabMWP, ASDiv, SingleEQ, SingleOP, AddSub, and MultiArith.\n### Tool-Integrated Reasoning\n\n\u003cp align=\"center\"\u003e\n    \u003cimg src=\"./docs/static/images/example.png\" width=\"800\"\u003e\n    \u003cbr\u003e\n    \u003cem\u003eFigure 2: A basic example of single-round tool interaction, which interleaves rationales with program-based tool use.\u003c/em\u003e\n\u003c/p\u003e\n\n### ToRA Training Pipeline\n\n\u003cp align=\"center\"\u003e\n    \u003cimg src=\"./docs/static/images/pipeline.png\" width=\"800\"\u003e\n    \u003cbr\u003e\n    \u003cem\u003eFigure 3: Training ToRA contains ① Imitation Learning, and ② output space shaping.\u003c/em\u003e\n\u003c/p\u003e\n\n\n## 🚀 Quick Start\n\n### ⚙️ Setup\n\nWe recommend using [Conda](https://docs.conda.io/projects/miniconda) to manage your environment. We use [vLLM](https://github.com/vllm-project/vllm) (0.1.4) to accelerate inference. Run the following commands to setup your environment:\n\n```sh\ngit clone https://github.com/microsoft/ToRA.git \u0026\u0026 cd ToRA/src\nconda create -n tora python=3.10\nconda activate tora\npip install packaging==22.0\npip install torch==2.0.1 --index-url https://download.pytorch.org/whl/cu118 # CUDA 11.8 for example\npip install -r requirements.txt\n```\n\n### 🪁 Inference\n\nWe provide a script for inference, simply config the `MODEL_NAME_OR_PATH` and `DATA` in [src/scripts/infer.sh](/src/scripts/infer.sh) and run the following command:\n\n```sh\nbash scritps/infer.sh\n```\n\nWe also open-source the [model outputs](/src/outputs/llm-agents/) from our best models (ToRA-Code-34B and ToRA-70B) in the `src/outputs/` folder.\n\n### ⚖️ Evaluation\n\nThe [src/eval/grader.py](/src/eval/) file contains the grading logic that assesses the accuracy of the predicted answer by comparing it to the ground truth. This logic is developed based on the Hendrycks' MATH grading system, which we have manually verified on the MATH dataset to minimize false positives and false negatives.\n\nTo evaluate the predicted answer, run the following command:\n\n```sh\npython -m eval.evaluate \\\n    --data_name \"math\" \\\n    --prompt_type \"tora\" \\\n    --file_path \"outputs/llm-agents/tora-code-34b-v1.0/math/test_tora_-1_seed0_t0.0_s0_e5000.jsonl\" \\\n    --execute\n```\n\nthen you will get:\n\n```\nNum samples: 5000\nNum scores: 5000\nTimeout samples: 0\nEmpty samples: 2\nMean score: [51.0]\nType scores: {'Algebra': 67.3, 'Counting \u0026 Probability': 42.2, 'Geometry': 26.1, 'Intermediate Algebra': 40.0, 'Number Theory': 59.3, 'Prealgebra': 63.8, 'Precalculus': 34.2}\n```\n\n### ⚡️ Training\n\nWe're currently undergoing an internal review to open-source the ToRA-Corpus-16k, stay tuned!\nWe also open-source our complete training scripts for the community, and you may construct your own dataset for training. We provide some example data in [data/tora/](/data/tora/).\n\nTo train a model, run the following command:\n\n```sh\nbash scripts/train.sh codellama 7b\n```\n\n\n## ☕️ Citation\n\nIf you find this repository helpful, please consider citing our paper:\n\n```\n@inproceedings{\ngou2024tora,\ntitle={To{RA}: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving},\nauthor={Zhibin Gou and Zhihong Shao and Yeyun Gong and yelong shen and Yujiu Yang and Minlie Huang and Nan Duan and Weizhu Chen},\nbooktitle={The Twelfth International Conference on Learning Representations},\nyear={2024},\nurl={https://openreview.net/forum?id=Ep0TtjVoap}\n}\n```\n\n## 🍀 Contributing\n\nThis project welcomes contributions and suggestions.  Most contributions require you to agree to a\nContributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us\nthe rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.\n\nThis project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/).\nFor more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or\ncontact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments.\n\n\n## 🌟 Star History\n\n[![Star History Chart](https://api.star-history.com/svg?repos=microsoft/ToRA\u0026type=Date)](https://star-history.com/#microsoft/ToRA\u0026Date)\n","funding_links":[],"categories":["Python","A01_文本生成_文本对话","Building","Agent Categories"],"sub_categories":["大语言对话模型及数据","Tools","\u003ca name=\"Unclassified\"\u003e\u003c/a\u003eUnclassified"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmicrosoft%2FToRA","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmicrosoft%2FToRA","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmicrosoft%2FToRA/lists"}