{"id":20216120,"url":"https://github.com/thudm/mathglm","last_synced_at":"2025-04-07T12:07:20.198Z","repository":{"id":194059158,"uuid":"689974187","full_name":"THUDM/MathGLM","owner":"THUDM","description":"Official Pytorch Implementation for MathGLM ","archived":false,"fork":false,"pushed_at":"2023-11-20T19:20:55.000Z","size":30514,"stargazers_count":320,"open_issues_count":10,"forks_count":27,"subscribers_count":10,"default_branch":"main","last_synced_at":"2025-03-31T10:08:33.484Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/THUDM.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-09-11T09:39:11.000Z","updated_at":"2025-02-27T19:22:32.000Z","dependencies_parsed_at":"2024-08-03T09:17:17.882Z","dependency_job_id":null,"html_url":"https://github.com/THUDM/MathGLM","commit_stats":null,"previous_names":["thudm/mathglm"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THUDM%2FMathGLM","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THUDM%2FMathGLM/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THUDM%2FMathGLM/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THUDM%2FMathGLM/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/THUDM","download_url":"https://codeload.github.com/THUDM/MathGLM/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247648977,"owners_count":20972945,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-14T06:26:28.488Z","updated_at":"2025-04-07T12:07:20.178Z","avatar_url":"https://github.com/THUDM.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"## GPT Can Solve Mathematical Problems Without a Calculator \u003cbr\u003e\u003csub\u003eOfficial Pytorch Implementation\u003c/sub\u003e\n\n![](resources/perf.jpg)\n![](resources/perf_mwp.jpg)\nPrevious studies have typically assumed that large language models are unable to accurately perform arithmetic operations, particularly multiplication of \u003e8 digits, and operations involving decimals and fractions, without the use of calculator tools. This paper aims to challenge this misconception. With sufficient training data, a 2 billion-parameter language model can accurately perform multi-digit arithmetic operations with almost 100% accuracy without data leakage, significantly surpassing GPT-4 (whose multi-digit multiplication accuracy is only 4.3%). We also demonstrate that our MathGLM, finetuned from GLM-10B on a dataset with additional multi-step arithmetic operations and math problems described in text, achieves similar performance to GPT-4 on a 5,000-samples Chinese math problem\ntest set.\n\n\n\nIf you want to find the detailed introduction, Read our paper: [GPT Can Solve Mathematical Problems Without a Calculator](https://arxiv.org/pdf/2309.03241v2.pdf).\n\n# Demo\n\n## Arithmetic Tasks\n\n\n\n## Math Word Problem\n\nYou can access the MathGLM-10B demo by visiting the [ModelScope](https://www.modelscope.cn/studios/ZhipuAI/MathGLM-demo/summary)\n\n# Model Download \n\n\n## Arithmetic Tasks\nFor arithmetic tasks, we provide various model sizes for our MathGLM. If you wish to use our MathGLM for inference, you can download it from the following links.                                                                                                                    \n| Model    | Download | \n| --------- | --------------- |\n| MathGLM-10M   |  [THU-Cloud](https://cloud.tsinghua.edu.cn/d/16a914d6db2a4b8f80f5/)          | \n| MathGLM-100M  |  [THU-Cloud](https://cloud.tsinghua.edu.cn/d/a6ca369a212c4df08359/)          | \n| MathGLM-500M  |  [THU-Cloud](https://cloud.tsinghua.edu.cn/d/c80046ec7e234be4831b/)          | \n| MathGLM-2B    |  [THU-Cloud](https://cloud.tsinghua.edu.cn/d/cf429216289948d889a6/)          | \n\n\n## Math Word Problem\nFor math word problem tasks, we leverage different backbone models to tune our MathGLM on the reconstructed Ape210K dataset. Here, we provide various model sizes for our MathGLM. If you wish to use our MathGLM for inference, you can download it from the following links.                                                                                                                    \n\n\n| Model   | Backbone Model | Model Size | Download | \n| --------- | ---------- | ---------------- |---------------- |\n| MathGLM-Large       | GLM-Large    |335M   | [THU-Cloud](https://cloud.tsinghua.edu.cn/d/3d138deaf93441b196fb/)         | \n| MathGLM-10B         | GLM-zh-10b   | 10B   | [THU-Cloud](https://cloud.tsinghua.edu.cn/d/bc3c7b5c46304c2b88f6/) [ModelScope](https://modelscope.cn/models/ZhipuAI/MathGLM/summary)        | \n| MathGLM-ChatGLM-6B  | ChatGLM-6B   | 6B    | [THU-Cloud](https://cloud.tsinghua.edu.cn/d/92127e3a1b4144db8d13/)         | \n| MathGLM-ChatGLM2-6B | ChatGLM2-6B  | 6B    | [THU-Cloud](https://cloud.tsinghua.edu.cn/d/ab7eaac27ebd4d088834/)         | \n\n\n# Setup\n\n### Environment\nOur MathGLM relies on sat(SwissArmyTransformer), please ``` pip install SwissArmyTransformer ```.\n\nDownload the repo and setup the environment with:\n\n```bash\ngit clone https://github.com/THUDM/MathGLM.git\ncd MathGLM\nconda env create -f env.yml\nconda activate mathglm\n```\n#### Note\nFor arithmetic tasks and MathGLM-10B: deepspeed==0.6.0; For math word problems on MathGLM-6B: deepspeed==0.9.5\n\n\n\n### Dataset\n\nFor arithmetic tasks, please download pre-training dataset from [MathGLM-dataset](https://cloud.tsinghua.edu.cn/d/8d9ee3e52bb54afd9c16/). For amth word problem, the reconstructed Ape210K dataset is provided in ```MathGLM_MWP/dataset/data.jsonl```\n\n\n## Inference \n\nFor arithmetic tashs, you can directly execute the following command to evaluate our MathGLM on the provided test dataset that contains 9,592 test cases:\n\n```bash\ncd MathGLM_Arithmetic\n./inference.sh\n```\n\nFor math word problem, you can evaluate our MathGLM on the Ape210K test dataset that contains 5,000 test Chinese math word problems. You can run the following  command:\n\n```bash\ncd MathGLM_MWP\n./inference.sh\n```\n\n\n### Performance Reproduction\n\nMathGLM achieves competitive results in comparison with the most powerful large language model GPT-4 and ChatGPT.\n\n| Model   | ACC | RE | \n| --------- | ---------- | ---------------- | \n| GPT-4 | 18.84%    | -             |\n| ChatGPT  | 10.00%    | -            | \n| MathGLM-10M  | 61.21%    | 97.83%            | \n| MathGLM-100M  | 70.28%    | 99.28%            | \n| MathGLM-500M  | 89.57%    | 99.41%            | \n| MathGLM-2B  | 93.03%    | 99.71%            | \n\nMathGLM-10B achieves similar performance to GPT-4 on a 5,000-samples Chinese math problem test set.\n| Model   | Arithmetic_ACC | Answer_ACC | \n| --------- | ---------- | ---------------- | \n| GPT-4 | -  | 59.57%            |\n| ChatGPT  | -   | 39.78%        | \n| MathGLM-Large  | 62.00%   | 50.80%            | \n| MathGLM-GLM-6B  | 64.60%   | 48.06%            | \n| MathGLM-10B  | 69.08%    | 58.68%            | \n| MathGLM-GLM2-6B  | 52.24%   | 45.48%           | \n| MathGLM-ChatGLM-6B  | 58.52%    | 42.28%           | \n| MathGLM-ChatGLM2-6B  | 50.38%    | 43.14%           | \n\n## Pre-training\n\nFor arithmetic tashs, run command:\n\n```bash\ncd MathGLM_Arithmetic\n./pretrain.sh\n```\n\nFor math word problem, run command:\n\n```bash\ncd MathGLM_MWP\n./continue.sh\n```\n\n\n## Citation\n\n```\n@article{yang2023gpt,\n  title={GPT Can Solve Mathematical Problems Without a Calculator},\n  author={Yang, Zhen and Ding, Ming and Lv, Qingsong and Jiang, Zhihuan and He, Zehai and Guo, Yuyi and Bai, Jinfeng and Tang, Jie},\n  journal={arXiv preprint arXiv:2309.03241},\n  year={2023}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthudm%2Fmathglm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthudm%2Fmathglm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthudm%2Fmathglm/lists"}