{"id":24879227,"url":"https://github.com/deepseek-ai/DeepSeek-Math","last_synced_at":"2025-10-15T22:30:59.252Z","repository":{"id":221085949,"uuid":"752957295","full_name":"deepseek-ai/DeepSeek-Math","owner":"deepseek-ai","description":null,"archived":false,"fork":false,"pushed_at":"2024-04-15T07:55:37.000Z","size":98735,"stargazers_count":550,"open_issues_count":13,"forks_count":23,"subscribers_count":11,"default_branch":"main","last_synced_at":"2024-04-23T00:11:44.304Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/deepseek-ai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE-CODE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-02-05T07:25:51.000Z","updated_at":"2024-04-22T23:13:31.000Z","dependencies_parsed_at":"2024-04-15T09:12:16.653Z","dependency_job_id":null,"html_url":"https://github.com/deepseek-ai/DeepSeek-Math","commit_stats":null,"previous_names":["deepseek-ai/deepseek-math"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deepseek-ai%2FDeepSeek-Math","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deepseek-ai%2FDeepSeek-Math/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deepseek-ai%2FDeepSeek-Math/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deepseek-ai%2FDeepSeek-Math/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/deepseek-ai","download_url":"https://codeload.github.com/deepseek-ai/DeepSeek-Math/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":236646157,"owners_count":19182608,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-02-01T10:03:23.034Z","updated_at":"2025-10-15T22:30:59.244Z","avatar_url":"https://github.com/deepseek-ai.png","language":"Python","readme":"\n\u003c!-- markdownlint-disable first-line-h1 --\u003e\n\u003c!-- markdownlint-disable html --\u003e\n\u003c!-- markdownlint-disable no-duplicate-header --\u003e\n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"images/logo.svg\" width=\"60%\" alt=\"DeepSeek LLM\" /\u003e\n\u003c/div\u003e\n\u003chr\u003e\n\u003cdiv align=\"center\"\u003e\n\n  \u003ca href=\"https://www.deepseek.com/\" target=\"_blank\"\u003e\n    \u003cimg alt=\"Homepage\" src=\"images/badge.svg\" /\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://chat.deepseek.com/\" target=\"_blank\"\u003e\n    \u003cimg alt=\"Chat\" src=\"https://img.shields.io/badge/🤖%20Chat-DeepSeek%20LLM-536af5?color=536af5\u0026logoColor=white\" /\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://huggingface.co/deepseek-ai\" target=\"_blank\"\u003e\n    \u003cimg alt=\"Hugging Face\" src=\"https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-DeepSeek%20AI-ffc107?color=ffc107\u0026logoColor=white\" /\u003e\n  \u003c/a\u003e\n   \u003ca href=\"https://replicate.com/cjwbw/deepseek-math-7b-base\" target=\"_parent\"\u003e\u003cimg src=\"https://replicate.com/cjwbw/deepseek-math-7b-base/badge\" alt=\"Replicate\"/\u003e\u003c/a\u003e \n\u003c/div\u003e\n\n\u003cdiv align=\"center\"\u003e\n\n  \u003ca href=\"https://discord.gg/Tc7c45Zzu5\" target=\"_blank\"\u003e\n    \u003cimg alt=\"Discord\" src=\"https://img.shields.io/badge/Discord-DeepSeek%20AI-7289da?logo=discord\u0026logoColor=white\u0026color=7289da\" /\u003e\n  \u003c/a\u003e\n  \u003ca href=\"images/qr.jpeg\" target=\"_blank\"\u003e\n    \u003cimg alt=\"Wechat\" src=\"https://img.shields.io/badge/WeChat-DeepSeek%20AI-brightgreen?logo=wechat\u0026logoColor=white\" /\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://twitter.com/deepseek_ai\" target=\"_blank\"\u003e\n    \u003cimg alt=\"Twitter Follow\" src=\"https://img.shields.io/badge/Twitter-deepseek_ai-white?logo=x\u0026logoColor=white\" /\u003e\n  \u003c/a\u003e\n\n\u003c/div\u003e\n\n\u003cdiv align=\"center\"\u003e\n\n  \u003ca href=\"LICENSE-CODE\"\u003e\n    \u003cimg alt=\"Code License\" src=\"https://img.shields.io/badge/Code_License-MIT-f5de53?\u0026color=f5de53\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"LICENSE-MODEL\"\u003e\n    \u003cimg alt=\"Model License\" src=\"https://img.shields.io/badge/Model_License-Model_Agreement-f5de53?\u0026color=f5de53\"\u003e\n  \u003c/a\u003e\n\u003c/div\u003e\n\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"#4-model-downloads\"\u003eModel Download\u003c/a\u003e |\n  \u003ca href=\"#2-evaluation-results\"\u003eEvaluation Results\u003c/a\u003e |\n  \u003ca href=\"#5-quick-start\"\u003eQuick Start\u003c/a\u003e |\n  \u003ca href=\"#6-license\"\u003eLicense\u003c/a\u003e |\n  \u003ca href=\"#7-citation\"\u003eCitation\u003c/a\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://arxiv.org/pdf/2402.03300.pdf\"\u003e\u003cb\u003ePaper Link\u003c/b\u003e👁️\u003c/a\u003e\n\u003c/p\u003e\n\n\n## 1. Introduction\n\nDeepSeekMath is initialized with [DeepSeek-Coder-v1.5 7B](https://huggingface.co/deepseek-ai/deepseek-coder-7b-base-v1.5) and continues pre-training on math-related tokens sourced from Common Crawl, together with natural language and code data for 500B tokens. DeepSeekMath 7B has achieved an impressive score of **51.7%** on the competition-level MATH benchmark without relying on external toolkits and voting techniques, approaching the performance level of Gemini-Ultra and GPT-4. For research purposes, we release [checkpoints](#4-model-downloads) of base, instruct, and RL models to the public.\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"images/math.png\" alt=\"table\" width=\"70%\"\u003e\n\u003c/p\u003e\n\n## 2. Evaluation Results\n\n### DeepSeekMath-Base 7B\n\nWe conduct a comprehensive assessment of the mathematical capabilities of DeepSeekMath-Base 7B, focusing on its ability to produce self-contained mathematical solutions without relying on external tools, solve math problems using tools, and conduct formal theorem proving. Beyond mathematics, we also provide a more general profile of the base model, including its performance of natural language understanding, reasoning, and programming skills.\n\n- **Mathematical problem solving with step-by-step reasoning**\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"images/base_results_1.png\" alt=\"table\" width=\"70%\"\u003e\n\u003c/p\u003e\n\n- **Mathematical problem solving with tool use**\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"images/base_results_2.png\" alt=\"table\" width=\"50%\"\u003e\n\u003c/p\u003e\n\n- **Natural Language Understanding, Reasoning, and Code**\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"images/base_results_3.png\" alt=\"table\" width=\"50%\"\u003e\n\u003c/p\u003e\n\nThe evaluation results from the tables above can be summarized as follows:\n  - **Superior Mathematical Reasoning:** On the competition-level MATH dataset, DeepSeekMath-Base 7B outperforms existing open-source base models by more than 10% in absolute terms through few-shot chain-of-thought prompting, and also surpasses Minerva 540B.\n  - **Strong Tool Use Ability:** Continuing pre-training with DeepSeekCoder-Base-7B-v1.5 enables DeepSeekMath-Base 7B to more effectively solve and prove mathematical problems by writing programs.\n  - **Comparable Reasoning and Coding Performance:** DeepSeekMath-Base 7B achieves performance in reasoning and coding that is comparable to that of DeepSeekCoder-Base-7B-v1.5.\n\n### DeepSeekMath-Instruct and -RL  7B\n\nDeepSeekMath-Instruct 7B is a mathematically instructed tuning model derived from DeepSeekMath-Base 7B, while DeepSeekMath-RL 7B is trained on the foundation of DeepSeekMath-Instruct 7B, utilizing our proposed Group Relative Policy Optimization (GRPO) algorithm.\n\nWe evaluate mathematical performance both without and with tool use, on 4 quantitative reasoning benchmarks in English and Chinese. As shown in Table, DeepSeekMath-Instruct 7B demonstrates strong performance of step-by-step reasoning, and DeepSeekMath-RL 7B approaches an accuracy of 60% on MATH with tool use, surpassing all existing open-source models.\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"images/instruct_results.png\" alt=\"table\" width=\"50%\"\u003e\n\u003c/p\u003e\n\n\n## 3. Data Collection\n\n- Step 1:  Select [OpenWebMath](https://arxiv.org/pdf/2310.06786.pdf), a collection of high-quality mathematical web texts, as our initial seed corpus for training a FastText model.\n- Step 2:  Use the FastText model to retrieve mathematical web pages from the deduplicated Common Crawl database.\n- Step 3:  Identify potential math-related domains through statistical analysis.\n- Step 4:  Manually annotate URLs within these identified domains that are associated with mathematical content.\n- Step 5:  Add web pages linked to these annotated URLs, but not yet collected, to the seed corpus. Jump to step 1 until four iterations.\n\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"images/data_pipeline.png\" alt=\"table\" width=\"80%\"\u003e\n\u003c/p\u003e\n\nAfter four iterations of data collection, we end up with **35.5M** mathematical web pages, totaling **120B** tokens. \n\n## 4. Model Downloads\n\nWe release the DeepSeekMath 7B, including base, instruct and RL models, to the public. To support a broader and more diverse range of research within both academic and commercial communities. Please **note** that the use of this model is subject to the terms outlined in [License section](#6-license). Commercial usage is permitted under these terms.\n\n### Huggingface\n\n| Model                    | Sequence Length |                           Download                           |\n| :----------------------- | :-------------: | :----------------------------------------------------------: |\n| DeepSeekMath-Base 7B     |      4096       | 🤗 [HuggingFace](https://huggingface.co/deepseek-ai/deepseek-math-7b-base) |\n| DeepSeekMath-Instruct 7B |      4096       | 🤗 [HuggingFace](https://huggingface.co/deepseek-ai/deepseek-math-7b-instruct) |\n| DeepSeekMath-RL 7B       |      4096       | 🤗 [HuggingFace](https://huggingface.co/deepseek-ai/deepseek-math-7b-rl) |\n\n## 5. Quick Start\n\nYou can directly employ [Huggingface's Transformers](https://github.com/huggingface/transformers) for model inference.\n\n**Text Completion**\n\n```python\nimport torch\nfrom transformers import AutoTokenizer, AutoModelForCausalLM, GenerationConfig\n\nmodel_name = \"deepseek-ai/deepseek-math-7b-base\"\ntokenizer = AutoTokenizer.from_pretrained(model_name)\nmodel = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.bfloat16, device_map=\"auto\")\nmodel.generation_config = GenerationConfig.from_pretrained(model_name)\nmodel.generation_config.pad_token_id = model.generation_config.eos_token_id\n\ntext = \"The integral of x^2 from 0 to 2 is\"\ninputs = tokenizer(text, return_tensors=\"pt\")\noutputs = model.generate(**inputs.to(model.device), max_new_tokens=100)\n\nresult = tokenizer.decode(outputs[0], skip_special_tokens=True)\nprint(result)\n```\n\n**Chat Completion**\n\n```python\nimport torch\nfrom transformers import AutoTokenizer, AutoModelForCausalLM, GenerationConfig\n\nmodel_name = \"deepseek-ai/deepseek-math-7b-instruct\"\ntokenizer = AutoTokenizer.from_pretrained(model_name)\nmodel = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.bfloat16, device_map=\"auto\")\nmodel.generation_config = GenerationConfig.from_pretrained(model_name)\nmodel.generation_config.pad_token_id = model.generation_config.eos_token_id\n\nmessages = [\n    {\"role\": \"user\", \"content\": \"what is the integral of x^2 from 0 to 2?\\nPlease reason step by step, and put your final answer within \\boxed{}.\"}\n]\ninput_tensor = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors=\"pt\")\noutputs = model.generate(input_tensor.to(model.device), max_new_tokens=100)\n\nresult = tokenizer.decode(outputs[0][input_tensor.shape[1]:], skip_special_tokens=True)\nprint(result)\n```\n\nAvoiding the use of the provided function `apply_chat_template`, you can also interact with our model following the sample template. Note that `messages` should be replaced by your input.\n\n```\nUser: {messages[0]['content']}\n\nAssistant: {messages[1]['content']}\u003c｜end▁of▁sentence｜\u003eUser: {messages[2]['content']}\n\nAssistant:\n```\n\n**Note:** By default (`add_special_tokens=True`), our tokenizer automatically adds a `bos_token` (`\u003c｜begin▁of▁sentence｜\u003e`) before the input text. Additionally, since the system prompt is not compatible with this version of our models, we DO NOT RECOMMEND including the system prompt in your input.\n\n❗❗❗ **Please use chain-of-thought prompt to test DeepSeekMath-Instruct and DeepSeekMath-RL:**\n\n- English questions: **{question}\\nPlease reason step by step, and put your final answer within \\\\boxed{}.**\n\n- Chinese questions: **{question}\\n请通过逐步推理来解答问题，并把最终答案放置于\\\\boxed{}中。**\n\n\n## 6. License\nThis code repository is licensed under the MIT License. The use of DeepSeekMath models is subject to the Model License. DeepSeekMath supports commercial use.\n\nSee the [LICENSE-CODE](LICENSE-CODE) and [LICENSE-MODEL](LICENSE-MODEL) for more details.\n\n## 7. Citation\n\n```\n@misc{deepseek-math,\n  author = {Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Mingchuan Zhang, Y.K. Li, Y. Wu, Daya Guo},\n  title = {DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models},\n  journal = {CoRR},\n  volume = {abs/2402.03300},\n  year = {2024},\n  url = {https://arxiv.org/abs/2402.03300},\n}\n```\n\n\n## 8. Contact\n\nIf you have any questions, please raise an issue or contact us at [service@deepseek.com](mailto:service@deepseek.com).\n","funding_links":[],"categories":["Repos","A01_文本生成_文本对话","Python"],"sub_categories":["大语言对话模型及数据"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeepseek-ai%2FDeepSeek-Math","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdeepseek-ai%2FDeepSeek-Math","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeepseek-ai%2FDeepSeek-Math/lists"}