{"id":28653872,"url":"https://github.com/tiger-ai-lab/general-reasoner","last_synced_at":"2025-06-13T07:07:58.221Z","repository":{"id":288124941,"uuid":"963673505","full_name":"TIGER-AI-Lab/General-Reasoner","owner":"TIGER-AI-Lab","description":"General Reasoner: Advancing LLM Reasoning Across All Domains","archived":false,"fork":false,"pushed_at":"2025-06-10T04:14:56.000Z","size":7856,"stargazers_count":132,"open_issues_count":1,"forks_count":6,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-06-10T05:21:28.457Z","etag":null,"topics":["llm","reasoning","rl"],"latest_commit_sha":null,"homepage":"https://tiger-ai-lab.github.io/General-Reasoner","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/TIGER-AI-Lab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-04-10T03:28:44.000Z","updated_at":"2025-06-10T04:14:59.000Z","dependencies_parsed_at":"2025-04-15T17:46:30.165Z","dependency_job_id":"bd21dcff-e9f3-41eb-a9e0-46cb4bd1fcc0","html_url":"https://github.com/TIGER-AI-Lab/General-Reasoner","commit_stats":null,"previous_names":["tiger-ai-lab/general-reasoner"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/TIGER-AI-Lab/General-Reasoner","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TIGER-AI-Lab%2FGeneral-Reasoner","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TIGER-AI-Lab%2FGeneral-Reasoner/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TIGER-AI-Lab%2FGeneral-Reasoner/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TIGER-AI-Lab%2FGeneral-Reasoner/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/TIGER-AI-Lab","download_url":"https://codeload.github.com/TIGER-AI-Lab/General-Reasoner/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TIGER-AI-Lab%2FGeneral-Reasoner/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":259599331,"owners_count":22882357,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["llm","reasoning","rl"],"created_at":"2025-06-13T07:07:56.638Z","updated_at":"2025-06-13T07:07:58.199Z","avatar_url":"https://github.com/TIGER-AI-Lab.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# General-Reasoner: Advancing LLM Reasoning Across All Domains (beyond Math)\n\n\n🪡 We introduce a novel framework within GRPO to utilize model-based verifier to support wider range of answer types. We train on WebInstruct-verfied, a dataset covering wide range of reasoning topics beyond math. We demonstrate substantial improvements over traditional binary rule-based rewards across diverse domains beyond math. \n\n✅ Model-based verifier to support verification of diverse answer types like math expression, string, list, fraction, matrix, etc;\u003cbr\u003e\n✅ Small 7B/14B models achieve robust cross-domain rewards; It boosts MMLU-Pro performance by 13%.\u003cbr\u003e\n✅ Our method does not require any additional SFT.\n\nCheck out our [Arxiv Paper](https://arxiv.org/abs/2505.14652) for the details!\n\n## Highlights\n\nOur experimental results are as follows:\n\n\u003cp float=\"left\"\u003e\n  \u003cimg src=\"assets/teaser.png\" width=\"65%\" /\u003e\n\u003c/p\u003e\n\nOur discpline distribution is described as follows:\n\n\u003cp float=\"left\"\u003e\n  \u003cimg src=\"assets/distribution.png\" width=\"65%\" /\u003e\n\u003c/p\u003e\n\nOur model-based verifier helps scaling the verifiable reasoning questions:\n\n\u003cp float=\"left\"\u003e\n  \u003cimg src=\"assets/verifier.png\" width=\"65%\" /\u003e\n\u003c/p\u003e\n\n\n---\n\n## Resources\n\n### Training Data\n\n|Data|Size|Link|\n|-|-|-|\n|WebInstruct-verified| 230k | [🤗](https://huggingface.co/datasets/TIGER-Lab/WebInstruct-verified)|\n\n\n\n### General Verifier\n\n|Model|Backbone|Link|\n|-|-|-|\n|General-Verifier|Qwen/Qwen2.5-Math-1.5B|[🤗](https://huggingface.co/TIGER-Lab/general-verifier)|\n\nCheck out HF page to learn how to use it. Feel free to plug this into your current RL-training framework. \n\n### Model Checkpoint\n\n|Model|Backbone|Link|\n|-|-|-|\n|General-Reasoner-Qwen2.5-7B|Qwen2.5-7B-Base|[🤗](https://huggingface.co/TIGER-Lab/General-Reasoner-Qwen2.5-7B)|\n|General-Reasoner-Qwen2.5-14B|Qwen2.5-14B-Base|[🤗](https://huggingface.co/TIGER-Lab/General-Reasoner-Qwen2.5-14B)|\n|General-Reasoner-Qwen3-4B|Qwen3-4B-Base|[🤗](https://huggingface.co/TIGER-Lab/General-Reasoner-Qwen3-4B)|\n|General-Reasoner-Qwen3-14B|Qwen3-14B-Base|[🤗](https://huggingface.co/TIGER-Lab/General-Reasoner-Qwen3-14B)|\n---\n\n## Installation\n\n```bash\npip install torch==2.4.0 --index-url https://download.pytorch.org/whl/cu124\npip install flash-attn --no-build-isolation\npip install -e ./verl\npip install vllm==0.8.3\npip install flashinfer-python\npip install math-verify\n```\n\n---\n\n## Training\n\n### 1. Prepare Data\n```bash\npython data_preprocess.py --local-dir \u003cdata_dir\u003e/webinstruct-verified\n```\n\n### 2. Download Verifier\n```bash\nhuggingface-cli download TIGER-Lab/general-verifier --local-dir \u003cdata_dir\u003e/general-reasoner-verifier\n```\n\n### 3. Download Backbone Model\n```bash\nhuggingface-cli download Qwen/Qwen2.5-7B --local-dir \u003cdata_dir\u003e/Qwen2.5-7B\n```\n\n### 4. Configure Training Script\nEdit the environment variables in `train_general_reasoner.sh` to fit your system setup.\n\n### 5. Launch Ray Cluster\n```bash\nray start --address \u003cMASTER-NODE-IP\u003e:6379\n```\n\n### 6. Start Training\n```bash\nbash train_general_reasoner.sh\n```\n\n---\n\n## Evaluation\n\n### MMLU-PRO:\n\n```bash\npython -m evaluation.eval_mmlupro \\\n    --model_path TIGER-Lab/General-Reasoner-14B \\\n    --output_file output-mmlupro-General-Reasoner-14B.json\n```\n\n### SuperGPQA:\n\n```bash\npython -m evaluation.eval_supergpqa \\\n    --model_path TIGER-Lab/General-Reasoner-14B \\\n    --output_file output-supergpqa-General-Reasoner-14B.json\n```\n\n### Math-Related Tasks \u0026 GPQA\nWe evaluate math and GPQA tasks using the `simple-eval` framework.\nFor non-multiple choice questions, answer equivalence is verified using `GPT-4o`.\n\n#### 1. Configure OpenAI Key\n```bash\nexport OPENAI_API_KEY=\u003creplace by OPENAI API KEY\u003e\n```\n\n#### 2. Serve the Model\n```bash\nvllm serve TIGER-Lab/General-Reasoner-14B --tensor-parallel-size 4\n```\n\n#### 3. Run Evaluation\n```bash\npython -m evaluation.simple-evals.run_simple_evals_qwen \\\n    --model General-Reasoner-14B\n```\n\n\u003e By default, the model uses greedy decoding.\n\u003e For AIME24 and AIME25, scores are averaged over 32 runs with temperature 1.\n\u003e For more configuration details, refer to `evaluation/simple-evals/run_simple_evals_qwen.py`.\n\n\n---\n\n## Detailed Results\n\nOur 7B and 14B models are trained from the corresponding base qwen models.\n\n\n### General results\n\n| Model Name                   | MMLU-Pro | GPQA     | SuperGPQA | TheoremQA | BBEH     |\n| ---------------------------- | -------- | -------- | --------- | --------- | -------- |\n| Qwen2.5-7B-Base              | 47.7     | 25.8     | 26.7      | 29.1      | 8.0      |\n| Qwen2.5-7B-Instruct          | 57.0     | 33.8     | 30.7      | 36.6      | 12.2     |\n| SimpleRL-Qwen2.5-7B-Zoo      | 51.5     | 24.2     | 29.9      | 38.0      | 11.9     |\n| **General-Reasoner-7B**      | **58.9** | **34.3** | **34.2**  | **45.3**  | **12.5** |\n|                              |          |          |           |           |          |\n| Qwen2.5-14B-Base             | 53.3     | 32.8     | 30.7      | 33.0      | 10.8     |\n| Qwen2.5-14B-Instruct         | 62.7     | 41.4     | 35.8      | 41.9      | 15.2     |\n| SimpleRL-Qwen2.5-14B-Zoo     | 64.0     | 39.4     | 35.7      | 40.8      | 13.6     |\n| **General-Reasoner-14B**     | **66.6** | **43.4** | **39.5**  | **44.3**  | **15.2** |\n|                              |          |          |           |           |          |\n| Qwen3-4B-Base                | 51.6     | 26.3     | 25.4      | 34.8      | 8.1      |\n| Qwen3-4B-Instruct            | 61.8     | 41.7     | 32.1      | 42.0      | **14.9** |\n| **General-Reasoner-Qw3-4B**  | **62.8** | **42.9** | **32.5**  | **48.3**  | 12.2     |\n|                              |          |          |           |           |          |\n| Qwen3-14B-Base               | 64.2     | 45.9     | 36.5      | 44.0      | 13.0     |\n| Qwen3-14B-Instruct           | **70.9** | 54.8     | 39.8      | 42.4      | **19.2** |\n| **General-Reasoner-Qw3-14B** | 70.3     | **56.1** | **39.9**  | **54.4**  | 17.3     |\n\n\n\n### Math-related results\n\n| Model Name                   | MATH-500 | Olympiad | Minerva | GSM8K | AMC  | AIME24x32 | AIME25x32 |\n| ---------------------------- | -------- | -------- | ------- | ----- | ---- | --------- | --------- |\n| Qwen2.5-7B-Base              | 60.2     | 28.6     | 36.0    | 83.1  | 30.0 | 3.8       | 1.4       |\n| Qwen2.5-7B-Instruct          | 75.0     | 39.4     | 45.2    | 90.9  | 52.5 | 12.5      | 8.5       |\n| SimpleRL-Qwen2.5-7B-Zoo      | 74.0     | 41.9     | 49.6    | 90.7  | 60.0 | 15.2      | 7.5       |\n| **General-Reasoner-7B**      | 76.0     | 37.9     | 54.0    | 92.7  | 55.0 | 13.8      | 10.4      |\n|                              |          |          |         |       |      |           |           |\n| Qwen2.5-14B-Base             | 65.4     | 33.5     | 24.3    | 91.6  | 37.5 | 3.6       | 2.9       |\n| Qwen2.5-14B-Instruct         | 77.4     | 44.7     | 52.2    | 94.5  | 57.5 | 12.2      | 11.0      |\n| SimpleRL-Qwen2.5-14B-Zoo     | 77.2     | 44.6     | 54.0    | 94.2  | 60.0 | 12.9      | 11.8      |\n| **General-Reasoner-14B**     | 78.6     | 42.1     | 58.1    | 94.2  | 70.0 | 17.5      | 16.9      |\n|                              |          |          |         |       |      |           |           |\n| Qwen3-4B-Base                | 68.2     | 34.8     | 42.3    | 72.6  | 47.5 | 10.3      | 6.7       |\n| Qwen3-4B-Instruct(non-think) | 80.4     | 49.0     | 57.0    | 92.0  | 62.5 | 22.5      | 16.1      |\n| **General-Reasoner-Qw3-4B**  | 80.6     | 47.7     | 57.7    | 92.2  | 60.0 | 20.0      | 15.4      |\n|                              |          |          |         |       |      |           |           |\n| Qwen3-14B-Base               | 74.6     | 44.3     | 55.9    | 93.2  | 55.0 | 14.7      | 11.4      |\n| Qwen3-14B-Instruct(non-think)| 82.0     | 52.4     | 59.9    | 93.9  | 57.5 | 28.5      | 25.1      |\n| **General-Reasoner-Qw3-14B** | 83.8     | 51.9     | 68.0    | 94.4  | 70.0 | 24.4      | 19.2      |\n\n\n\n\n## Acknowledgements\n\nThis project is built upon the following open-source projects:\n\n- [VERL](https://github.com/volcengine/verl/tree/main/verl)  \n- [simpleRL-reason](https://github.com/hkust-nlp/simpleRL-reason)  \n- [simple-evals](https://github.com/openai/simple-evals)\n\n\n## Citation\n\n```tex\n@article{general-reasoner,\n      title={General-Reasoner: Advancing LLM Reasoning Across All Domains}, \n      author={Xueguang Ma and Qian Liu and Dongfu Jiang and Ge Zhang and Zejun Ma and Wenhu Chen},\n      year={2025},\n      journal={arXiv:2505.14652},\n      url={https://arxiv.org/abs/2505.14652}, \n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftiger-ai-lab%2Fgeneral-reasoner","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftiger-ai-lab%2Fgeneral-reasoner","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftiger-ai-lab%2Fgeneral-reasoner/lists"}