{"id":31034646,"url":"https://github.com/freedomintelligence/question-free-fine-tuning","last_synced_at":"2025-09-14T02:46:41.027Z","repository":{"id":299755804,"uuid":"1004078348","full_name":"FreedomIntelligence/Question-Free-Fine-Tuning","owner":"FreedomIntelligence","description":"The official code for paper: QFFT, Question-Free Fine-Tuning for Adaptive Reasoning","archived":false,"fork":false,"pushed_at":"2025-06-18T05:06:20.000Z","size":19076,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-06-18T06:19:35.707Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/FreedomIntelligence.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-06-18T04:59:05.000Z","updated_at":"2025-06-18T05:06:45.000Z","dependencies_parsed_at":"2025-06-18T06:29:49.663Z","dependency_job_id":null,"html_url":"https://github.com/FreedomIntelligence/Question-Free-Fine-Tuning","commit_stats":null,"previous_names":["freedomintelligence/question-free-fine-tuning"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/FreedomIntelligence/Question-Free-Fine-Tuning","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FreedomIntelligence%2FQuestion-Free-Fine-Tuning","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FreedomIntelligence%2FQuestion-Free-Fine-Tuning/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FreedomIntelligence%2FQuestion-Free-Fine-Tuning/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FreedomIntelligence%2FQuestion-Free-Fine-Tuning/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/FreedomIntelligence","download_url":"https://codeload.github.com/FreedomIntelligence/Question-Free-Fine-Tuning/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FreedomIntelligence%2FQuestion-Free-Fine-Tuning/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":275054971,"owners_count":25397576,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-14T02:00:10.474Z","response_time":75,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-09-14T02:46:39.519Z","updated_at":"2025-09-14T02:46:40.997Z","avatar_url":"https://github.com/FreedomIntelligence.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# QFFT, Question-Free Fine-Tuning for Adaptive Reasoning\n\n\u003cp align=\"center\"\u003e\n📃 \u003ca href=\"assets/paper.pdf\" target=\"assets/paper.pdf\"\u003ePaper\u003c/a\u003e ｜ 🤗 \u003ca href=\"https://huggingface.co/lwl-uestc/QFFT-S1-7B\" target=\"_blank\"\u003eQFFT-7B\u003c/a\u003e ｜ 🤗 \u003ca href=\"https://huggingface.co/lwl-uestc/QFFT-S1-32B\" target=\"_blank\"\u003eQFFT-32B\u003c/a\u003e ｜ 📚 \u003ca href=\"https://huggingface.co/datasets/lwl-uestc/S1_QFFT\"\u003eQFFT Datasets\u003c/a\u003e\n\u003c/p\u003e\n\n---\n\n[![Paper](https://img.shields.io/badge/arXiv-2506.12860-b31b1b.svg)](https://arxiv.org/abs/2506.12860)\n\nThe complete code is coming soon! \n\n---\n\n\n## ⚡ Introduction\n\nWelcome to the official repository for **QFFT, Question-Free Fine-Tuning for Adaptive Reasoning**!\n\n\nQFFT introduces a novel and efficient fine-tuning method designed to empower large language models with **adaptive reasoning ability**. Instead of training models on (Question, Reasoning) pairs like traditional Supervised Fine-Tuning (SFT), **QFFT discards the question input and learns solely from the reasoning response**—especially Long CoT outputs.\n\nQFFT enables models to:\n\n- **Preserve Short CoT** for simple tasks (efficiency)\n- **Trigger Long CoT** only when needed (effectiveness)\n- **Reduce overthinking** by minimizing unnecessary reasoning\n- **Improve robustness** in noisy, low-resource, and out-of-domain scenarios\n\nWe open-sourced our models, data, and code here.\n\n---\n\n## 💭 Environment\n\n### Training Environment (LLaMA-Factory)\n\n```bash\ncd LLaMA-Factory\npip install -e \".[torch,metrics]\" --no-build-isolation\n```\n\n### Evaluation Environment (VLLM)\n\n```bash\npip install vllm bitsandbytes flashinfer-python==0.2.2.post1\npip install latex2sympy2 word2number\n```\n\n---\n\n## 💻 Model\n\n| Model Name           | Base LLM              | Link                                                                   |\n|----------------------|-----------------------|------------------------------------------------------------------------|\n| **QFFT-S1-7B**       | Qwen2.5-7B-Instruct    | [HF Link](https://huggingface.co/lwl-uestc/QFFT-S1-7B)                |\n| **QFFT-S1-32B**      | Qwen2.5-32B-Instruct   | [HF Link](https://huggingface.co/lwl-uestc/QFFT-S1-32B)               |\n| **QFFT-LIMO-7B**     | Qwen2.5-7B-Instruct    | [HF Link](https://huggingface.co/lwl-uestc/QFFT-LIMO-7B)              |\n| **QFFT-LIMO-32B**    | Qwen2.5-32B-Instruct   | [HF Link](https://huggingface.co/lwl-uestc/QFFT-LIMO-32B)             |\n\n---\n\n## 📚 Datasets\n\nQFFT uses distilled responses from strong Long CoT models (e.g., DeepSeek-R1). During QFFT, the input questions are removed entirely.\n\n| Dataset             | Size   | Link                        |\n|---------------------|--------|------------------------------------|\n| S1.1                | 1k     | [HF Link](https://huggingface.co/datasets/lwl-uestc/S1_QFFT)     |\n| LIMO                | 871    | [HF Link](https://huggingface.co/datasets/lwl-uestc/LIMO_QFFT)         |\n\n---\n\n## 🛠️ Training\n\n### Getting Started\n\nTo train a model using QFFT, you can use `llamafactory-cli` and the provided YAML configs:\n\n```bash\nllamafactory-cli train examples/train_qfft/train_s1_qfft.yaml\nllamafactory-cli train examples/train_qfft/train_limo_qfft.yaml\n```\n\n### Our Modifications\n\nThis codebase is based on [LLaMA-Factory](https://github.com/hiyouga/LLaMA-Factory).  \nOur key modification lies in the template system. We implement a new QFFT template in:\n\n```\n/src/llamafactory/data/template.py\n```\n\nFor details, please refer line **1569**.\n\n---\n\n## 🧪 Evaluation\n\nYou can evaluate QFFT models on benchmarks (e.g., GSM8K, MATH, AIME) with tools like `vllm` or `Sglang`.  \nWe also propose a novel metric **RAK** (Reasoning Adaptability Kappa) to evaluate the reasoning adaptability.\n\nThe evaluation code is coming soon!\n\n---\n\n## 📊 Results\n\nHere are the main results comparing SFT and QFFT on 3 mathematical reasoning benchmarks:\n\n### 📌 7B Models (Qwen2.5-7B-Instruct)\n\n| Dataset | Method | GSM8K Acc | GSM8K Tokens | MATH Acc | MATH Tokens | AIME25 Acc | AIME25 Tokens | Avg Acc | Avg Tokens |\n|---------|--------|-----------|--------------|----------|-------------|------------|----------------|---------|-------------|\n| S1.1    | SFT    | 90.6      | 1.7K         | 80.8     | 5.3K        | 18.2       | 17.7K          | 63.2    | 8.2K        |\n|         | QFFT   | 91.0      | 0.4K         | 80.2     | 2.8K        | 17.2       | 12.8K          | 62.8    | 5.3K        |\n|         | Δ      | +0.4      | -76.5%       | -0.6     | -47.2%      | -1.0       | -27.7%         | -0.4    | -50.5%      |\n\n| Dataset | Method | GSM8K Acc | GSM8K Tokens | MATH Acc | MATH Tokens | AIME25 Acc | AIME25 Tokens | Avg Acc | Avg Tokens |\n|---------|--------|-----------|--------------|----------|-------------|------------|----------------|---------|-------------|\n| LIMO    | SFT    | 88.2      | 1.8K         | 80.4     | 5.8K        | 16.8       | 17.1K          | 61.8    | 8.2K        |\n|         | QFFT   | 88.0      | 0.7K         | 80.6     | 4.1K        | 17.2       | 15.6K          | 61.9    | 6.8K        |\n|         | Δ      | -0.2      | -61.1%       | +0.2     | -29.3%      | +0.4       | -8.8%          | +0.1    | -33.1%      |\n\n### 📌 32B Models (Qwen2.5-32B-Instruct)\n\n| Dataset | Method | GSM8K Acc | GSM8K Tokens | MATH Acc | MATH Tokens | AIME25 Acc | AIME25 Tokens | Avg Acc | Avg Tokens |\n|---------|--------|-----------|--------------|----------|-------------|------------|----------------|---------|-------------|\n| S1.1    | SFT    | 92.8      | 2.1K         | 93.1     | 4.1K        | 48.6       | 16.2K          | 78.2    | 7.5K        |\n|         | QFFT   | 93.6      | 0.6K         | 92.2     | 2.4K        | 46.8       | 12.9K          | 77.5    | 5.3K        |\n|         | Δ      | +0.8      | -71.4%       | -0.9     | -41.5%      | -1.8       | -20.4%         | -0.6    | -44.4%      |\n\n| Dataset | Method | GSM8K Acc | GSM8K Tokens | MATH Acc | MATH Tokens | AIME25 Acc | AIME25 Tokens | Avg Acc | Avg Tokens |\n|---------|--------|-----------|--------------|----------|-------------|------------|----------------|---------|-------------|\n| LIMO    | SFT    | 91.2      | 1.9K         | 93.0     | 3.9K        | 45.8       | 13.2K          | 76.6    | 6.3K        |\n|         | QFFT   | 92.6      | 0.8K         | 92.6     | 2.9K        | 45.0       | 12.5K          | 76.7    | 5.4K        |\n|         | Δ      | +1.4      | -57.9%       | -0.4     | -25.6%      | -0.8       | -5.3%          | +0.1    | -29.6%      |\n\n\n\n---\n\n## 📖 Citation\n\n```\n@misc{liu2025qfft,\n  title={QFFT, Question-Free Fine-Tuning for Adaptive Reasoning},\n  author={Wanlong Liu and Junxiao Xu and Fei Yu and Yukang Lin and Ke Ji and Wenyu Chen and Yan Xu and Yasheng Wang and Lifeng Shang and Benyou Wang},\n  year={2025},\n  eprint={2506.12860},\n  archivePrefix={arXiv},\n  primaryClass={cs.CL},\n  url={https://arxiv.org/abs/2506.12860},\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffreedomintelligence%2Fquestion-free-fine-tuning","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffreedomintelligence%2Fquestion-free-fine-tuning","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffreedomintelligence%2Fquestion-free-fine-tuning/lists"}