{"id":27093590,"url":"https://github.com/akshint0407/nano-r1","last_synced_at":"2026-04-28T17:32:31.941Z","repository":{"id":286071156,"uuid":"960258888","full_name":"Akshint0407/Nano-R1","owner":"Akshint0407","description":" This project demonstrates the process of fine-tuning the Qwen2.5-3B-Instruct model using GRPO (Generalized Reward Policy Optimization) on the GSM8K dataset. ","archived":false,"fork":false,"pushed_at":"2025-04-07T07:23:25.000Z","size":787,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-06-04T18:54:16.273Z","etag":null,"topics":["adapters","grpo","huggingface","python","qwen2-5","safetensors","text-generation-inference","transformer","trl","unsloth"],"latest_commit_sha":null,"homepage":"https://huggingface.co/Akshint47/Nano_R1_Model","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Akshint0407.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-04-04T06:00:58.000Z","updated_at":"2025-05-13T09:06:24.000Z","dependencies_parsed_at":"2025-04-04T07:28:30.711Z","dependency_job_id":"3751f3dd-58cd-424f-8f8a-258cc37d5480","html_url":"https://github.com/Akshint0407/Nano-R1","commit_stats":null,"previous_names":["akshint0407/nano-r1"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Akshint0407/Nano-R1","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Akshint0407%2FNano-R1","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Akshint0407%2FNano-R1/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Akshint0407%2FNano-R1/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Akshint0407%2FNano-R1/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Akshint0407","download_url":"https://codeload.github.com/Akshint0407/Nano-R1/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Akshint0407%2FNano-R1/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32392293,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-28T14:34:11.604Z","status":"ssl_error","status_checked_at":"2026-04-28T14:32:37.009Z","response_time":56,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["adapters","grpo","huggingface","python","qwen2-5","safetensors","text-generation-inference","transformer","trl","unsloth"],"created_at":"2025-04-06T08:38:05.009Z","updated_at":"2026-04-28T17:32:31.926Z","avatar_url":"https://github.com/Akshint0407.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Nano-R1\n# Fine-Tuning Qwen2.5-3B-Instruct with GRPO for Mathematical Reasoning\n\n![Python](https://img.shields.io/badge/python-3.11%2B-blue?style=for-the-badge\u0026logo=python\u0026logoColor=white)\n![License](https://img.shields.io/badge/license-Apache_2.0-red?style=for-the-badge\u0026logo=apache\u0026logoColor=white)\n![Hugging Face](https://img.shields.io/badge/Hugging_Face-Deployed-yellow?style=for-the-badge\u0026logo=huggingface\u0026logoColor=yellow)\n\n\nThis repository contains code for fine-tuning the **Qwen2.5-3B-Instruct** model using **GRPO (Generalized Reward Policy Optimization)** on the **GSM8K** dataset. The goal is to improve the model's ability to solve mathematical reasoning problems through reinforcement learning with custom reward functions.\n\n## 🚀 Deployment\n\nThe fine-tuned model is deployed on Hugging Face and can be accessed here:  \n🔗 **[Hugging Face Model Hub](https://huggingface.co/Akshint47/Nano_R1_Model)** \n\nYou can interact with the model directly or integrate it into your projects using the Hugging Face `transformers` library.\n\n## ✨ Features\n\n- **Efficient Fine-Tuning**: Uses Unsloth and LoRA for faster training with reduced GPU memory.\n- **Custom Reward Engineering**:\n  - Correctness (answer accuracy)\n  - Format adherence (XML-structured reasoning)\n  - Integer validation\n  - XML completeness scoring\n- **vLLM Integration**: Accelerates inference during training.\n- **GSM8K Focus**: Optimized for mathematical word problems.\n\n## 📋 Requirements\n\n```bash\n# Core packages\npip install unsloth vllm trl datasets\n```\n## Additional dependencies\n```bash\npip install torch transformers sentence piece accelerate\n```\n\n## Hardware Recommendations:\n\nGPU with ≥16GB VRAM (e.g., NVIDIA T4, A10G, or better)\n\nRecommended: CUDA 12.x and cuDNN 8.6+\n\n## 🛠️ Setup \u0026 Usage\nInstall dependencies:\n\n```bash\ngit clone https://github.com/your-username/your-repo.git\ncd your-repo\npip install -r requirements.txt\n```\n\nRun the notebook:\n\n```bash\njupyter notebook nano_r1_train_v2.ipynb\n```\nKey Configuration (in notebook):\n```python```\n```\nmodel, tokenizer = FastLanguageModel.from_pretrained(\n    model_name = \"Qwen/Qwen2.5-3B-Instruct\",\n    max_seq_length = 1024,\n    load_in_4bit = True,\n    max_lora_rank = 64\n)\n```\n\n## 📊 Training Process\nThe GRPO trainer optimizes for:\n\n-  Reward Maximization: Combined score from all reward functions\n  \n-  KL Regularization: Maintains policy stability\n  \n-  Efficiency: Processes 8 generations per batch\n  \n-  Training Progress (replace with actual metrics screenshot)\n\n## 📜 License  \nThis project is licensed under the **Apache License 2.0** - see the [LICENSE](LICENSE) file for full terms.  \n\n## 🙏 Acknowledgments\n- Unsloth for optimization tools\n\n- Hugging Face for models and datasets\n\n- vLLM for fast inference\n\n- OpenAI for the GSM8K dataset\n\n## 🤝 Contributing\n- Contributions are welcome! Please open an issue or PR for:\n\n- Bug fixes\n\n- Additional reward functions\n\n- Performance improvements\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fakshint0407%2Fnano-r1","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fakshint0407%2Fnano-r1","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fakshint0407%2Fnano-r1/lists"}