{"id":27156672,"url":"https://github.com/mikesterner87/nano-r1","last_synced_at":"2025-09-03T20:45:15.537Z","repository":{"id":286801000,"uuid":"962558652","full_name":"Mikesterner87/Nano-R1","owner":"Mikesterner87","description":" This project demonstrates the process of fine-tuning the Qwen2.5-3B-Instruct model using GRPO (Generalized Reward Policy Optimization) on the GSM8K dataset. ","archived":false,"fork":false,"pushed_at":"2025-07-18T05:38:52.000Z","size":112,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-07-18T07:20:25.594Z","etag":null,"topics":["adapters","build","grpo","huggingface","nanopi","nanopi-r1","nanopi-r1s","openwrt","python","safetensors","text-generation-inference","transformer","trl","unsloth"],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Mikesterner87.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-04-08T10:34:30.000Z","updated_at":"2025-07-18T05:38:55.000Z","dependencies_parsed_at":"2025-04-10T02:41:03.990Z","dependency_job_id":"b668b982-c5cf-4d33-8e9c-3fa039079aa5","html_url":"https://github.com/Mikesterner87/Nano-R1","commit_stats":null,"previous_names":["mikesterner87/nano-r1"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/Mikesterner87/Nano-R1","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mikesterner87%2FNano-R1","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mikesterner87%2FNano-R1/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mikesterner87%2FNano-R1/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mikesterner87%2FNano-R1/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Mikesterner87","download_url":"https://codeload.github.com/Mikesterner87/Nano-R1/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mikesterner87%2FNano-R1/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":267007833,"owners_count":24020267,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-25T02:00:09.625Z","response_time":70,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["adapters","build","grpo","huggingface","nanopi","nanopi-r1","nanopi-r1s","openwrt","python","safetensors","text-generation-inference","transformer","trl","unsloth"],"created_at":"2025-04-08T20:32:02.808Z","updated_at":"2025-07-25T12:39:21.769Z","avatar_url":"https://github.com/Mikesterner87.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Nano-R1: Fine-Tuning Qwen2.5-3B-Instruct Model\n\n![Nano-R1](https://img.shields.io/badge/Nano--R1-Fine--Tuning--Model-brightgreen)\n\nWelcome to the **Nano-R1** repository! This project demonstrates the process of fine-tuning the **Qwen2.5-3B-Instruct** model using **Generalized Reward Policy Optimization (GRPO)** on the **GSM8K dataset**. This README provides all the information you need to get started with the project.\n\n## Table of Contents\n\n- [Introduction](#introduction)\n- [Getting Started](#getting-started)\n- [Features](#features)\n- [Installation](#installation)\n- [Usage](#usage)\n- [Contributing](#contributing)\n- [License](#license)\n- [Releases](#releases)\n- [Contact](#contact)\n\n## Introduction\n\nIn the realm of natural language processing, fine-tuning models for specific tasks is essential for achieving high performance. The **Qwen2.5-3B-Instruct** model is designed to understand and generate human-like text. This project focuses on fine-tuning this model using **GRPO** to enhance its performance on the **GSM8K dataset**, which consists of various mathematical problems.\n\n## Getting Started\n\nTo get started with the **Nano-R1** project, you will need to clone the repository and install the necessary dependencies. Follow the steps below to set up your environment.\n\n### Prerequisites\n\n- Python 3.7 or higher\n- pip (Python package installer)\n- Git\n\n### Clone the Repository\n\nYou can clone the repository using the following command:\n\n```bash\ngit clone https://github.com/Mikesterner87/Nano-R1.git\ncd Nano-R1\n```\n\n### Install Dependencies\n\nAfter cloning the repository, install the required packages:\n\n```bash\npip install -r requirements.txt\n```\n\n## Features\n\n- Fine-tuning of the **Qwen2.5-3B-Instruct** model.\n- Implementation of **GRPO** for effective training.\n- Utilization of the **GSM8K dataset** for model evaluation.\n- Support for various adapters and configurations.\n- Easy integration with Hugging Face libraries.\n\n## Installation\n\nTo run the project, you need to install the required libraries. The `requirements.txt` file contains all necessary dependencies. Use the following command to install them:\n\n```bash\npip install -r requirements.txt\n```\n\nMake sure to have the following libraries installed:\n\n- `transformers`\n- `torch`\n- `safetensors`\n- `trl`\n- `text-generation-inference`\n\n## Usage\n\nOnce you have set up the environment, you can start fine-tuning the model. Use the following command to begin the training process:\n\n```bash\npython train.py --dataset gsm8k --model qwen2-5\n```\n\nYou can adjust parameters in the `train.py` script to customize your training process. Refer to the comments in the code for guidance.\n\n### Example of Fine-Tuning\n\nHere’s a simple example of how to use the fine-tuned model for text generation:\n\n```python\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\n\nmodel_name = \"your_fine_tuned_model\"\ntokenizer = AutoTokenizer.from_pretrained(model_name)\nmodel = AutoModelForCausalLM.from_pretrained(model_name)\n\ninput_text = \"What is 7 + 5?\"\ninputs = tokenizer(input_text, return_tensors=\"pt\")\noutputs = model.generate(**inputs)\nprint(tokenizer.decode(outputs[0], skip_special_tokens=True))\n```\n\nThis code snippet demonstrates how to load your fine-tuned model and generate text based on a prompt.\n\n## Contributing\n\nWe welcome contributions to the **Nano-R1** project. If you would like to contribute, please follow these steps:\n\n1. Fork the repository.\n2. Create a new branch (`git checkout -b feature/YourFeature`).\n3. Make your changes.\n4. Commit your changes (`git commit -m 'Add some feature'`).\n5. Push to the branch (`git push origin feature/YourFeature`).\n6. Open a pull request.\n\nPlease ensure that your code adheres to the existing style and includes tests where applicable.\n\n## License\n\nThis project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.\n\n## Releases\n\nYou can find the latest releases and downloadable files [here](https://github.com/Mikesterner87/Nano-R1/releases). Please download and execute the files as needed to get the most out of this project.\n\nFor detailed information about each release, visit the [Releases](https://github.com/Mikesterner87/Nano-R1/releases) section.\n\n## Contact\n\nFor any questions or feedback, feel free to reach out:\n\n- GitHub: [Mikesterner87](https://github.com/Mikesterner87)\n- Email: mikesterner87@example.com\n\nThank you for your interest in the **Nano-R1** project! We hope you find it useful for your fine-tuning tasks.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmikesterner87%2Fnano-r1","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmikesterner87%2Fnano-r1","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmikesterner87%2Fnano-r1/lists"}