{"id":30874703,"url":"https://github.com/jchip/lm-finetune","last_synced_at":"2025-09-08T01:07:03.989Z","repository":{"id":307575522,"uuid":"1029992990","full_name":"jchip/lm-finetune","owner":"jchip","description":"python project to fine tune LLM","archived":false,"fork":false,"pushed_at":"2025-07-31T23:17:39.000Z","size":41,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-08-01T01:33:36.357Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jchip.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-07-31T23:14:18.000Z","updated_at":"2025-07-31T23:17:42.000Z","dependencies_parsed_at":"2025-08-01T01:33:38.887Z","dependency_job_id":"04d75260-b906-4418-ab36-8a429789a193","html_url":"https://github.com/jchip/lm-finetune","commit_stats":null,"previous_names":["jchip/lm-finetune"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/jchip/lm-finetune","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jchip%2Flm-finetune","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jchip%2Flm-finetune/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jchip%2Flm-finetune/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jchip%2Flm-finetune/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jchip","download_url":"https://codeload.github.com/jchip/lm-finetune/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jchip%2Flm-finetune/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":274117706,"owners_count":25225143,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-07T02:00:09.463Z","response_time":67,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-09-08T01:07:01.512Z","updated_at":"2025-09-08T01:07:03.979Z","avatar_url":"https://github.com/jchip.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Local 7B LLM Fine-tuning and GGUF Conversion\n\nA complete pipeline for fine-tuning 7B language models using QLoRA and converting them to GGUF format.\n\n## Features\n\n- 🔧 **QLoRA Fine-tuning**: Efficient fine-tuning with 4-bit quantization\n- 🚀 **GPU Acceleration**: Optimized for consumer-grade GPUs\n- 🔄 **GGUF Conversion**: Convert models to GGUF format for llama.cpp inference\n- 📊 **Training Monitoring**: Integration with Weights \u0026 Biases and TensorBoard\n- 🎯 **Easy Setup**: Automated environment setup and dependency management\n\n## Prerequisites\n\n### System Requirements\n\n- Python 3.9+\n- CUDA Toolkit (for GPU acceleration)\n\n### Hardware Requirements\n\n- **Minimum**: 16GB RAM, CPU-only training\n- **Recommended**: 24GB+ RAM, NVIDIA GPU with 8GB+ VRAM\n\n## Quick Start\n\n1. **Clone and Setup**\n\n   ```bash\n   git clone \u003crepository-url\u003e\n   cd lm-finetune\n   python -m venv venv\n   venv\\Scripts\\activate\n   pip install -r requirements.txt\n   ```\n\n2. **Prepare Training Data**\n   Create a JSONL file with your training data:\n\n   ```json\n   {\"text\": \"\u003cs\u003e[INST] What is machine learning? [/INST] Machine learning is...\"}\n   {\"text\": \"\u003cs\u003e[INST] Explain neural networks [/INST] Neural networks are...\"}\n   ```\n\n3. **Run Fine-tuning**\n\n   ```bash\n   python scripts/finetune.py --model_name_or_path meta-llama/Llama-2-7b-hf --data_path data/training.jsonl --output_dir models/finetuned\n   ```\n\n4. **Convert to GGUF**\n\n   ```bash\n   python scripts/convert_to_gguf.py --model_path models/finetuned --output_path models/final.gguf\n   ```\n\n5. **Test Inference**\n   ```bash\n   python scripts/inference.py --model_path models/final.gguf --prompt \"What is machine learning?\"\n   ```\n\n## Project Structure\n\n```\nlm-finetune/\n├── scripts/\n│   ├── finetune.py          # QLoRA fine-tuning script\n│   ├── convert_to_gguf.py   # GGUF conversion script\n│   ├── inference.py         # Inference testing script\n│   └── setup_env.py         # Environment setup\n├── utils/\n│   ├── data_processing.py   # Data loading and preprocessing\n│   ├── model_utils.py       # Model loading utilities\n│   └── training_utils.py    # Training helpers\n├── configs/\n│   └── training_config.py   # Training configuration\n├── data/                    # Training data directory\n├── models/                  # Output models directory\n├── requirements.txt\n└── README.md\n```\n\n## Configuration\n\n### Training Parameters\n\nKey parameters can be configured in `configs/training_config.py`:\n\n- **Model**: Base model to fine-tune\n- **LoRA**: Rank, alpha, dropout settings\n- **Training**: Learning rate, batch size, epochs\n- **Quantization**: Bits, group size settings\n\n### Data Format\n\nTraining data should be in JSONL format with a \"text\" field containing complete prompts including model-specific tokens:\n\n```json\n{ \"text\": \"\u003cs\u003e[INST] User question here [/INST] Assistant response here\u003c/s\u003e\" }\n```\n\n## Advanced Usage\n\n### Custom Training Configuration\n\n```python\nfrom configs.training_config import TrainingConfig\n\nconfig = TrainingConfig(\n    model_name=\"meta-llama/Llama-2-7b-hf\",\n    lora_r=16,\n    lora_alpha=32,\n    learning_rate=2e-4,\n    num_epochs=3,\n    batch_size=4\n)\n```\n\n### Multi-GPU Training\n\n```bash\naccelerate launch --multi_gpu scripts/finetune.py --config_path configs/multi_gpu.yaml\n```\n\n## Troubleshooting\n\n### Common Issues\n\n1. **CUDA Out of Memory**: Reduce batch size or use gradient accumulation\n2. **Model Loading Errors**: Ensure you have access to the base model\n3. **GGUF Conversion Failures**: Check model format and llama.cpp installation\n\n### Performance Tips\n\n- Use gradient checkpointing for memory efficiency\n- Enable mixed precision training\n- Monitor GPU memory usage with `nvidia-smi`\n\n## Contributing\n\n1. Fork the repository\n2. Create a feature branch\n3. Make your changes\n4. Add tests\n5. Submit a pull request\n\n## License\n\nThis project is licensed under the MIT License - see the LICENSE file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjchip%2Flm-finetune","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjchip%2Flm-finetune","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjchip%2Flm-finetune/lists"}