https://github.com/deep-div/fine-tuning-llms-and-visionmodels

Fine-Tuning LLMs (Gemma, LLaMA, Mistral, etc.) A practical guide to fine-tuning various large language models using popular frameworks. Includes examples, scripts, and tips for efficient training on custom datasets.
https://github.com/deep-div/fine-tuning-llms-and-visionmodels

deepseek finetuning-llms gemma generative-ai huggingface keras-finetuning large-language-models llama llama-factory llm transformers unsloth

Last synced: 12 months ago
JSON representation

Fine-Tuning LLMs (Gemma, LLaMA, Mistral, etc.) A practical guide to fine-tuning various large language models using popular frameworks. Includes examples, scripts, and tips for efficient training on custom datasets.

• Host: GitHub
• URL: https://github.com/deep-div/fine-tuning-llms-and-visionmodels
• Owner: deep-div
• License: apache-2.0
• Created: 2025-05-23T06:16:14.000Z (about 1 year ago)
• Default Branch: main
• Last Pushed: 2025-06-02T11:55:52.000Z (about 1 year ago)
• Last Synced: 2025-06-02T23:56:30.771Z (about 1 year ago)
• Topics: deepseek, finetuning-llms, gemma, generative-ai, huggingface, keras-finetuning, large-language-models, llama, llama-factory, llm, transformers, unsloth
• Language: Jupyter Notebook
• Homepage:
• Size: 540 KB
• Stars: 13
• Watchers: 1
• Forks: 4
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# 🧠 Fine-Tuning LLMs (Gemma, LLaMA, Mistral, and More)

Welcome to this curated repository showcasing the fine-tuning of various open-source large language models (LLMs) such as **Gemma**, **LLaMA**, **Mistral**, and others using **Hugging Face Transformers**, **PEFT (LoRA/QLoRA)**, and other modern libraries.

This repo is designed for researchers, ML engineers, and enthusiasts looking to explore or build on top of custom fine-tuned LLMs.

## 🔧 Features

* ✅ Fine-tuning with Hugging Face Trainer and PEFT (LoRA / QLoRA)

* ✅ Dataset loading and preprocessing

* ✅ Tokenization and model configuration

* ✅ Evaluation with custom metrics

* ✅ Easy-to-edit configs for reproducibility

* ✅ Support for mixed precision (fp16, bf16)

## 🧠 Contributing

Got improvements, additional models, or tips? Contributions are welcome! Just open an issue or submit a pull request.