https://github.com/codeofrahul/fine_tuning_gemma_model

This project demonstrates how to fine-tune the Gemma-2B language model for conversation summarization using QLoRA (Quantized Low-Rank Adaptation) on the SAMSum dataset. It leverages Hugging Face Transformers, PEFT, and BitsAndBytes libraries for efficient and effective fine-tuning.
https://github.com/codeofrahul/fine_tuning_gemma_model

finetuning-large-language-models finetuning-llms gemma-2b huggingface

Last synced: about 2 months ago
JSON representation

This project demonstrates how to fine-tune the Gemma-2B language model for conversation summarization using QLoRA (Quantized Low-Rank Adaptation) on the SAMSum dataset. It leverages Hugging Face Transformers, PEFT, and BitsAndBytes libraries for efficient and effective fine-tuning.

• Host: GitHub
• URL: https://github.com/codeofrahul/fine_tuning_gemma_model
• Owner: CodeofRahul
• License: mit
• Created: 2025-02-20T08:23:12.000Z (8 months ago)
• Default Branch: main
• Last Pushed: 2025-02-20T08:32:18.000Z (8 months ago)
• Last Synced: 2025-07-08T13:12:14.076Z (3 months ago)
• Topics: finetuning-large-language-models, finetuning-llms, gemma-2b, huggingface
• Language: Jupyter Notebook
• Homepage:
• Size: 41 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# Gemma-2B Fine-tuned for Summarization using QLoRA

This project demonstrates how to fine-tune the Gemma-2B language model for conversation summarization using QLoRA (Quantized Low-Rank Adaptation) on the SAMSum dataset. It leverages Hugging Face Transformers, PEFT, and BitsAndBytes libraries for efficient and effective fine-tuning.

## Project Overview

The goal of this project is to create a model capable of generating concise and accurate summaries of conversations. We achieve this by fine-tuning a pre-trained Gemma-2B model using QLoRA, a technique that reduces memory footprint and speeds up training while maintaining performance. The SAMSum dataset, consisting of dialogues and their corresponding summaries, is used for training and evaluation.

## Key Features

- **Fine-tuning Gemma-2B:** Leverages the powerful Gemma-2B language model as the foundation.

- **QLoRA for Efficiency:** Employs Quantized Low-Rank Adaptation for memory-efficient fine-tuning.

- **SAMSum Dataset:** Utilizes a well-established dataset for conversation summarization.

- **Hugging Face Ecosystem:** Integrates seamlessly with Hugging Face Transformers, PEFT, and Datasets libraries.

- **Detailed Code:** Provides clear and well-commented code for easy understanding and reproducibility.

## Getting Started

1. **Clone the repository:**

2. **Install dependencies:**

3. **Authenticate with Hugging Face Hub:**

4. **Run the Colab notebook:**

   Open the notebook in Google Colab and execute the cells. This will download the dataset, fine-tune the model, and save the results.

## Usage

After fine-tuning, you can use the model to generate summaries of conversations:

## Results

The fine-tuned model achieves promising results on the SAMSum dataset, demonstrating its ability to generate informative and concise summaries.

## Contributing

Contributions are welcome! If you find any issues or have suggestions for improvements, please feel free to open an issue or submit a pull request.