https://github.com/samadpls/sentimentfinetuning
Efficient fine-tuned large language model (LLM) for the task of sentiment analysis using the IMDB dataset.
https://github.com/samadpls/sentimentfinetuning
finetuning huggingface llm low-rank-adaptation opensource sentiment-analysis transformer
Last synced: about 1 year ago
JSON representation
Efficient fine-tuned large language model (LLM) for the task of sentiment analysis using the IMDB dataset.
- Host: GitHub
- URL: https://github.com/samadpls/sentimentfinetuning
- Owner: samadpls
- License: apache-2.0
- Created: 2024-06-02T09:54:51.000Z (about 2 years ago)
- Default Branch: main
- Last Pushed: 2024-06-03T08:11:26.000Z (about 2 years ago)
- Last Synced: 2025-02-12T07:49:34.230Z (over 1 year ago)
- Topics: finetuning, huggingface, llm, low-rank-adaptation, opensource, sentiment-analysis, transformer
- Language: Python
- Homepage:
- Size: 7.81 KB
- Stars: 7
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Fine-Tuning LLM for Sentiment Analysis
This repository includes the steps I took to fine-tune a large language model (LLM) for sentiment analysis using the IMDB dataset. The process leverages Hugging Face's `transformers` and `datasets` libraries, as well as the `peft` library for parameter-efficient fine-tuning using LoRA (Low-Rank Adaptation).
The fine-tuned sentiment analysis model can be found on Hugging Face at: [samadpls/sentiment-analysis](https://huggingface.co/samadpls/sentiment-analysis).
## Model
We use the `distilbert-base-uncased` model as the base model for fine-tuning. The model is configured for sequence classification with two labels (positive and negative sentiment).
## LoRA Configuration
LoRA (Low-Rank Adaptation) is used to fine-tune the model efficiently by adding trainable low-rank adaptation matrices to certain model layers. The configuration parameters for LoRA used in this project are:
By leveraging LoRA, we achieve efficient fine-tuning with reduced computational resources, making it a key component of this project.
## Evaluation
The model's performance is evaluated using accuracy as the metric. The trained model is tested on a set of example texts to verify its predictions.
## Conclusion
I developed this project in 2023 while learning to run and fine-tune LLMs. This project serves as a starting point for fine-tuning LLMs. Using LoRA, we can efficiently fine-tune large models with reduced computational resources. Feel free to experiment with various models, datasets, and configurations to enhance your understanding and achieve better results.