https://github.com/nav3005/dl-1-section-2
Project to implement fine-tuning Language model with Python libraries such as Pandas, transformers with the use of LED Model
https://github.com/nav3005/dl-1-section-2
Last synced: 12 months ago
JSON representation
Project to implement fine-tuning Language model with Python libraries such as Pandas, transformers with the use of LED Model
- Host: GitHub
- URL: https://github.com/nav3005/dl-1-section-2
- Owner: Nav3005
- Created: 2024-09-08T14:24:10.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2024-09-08T14:34:08.000Z (almost 2 years ago)
- Last Synced: 2025-05-17T19:38:25.969Z (about 1 year ago)
- Language: Python
- Homepage:
- Size: 4.88 KB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Blog Generation Project
## Section 2: Fine-Tuning the Language Model
### Overview
In this section, the language model is fine-tuned to generate subheadings for paragraphs using the dataset created in Section 1. The process includes data preprocessing, setting up the Longformer Encoder Decoder (LED) model, and training it on Google Colab.
### Steps
1. **Data Preprocessing**: Clean the dataset by handling missing values, removing duplicates, and eliminating outliers.
2. **Model Setup**: Set up the Longformer Encoder Decoder (LED) model for fine-tuning, including tokenization and batch size configuration.
3. **Training**: Train the model using Google Colab, ensuring proper use of GPU resources.
4. **Model Checkpoints**: Save checkpoints during training for later evaluation and testing.
### Tools Used
- **Google Colab**: For training the model using GPU resources.
- **Pandas**: For data preprocessing and handling.
- **Transformers Library**: To fine-tune the LED model.