https://github.com/furk4neg3/ibm-transformer-recommender-system

A PyTorch-based project exploring pre-training and fine-tuning strategies for building a recommender system on small, domain-specific datasets. Demonstrates transformer model pretraining on a general dataset and fine-tuning techniques to optimize recommendation accuracy with limited data.
https://github.com/furk4neg3/ibm-transformer-recommender-system

Last synced: about 2 months ago
JSON representation

A PyTorch-based project exploring pre-training and fine-tuning strategies for building a recommender system on small, domain-specific datasets. Demonstrates transformer model pretraining on a general dataset and fine-tuning techniques to optimize recommendation accuracy with limited data.

• Host: GitHub
• URL: https://github.com/furk4neg3/ibm-transformer-recommender-system
• Owner: furk4neg3
• Created: 2024-11-11T12:12:59.000Z (about 2 months ago)
• Default Branch: main
• Last Pushed: 2024-11-11T12:15:43.000Z (about 2 months ago)
• Last Synced: 2024-11-11T13:22:10.777Z (about 2 months ago)
• Language: Jupyter Notebook
• Size: 223 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# Pre-Training and Fine-Tuning Transformer Models with PyTorch

This project explores the development of a recommender system for movie reviews using transformer-based neural networks in PyTorch. The project leverages transfer learning by pretraining on a large general-domain text corpus (magazine articles) and fine-tuning on a smaller, domain-specific dataset of movie reviews. By testing different fine-tuning techniques, this project aims to maximize recommendation accuracy while balancing computational efficiency.

## Overview

This project covers:

- **Pretraining** a transformer model for language understanding on a large dataset

- **Fine-tuning** strategies on a smaller, specialized dataset

- Comparison of approaches: full fine-tuning, last-layer tuning, and selective layer unfreezing

- Trade-offs between model generalization and specialization

## Table of Contents

1. [Introduction](#introduction)

2. [Objectives](#objectives)

6. [Requirements](#requirements)

7. [References](#references)

## Introduction

In this project, we build a recommender system for a streaming site based on written movie reviews. Due to the small size of the movie review dataset, we utilize a larger, general-domain text corpus for pretraining to capture broad language patterns. The model is then fine-tuned on the specific dataset of movie reviews, making it better suited for recommendation tasks.

## Objectives

By completing this project, you will:

1. Define and pretrain a transformer-based neural network using PyTorch

2. Fully fine-tune the pretrained model on a different dataset

3. Compare results by fine-tuning only the final layer of the model and selectively unfreezing layers

## Requirements

- Python 3.7+

- PyTorch

- Transformers Library (Hugging Face)

## References

- [IBM AI Engineering Professional Certificate](https://www.coursera.org/professional-certificates/ai-engineer?)

- [Generative AI Engineering with LLMs Specialization](https://www.coursera.org/specializations/generative-ai-engineering-with-llms)

- [Generative AI Engineering and Fine-Tuning Transformers](https://www.coursera.org/learn/generative-ai-engineering-and-fine-tuning-transformers?specialization=generative-ai-engineering-with-llms)