https://github.com/rasbt/pytorch-memory-optim
This code repository contains the code used for my "Optimizing Memory Usage for Training LLMs and Vision Transformers in PyTorch" blog post.
https://github.com/rasbt/pytorch-memory-optim
deep-learning llm memory-optimization pytorch vision vit
Last synced: 2 months ago
JSON representation
This code repository contains the code used for my "Optimizing Memory Usage for Training LLMs and Vision Transformers in PyTorch" blog post.
- Host: GitHub
- URL: https://github.com/rasbt/pytorch-memory-optim
- Owner: rasbt
- License: apache-2.0
- Created: 2023-06-27T18:16:56.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2023-07-14T20:11:55.000Z (over 2 years ago)
- Last Synced: 2025-03-12T17:51:55.596Z (8 months ago)
- Topics: deep-learning, llm, memory-optimization, pytorch, vision, vit
- Language: Python
- Homepage: https://lightning.ai/pages/community/tutorial/pytorch-memory-vit-llm/
- Size: 1.14 MB
- Stars: 87
- Watchers: 3
- Forks: 10
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE.txt
Awesome Lists containing this project
README
# Optimizing PyTorch Memory Usage
This code repository contains the code used for my "Optimizing Memory Usage for Training LLMs and Vision Transformers in PyTorch" blog post.
You can install the dependencies via
```bash
pip install -r requirements.txt
```
The scripts are all standalone scripts and can be run by executing
```bash
python 01_pytorch-vit.py
```
and so forth. The only requirement is to have the [`local_utilities.py`](local_utilities.py) in the same folder as the script as it contains some data loading utilities that are reused across all the scripts.
I tracked the script outputs in the [`logs.md`](logs.md) file.
