https://github.com/tcapelle/resnet_checkpoints

Exploring gradient chekcpoints resnets with fastai V2
https://github.com/tcapelle/resnet_checkpoints

Last synced: about 1 month ago
JSON representation

Exploring gradient chekcpoints resnets with fastai V2

• Host: GitHub
• URL: https://github.com/tcapelle/resnet_checkpoints
• Owner: tcapelle
• License: apache-2.0
• Created: 2020-06-03T09:27:35.000Z (over 4 years ago)
• Default Branch: master
• Last Pushed: 2023-04-12T03:18:27.000Z (over 1 year ago)
• Last Synced: 2024-06-11T19:15:46.506Z (5 months ago)
• Language: Jupyter Notebook
• Size: 1.47 MB
• Stars: 7
• Watchers: 2
• Forks: 0
• Open Issues: 4
• Metadata Files:
  • Readme: README.md
  • Contributing: CONTRIBUTING.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Pytorch Model Checkpoints

> Reducing memory on torchvision models.

This is a fastai2 exploraton on hooks and pytorch `checkpoint_sequential`.

- Hooks: They let you access the model at training/inference time. (https://pytorch.org/docs/stable/nn.html?highlight=register_forward#torch.nn.Module.register_forward_pre_hook)

- Checkpoints: Stores intermediate parameters to reduce GPU memory usage (https://pytorch.org/docs/stable/checkpoint.html?highlight=checkpoint_sequential#torch.utils.checkpoint.checkpoint_sequential)

You can check how to cut memory usage of your resnet in half.

## Install

You only need fastai2 (and pytorch)

```bash

pip install fastai2

```

## Experiments

Everything is in the `00_core.ipynb` for the moment