https://github.com/xl0/dl-experiments

Experiments in Deep Learning and Neural Networks
https://github.com/xl0/dl-experiments

Last synced: 17 days ago
JSON representation

Experiments in Deep Learning and Neural Networks

• Host: GitHub
• URL: https://github.com/xl0/dl-experiments
• Owner: xl0
• License: mit
• Created: 2022-09-15T04:29:35.000Z (over 2 years ago)
• Default Branch: master
• Last Pushed: 2022-09-22T13:51:38.000Z (over 2 years ago)
• Last Synced: 2024-10-30T08:26:02.480Z (2 months ago)
• Language: Python
• Size: 25.4 KB
• Stars: 0
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# dl-experiments

Experiments in Deep Learning and Neural Networks

# VGG net training / inference / experiments

- [x] Implement VGG 11/13/16/19 as in the paper.

- [x] Trainig 1

    - [x] Imagenet, imagenette, MNIST datasets

    - [x] Train / val loop

    - [x] Log metrics

- [x] Testing 1 (VGG11)

    - [x] Overfit 1 batch imagenette

    - [x] Overfit imagenette

    - [x] Overgit 1 batch ImageNet

    - [x] Overfit ImageNet

    - [x] Overfit ImageNet with dropout

- [ ] Training 2

    - [x] W&B logging

    - [x] AMP

    - [x] Gradient Accumulation

    - [ ] Multiple GPUs

    - [X] Data Augmentaiton as in paper

    - [ ] Inference time augmentation

- [ ] Testing 2 (VGG11)

    - [X] Train on imagenette

    - [ ] Train on ImageNet

VGG11 can't train at all with the paper's original parameter intialization, but works fine with Glorot initializaiton that they also discovered worked better.

Runs

- Overfit on ImageNet in 6 epochs (glorot, dropout=0, wd, no augmentation):

https://wandb.ai/xl0/vgg/runs/30zipp5v