Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/mgrankin/over9000
Over9000 optimizer
https://github.com/mgrankin/over9000
Last synced: 2 months ago
JSON representation
Over9000 optimizer
- Host: GitHub
- URL: https://github.com/mgrankin/over9000
- Owner: mgrankin
- License: apache-2.0
- Created: 2019-08-25T09:48:41.000Z (over 5 years ago)
- Default Branch: master
- Last Pushed: 2022-11-22T11:25:31.000Z (about 2 years ago)
- Last Synced: 2024-08-02T15:20:42.956Z (5 months ago)
- Language: Jupyter Notebook
- Size: 617 KB
- Stars: 423
- Watchers: 20
- Forks: 57
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-fastai - mgrankin/over9000: Over9000 optimizer
- awesome-fastai - mgrankin/over9000: Over9000 optimizer
- awesome-fastai - Over9000 - different SOTA optimizers and fastai training script (Github Repositories<a name="github"></a> / Other libraries)
README
# Optimizers and tests
Every result is avg of 20 runs.
Dataset | LR Schedule| Imagenette size 128, 5 epoch | Imagewoof size 128, 5 epoch
--- | -- | --- | ---
Adam - baseline |OneCycle| 0.8493 | 0.6125
RangerLars (RAdam + LARS + Lookahead) |Flat and anneal| 0.8732 | 0.6523
Ralamb (RAdam + LARS) |Flat and anneal| 0.8675 | 0.6367
Ranger (RAdam + Lookahead) |Flat and anneal| 0.8594 | 0.5946
Novograd |Flat and anneal| 0.8711 | 0.6126
Radam |Flat and anneal| 0.8444 | 0.537
Lookahead |OneCycle| 0.8578 | 0.6106
Lamb |OneCycle| 0.8400 | 0.5597
DiffGrad |OneCycle| 0.8527 | 0.5912
AdaMod |OneCycle| 0.8473 | 0.6132