https://github.com/rzhu3/YellowFin_MXNet

auto-tuning momentum SGD optimizer
https://github.com/rzhu3/YellowFin_MXNet

Last synced: 2 months ago
JSON representation

auto-tuning momentum SGD optimizer

• Host: GitHub
• URL: https://github.com/rzhu3/YellowFin_MXNet
• Owner: rzhu3
• License: apache-2.0
• Created: 2017-07-04T20:54:05.000Z (over 7 years ago)
• Default Branch: master
• Last Pushed: 2017-07-14T01:45:37.000Z (over 7 years ago)
• Last Synced: 2024-05-22T04:22:04.270Z (8 months ago)
• Language: Python
• Size: 22 MB
• Stars: 23
• Watchers: 5
• Forks: 5
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

• Awesome-MXNet - YellowFin

README

        
# YellowFin

YellowFin is an auto-tuning optimizer based on momentum SGD **which requires no manual specification of learning rate and momentum**. It measures the objective landscape on-the-fly and tune momentum as well as learning rate using local quadratic approximation.

The implementation here can be **a drop-in replacement for any optimizer in MXNet** (So far we only implemented and tested upon SGD and other optimizers are in the to-do list).

For more technical details, please refer to the paper [YellowFin and the Art of Momentum Tuning](https://arxiv.org/abs/1706.03471).