https://github.com/wkcn/LookaheadOptimizer-mx

Lookahead Optimizer: k steps forward, 1step back for MXNet
https://github.com/wkcn/LookaheadOptimizer-mx

lookahead mxnet optimizer

Last synced: about 2 months ago
JSON representation

Lookahead Optimizer: k steps forward, 1step back for MXNet

• Host: GitHub
• URL: https://github.com/wkcn/LookaheadOptimizer-mx
• Owner: wkcn
• Created: 2019-07-26T07:44:10.000Z (about 5 years ago)
• Default Branch: master
• Last Pushed: 2020-06-28T15:06:11.000Z (about 4 years ago)
• Last Synced: 2024-06-12T08:13:00.219Z (3 months ago)
• Topics: lookahead, mxnet, optimizer
• Language: Python
• Homepage:
• Size: 14.6 KB
• Stars: 21
• Watchers: 4
• Forks: 3
• Open Issues: 2
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

• Awesome-MXNet - **LookaheadOptimizer**

README

        
# Lookahead Optimizer: k steps forward, 1step back

It is a MXNet implementation of Lookahead Optimizer.

The link of the paper: https://arxiv.org/abs/1907.08610

## Usage

Import `optimizer.py`, then add the prefix `Lookahead` before the name of [arbitrary optimizer](http://mxnet.incubator.apache.org/api/python/optimization/optimization.html?highlight=opt#module-mxnet.optimizer).

```python

import optimizer

optimizer.LookaheadSGD(k=5, alpha=0.5, learning_rate=1e-3)

```

## Example

```bash

python mnist.py --optimizer sgd --seed 42

python mnist.py --optimizer lookaheadsgd --seed 42

```