https://github.com/nicklhy/cnn_loss_surface

https://github.com/nicklhy/cnn_loss_surface

Last synced: 2 months ago
JSON representation

• Host: GitHub
• URL: https://github.com/nicklhy/cnn_loss_surface
• Owner: nicklhy
• Created: 2017-01-16T08:55:50.000Z (almost 8 years ago)
• Default Branch: master
• Last Pushed: 2017-01-18T01:35:12.000Z (almost 8 years ago)
• Last Synced: 2024-08-01T22:41:39.584Z (5 months ago)
• Language: Jupyter Notebook
• Size: 333 KB
• Stars: 7
• Watchers: 2
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

• Awesome-MXNet - Loss Surfaces

README

        
## AN EMPIRICAL ANALYSIS OF DEEP NETWORK LOSS SURFACES WITH MXNET

----------------

A MXNet implementation of [AN EMPIRICAL ANALYSIS OF DEEP NETWORK LOSS SURFACES](https://arxiv.org/abs/1612.04010).

**Important: This code can not generate absolutely the same visualization results with the original paper yet.**

### Usage

1. To train a ResNet-32 network on cifar10 with adam, run the following command:

```

$ python train_cifar10.py --model-prefix models/cifar10/resnet32_adam --network resnet --num-layers 32 --optimizer adam --model-period 100 --params models/cifar10/resnet32_sgd-0000.params --gpus 0

```

2. To compute the train/val error curve(50 points) between two different models:

```

$ python linear_interpolate_model.py --net-json models/cifar10/resnet_sgd-symbol.json --params1 models/cifar10/resnet_sgd-0300.params --params2 models/cifar10/resnet_adam-0300.params --alpha-num 50 --batch-size 16 --alpha-num 50 --gpus 0

```

3. To compute the train/val error mesh(alpha num = 50, beta num = 50) among three different models using the barycentric interpolate method:

```

$ python barycentric_interpolate_model.py --net-json models/cifar10/resnet_sgd-symbol.json --params1 models/cifar10/resnet_rmsprop-0300.params --params2 models/cifar10/resnet_adam-0300.params --params3 models/cifar10/resnet_sgd-0300.params --alpha-num 50 --beta-num 50 --gpus 0

```

4. To compute the train/val error mesh(alpha num = 50, beta num = 50) among four different models using the bilinear interpolate method:

```

$ python bilinear_interpolate_model.py --net-json models/cifar10/resnet_sgd-symbol.json --params1 models/cifar10/resnet_adam-0000.params --params2 models/cifar10/resnet_sgd-0300.params --params3 models/cifar10/resnet_adam-0300.params --params4 models/cifar10/resnet_rmsprop-0300.params --alpha-num 50 --beta-num 50 --gpus 0

```

5. To plot the train/val error curve between two models, please use visualize\_interpolate\_2d.ipynb.

6. To plot the train/val error surface among three or four models, please use visualize\_interpolate\_3d.ipynb.