https://github.com/yeonghyeon/shake-shake
TensorFlow implementation of Shake-Shake Regularization.
https://github.com/yeonghyeon/shake-shake
cnn convolutional-neural-networks regularization tensorflow
Last synced: about 1 month ago
JSON representation
TensorFlow implementation of Shake-Shake Regularization.
- Host: GitHub
- URL: https://github.com/yeonghyeon/shake-shake
- Owner: YeongHyeon
- License: mit
- Created: 2020-01-10T04:23:21.000Z (over 6 years ago)
- Default Branch: master
- Last Pushed: 2020-02-04T00:18:43.000Z (over 6 years ago)
- Last Synced: 2025-02-26T05:32:55.584Z (over 1 year ago)
- Topics: cnn, convolutional-neural-networks, regularization, tensorflow
- Language: Python
- Homepage:
- Size: 1.15 MB
- Stars: 2
- Watchers: 3
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
Shake-Shake Regularization
=====
TensorFlow implementation of Shake-Shake Regularization.
## Concept
The concept of Shake-Shake Regularization [1].
## Procedure
The whole procedure for using Shake-Shake Regularization is shown as below. All the figures are redesigned by YeongHyeon.
Phase 0. Preparing for Shake-Shake.
Phase 1. Forward propagation in training.
Phase 2. Backward propagation in training.
Phase 3. Forward propagation in test.
## Performance
The performance is measured using below two CNN architectures.
Two Convolutional Neural Networks for experiment.
| |ConvNet8|ConvNet8 with S-S|
|:---|:---:|:---:|
|Accuracy|0.99340|0.99420|
|Precision|0.99339|0.99416|
|Recall|0.99329|0.99413|
|F1-Score|0.99334|0.99414|
### ConvNet8
```
Confusion Matrix
[[ 979 0 0 0 0 0 0 1 0 0]
[ 0 1132 0 1 0 0 1 1 0 0]
[ 0 0 1029 0 0 0 0 3 0 0]
[ 0 0 1 1006 0 3 0 0 0 0]
[ 0 0 1 0 975 0 2 0 0 4]
[ 1 0 0 7 0 882 1 0 0 1]
[ 4 2 0 0 0 1 950 0 1 0]
[ 1 3 3 2 0 0 0 1018 1 0]
[ 3 0 1 1 0 1 0 0 966 2]
[ 0 0 0 1 6 2 0 3 0 997]]
Class-0 | Precision: 0.99089, Recall: 0.99898, F1-Score: 0.99492
Class-1 | Precision: 0.99560, Recall: 0.99736, F1-Score: 0.99648
Class-2 | Precision: 0.99420, Recall: 0.99709, F1-Score: 0.99565
Class-3 | Precision: 0.98821, Recall: 0.99604, F1-Score: 0.99211
Class-4 | Precision: 0.99388, Recall: 0.99287, F1-Score: 0.99338
Class-5 | Precision: 0.99213, Recall: 0.98879, F1-Score: 0.99045
Class-6 | Precision: 0.99581, Recall: 0.99165, F1-Score: 0.99372
Class-7 | Precision: 0.99220, Recall: 0.99027, F1-Score: 0.99124
Class-8 | Precision: 0.99793, Recall: 0.99179, F1-Score: 0.99485
Class-9 | Precision: 0.99303, Recall: 0.98811, F1-Score: 0.99056
Total | Accuracy: 0.99340, Precision: 0.99339, Recall: 0.99329, F1-Score: 0.99334
```
### ConvNet8 with S-S (ConvNet8 + Shake-Shake Regularization)
```
Confusion Matrix
[[ 978 1 0 0 0 0 0 1 0 0]
[ 0 1131 0 0 0 0 2 1 1 0]
[ 1 1 1027 0 0 0 0 2 1 0]
[ 0 0 0 1008 0 2 0 0 0 0]
[ 0 0 0 0 979 0 1 0 0 2]
[ 1 0 0 6 0 884 1 0 0 0]
[ 3 2 0 0 2 1 948 0 2 0]
[ 0 1 4 0 1 0 0 1020 1 1]
[ 2 0 2 0 0 1 0 0 967 2]
[ 0 0 0 0 4 3 0 1 1 1000]]
Class-0 | Precision: 0.99289, Recall: 0.99796, F1-Score: 0.99542
Class-1 | Precision: 0.99560, Recall: 0.99648, F1-Score: 0.99604
Class-2 | Precision: 0.99419, Recall: 0.99516, F1-Score: 0.99467
Class-3 | Precision: 0.99408, Recall: 0.99802, F1-Score: 0.99605
Class-4 | Precision: 0.99290, Recall: 0.99695, F1-Score: 0.99492
Class-5 | Precision: 0.99214, Recall: 0.99103, F1-Score: 0.99159
Class-6 | Precision: 0.99580, Recall: 0.98956, F1-Score: 0.99267
Class-7 | Precision: 0.99512, Recall: 0.99222, F1-Score: 0.99367
Class-8 | Precision: 0.99383, Recall: 0.99281, F1-Score: 0.99332
Class-9 | Precision: 0.99502, Recall: 0.99108, F1-Score: 0.99305
Total | Accuracy: 0.99420, Precision: 0.99416, Recall: 0.99413, F1-Score: 0.99414
```
## Requirements
* Python 3.6.8
* Tensorflow 1.14.0
* Numpy 1.17.1
* Matplotlib 3.1.1
## Reference
[1] Gastaldi, Xavier. Shake-Shake Regularization. arXiv preprint arXiv:1705.07485 (2017).