{"id":20503203,"url":"https://github.com/curt-park/handwritten_digit_recognition","last_synced_at":"2025-04-13T20:16:06.806Z","repository":{"id":134252441,"uuid":"92254104","full_name":"Curt-Park/handwritten_digit_recognition","owner":"Curt-Park","description":"Handwritten digit recognition with MNIST \u0026 Keras","archived":false,"fork":false,"pushed_at":"2020-10-02T10:48:31.000Z","size":42436,"stargazers_count":78,"open_issues_count":1,"forks_count":31,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-04-13T20:16:03.566Z","etag":null,"topics":["deep-convolutional-networks","ensemble-methods","handwritten-digit-recognition","image-recognition","mnist","mobilenet","python","residual-networks","resnet","super-learner","vgg16","wide-residual-networks"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Curt-Park.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-05-24T05:41:07.000Z","updated_at":"2025-04-03T09:58:29.000Z","dependencies_parsed_at":null,"dependency_job_id":"b1dc9f0f-013f-45f4-a3f7-54a9908ee291","html_url":"https://github.com/Curt-Park/handwritten_digit_recognition","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Curt-Park%2Fhandwritten_digit_recognition","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Curt-Park%2Fhandwritten_digit_recognition/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Curt-Park%2Fhandwritten_digit_recognition/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Curt-Park%2Fhandwritten_digit_recognition/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Curt-Park","download_url":"https://codeload.github.com/Curt-Park/handwritten_digit_recognition/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248774965,"owners_count":21159534,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-convolutional-networks","ensemble-methods","handwritten-digit-recognition","image-recognition","mnist","mobilenet","python","residual-networks","resnet","super-learner","vgg16","wide-residual-networks"],"created_at":"2024-11-15T19:29:41.302Z","updated_at":"2025-04-13T20:16:06.800Z","avatar_url":"https://github.com/Curt-Park.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Handwritten digit recognition with MNIST and Keras\n\nThis repository is for practice of implementing well-known network architectures and ensembling methods, including the followings:\n\n#### Architectures\n- [Mobilenet](https://arxiv.org/abs/1704.04861) - [[structure]](images/MobileNet.png) [[training progress]](images/MobileNet_9968_plot.png)\n- [VGG16](https://arxiv.org/abs/1409.1556) - [[structure]](images/VGG16.png) [[training progress]](images/VGG16_9968_plot.png)\n- [Resnet164](https://arxiv.org/abs/1603.05027) - [[structure]](images/ResNet164.png) [[training progress]](images/ResNet164_9970_plot.png)\n- [WideResnet28-10](https://arxiv.org/abs/1603.05027) - [[structure]](images/WideResNet28_10.png) [[training progress]](images/WideResNet28_10_9976_plot.png)\n\n#### Ensembling methods\n- Unweighted average\n- Majority voting\n- [Super Learner](https://arxiv.org/abs/1704.01664) - [[structure]](images/SuperLearner.png)\n\n#### Others\n- Channel-wise normalization of input images: substracted by mean and divided by std\n- Data augmentation: rotation, width shift, height shift, shearing, zooming\n\n## Environment\n- MacOS High Sierra 10.13.1 for implementation / Ubuntu 14.04 for training\n- Python 3.6.3\n- Keras 2.1.2 (Tensorflow backend)\n\n## Evaluation\nThe best single model and the best ensemble method achieve **99.76%** and **99.77%** on the test set respectively.\n\n|  **Model**          |  **On the validation set**  | **On the test set** |\n|:-------------------:|:---------------------------:|:-------------------:|\n|  Mobilenet          |          99.63%             |       99.68%        |\n|  VGG16              |          99.61%             |       99.68%        |\n|  Resnet164          |        **99.72%**           |       99.70%        |\n|  WideResnet28-10    |        **99.72%**           |     **99.76%**      |\n\n|  **Ensemble (all)** |  **On the validation set**  | **On the test set** |\n|:-------------------:|:---------------------------:|:-------------------:|\n|  Unweighted average |          99.70%             |       99.75%        |\n|  Majority voting    |          99.71%             |       99.76%        |\n|  Super Learner      |        **99.73%**           |     **99.77%**      |\n\nIn order to run the evaluation, it requires pre-trained weights for each model, which can be downloaded [here](https://drive.google.com/drive/folders/1kBvOL019Gcx00vwUM1BhtaoWQDlfPPim?usp=sharing).\n\n**\\*All pre-trained weights should be stored in './models'.**\n\n#### How to run\n```bash\npython evaluate.py [options]\n```\n#### Options\n```bash\n$ python evaluate.py --help\nusage: evaluate.py [-h] [--dataset DATASET]\n\noptional arguments:\n  -h, --help         show this help message and exit\n  --dataset DATASET  training set: 0, validation set: 1, test set: 2\n```\n\n## Training\nThe training can be executed by the following command. Every model has the same options.\n\n#### How to run\n```bash\n$ python vgg16.py [options]\n```\n\n#### Options\n```bash\n$ python vgg16.py --help\nusage: vgg16.py [-h] [--epochs EPOCHS] [--batch_size BATCH_SIZE]\n                [--path_for_weights PATH_FOR_WEIGHTS]\n                [--path_for_image PATH_FOR_IMAGE]\n                [--path_for_plot PATH_FOR_PLOT]\n                [--data_augmentation DATA_AUGMENTATION]\n                [--save_model_and_weights SAVE_MODEL_AND_WEIGHTS]\n                [--load_weights LOAD_WEIGHTS]\n                [--plot_training_progress PLOT_TRAINING_PROGRESS]\n                [--save_model_to_image SAVE_MODEL_TO_IMAGE]\n\noptional arguments:\n  -h, --help            show this help message and exit\n  --epochs EPOCHS       How many epochs you need to run (default: 10)\n  --batch_size BATCH_SIZE\n                        The number of images in a batch (default: 64)\n  --path_for_weights PATH_FOR_WEIGHTS\n                        The path from where the weights will be saved or\n                        loaded (default: ./models/VGG16.h5)\n  --path_for_image PATH_FOR_IMAGE\n                        The path from where the model image will be saved\n                        (default: ./images/VGG16.png)\n  --path_for_plot PATH_FOR_PLOT\n                        The path from where the training progress will be\n                        plotted (default: ./images/VGG16_plot.png)\n  --data_augmentation DATA_AUGMENTATION\n                        0: No, 1: Yes (default: 1)\n  --save_model_and_weights SAVE_MODEL_AND_WEIGHTS\n                        0: No, 1: Yes (default: 1)\n  --load_weights LOAD_WEIGHTS\n                        0: No, 1: Yes (default: 0)\n  --plot_training_progress PLOT_TRAINING_PROGRESS\n                        0: No, 1: Yes (default: 1)\n  --save_model_to_image SAVE_MODEL_TO_IMAGE\n                        0: No, 1: Yes (default: 1)\n```\n\n## File descriptions\n```bash\n├── images/ # model architectures and training progresses\n├── predictions/ # prediction results to be used for fast inference\n├── models/ # model weights (not included in this repo)\n├── README.md\n├── base_model.py # base model interface\n├── evaluate.py # for evaluation\n├── utils.py # helper functions\n├── mobilenet.py\n├── vgg16.py\n├── resnet164.py\n├── wide_resnet_28_10.py\n└── super_learner.py\n```\n\n## References\n#### Papers\n- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://arxiv.org/abs/1409.1556)\n- [Deep Residual Learning for Image Recognition](https://arxiv.org/abs/1512.03385)\n- [Identity Mappings in Deep Residual Networks](https://arxiv.org/abs/1603.05027)\n- [Wide Residual Networks](https://arxiv.org/abs/1605.07146)\n- [The Relative Performance of Ensemble Methods with Deep Convolutional Neural Networks for Image Classification](https://arxiv.org/abs/1704.01664)\n\n#### Implementation\n- [ResNet Author's Implementation](https://github.com/KaimingHe/resnet-1k-layers/blob/master/resnet-pre-act.lua)\n- [WideResNet Author's implementation](https://github.com/szagoruyko/wide-residual-networks)\n- [MobileNet in Keras](https://github.com/keras-team/keras/blob/master/keras/applications/mobilenet.py)\n- [How far can we go with MNIST??](https://github.com/hwalsuklee/how-far-can-we-go-with-MNIST)\n\n#### Others\n- [Global weight decay in keras? - Stackoverflow](https://stackoverflow.com/questions/41260042/global-weight-decay-in-keras)\n- [Best up to date result on MNIST dataset - Kaggle](https://www.kaggle.com/c/digit-recognizer/discussion/23999#138390)\n- [Batch Normalization before or after ReLU? - Reddit](https://www.reddit.com/r/MachineLearning/comments/67gonq/d_batch_normalization_before_or_after_relu/)\n- [Depth-wise Conv2D - Tensorflow Document](https://www.tensorflow.org/api_docs/python/tf/nn/depthwise_conv2d)\n- [A Complete Tutorial on Ridge and Lasso Regression in Python](https://www.analyticsvidhya.com/blog/2016/01/complete-tutorial-ridge-lasso-regression-python/)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcurt-park%2Fhandwritten_digit_recognition","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcurt-park%2Fhandwritten_digit_recognition","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcurt-park%2Fhandwritten_digit_recognition/lists"}