{"id":13436253,"url":"https://github.com/KaimingHe/deep-residual-networks","last_synced_at":"2025-03-18T20:31:05.343Z","repository":{"id":38628738,"uuid":"50839932","full_name":"KaimingHe/deep-residual-networks","owner":"KaimingHe","description":"Deep Residual Learning for Image Recognition ","archived":false,"fork":false,"pushed_at":"2017-10-28T15:26:22.000Z","size":38,"stargazers_count":6500,"open_issues_count":57,"forks_count":2220,"subscribers_count":301,"default_branch":"master","last_synced_at":"2025-02-21T00:01:55.685Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/KaimingHe.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-02-01T13:07:00.000Z","updated_at":"2025-02-19T13:25:39.000Z","dependencies_parsed_at":"2022-07-09T18:01:03.019Z","dependency_job_id":null,"html_url":"https://github.com/KaimingHe/deep-residual-networks","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KaimingHe%2Fdeep-residual-networks","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KaimingHe%2Fdeep-residual-networks/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KaimingHe%2Fdeep-residual-networks/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KaimingHe%2Fdeep-residual-networks/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/KaimingHe","download_url":"https://codeload.github.com/KaimingHe/deep-residual-networks/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244301363,"owners_count":20430929,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T03:00:45.917Z","updated_at":"2025-03-18T20:31:05.085Z","avatar_url":"https://github.com/KaimingHe.png","language":null,"funding_links":[],"categories":["Uncategorized","Others","\u003ca name=\"Vision\"\u003e\u003c/a\u003e2. Vision","📚 Project Purpose"],"sub_categories":["Uncategorized","Machine Learning (Intermediate-Level"],"readme":"# Deep Residual Networks\n\nBy [Kaiming He](http://kaiminghe.com), [Xiangyu Zhang](https://scholar.google.com/citations?user=yuB-cfoAAAAJ\u0026hl=en), [Shaoqing Ren](http://home.ustc.edu.cn/~sqren/), [Jian Sun](http://research.microsoft.com/en-us/people/jiansun/).\n\nMicrosoft Research Asia (MSRA).\n\n### Table of Contents\n0. [Introduction](#introduction)\n0. [Citation](#citation)\n0. [Disclaimer and known issues](#disclaimer-and-known-issues)\n0. [Models](#models)\n0. [Results](#results)\n0. [Third-party re-implementations](#third-party-re-implementations)\n\n### Introduction\n\nThis repository contains the original models (ResNet-50, ResNet-101, and ResNet-152) described in the paper \"Deep Residual Learning for Image Recognition\" (http://arxiv.org/abs/1512.03385). These models are those used in [ILSVRC] (http://image-net.org/challenges/LSVRC/2015/) and [COCO](http://mscoco.org/dataset/#detections-challenge2015) 2015 competitions, which won the 1st places in: ImageNet classification, ImageNet detection, ImageNet localization, COCO detection, and COCO segmentation.\n\n**Note**\n\n0. Re-implementations with **training code** and models from Facebook AI Research (FAIR): [blog](http://torch.ch/blog/2016/02/04/resnets.html), [code](https://github.com/facebook/fb.resnet.torch)\n0. Code of improved **1K-layer ResNets** with 4.62% test error on CIFAR-10 in our new arXiv paper: https://github.com/KaimingHe/resnet-1k-layers\n\n### Citation\n\nIf you use these models in your research, please cite:\n\n\t@article{He2015,\n\t\tauthor = {Kaiming He and Xiangyu Zhang and Shaoqing Ren and Jian Sun},\n\t\ttitle = {Deep Residual Learning for Image Recognition},\n\t\tjournal = {arXiv preprint arXiv:1512.03385},\n\t\tyear = {2015}\n\t}\n\n### Disclaimer and known issues\n\n0. These models are converted from our own implementation to a recent version of Caffe (2016/2/3, b590f1d). The numerical results using this code are as in the tables below.\n0. These models are for the usage of testing or fine-tuning.\n0. These models were **not** trained using this version of Caffe.\n0. If you want to train these models using this version of Caffe without modifications, please notice that:\n\t- GPU memory might be insufficient for extremely deep models.\n\t- Changes of mini-batch size should impact accuracy (we use a mini-batch of 256 images on 8 GPUs, that is, 32 images per GPU).\n\t- Implementation of data augmentation might be different (see our paper about the data augmentation we used).\n\t- We randomly shuffle data at the beginning of every epoch.\n\t- There might be some other untested issues.\n0. In our BN layers, the provided mean and variance are strictly computed using average (**not** moving average) on a sufficiently large training batch after the training procedure. The numerical results are very stable (variation of val error \u003c 0.1%). Using moving average might lead to different results. \n0. In the BN paper, the BN layer learns gamma/beta. To implement BN in this version of Caffe, we use its provided \"batch_norm_layer\" (which has no gamma/beta learned) followed by \"scale_layer\" (which learns gamma/beta).\n0. We use Caffe's implementation of SGD with momentum: v := momentum\\*v + lr\\*g. **If you want to port these models to other libraries (e.g., Torch, CNTK), please pay careful attention to the possibly different implementation of SGD with momentum**: v := momentum\\*v + (1-momentum)\\*lr\\*g, which changes the effective learning rates.\n\n\t\n### Models\n\n0. Visualizations of network structures (tools from [ethereon](http://ethereon.github.io/netscope/quickstart.html)):\n\t- [ResNet-50] (http://ethereon.github.io/netscope/#/gist/db945b393d40bfa26006)\n\t- [ResNet-101] (http://ethereon.github.io/netscope/#/gist/b21e2aae116dc1ac7b50)\n\t- [ResNet-152] (http://ethereon.github.io/netscope/#/gist/d38f3e6091952b45198b)\n\n0. Model files:\n\t- ~~MSR download: [link] (http://research.microsoft.com/en-us/um/people/kahe/resnet/models.zip)~~\n\t- OneDrive download: [link](https://onedrive.live.com/?authkey=%21AAFW2-FVoxeVRck\u0026id=4006CBB8476FF777%2117887\u0026cid=4006CBB8476FF777)\n\n### Results\n0. Curves on ImageNet (solid lines: 1-crop val error; dashed lines: training error):\n\t![Training curves](https://cloud.githubusercontent.com/assets/11435359/13046277/e904c04c-d412-11e5-9260-efc5b8301e2f.jpg)\n\n0. 1-crop validation error on ImageNet (center 224x224 crop from resized image with shorter side=256):\n\n\tmodel|top-1|top-5\n\t:---:|:---:|:---:\n\t[VGG-16](http://www.vlfeat.org/matconvnet/pretrained/)|[28.5%](http://www.vlfeat.org/matconvnet/pretrained/)|[9.9%](http://www.vlfeat.org/matconvnet/pretrained/)\n\tResNet-50|24.7%|7.8%\n\tResNet-101|23.6%|7.1%\n\tResNet-152|23.0%|6.7%\n\t\n0. 10-crop validation error on ImageNet (averaging softmax scores of 10 224x224 crops from resized image with shorter side=256), the same as those in the paper:\n\n\tmodel|top-1|top-5\n\t:---:|:---:|:---:\n\tResNet-50|22.9%|6.7%\n\tResNet-101|21.8%|6.1%\n\tResNet-152|21.4%|5.7%\n\t\n### Third-party re-implementations\n\nDeep residual networks are very easy to implement and train. We recommend to see also the following third-party re-implementations and extensions:\n\n0. By Facebook AI Research (FAIR), with **training code in Torch and pre-trained ResNet-18/34/50/101 models for ImageNet**: [blog](http://torch.ch/blog/2016/02/04/resnets.html), [code](https://github.com/facebook/fb.resnet.torch)\n0. Torch, CIFAR-10, with ResNet-20 to ResNet-110, training code, and curves: [code](https://github.com/gcr/torch-residual-networks)\n0. Lasagne, CIFAR-10, with ResNet-32 and ResNet-56 and training code: [code](https://github.com/Lasagne/Recipes/tree/master/papers/deep_residual_learning)\n0. Neon, CIFAR-10, with pre-trained ResNet-32 to ResNet-110 models, training code, and curves: [code](https://github.com/apark263/cfmz)\n0. Torch, MNIST, 100 layers: [blog](https://deepmlblog.wordpress.com/2016/01/05/residual-networks-in-torch-mnist/), [code](https://github.com/arunpatala/residual.mnist)\n0. A winning entry in Kaggle's right whale recognition challenge: [blog](http://blog.kaggle.com/2016/02/04/noaa-right-whale-recognition-winners-interview-2nd-place-felix-lau/), [code](https://github.com/felixlaumon/kaggle-right-whale)\n0. Neon, Place2 (mini), 40 layers: [blog](http://www.nervanasys.com/using-neon-for-scene-recognition-mini-places2/), [code](https://github.com/hunterlang/mpmz/)\n0. MatConvNet, CIFAR-10, with ResNet-20 to ResNet-110, training code, and curves: [code](https://github.com/suhangpro/matresnet)\n0. TensorFlow, CIFAR-10, with ResNet-32,110,182 training code and curves:\n[code](https://github.com/ppwwyyxx/tensorpack/tree/master/examples/ResNet)\n0. MatConvNet, reproducing CIFAR-10 and ImageNet experiments (supporting official MatConvNet), training code and curves: [blog](https://zhanghang1989.github.io/ResNet/), [code](https://github.com/zhanghang1989/ResNet-Matconvnet)\n0. Keras, ResNet-50: [code](https://github.com/raghakot/keras-resnet)\n\nConverters:\n\n0. MatConvNet: [url](http://www.vlfeat.org/matconvnet/pretrained/#imagenet-ilsvrc-classification)\n0. TensorFlow: [url](https://github.com/ry/tensorflow-resnet)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FKaimingHe%2Fdeep-residual-networks","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FKaimingHe%2Fdeep-residual-networks","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FKaimingHe%2Fdeep-residual-networks/lists"}