{"id":13499021,"url":"https://github.com/tornadomeet/ResNet","last_synced_at":"2025-03-29T03:32:16.381Z","repository":{"id":96016070,"uuid":"60523884","full_name":"tornadomeet/ResNet","owner":"tornadomeet","description":"Reproduce ResNet-v2(Identity Mappings in Deep Residual Networks) with MXNet","archived":false,"fork":false,"pushed_at":"2018-03-04T22:40:59.000Z","size":3844,"stargazers_count":556,"open_issues_count":20,"forks_count":199,"subscribers_count":40,"default_branch":"master","last_synced_at":"2024-10-31T17:39:05.265Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tornadomeet.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2016-06-06T11:49:59.000Z","updated_at":"2024-09-28T13:23:27.000Z","dependencies_parsed_at":"2023-07-09T12:31:51.809Z","dependency_job_id":null,"html_url":"https://github.com/tornadomeet/ResNet","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tornadomeet%2FResNet","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tornadomeet%2FResNet/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tornadomeet%2FResNet/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tornadomeet%2FResNet/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tornadomeet","download_url":"https://codeload.github.com/tornadomeet/ResNet/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246135753,"owners_count":20729056,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T22:00:26.264Z","updated_at":"2025-03-29T03:32:15.475Z","avatar_url":"https://github.com/tornadomeet.png","language":"Python","funding_links":[],"categories":["Papers\u0026Codes","\u003ca name=\"Vision\"\u003e\u003c/a\u003e2. Vision"],"sub_categories":["PreActResNet","2.1 Image Classification"],"readme":"Reproduce ResNet-v2 using MXNet\r\n=====================================\r\n## Requirements\r\n- Install [MXNet](http://mxnet.io/get_started/install.html) on a machine with CUDA GPU, and it's better also installed with [cuDNN v5](https://developer.nvidia.com/cudnn)\r\n- Please fix the randomness if you want to train your own model and using this [pull request](https://github.com/dmlc/mxnet/pull/3001/files)\r\n\r\n## Trained models\r\n\r\nThe trained ResNet models achieve better error rates than the [original ResNet-v1 models](https://github.com/KaimingHe/deep-residual-networks).\r\n\r\n### ImageNet 1K\r\n\r\nImagenet 1000 class dataset with 1.2 million images.\r\n\r\nsingle center crop (224x224) validation error rate(%)\r\n\r\n| Network       | Top-1 error | Top-5 error | Traind Model |\r\n| :------------ | :---------: | :---------: | :-------------: |\r\n| ResNet-18     | 30.48       | 10.92       |[data.dmlc.ml](http://data.dmlc.ml/mxnet/models/imagenet/resnet/18-layers/) |\r\n| ResNet-34     | 27.20      | 8.86        | [data.dmlc.ml](http://data.dmlc.ml/mxnet/models/imagenet/resnet/34-layers/) |\r\n| ResNet-50     | 24.39   | 7.24   | [data.dmlc.ml](http://data.dmlc.ml/mxnet/models/imagenet/resnet/50-layers/) |\r\n| ResNet-101    | 22.68      | 6.58        | [data.dmlc.ml](http://data.dmlc.ml/mxnet/models/imagenet/resnet/101-layers/) |\r\n| ResNet-152    | 22.25       | 6.42        | [data.dmlc.ml](http://data.dmlc.ml/mxnet/models/imagenet/resnet/152-layers/) |\r\n| ResNet-200    | 22.14       | 6.16        | [data.dmlc.ml](http://data.dmlc.ml/mxnet/models/imagenet/resnet/200-layers/) |\r\n\r\n### ImageNet 11K:\r\n\r\n Full imagenet dataset: `fall11_whole.tar` from\r\n http://www.image-net.org/download-images.\r\n\r\n We removed classes with less than 500 images. The filtered dataset contains\r\n 11221 classes and 12.4 millions images. We randomly pick 50 images from each\r\n class as the validation set. The split is available at http://data.dmlc.ml/mxnet/models/imagenet-11k/\r\n\r\n| Network       | Top-1 error | Top-5 error | Traind Model |\r\n| :------------ | :---------: | :---------: | :-------------: |\r\n| ResNet-200 | 58.4 | 28.8 |  |\r\n\r\n\r\n### cifar10: single crop validation error rate(%):\r\n\r\n| Network    | top-1 |\r\n| :------:   | :---: |\r\n| ResNet-164  | 4.68 |\r\n\r\n### Training Curve\r\nThe following curve is ResNet-v2 trainined on imagenet-1k, all the training detail you can found [here](./log), which include gpu information, lr schedular, batch-size etc, and you can also see the training speed with the corresponding logs.   \r\n\r\n\u003cdiv align=\"left\"\u003e\r\n  \u003cimg src=\"./log/training-curve.png\"/\u003e\r\n\u003c/div\u003e\r\n\r\nyou can get the curve by run:  \r\n```cd log \u0026\u0026 python  plot_curve.py --logs=resnet-18.log,resnet-34.log,resnet-50.log,resnet-101.log,resnet-152.log,resnet-200.log```\r\n\r\n## How to Train\r\n\r\n### imagenet\r\nfirst you should prepare the `train.lst` and `val.lst`, you can generate this list files by yourself(please ref.[make-the-image-list]( http://mxnet.readthedocs.io/en/latest/packages/python/io.html#make-the-image-list), and do not forget to shuffle the list files!), or just download the provided version from [here](http://data.dmlc.ml/mxnet/models/imagenet/resnet/).\r\n\r\nthen you can create the ```*.rec``` file, i recommend use this cmd parameters:\r\n```shell\r\n$im2rec_path train.lst train/ data/imagenet/train_480_q90.rec resize=480 quality=90\r\n```\r\nset ```resize=480``` and ```quality=90```(```quality=100``` will be best i think:)) here may use more disk memory(about ~103G), but this is very useful with scale augmentation during training[1][2], and can help reproducing a good result.\r\n\r\nbecause you are training imagenet , so we should set ```data-type = imagenet```, then the training cmd is like this(here i use 6 gpus for training):\r\n```shell\r\npython -u train_resnet.py --data-dir data/imagenet \\\r\n--data-type imagenet --depth 50 --batch-size 256  --gpus=0,1,2,3,4,5\r\n```\r\nchange depth to different number to support different model, currently support ResNet-18, ResNet-34, ResNet-50, ResNet-101, ResNet-152, ResNet-200.\r\n\r\n### cifar10\r\nsame as above, first you should use ```im2rec``` to create the .rec file, then training with cmd like this:\r\n```shell\r\npython -u train_resnet.py --data-dir data/cifar10 --data-type cifar10 \\\r\n  --depth 164 --batch-size 128 --num-examples 50000 --gpus=0,1\r\n```\r\nchange ```depth``` when training different model, only support```(depth-2)%9==0```, such as RestNet-110, ResNet-164, ResNet-1001...\r\n\r\n### retrain\r\nWhen training large dataset(like imagenet), it's better for us to change learning rate manually, or the training is killed by some other reasons, so retrain is very important.\r\nthe code here support retrain, suppose you want to retrain your resnet-50 model from epoch 70 and want to change lr=0.0005, wd=0.001, batch-size=256 using 8gpu, then you can try this cmd:\r\n```shell\r\npython -u train_resnet.py --data-dir data/imagenet --data-type imagenet --depth 50 --batch-size 256 \\\r\n--gpus=0,1,2,3,4,5,6,7 --model-load-epoch=70 --lr 0.0005 --wd 0.001 --retrain\r\n```\r\n\r\n### Notes\r\n* it's better training the model in imagenet with epoch \u003e 110, because this will lead better result.\r\n* when epoch is about 95, cancel the scale/color/aspect augmentation during training, this can be done by only comment out 6 lines of the code, like this:\r\n```python\r\ntrain = mx.io.ImageRecordIter(\r\n        # path_imgrec         = os.path.join(args.data_dir, \"train_480_q90.rec\"),\r\n        path_imgrec         = os.path.join(args.data_dir, \"train_256_q90.rec\"),\r\n        label_width         = 1,\r\n        data_name           = 'data',\r\n        label_name          = 'softmax_label',\r\n        data_shape          = (3, 32, 32) if args.data_type==\"cifar10\" else (3, 224, 224),\r\n        batch_size          = args.batch_size,\r\n        pad                 = 4 if args.data_type == \"cifar10\" else 0,\r\n        fill_value          = 127,  # only used when pad is valid\r\n        rand_crop           = True,\r\n        # max_random_scale    = 1.0 if args.data_type == \"cifar10\" else 1.0,  # 480\r\n        # min_random_scale    = 1.0 if args.data_type == \"cifar10\" else 0.533,  # 256.0/480.0\r\n        # max_aspect_ratio    = 0 if args.data_type == \"cifar10\" else 0.25,\r\n        # random_h            = 0 if args.data_type == \"cifar10\" else 36,  # 0.4*90\r\n        # random_s            = 0 if args.data_type == \"cifar10\" else 50,  # 0.4*127\r\n        # random_l            = 0 if args.data_type == \"cifar10\" else 50,  # 0.4*127\r\n        rand_mirror         = True,\r\n        shuffle             = True,\r\n        num_parts           = kv.num_workers,\r\n        part_index          = kv.rank)\r\n```\r\nbut you should prepare one ```train_256_q90.rec```  using ```im2rec```  like:\r\n```shell\r\n$im2rec_path train.lst train/ data/imagenet/train_256_q90.rec resize=256 quality=90\r\n```\r\ncancel this scale/color/aspect augmentation can be done easily by using ```--aug-level=1``` in your cmd.\r\n* it's better for running longer than 30 epoch before first decrease the ```lr```(such as 60), so you may decide  the epoch number by observe the val-acc curve, and set lr with ```retrain```.\r\n\r\n## Training ResNet-200 by only one gpu with 'dark knowledge' of mxnet\r\nyou can training ResNet-200 or even ResNet-1000 on imaget with only one gpu! for example, we can train ResNet-200 with batch-size=128 on one gpu(=12G), or if your gpu memory is less than 12G, you should decrease the batch-size by a little. here is the way of how to using 'dark knowledge' of mxnet:\r\n* download the file  https://github.com/dmlc/mxnet-memonger/blob/master/memonger.py, and put it on the root path of this resnet repo.\r\n* add `--memonger` to your training cmd, i.e. `python train_resnet.py --memonger ...`.  \r\n\r\nwhen turn on memonger, the trainning speed will be about 25% slower, but we can training more depth network, have fun!\r\n\r\n## ResNet-v2 vs ResNet-v1\r\n**Does ResNet-v2 always achieve better result than ResNet-v1 on imagnet?**\r\nThe answer is **NO**, ResNet-v2 has no advantage or even has disadvantage than ResNet-v1 when  `depth\u003c152`, we can get the following result from paper[2].(why?)\r\n\r\nImageNet: single center crop validation error rate(%)\r\n\r\n| Network    |crop-size | top-1 |  top-5 |\r\n| :------:   | :---: | :---: |:---: |\r\n|ResNet-101-v1  | 224x224 |23.6|7.1|\r\n|ResNet-101-v2  | 224x224 |24.6|7.5|\r\n|ResNet-152-v1  | 320x320 |21.3|5.5|\r\n|ResNet-152-v2  | 320x320 |21.1|5.5|\r\n\r\nwe can see that:\r\n* when `depth=101`, ResNet-v2 is 1% worse than ResNet-v1 on top-1 and 0.4% worse on top-5.\r\n* when `depth=152`, ResNet-v2 is only 0.2% better than ResNet-v1 on top-1 and owns the same performance on top-5 even when crop-size=320x320.\r\n\r\n\r\n## How to use Trained Models\r\nwe can use the pre-trained model to classify one input image, the step is easy:\r\n* download the pre-trained model form [data.dml.ml](http://data.dmlc.ml/mxnet/models/imagenet/resnet/) and put it into the predict directory.\r\n* ```cd predict``` and run ```python -u predict.py --img test.jpg --prefix resnet-50 --gpu 0```, this means you want to recgnition test.jpg using model resnet-50-0000.params and gpu 0, then it will output the classification result.\r\n\r\n## Reference\r\n[1] Kaiming He, et al. \"Deep Residual Learning for Image Recognition.\" arXiv arXiv:1512.03385 (2015).  \r\n[2] Kaiming He, et al. \"Identity Mappings in Deep Residual Networks\" arXiv:1603.05027 (2016).  \r\n[3] caffe official training code and model, https://github.com/KaimingHe/deep-residual-networks  \r\n[4] torch training code and model provided by facebook, https://github.com/facebook/fb.resnet.torch  \r\n[5] MXNet resnet-v1 cifar10 examples,https://github.com/dmlc/mxnet/blob/master/example/image-classification/train_cifar10_resnet.py\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftornadomeet%2FResNet","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftornadomeet%2FResNet","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftornadomeet%2FResNet/lists"}