{"id":22354760,"url":"https://github.com/pprp/mutablenas","last_synced_at":"2025-08-07T15:19:24.271Z","repository":{"id":112699114,"uuid":"351025613","full_name":"pprp/MutableNAS","owner":"pprp","description":"Channel Search for ResNet20; cvpr21 NAS workshop 16th","archived":false,"fork":false,"pushed_at":"2022-02-26T06:57:41.000Z","size":177333,"stargazers_count":2,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-01-31T13:43:57.532Z","etag":null,"topics":["cifar10","cifar100","nas","resnet"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pprp.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-03-24T09:48:34.000Z","updated_at":"2024-06-18T03:45:10.000Z","dependencies_parsed_at":null,"dependency_job_id":"fcf8a727-d42f-4f3c-94bc-44f4e249d4b9","html_url":"https://github.com/pprp/MutableNAS","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pprp%2FMutableNAS","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pprp%2FMutableNAS/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pprp%2FMutableNAS/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pprp%2FMutableNAS/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pprp","download_url":"https://codeload.github.com/pprp/MutableNAS/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245658981,"owners_count":20651519,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cifar10","cifar100","nas","resnet"],"created_at":"2024-12-04T13:15:05.478Z","updated_at":"2025-03-26T12:41:55.045Z","avatar_url":"https://github.com/pprp.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Channel Search for ResNet20\r\n\r\nCVPR2021 Workshop Neural Architecture Search 1st lightweight NAS challenge and moving beyond\r\n\r\n## Feature\r\n\r\n- [x] support channel search for resnet20\r\n- [x] support artificial design model such as resnet, densenet, mobilenet,shufflenet,DLA,DPN etc. \r\n- [ ] support automl models such as regnet, pnasnet\r\n- [ ] support data augmentation tricks:\r\n\t- [ ] Cutout\r\n\t- [ ] AutoAugmentation\r\n\t- [ ] Shake-Shake\r\n\t- [ ] Mixup\r\n\t- [ ] random erasing\r\n\t- [ ] auto-augment\r\n\t- [ ] rand-augment\r\n- [ ] experimental results on cifar100 and cifar10\r\n- [ ] distributed data parrallel to support\r\n- [ ] support transformer models\r\n- [ ] support differenet datasets such as tiny imagenet\r\n\r\n\r\n## Experimental Results\r\n\r\nTraining Details:\r\n\r\n- total epoch: 300 \r\n- lr scheduler: CosineAnnealingLR\r\n- weight decay: 5e-4\r\n\r\n### CIFAR10\r\n\r\n| Network                                   | Params(M) | Train loss | Train top1 | Val loss | Val top1 | Hyper                                   | GPU(M) |\r\n| ----------------------------------------- | --------- | ---------- | ---------- | -------- | -------- | --------------------------------------- | ------ |\r\n| densenet_cifar                            | 4.4       | 0.00156    | 99.99%     | 0.24     | 94.83%   | 0.1/256/w/o cutout                      | 7303   |\r\n| dla                                       | 63        | 0.00164    | 99.99%     | 0.20     | 95.57%   | 0.1/256/w/o cutout                      | 5555   |\r\n| resnet50                                  | 91        | 0.00105    | 100.0%     | 0.19     | 95.74%   | 0.1/256/w/o cutout                      | 10895  |\r\n| attention52                               | 214       | 0.00109    | 99.99%     | 0.49     | 90.62%   | 0.01/256/w/o cutout                     | 5691   |\r\n| dpn26                                     | 45        | 0.00195    | 100.0%     | 0.16     | 95.43%   | 0.1/256/w/o cutout                      | 10260  |\r\n| resnet50_cutout                           | 91        | 0.00103    | 100.0%     | 0.18     | 95.87%   | 0.1/128/ cutout=0.5                     | 10895  |\r\n| efficientnetb0                            | 15        | 0.02396    | 99.32%     | 0.35     | 91.52%   | 0.1/128/w/o cutout                      | 3961   |\r\n| googlenet                                 | 25        | 0.00216    | 100.0%     | 0.17     | 95.18%   | 0.1/128/w/o cutout                      | 7689   |\r\n| inceptionv3                               | 86        | 0.00183    | 100.0%     | 0.19     | 95.27%   | 0.1/128/w/o cutout                      | 8053   |\r\n| inceptionv4                               | 159       | 0.00292    | 99.99%     | 0.24     | 93.50%   | 0.1/64/w/o cutout                       | 7557   |\r\n| inception_resnet_v2                       | 251       | 0.01001    | 99.79%     | 0.31     | 92.22%   | 0.1/64/w/o cutout                       | 8237   |\r\n| mobilenet                                 | 13        | 0.00904    | 99.78%     | 0.37     | 91.94%   | 0.1/128/w/o cutout                      | 2655   |\r\n| mobilenetv2                               |           | 0.00427    | 99.93%     | 0.24     | 94.00%   |                                         |        |\r\n| shake_resnet26_2x32d                      | 23        | 0.16430    | 94.31%     | 0.12     | 95.94%   | 0.1/128/w/o cutout w/o mixup            | 2253   |\r\n| shake_resnet26_2x64d                      | 91        | 0.10775    | 96.41%     | 0.10     | 96.94%   | 0.1/128/w/o cutout w/o mixup            | 3779   |\r\n| shake_resnet26_2x64d_mixup                | 91        | 0.97755    | 70.70%     | 0.27     | 96.53%   | 0.1/128/w/o cutout w mixup              | 3779   |\r\n| shake_resnet26_2x64d_cutout               | 91        | 0.10788    | 96.37%     | 0.10     | 96.89%   | 0.1/128/w cutout w/o mixup              | 3779   |\r\n| shake_resnet26_2x64d_autoaug              | 91        | 0.10775    | 96.41%     | 0.10     | 96.94%   | 0.1/128/w/o cutout w/o mixup w/ autoaug | 3779   |\r\n| shake_resnet26_2x64d_autoaug_mixup        | 91        | 0.97755    | 70.07%     | 0.27     | 96.53%   | 0.1/128/w/o cutout w/ mixup w/ autoaug  | 3779   |\r\n| shake_resnet26_2x64d_autoaug_cutout       | 91        | 0.10788    | 96.37%     | 0.10     | 96.89%   | 0.1/128/w cutout w/o mixup w/ autoaug   | 3779   |\r\n| shake_resnet26_2x64d_autoaug_cutout_mixup | 91        | 0.97755    | 70.07%     | 0.27     | 96.53%   | 0.1/128/w cutout w/ mixup w/ autoaug    | 3779   |\r\n| resnet50_mixup                            | 91        | 0.68908    | 76.88%     | 0.26     | 96.44%   | 0.1/128/w/o cutout/ w mixup             | 10895  |\r\n| resnet50_cutout_mixup                     | 91        | 0.69914    | 76.15%     | 0.26     | 96.44%   | 0.1/128/cutout=0.5 /w mixup             | 10895  |\r\n| resnet50_autoaug                          | 91        | 0.06838    | 97.63%     | 0.14     | 96.10%   | 0.1/128/w/o cutout w/o mixup/ w autoaug | 6479   |\r\n| resnet50_autoaug_mixup                    | 91        | 0.86331    | 72.5%      | 0.28     | 96.95%   | 0.1/128/w/o cutout w/mixup w/ autoaug   | 6101   |\r\n\r\n\r\n\r\npython train.py \r\n\r\n\r\n\r\n### CIFAR100\r\n\r\n| dataset  | network            | params | top1 err | top5 err | epoch(lr = 0.1) | epoch(lr = 0.02) | epoch(lr = 0.004) | epoch(lr = 0.0008) | total epoch |\r\n| -------- | ------------------ | ------ | -------- | -------- | --------------- | ---------------- | ----------------- | ------------------ | ----------- |\r\n| cifar100 | mobilenet          | 3.3M   | 34.02    | 10.56    | 60              | 60               | 40                | 40                 | 200         |\r\n| cifar100 | mobilenetv2        | 2.36M  | 31.92    | 09.02    | 60              | 60               | 40                | 40                 | 200         |\r\n| cifar100 | squeezenet         | 0.78M  | 30.59    | 8.36     | 60              | 60               | 40                | 40                 | 200         |\r\n| cifar100 | shufflenetv2       | 1.3M   | 30.49    | 8.49     | 60              | 60               | 40                | 40                 | 200         |\r\n| cifar100 | vgg13_bn           | 28.7M  | 28.00    | 9.71     | 60              | 60               | 40                | 40                 | 200         |\r\n| cifar100 | vgg16_bn           | 34.0M  | 27.07    | 8.84     | 60              | 60               | 40                | 40                 | 200         |\r\n| cifar100 | resnet18           | 11.2M  | 24.39    | 6.95     | 60              | 60               | 40                | 40                 | 200         |\r\n| cifar100 | resnet34           | 21.3M  | 23.24    | 6.63     | 60              | 60               | 40                | 40                 | 200         |\r\n| cifar100 | resnet50           | 23.7M  | 22.61    | 6.04     | 60              | 60               | 40                | 40                 | 200         |\r\n| cifar100 | resnet101          | 42.7M  | 22.22    | 5.61     | 60              | 60               | 40                | 40                 | 200         |\r\n| cifar100 | resnet152          | 58.3M  | 22.31    | 5.81     | 60              | 60               | 40                | 40                 | 200         |\r\n| cifar100 | preactresnet18     | 11.3M  | 27.08    | 8.53     | 60              | 60               | 40                | 40                 | 200         |\r\n| cifar100 | preactresnet34     | 21.5M  | 24.79    | 7.68     | 60              | 60               | 40                | 40                 | 200         |\r\n| cifar100 | preactresnet50     | 23.9M  | 25.73    | 8.15     | 60              | 60               | 40                | 40                 | 200         |\r\n| cifar100 | preactresnet101    | 42.9M  | 24.84    | 7.83     | 60              | 60               | 40                | 40                 | 200         |\r\n| cifar100 | preactresnet152    | 58.6M  | 22.71    | 6.62     | 60              | 60               | 40                | 40                 | 200         |\r\n| cifar100 | resnext50          | 14.8M  | 22.23    | 6.00     | 60              | 60               | 40                | 40                 | 200         |\r\n| cifar100 | resnext101         | 25.3M  | 22.22    | 5.99     | 60              | 60               | 40                | 40                 | 200         |\r\n| cifar100 | resnext152         | 33.3M  | 22.40    | 5.58     | 60              | 60               | 40                | 40                 | 200         |\r\n| cifar100 | attention59        | 55.7M  | 33.75    | 12.90    | 60              | 60               | 40                | 40                 | 200         |\r\n| cifar100 | attention92        | 102.5M | 36.52    | 11.47    | 60              | 60               | 40                | 40                 | 200         |\r\n| cifar100 | densenet121        | 7.0M   | 22.99    | 6.45     | 60              | 60               | 40                | 40                 | 200         |\r\n| cifar100 | densenet161        | 26M    | 21.56    | 6.04     | 60              | 60               | 60                | 40                 | 200         |\r\n| cifar100 | densenet201        | 18M    | 21.46    | 5.9      | 60              | 60               | 40                | 40                 | 200         |\r\n| cifar100 | googlenet          | 6.2M   | 21.97    | 5.94     | 60              | 60               | 40                | 40                 | 200         |\r\n| cifar100 | inceptionv3        | 22.3M  | 22.81    | 6.39     | 60              | 60               | 40                | 40                 | 200         |\r\n| cifar100 | inceptionv4        | 41.3M  | 24.14    | 6.90     | 60              | 60               | 40                | 40                 | 200         |\r\n| cifar100 | inceptionresnetv2  | 65.4M  | 27.51    | 9.11     | 60              | 60               | 40                | 40                 | 200         |\r\n| cifar100 | xception           | 21.0M  | 25.07    | 7.32     | 60              | 60               | 40                | 40                 | 200         |\r\n| cifar100 | seresnet18         | 11.4M  | 23.56    | 6.68     | 60              | 60               | 40                | 40                 | 200         |\r\n| cifar100 | seresnet34         | 21.6M  | 22.07    | 6.12     | 60              | 60               | 40                | 40                 | 200         |\r\n| cifar100 | seresnet50         | 26.5M  | 21.42    | 5.58     | 60              | 60               | 40                | 40                 | 200         |\r\n| cifar100 | seresnet101        | 47.7M  | 20.98    | 5.41     | 60              | 60               | 40                | 40                 | 200         |\r\n| cifar100 | seresnet152        | 66.2M  | 20.66    | 5.19     | 60              | 60               | 40                | 40                 | 200         |\r\n| cifar100 | nasnet             | 5.2M   | 22.71    | 5.91     | 60              | 60               | 40                | 40                 | 200         |\r\n| cifar100 | stochasticdepth18  | 11.22M | 31.40    | 8.84     | 60              | 60               | 40                | 40                 | 200         |\r\n| cifar100 | stochasticdepth34  | 21.36M | 27.72    | 7.32     | 60              | 60               | 40                | 40                 | 200         |\r\n| cifar100 | stochasticdepth50  | 23.71M | 23.35    | 5.76     | 60              | 60               | 40                | 40                 | 200         |\r\n| cifar100 | stochasticdepth101 | 42.69M | 21.28    | 5.39     | 60              | 60               | 40                | 40                 | 200         |\r\n\r\n\r\n\r\n\r\n\r\n## Acknowledgement\r\n\r\n- https://github.com/BIGBALLON/CIFAR-ZOO/\r\n\r\n- https://github.com/KaiyangZhou/deep-person-reid\r\n\r\n- https://github.com/weiaicunzai/pytorch-cifar100\r\n\r\n- https://github.com/kuangliu/pytorch-cifar\r\n\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpprp%2Fmutablenas","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpprp%2Fmutablenas","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpprp%2Fmutablenas/lists"}