{"id":22354745,"url":"https://github.com/pprp/nas-bench-201-refactor","last_synced_at":"2025-03-26T12:41:54.348Z","repository":{"id":112699129,"uuid":"513716639","full_name":"pprp/NAS-Bench-201-refactor","owner":"pprp","description":"Original version of NAS-Bench-201 from AngleNAS","archived":false,"fork":false,"pushed_at":"2022-11-03T07:25:35.000Z","size":6036,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-31T13:43:54.761Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pprp.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-07-14T01:06:50.000Z","updated_at":"2024-06-18T03:45:11.000Z","dependencies_parsed_at":null,"dependency_job_id":"a2e0f2dc-7a89-40dc-aeda-1be56e1b11bc","html_url":"https://github.com/pprp/NAS-Bench-201-refactor","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pprp%2FNAS-Bench-201-refactor","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pprp%2FNAS-Bench-201-refactor/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pprp%2FNAS-Bench-201-refactor/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pprp%2FNAS-Bench-201-refactor/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pprp","download_url":"https://codeload.github.com/pprp/NAS-Bench-201-refactor/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245658981,"owners_count":20651519,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-04T13:15:01.927Z","updated_at":"2025-03-26T12:41:54.341Z","avatar_url":"https://github.com/pprp.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Neural Architecture Search (NAS)\n\nThis project contains the following neural architecture search (NAS) algorithms, implemented in [PyTorch](http://pytorch.org).\nMore NAS resources can be found in [Awesome-NAS](https://github.com/D-X-Y/Awesome-NAS).\n\n- NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search, ICLR 2020\n- Network Pruning via Transformable Architecture Search, NeurIPS 2019\n- One-Shot Neural Architecture Search via Self-Evaluated Template Network, ICCV 2019\n- Searching for A Robust Neural Architecture in Four GPU Hours, CVPR 2019\n- 10 NAS algorithms for the neural topology in `exps/algos` (see [NAS-Bench-201.md](https://github.com/D-X-Y/NAS-Projects/blob/master/NAS-Bench-201.md) for more details)\n- Several typical classification models, e.g., ResNet and DenseNet (see [BASELINE.md](https://github.com/D-X-Y/NAS-Projects/blob/master/BASELINE.md))\n\n\n## Requirements and Preparation\n\nPlease install `PyTorch\u003e=1.2.0`, `Python\u003e=3.6`, and `opencv`.\n\nCIFAR and ImageNet should be downloaded and extracted into `$TORCH_HOME`.\nSome methods use knowledge distillation (KD), which require pre-trained models. Please download these models from [Google Driver](https://drive.google.com/open?id=1ANmiYEGX-IQZTfH8w0aSpj-Wypg-0DR-) (or train by yourself) and save into `.latent-data`.\n\n### Usefull tools\n1. Compute the number of parameters and FLOPs of a model:\n```\nfrom utils import get_model_infos\nflop, param  = get_model_infos(net, (1,3,32,32))\n```\n\n2. Different NAS-searched architectures are defined [here](https://github.com/D-X-Y/NAS-Projects/blob/master/lib/nas_infer_model/DXYs/genotypes.py).\n\n\n## [NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search](https://openreview.net/forum?id=HJxyZkBKDr)\n\nWe build a new benchmark for neural architecture search, please see more details in [NAS-Bench-201.md](https://github.com/D-X-Y/NAS-Projects/blob/master/NAS-Bench-201.md).\n\nThe benchmark data file (v1.0) is `NAS-Bench-201-v1_0-e61699.pth`, which can be downloaded from [Google Drive](https://drive.google.com/open?id=1SKW0Cu0u8-gb18zDpaAGi0f74UdXeGKs).\n\nNow you can simply use our API by `pip install nas-bench-201`.\n\n## [Network Pruning via Transformable Architecture Search](https://arxiv.org/abs/1905.09717)\n[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/network-pruning-via-transformable/network-pruning-on-cifar-100)](https://paperswithcode.com/sota/network-pruning-on-cifar-100?p=network-pruning-via-transformable)\n\nIn this paper, we proposed a differentiable searching strategy for transformable architectures, i.e., searching for the depth and width of a deep neural network.\nYou could see the highlight of our Transformable Architecture Search (TAS) at our [project page](https://xuanyidong.com/assets/projects/NeurIPS-2019-TAS.html).\n\n\u003cp float=\"left\"\u003e\n\u003cimg src=\"https://d-x-y.github.com/resources/paper-icon/NIPS-2019-TAS.png\" width=\"680px\"/\u003e\n\u003cimg src=\"https://d-x-y.github.com/resources/videos/NeurIPS-2019-TAS/TAS-arch.gif?raw=true\" width=\"180px\"/\u003e\n\u003c/p\u003e\n\n\n### Usage\n\nUse `bash ./scripts/prepare.sh` to prepare data splits for `CIFAR-10`, `CIFARR-100`, and `ILSVRC2012`.\nIf you do not have `ILSVRC2012` data, pleasee comment L12 in `./scripts/prepare.sh`.\n\nargs: `cifar10` indicates the dataset name, `ResNet56` indicates the basemodel name, `CIFARX` indicates the searching hyper-parameters, `0.47/0.57` indicates the expected FLOP ratio, `-1` indicates the random seed.\n\n#### Search for the depth configuration of ResNet:\n```\nCUDA_VISIBLE_DEVICES=0,1 bash ./scripts-search/search-depth-gumbel.sh cifar10 ResNet110 CIFARX 0.57 -1\n```\n\n#### Search for the width configuration of ResNet:\n```\nCUDA_VISIBLE_DEVICES=0,1 bash ./scripts-search/search-width-gumbel.sh cifar10 ResNet110 CIFARX 0.57 -1\n```\n\n#### Search for both depth and width configuration of ResNet:\n```\nCUDA_VISIBLE_DEVICES=0,1 bash ./scripts-search/search-shape-cifar.sh cifar10 ResNet56  CIFARX 0.47 -1\n```\n\n#### Training the searched shape config from TAS\nIf you want to directly train a model with searched configuration of TAS, try these:\n```\nCUDA_VISIBLE_DEVICES=0,1 bash ./scripts/tas-infer-train.sh cifar10  C010-ResNet32 -1\nCUDA_VISIBLE_DEVICES=0,1 bash ./scripts/tas-infer-train.sh cifar100 C100-ResNet32 -1\n```\n\n### Model Configuration\nThe searched shapes for ResNet-20/32/56/110/164 in Table 3 in the original paper are listed in [`configs/NeurIPS-2019`](https://github.com/D-X-Y/NAS-Projects/tree/master/configs/NeurIPS-2019).\n\n\n## [One-Shot Neural Architecture Search via Self-Evaluated Template Network](https://arxiv.org/abs/1910.05733)\n\n\u003cimg align=\"right\" src=\"https://d-x-y.github.com/resources/paper-icon/ICCV-2019-SETN.png\" width=\"450\"\u003e\n\n\u003cstrong\u003eHighlight\u003c/strong\u003e: we equip one-shot NAS with an architecture sampler and train network weights using uniformly sampling.\n\n\n### Usage\n\nPlease use the following scripts to train the searched SETN-searched CNN on CIFAR-10, CIFAR-100, and ImageNet.\n```\nCUDA_VISIBLE_DEVICES=0 bash ./scripts/nas-infer-train.sh cifar10  SETN 96 -1\nCUDA_VISIBLE_DEVICES=0 bash ./scripts/nas-infer-train.sh cifar100 SETN 96 -1\nCUDA_VISIBLE_DEVICES=0,1,2,3 bash ./scripts/nas-infer-train.sh imagenet-1k SETN  256 -1\n```\n\nThe searching codes of SETN on a small search space:\n```\nCUDA_VISIBLE_DEVICES=0 bash ./scripts-search/algos/SETN.sh cifar10 -1\n```\n\n\n## [Searching for A Robust Neural Architecture in Four GPU Hours](https://arxiv.org/abs/1910.04465)\n\n\n\u003cimg align=\"right\" src=\"https://d-x-y.github.com/resources/paper-icon/CVPR-2019-GDAS.png\" width=\"300\"\u003e\n\nWe proposed a Gradient-based searching algorithm using Differentiable Architecture Sampling (GDAS). GDAS is baseed on DARTS and improves it with Gumbel-softmax sampling.\nExperiments on CIFAR-10, CIFAR-100, ImageNet, PTB, and WT2 are reported.\n\n\n### Usage\n\n#### Reproducing the results of our searched architecture in GDAS\nPlease use the following scripts to train the searched GDAS-searched CNN on CIFAR-10, CIFAR-100, and ImageNet.\n```\nCUDA_VISIBLE_DEVICES=0 bash ./scripts/nas-infer-train.sh cifar10  GDAS_V1 96 -1\nCUDA_VISIBLE_DEVICES=0 bash ./scripts/nas-infer-train.sh cifar100 GDAS_V1 96 -1\nCUDA_VISIBLE_DEVICES=0,1,2,3 bash ./scripts/nas-infer-train.sh imagenet-1k GDAS_V1 256 -1\n```\n\n#### Searching on the NASNet search space\nPlease use the following scripts to use GDAS to search as in the original paper:\n```\nCUDA_VISIBLE_DEVICES=0 bash ./scripts-search/GDAS-search-NASNet-space.sh cifar10 1 -1\n```\n\n#### Searching on a small search space (NAS-Bench-201)\nThe GDAS searching codes on a small search space:\n```\nCUDA_VISIBLE_DEVICES=0 bash ./scripts-search/algos/GDAS.sh cifar10 -1\n```\n\nThe baseline searching codes are DARTS:\n```\nCUDA_VISIBLE_DEVICES=0 bash ./scripts-search/algos/DARTS-V1.sh cifar10 -1\nCUDA_VISIBLE_DEVICES=0 bash ./scripts-search/algos/DARTS-V2.sh cifar10 -1\n```\n\n#### Training the searched architecture\nTo train the searched architecture found by the above scripts, please use the following codes:\n```\nCUDA_VISIBLE_DEVICES=0 bash ./scripts-search/NAS-Bench-201/train-a-net.sh '|nor_conv_3x3~0|+|nor_conv_3x3~0|nor_conv_3x3~1|+|skip_connect~0|skip_connect~1|skip_connect~2|' 16 5\n```\n`|nor_conv_3x3~0|+|nor_conv_3x3~0|nor_conv_3x3~1|+|skip_connect~0|skip_connect~1|skip_connect~2|` represents the structure of a searched architecture. My codes will automatically print it during the searching procedure.\n\n\n# Citation\n\nIf you find that this project helps your research, please consider citing some of the following papers:\n```\n@inproceedings{dong2020nasbench201,\n  title     = {NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search},\n  author    = {Dong, Xuanyi and Yang, Yi},\n  booktitle = {International Conference on Learning Representations (ICLR)},\n  url       = {https://openreview.net/forum?id=HJxyZkBKDr},\n  year      = {2020}\n}\n@inproceedings{dong2019tas,\n  title     = {Network Pruning via Transformable Architecture Search},\n  author    = {Dong, Xuanyi and Yang, Yi},\n  booktitle = {Neural Information Processing Systems (NeurIPS)},\n  year      = {2019}\n}\n@inproceedings{dong2019one,\n  title     = {One-Shot Neural Architecture Search via Self-Evaluated Template Network},\n  author    = {Dong, Xuanyi and Yang, Yi},\n  booktitle = {Proceedings of the IEEE International Conference on Computer Vision (ICCV)},\n  pages     = {3681--3690},\n  year      = {2019}\n}\n@inproceedings{dong2019search,\n  title     = {Searching for A Robust Neural Architecture in Four GPU Hours},\n  author    = {Dong, Xuanyi and Yang, Yi},\n  booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},\n  pages     = {1761--1770},\n  year      = {2019}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpprp%2Fnas-bench-201-refactor","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpprp%2Fnas-bench-201-refactor","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpprp%2Fnas-bench-201-refactor/lists"}