{"id":13935875,"url":"https://github.com/naszilla/naszilla","last_synced_at":"2025-04-05T08:08:35.341Z","repository":{"id":38410793,"uuid":"217753524","full_name":"naszilla/naszilla","owner":"naszilla","description":"Naszilla is a Python library for neural architecture search (NAS)","archived":false,"fork":false,"pushed_at":"2022-12-29T03:08:50.000Z","size":1802,"stargazers_count":307,"open_issues_count":5,"forks_count":46,"subscribers_count":11,"default_branch":"master","last_synced_at":"2025-03-29T07:09:13.148Z","etag":null,"topics":["automl","bananas","cifar10","convolutional-networks","convolutional-neural-networks","deep-learning","meta-neural-network","nas-algorithms","neural-architecture-search","pytorch","tensorflow"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/naszilla.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-10-26T18:39:46.000Z","updated_at":"2025-03-11T15:23:02.000Z","dependencies_parsed_at":"2023-01-31T08:01:21.869Z","dependency_job_id":null,"html_url":"https://github.com/naszilla/naszilla","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/naszilla%2Fnaszilla","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/naszilla%2Fnaszilla/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/naszilla%2Fnaszilla/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/naszilla%2Fnaszilla/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/naszilla","download_url":"https://codeload.github.com/naszilla/naszilla/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247305935,"owners_count":20917208,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["automl","bananas","cifar10","convolutional-networks","convolutional-neural-networks","deep-learning","meta-neural-network","nas-algorithms","neural-architecture-search","pytorch","tensorflow"],"created_at":"2024-08-07T23:02:09.885Z","updated_at":"2025-04-05T08:08:35.310Z","avatar_url":"https://github.com/naszilla.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\u003cimg src=\"img/naszilla_banner.png\" width=700 /\u003e\u003c/p\u003e\n\n[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](LICENSE.md)\n\nA repository to compare many popular NAS algorithms seamlessly across three popular\nbenchmarks (NASBench 101, 201, and 301). You can implement your own NAS algorithm, and\nthen easily compare it with eleven algorithms across three benchmarks.\n\nThis repository contains the official code for the following three papers:\n\n\u003ctable\u003e\n \u003ctbody\u003e\n    \u003ctr align=\"center\" valign=\"bottom\"\u003e\n      \u003cth\u003ePaper\u003c/th\u003e\n      \u003cth\u003eREADME\u003c/th\u003e\n      \u003cth\u003eBlog Post\u003c/th\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e \u003c!-- (1st row) --\u003e\n    \u003ctd rowspan=\"1\" align=\"center\" valign=\"middle\" halign=\"middle\"\u003e \u003ca href=\"https://arxiv.org/abs/2007.04965\" target=\"_blank\"\u003eA Study on Encodings for Neural Architecture Search\u003c/a\u003e \u003c/td\u003e\n    \u003ctd align=\"center\" valign=\"middle\"\u003e \u003ca href=\"docs/encodings.md\"\u003eencodings.md\u003c/a\u003e \u003c/td\u003e\n    \u003ctd align=\"center\" valign=\"middle\"\u003e \u003ca href=\"https://abacus.ai/blog/2020/10/02/a-study-on-encodings-for-nas/\" target=\"_blank\"\u003eBlog Post\u003c/a\u003e \u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e \u003c!-- (2nd row) --\u003e\n    \u003ctd rowspan=\"1\" align=\"center\" valign=\"middle\" halign=\"middle\"\u003e \u003ca href=\"https://arxiv.org/abs/1910.11858\" target=\"_blank\"\u003eBANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Search\u003c/a\u003e \u003c/td\u003e\n    \u003ctd align=\"center\" valign=\"middle\"\u003e \u003ca href=\"docs/bananas.md\"\u003ebananas.md\u003c/a\u003e \u003c/td\u003e\n    \u003ctd align=\"center\" valign=\"middle\"\u003e \u003ca href=\"https://medium.com/reality-engines/bananas-a-new-method-for-neural-architecture-search-192d21959c0c\" target=\"_blank\"\u003eBlog Post\u003c/a\u003e \u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e \u003c!-- (3rd row) --\u003e\n    \u003ctd rowspan=\"1\" align=\"center\" valign=\"middle\" halign=\"middle\"\u003e \u003ca href=\"https://arxiv.org/abs/2005.02960\" target=\"_blank\"\u003eExploring the Loss Landscape in Neural Architecture Search\u003c/a\u003e \u003c/td\u003e\n    \u003ctd align=\"center\" valign=\"middle\"\u003e \u003ca href=\"docs/local_search.md\"\u003elocal_search.md\u003c/a\u003e \u003c/td\u003e\n    \u003ctd align=\"center\" valign=\"middle\"\u003e \u003ca href=\"https://abacus.ai/blog/local-search-is-state-of-the-art-for-neural-architecture-search-benchmarks/\" target=\"_blank\"\u003eBlog Post\u003c/a\u003e \u003c/td\u003e\n    \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\n## Installation\nClone this repository and install its requirements (which includes [nasbench](https://github.com/google-research/nasbench), \n[nas-bench-201](https://github.com/D-X-Y/NAS-Bench-201), and [nasbench301](https://github.com/automl/nasbench301)). \nIt may take a few minutes.\n```bash\ngit clone https://github.com/naszilla/naszilla\ncd naszilla\ncat requirements.txt | xargs -n 1 -L 1 pip install\npip install -e .\n```\nYou might need to replace line 32 of `src/nasbench301/surrogate_models/surrogate_models.py`\nwith a new path to the configspace file:\n```python\nself.config_loader = utils.ConfigLoader(os.path.expanduser('~/naszilla/src/nasbench301/configspace.json'))\n```\nNext, download the nas benchmark datasets (either with the terminal commands below,\nor from their respective websites\n([nasbench](https://github.com/google-research/nasbench),\n[nas-bench-201](https://github.com/D-X-Y/NAS-Bench-201), and\n[nasbench301](https://github.com/automl/nasbench301)).\nThe versions recommended for use with naszilla are `nasbench_only108.tfrecord`, `NAS-Bench-201-v1_0-e61699.pth`, and `nasbench301_models_v0.9.zip`.\nIf you use a different version, you might need to edit some of the naszilla code.\n```bash\n# these files are 0.5GB, 2.1GB, and 1.6GB, respectively\nwget https://storage.googleapis.com/nasbench/nasbench_only108.tfrecord\nwget https://ndownloader.figshare.com/files/25506206?private_link=7d47bf57803227af4909 -O NAS-Bench-201-v1_0-e61699.pth\nwget https://ndownloader.figshare.com/files/24693026 -O nasbench301_models_v0.9.zip\nunzip nasbench301_models_v0.9.zip\n```\nPlace the three downloaded benchmark data files in `~/nas_benchmark_datasets` (or choose\nanother directory and edit line 15 of `naszilla/nas_benchmarks.py` accordingly).\n\nNow you have successfully installed all of the requirements to run **eleven NAS\nalgorithms** on **three benchmark search spaces**!\n\n### Test Installation\n\nYou can test the installation by running these commands:\n```bash\ncd naszilla\npython naszilla/run_experiments.py --search_space nasbench_101 --algo_params all_algos --queries 30 --trials 1\npython naszilla/run_experiments.py --search_space nasbench_201 --algo_params all_algos --queries 30 --trials 1\npython naszilla/run_experiments.py --search_space nasbench_301 --algo_params all_algos --queries 30 --trials 1\n```\n\nThese experiments should finish running within a few minutes.\n\u003cbr\u003e\u003cbr\u003e\n\n## Run NAS experiments on NASBench-101/201/301 search spaces\n\n```bash\ncd naszilla\npython naszilla/run_experiments.py --search_space nasbench_201 --dataset cifar100 --queries 100 --trials 100\n```\nThis will test several NAS algorithms against each other on the NASBench-201 search\nspace. Note that NASBench-201 allows you to specify one of three datasets: cifar10, cifar100, or imagenet. \nTo customize your experiment, open `naszilla/params.py`. Here, you can change the\nalgorithms and their hyperparameters. For details on running specific methods,\nsee [these docs](docs/naszilla.md).\n\n## Contributions\nContributions are welcome!\n\n## Reproducibility\nIf you have any questions about reproducing an experiment, please [open an issue](https://github.com/naszilla/naszilla/issues) \nor email `colin@abacus.ai`.\n\n## Citation\nPlease cite our papers if you use code from this repo:\n```bibtex\n@inproceedings{white2020study,\n  title={A Study on Encodings for Neural Architecture Search},\n  author={White, Colin and Neiswanger, Willie and Nolen, Sam and Savani, Yash},\n  booktitle={Advances in Neural Information Processing Systems},\n  year={2020}\n}\n\n@inproceedings{white2021bananas,\n  title={BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Search},\n  author={White, Colin and Neiswanger, Willie and Savani, Yash},\n  booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},\n  year={2021}\n}\n\n@inproceedings{white2021exploring,\n  title={Exploring the Loss Landscape in Neural Architecture Search},\n  author={White, Colin and Nolen, Sam and Savani, Yash},\n  booktitle={Uncertainty in Artificial Intelligence},\n  organization={PMLR},\n  year={2021}\n}\n```\n\n## Contents\n\nThis repo contains [encodings](docs/encodings.md) for neural architecture search, a\nvariety of NAS methods (including [BANANAS](docs/bananas.md), a neural predictor\nBayesian optimization method, and [local search](docs/local_search.md) for NAS), and an\neasy interface for using multiple NAS benchmarks.\n\nEncodings:\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"img/encodings.png\" alt=\"encodings\" width=\"90%\"\u003e\n\u003c/p\u003e\n\nBANANAS:\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"img/metann_adj_train.png\" alt=\"adj_train\" width=\"24%\"\u003e\n  \u003cimg src=\"img/metann_adj_test.png\" alt=\"adj_test\" width=\"24%\"\u003e\n  \u003cimg src=\"img/metann_path_train.png\" alt=\"path_train\" width=\"24%\"\u003e\n  \u003cimg src=\"img/metann_path_test.png\" alt=\"path_test\" width=\"24%\"\u003e\n\u003c/p\u003e\n\nLocal search:\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"img/local_search.png\" alt=\"local_search\" width=\"65%\"\u003e\n\u003c/p\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnaszilla%2Fnaszilla","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnaszilla%2Fnaszilla","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnaszilla%2Fnaszilla/lists"}