{"id":22449647,"url":"https://github.com/naszilla/bananas","last_synced_at":"2025-08-01T23:31:06.011Z","repository":{"id":119440374,"uuid":"309543517","full_name":"naszilla/bananas","owner":"naszilla","description":"Bayesian Optimization with Neural Architectures for Neural Architecture Search - https://arxiv.org/abs/1910.11858","archived":false,"fork":false,"pushed_at":"2021-04-05T03:43:03.000Z","size":1019,"stargazers_count":27,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-08T09:42:38.831Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/naszilla.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2020-11-03T01:52:23.000Z","updated_at":"2024-11-08T11:13:34.000Z","dependencies_parsed_at":null,"dependency_job_id":"a38197a1-6a59-47d5-acbb-ca3b02d70121","html_url":"https://github.com/naszilla/bananas","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/naszilla/bananas","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/naszilla%2Fbananas","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/naszilla%2Fbananas/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/naszilla%2Fbananas/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/naszilla%2Fbananas/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/naszilla","download_url":"https://codeload.github.com/naszilla/bananas/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/naszilla%2Fbananas/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":268311133,"owners_count":24230286,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-01T02:00:08.611Z","response_time":67,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-06T05:10:35.973Z","updated_at":"2025-08-01T23:31:05.685Z","avatar_url":"https://github.com/naszilla.png","language":"Python","funding_links":[],"categories":["Projects"],"sub_categories":["Distributed Frameworks"],"readme":"# BANANAS\n\n**Note: our naszilla/bananas repo has been extended and renamed to [naszilla/naszilla](https://github.com/naszilla/naszilla), and this repo is deprecated and not maintained. Please use [naszilla/naszilla](https://github.com/naszilla/naszilla), which has more functionality.**\n\n[BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Search](https://arxiv.org/abs/1910.11858)\\\nColin White, Willie Neiswanger, and Yash Savani.\\\n_arXiv:1910.11858_.\n\n## A new method for neural architecture search\nBANANAS is a neural architecture search (NAS) algorithm which uses Bayesian optimization with a meta neural network to predict the validation accuracy of neural architectures. We use a path-based encoding scheme to featurize the neural architectures that are used to train the neural network model. After training on just 200 architectures, we are able to predict the validation accuracy of new architectures to within one percent on average. The full NAS algorithm beats the state of the art on the NASBench and the DARTS search spaces. On the NASBench search space, BANANAS is over 100x more efficient than random search, and 3.8x more efficent than the next-best algorithm we tried. On the DARTS search space, BANANAS finds an architecture with a test error of 2.57%.\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"img/bananas_fig.png\" alt=\"bananas_fig\" width=\"70%\"\u003e\n\u003c/p\u003e\n\n## Requirements\n- jupyter\n- tensorflow == 1.14.0 (used for all experiments)\n- nasbench (follow the installation instructions [here](https://github.com/google-research/nasbench))\n- nas-bench-201 (follow the installation instructions [here](https://github.com/D-X-Y/NAS-Bench-201))\n- pytorch == 1.2.0, torchvision == 0.4.0 (used for experiments on the DARTS search space)\n- pybnn (used only for the DNGO baselien algorithm. Installation instructions [here](https://github.com/automl/pybnn))\n\nIf you run experiments on DARTS, you will need our fork of the darts repo:\n- Download the repo: https://github.com/naszilla/darts\n- If the repo is not in your home directory, i.e., `~/darts`, then update line 5 of `bananas/darts/arch.py` and line 8 of `bananas/train_arch_runner.py` with the correct path to this repo\n\n\n## Train a meta neural network with a notebook on the NASBench dataset\n- Download the nasbench_only108 tfrecord file (size 499MB) [here](https://storage.googleapis.com/nasbench/nasbench_only108.tfrecord)\n- Place `nasbench_only108.tfrecord` in the top level folder of this repo\n- Open and run `meta_neuralnet.ipynb` to reproduce Table 1 and Figure A.1 of our paper\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"img/metann_adj_train.png\" alt=\"bananas_fig\" width=\"24%\"\u003e\n  \u003cimg src=\"img/metann_adj_test.png\" alt=\"bananas_fig\" width=\"24%\"\u003e\n  \u003cimg src=\"img/metann_path_train.png\" alt=\"bananas_fig\" width=\"24%\"\u003e\n  \u003cimg src=\"img/metann_path_test.png\" alt=\"bananas_fig\" width=\"24%\"\u003e\n\u003c/p\u003e\n\n## Evaluate pretrained BANANAS architecture\nThe best architecture found by BANANAS on the DARTS search space achieved 2.57% test error. To evaluate our pretrained neural architecture, download the weights [bananas.pt](https://drive.google.com/file/d/1d8jnI0R9fvXBjkIY7CRogyxynEh6TWu_/view?usp=sharing) and put it inside the folder `\u003cpath-to-darts\u003e/cnn`\n\n```bash\ncd \u003cpath-to-darts\u003e/cnn; python test.py --model_path bananas.pt\n```\n\nThe error on the test set should be 2.57%. This can be run on a CPU or GPU, but it will be faster on a GPU.\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"img/bananas_normal.png\" alt=\"bananas_normal\" width=\"42%\"\u003e\n\u003cimg src=\"img/bananas_reduction.png\" alt=\"bananas_reduction\" width=\"47%\"\u003e\n\u003c/p\u003e\n\u003cp align=\"center\"\u003e\nThe best neural architecture found by BANANAS on CIFAR-10. Convolutional cell (left), and reduction cell (right).\n\u003c/p\u003e\n\n## Train BANANAS architecture\nTrain the best architecture found by BANANAS.\n\n```bash\ncd \u003cpath-to-darts\u003e/cnn; python train.py --auxiliary --cutout\n```\n\nThis will train the architecture from scratch, which takes about 34 hours on an NVIDIA V100 GPU. \nThe final test error should be 2.59%.\nSetting the random seed to 4 by adding `--seed 4` will result in a test error of 2.57%.\nWe report the random seeds and hardware used in Table 2 of our paper [here](https://docs.google.com/spreadsheets/d/1z6bHUgX8r0y9Bh9Zxot_B9nT_9qLWJoD0Um0fTYdpus/edit?usp=sharing).\n\n## Run BANANAS on the NASBench search space\nTo run BANANAS on NASBench, download `nasbench_only108.tfrecord` and place it in the top level folder of this repo.\n\n```bash\npython run_experiments_sequential.py\n```\n\nThis will test the nasbench algorithm against several other NAS algorithms on the NASBench search space.\nTo customize your experiment, open `params.py`. Here, you can change the hyperparameters and the algorithms to run.\nTo run experiments with NAS-Bench-201, download `NAS-Bench-201-v1_0-e61699.pth` and place it in the top level folder of this repo.\nChoose between cifar10, cifar100, and imagenet. For example,\n\n```bash\npython run_experiments_sequential.py --search_space nasbench_201_cifar10\n```\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"img/nasbench_plot.png\" alt=\"nasbench_plot\" width=\"70%\"\u003e\n\u003c/p\u003e\n\n## Run BANANAS on the DARTS search space\nWe highly recommend using multiple GPUs to run BANANAS on the DARTS search space. You can run BANANAS in parallel on GCP using the shell script:\n\n```bash\nrun_experiments_parallel.sh\n```\n\n## Contributions\nWe welcome community contributions to this repo!\n\n## Citation\nPlease cite [our paper](https://arxiv.org/abs/1910.11858) if you use code from this repo:\n\n```bibtex\n@inproceedings{white2019bananas,\n  title={BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Search},\n  author={White, Colin and Neiswanger, Willie and Savani, Yash},\n  booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},\n  year={2021}\n}\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnaszilla%2Fbananas","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnaszilla%2Fbananas","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnaszilla%2Fbananas/lists"}