{"id":13578995,"url":"https://github.com/carpedm20/ENAS-pytorch","last_synced_at":"2025-04-05T20:32:55.190Z","repository":{"id":37390665,"uuid":"121592531","full_name":"carpedm20/ENAS-pytorch","owner":"carpedm20","description":"PyTorch implementation of \"Efficient Neural Architecture Search via Parameters Sharing\"","archived":false,"fork":false,"pushed_at":"2023-07-06T21:33:33.000Z","size":13520,"stargazers_count":2713,"open_issues_count":40,"forks_count":491,"subscribers_count":107,"default_branch":"master","last_synced_at":"2025-03-30T23:11:12.496Z","etag":null,"topics":["google-brain","neural-architecture-search","pytorch"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/carpedm20.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-02-15T04:54:37.000Z","updated_at":"2025-03-25T11:23:07.000Z","dependencies_parsed_at":"2024-08-01T15:30:06.462Z","dependency_job_id":"da71b822-b7a8-4a46-a01e-51b1f6895f25","html_url":"https://github.com/carpedm20/ENAS-pytorch","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/carpedm20%2FENAS-pytorch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/carpedm20%2FENAS-pytorch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/carpedm20%2FENAS-pytorch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/carpedm20%2FENAS-pytorch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/carpedm20","download_url":"https://codeload.github.com/carpedm20/ENAS-pytorch/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247399818,"owners_count":20932875,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["google-brain","neural-architecture-search","pytorch"],"created_at":"2024-08-01T15:01:35.662Z","updated_at":"2025-04-05T20:32:50.180Z","avatar_url":"https://github.com/carpedm20.png","language":"Python","funding_links":[],"categories":["Python","Neural Architecture Search","AutoML","1.) Neural Architecture Search","超参数优化和AutoML","Paper implementations","Paper implementations｜论文实现"],"sub_categories":["**[Papers]**","Other libraries｜其他库:"],"readme":"# Efficient Neural Architecture Search (ENAS) in PyTorch\n\nPyTorch implementation of [Efficient Neural Architecture Search via Parameters Sharing](https://arxiv.org/abs/1802.03268).\n\n\u003cp align=\"center\"\u003e\u003cimg src=\"assets/ENAS_rnn.png\" alt=\"ENAS_rnn\" width=\"60%\"\u003e\u003c/p\u003e\n\n**ENAS** reduce the computational requirement (GPU-hours) of [Neural Architecture Search](https://arxiv.org/abs/1611.01578) (**NAS**) by 1000x via parameter sharing between models that are subgraphs within a large computational graph. SOTA on `Penn Treebank` language modeling.\n\n**\\*\\*[Caveat] Use official code from the authors: [link](https://github.com/melodyguan/enas)\\*\\***\n\n\n## Prerequisites\n\n- Python 3.6+\n- [PyTorch==0.3.1](https://pytorch.org/get-started/previous-versions/)\n- tqdm, scipy, imageio, graphviz, tensorboardX\n\n## Usage\n\nInstall prerequisites with:\n\n    conda install graphviz\n    pip install -r requirements.txt\n\nTo train **ENAS** to discover a recurrent cell for RNN:\n\n    python main.py --network_type rnn --dataset ptb --controller_optim adam --controller_lr 0.00035 \\\n                   --shared_optim sgd --shared_lr 20.0 --entropy_coeff 0.0001\n\n    python main.py --network_type rnn --dataset wikitext\n\nTo train **ENAS** to discover CNN architecture (in progress):\n\n    python main.py --network_type cnn --dataset cifar --controller_optim momentum --controller_lr_cosine=True \\\n                   --controller_lr_max 0.05 --controller_lr_min 0.0001 --entropy_coeff 0.1\n\nor you can use your own dataset by placing images like:\n\n    data\n    ├── YOUR_TEXT_DATASET\n    │   ├── test.txt\n    │   ├── train.txt\n    │   └── valid.txt\n    ├── YOUR_IMAGE_DATASET\n    │   ├── test\n    │   │   ├── xxx.jpg (name doesn't matter)\n    │   │   ├── yyy.jpg (name doesn't matter)\n    │   │   └── ...\n    │   ├── train\n    │   │   ├── xxx.jpg\n    │   │   └── ...\n    │   └── valid\n    │       ├── xxx.jpg\n    │       └── ...\n    ├── image.py\n    └── text.py\n\nTo generate `gif` image of generated samples:\n\n    python generate_gif.py --model_name=ptb_2018-02-15_11-20-02 --output=sample.gif\n\nMore configurations can be found [here](config.py).\n\n\n## Results\n\nEfficient Neural Architecture Search (**ENAS**) is composed of two sets of learnable parameters, controller LSTM *θ* and the shared parameters *ω*. These two parameters are alternatively trained and only trained controller is used to derive novel architectures.\n\n### 1. Discovering Recurrent Cells\n\n![rnn](./assets/rnn.png)\n\nController LSTM decide 1) what activation function to use and 2) which previous node to connect.\n\nThe RNN cell **ENAS** discovered for `Penn Treebank` and `WikiText-2` dataset:\n\n\u003cimg src=\"assets/ptb.gif\" alt=\"ptb\" width=\"45%\"\u003e \u003cimg src=\"assets/wikitext.gif\" alt=\"wikitext\" width=\"45%\"\u003e\n\nBest discovered ENAS cell for `Penn Treebank` at epoch 27:\n\n\u003cimg src=\"assets/best_rnn_epoch27.png\" alt=\"ptb\" width=\"30%\"\u003e\n\nYou can see the details of training (e.g. `reward`, `entropy`, `loss`) with:\n\n    tensorboard --logdir=logs --port=6006\n\n\n### 2. Discovering Convolutional Neural Networks\n\n![cnn](./assets/cnn.png)\n\nController LSTM samples 1) what computation operation to use and 2) which previous node to connect.\n\nThe CNN network **ENAS** discovered for `CIFAR-10` dataset:\n\n(in progress)\n\n\n### 3. Designing Convolutional Cells\n\n(in progress)\n\n\n## Reference\n\n- [Neural Architecture Search with Reinforcement Learning](https://arxiv.org/abs/1611.01578)\n- [Neural Optimizer Search with Reinforcement Learning](https://arxiv.org/abs/1709.07417)\n\n\n## Author\n\nTaehoon Kim / [@carpedm20](http://carpedm20.github.io/)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcarpedm20%2FENAS-pytorch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcarpedm20%2FENAS-pytorch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcarpedm20%2FENAS-pytorch/lists"}