{"id":13435754,"url":"https://github.com/peteryuX/pcdarts-tf2","last_synced_at":"2025-03-18T11:32:11.199Z","repository":{"id":39739590,"uuid":"241377295","full_name":"peteryuX/pcdarts-tf2","owner":"peteryuX","description":"PC-DARTS (PC-DARTS: Partial Channel Connections for Memory-Efficient Differentiable Architecture Search, published in ICLR 2020) implemented in Tensorflow 2.0+. This is an unofficial implementation.","archived":false,"fork":false,"pushed_at":"2023-03-24T22:41:01.000Z","size":276,"stargazers_count":27,"open_issues_count":1,"forks_count":8,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-10-27T18:59:07.000Z","etag":null,"topics":["cifar-10","cifar10","darts","iclr2020","neural-architecture-search","pc-darts","pcdarts","tensorflow","tensorflow2","tf2"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/peteryuX.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2020-02-18T14:07:14.000Z","updated_at":"2024-08-23T12:27:54.000Z","dependencies_parsed_at":"2022-08-20T17:10:12.296Z","dependency_job_id":"d15f712d-65a2-4805-b6de-c27bfedabe8c","html_url":"https://github.com/peteryuX/pcdarts-tf2","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/peteryuX%2Fpcdarts-tf2","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/peteryuX%2Fpcdarts-tf2/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/peteryuX%2Fpcdarts-tf2/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/peteryuX%2Fpcdarts-tf2/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/peteryuX","download_url":"https://codeload.github.com/peteryuX/pcdarts-tf2/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244211155,"owners_count":20416589,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cifar-10","cifar10","darts","iclr2020","neural-architecture-search","pc-darts","pcdarts","tensorflow","tensorflow2","tf2"],"created_at":"2024-07-31T03:00:38.718Z","updated_at":"2025-03-18T11:32:06.191Z","avatar_url":"https://github.com/peteryuX.png","language":"Python","funding_links":[],"categories":["Sample Codes / Projects \u003ca name=\"sample\" /\u003e ⛏️📐📁"],"sub_categories":["Reinforcement Learning \u003ca name=\"RL\" /\u003e🔮"],"readme":"# [pcdarts-tf2](https://github.com/peteryuX/pcdarts-tf2)\n\n[![Language grade: Python](https://img.shields.io/lgtm/grade/python/g/peteryuX/pcdarts-tf2.svg?logo=lgtm\u0026logoWidth=18)](https://lgtm.com/projects/g/peteryuX/pcdarts-tf2/context:python)\n![Star](https://img.shields.io/github/stars/peteryuX/pcdarts-tf2)\n![Fork](https://img.shields.io/github/forks/peteryuX/pcdarts-tf2)\n![License](https://img.shields.io/github/license/peteryuX/pcdarts-tf2)\n\n:fire: PC-DARTS (PC-DARTS: Partial Channel Connections for Memory-Efficient Differentiable Architecture Search, published in ICLR 2020) implemented in Tensorflow 2.0+. This is an unofficial implementation. :fire:\n\n\u003e PC-DARTS is a memory efficient differentiable architecture search method, which can be trained with a larger batch size and, consequently, enjoys both faster speed and higher training stability. Experimental results achieve an error rate of **2.57%** on CIFAR10 with merely **0.1 GPU-days** for architecture search.\n\nOriginal Paper: \u0026nbsp; [Arxiv](https://arxiv.org/abs/1907.05737) \u0026nbsp; [OpenReview](https://openreview.net/forum?id=BJlS634tPr)\n\nOffical Implementation: \u0026nbsp; [PyTorch](https://github.com/yuhuixu1993/PC-DARTS)\n\n\u003cp align=\"center\"\u003e\n    \u003cimg src=\"photo/architecture.jpg\" width=\"75%\"\u003e\n\u003c/p\u003e\n\n****\n\n## Contents\n:bookmark_tabs:\n\n* [Installation](#Installation)\n* [Usage](#Training-and-Testing)\n* [Benchmark](#Benchmark)\n* [Models](#Models)\n* [References](#References)\n\n***\n\n## Installation\n:pizza:\n\nCreate a new python virtual environment by [Anaconda](https://www.anaconda.com/) or just use pip in your python environment and then clone this repository as following.\n\n### Clone this repo\n```bash\ngit clone https://github.com/peteryuX/pcdarts-tf2.git\ncd pcdarts-tf2\n```\n\n### Conda\n```bash\nconda env create -f environment.yml\nconda activate pcdarts-tf2\n```\n\n### Pip\n\n```bash\npip install -r requirements.txt\n```\n\n****\n\n## Usage\n:lollipop:\n\n### Config File\nYou can modify your own dataset path or other settings of model in [./configs/*.yaml](https://github.com/peteryuX/pcdarts-tf2/tree/master/configs) for training and testing, which would like below.\n\n```python\n# general setting\nbatch_size: 128\ninput_size: 32\ninit_channels: 36\nlayers: 20\nnum_classes: 10\nauxiliary_weight: 0.4\ndrop_path_prob: 0.3\narch: PCDARTS\nsub_name: 'pcdarts_cifar10'\nusing_normalize: True\n\n# training dataset\ndataset_len: 50000  # number of training samples\nusing_crop: True\nusing_flip: True\nusing_cutout: True\ncutout_length: 16\n\n# training setting\nepoch: 600\ninit_lr: 0.025\nlr_min: 0.0\nmomentum: 0.9\nweights_decay: !!float 3e-4\ngrad_clip: 5.0\n\nval_steps: 1000\nsave_steps: 1000\n```\n\nNote:\n- The `sub_name` is the name of outputs directory used in checkpoints and logs folder. (make sure of setting it unique to other models)\n- The `save_steps` is the number interval steps of saving checkpoint file.\n- The [./configs/pcdarts_cifar10_search.yaml](https://github.com/peteryuX/pcdarts-tf2/tree/master/configs/pcdarts_cifar10_search.yaml) and [./configs/pcdarts_cifar10.yaml](https://github.com/peteryuX/pcdarts-tf2/tree/master/configs/pcdarts_cifar10.yaml) are used by [train_search.py](https://github.com/peteryuX/pcdarts-tf2/tree/master/train_search.py) and [train.py](https://github.com/peteryuX/pcdarts-tf2/tree/master/train.py) respectively, which have different settings for small proxy model training(architecture searching) and full-size model training. Please make sure you use the correct config file in related script. (The example yaml script above is [./configs/pcdarts_cifar10.yaml](https://github.com/peteryuX/pcdarts-tf2/tree/master/configs/pcdarts_cifar10.yaml).)\n\n### Architecture Searching on CIFAR-10 (using small proxy model)\n\n**Step1**: Search cell architecture on CIFAR-10 using small proxy model.\n\n```bash\npython train_search.py --cfg_path=\"./configs/pcdarts_cifar10_search.yaml\" --gpu=0\n```\n\nNote:\n- The `--gpu` is used to choose the id of your avaliable GPU devices with `CUDA_VISIBLE_DEVICES` system varaible.\n- You can visualize the training status on tensorboard by running \"`tensorboard --logdir=./logs/`\". My logs can be found from [search_log](https://tensorboard.dev/experiment/SWu96NYrSlyIk7VRmCqjgw/) and [full_train_log](https://tensorboard.dev/experiment/lyI64MKgRAODUqv3V4rzjg/).\n- You can visualize the learning rate scheduling by running \"`python ./modules/lr_scheduler.py`\".\n- You can visualize the dataset augmantation by running \"`python ./dataset_checker.py`\".\n\n**Step2**: After the searching completed, you can find the result genotypes in `./logs/{sub_name}/search_arch_genotype.py`. Open it and copy the latest genotype into the [./modules/genotypes.py](https://github.com/peteryuX/pcdarts-tf2/tree/master/modules/genotypes.py), which will be used for further training later. The genotype like bellow:\n\n```python\nTheNameYouWantToCall = Genotype(\n    normal=[\n        ('sep_conv_3x3', 1),\n        ('skip_connect', 0),\n        ('sep_conv_3x3', 0),\n        ('dil_conv_3x3', 1),\n        ('sep_conv_5x5', 0),\n        ('sep_conv_3x3', 1),\n        ('avg_pool_3x3', 0),\n        ('dil_conv_3x3', 1)],\n    normal_concat=range(2, 6),\n    reduce=[\n        ('sep_conv_5x5', 1),\n        ('max_pool_3x3', 0),\n        ('sep_conv_5x5', 1),\n        ('sep_conv_5x5', 2),\n        ('sep_conv_3x3', 0),\n        ('sep_conv_3x3', 3),\n        ('sep_conv_3x3', 1),\n        ('sep_conv_3x3', 2)],\n    reduce_concat=range(2, 6))\n```\n\nNote:\n- You can visualize the genotype by running \"`python ./visualize_genotype.py TheNameYouWantToCall`\".\n\u003cp align=\"center\"\u003e\n    \u003cimg src=\"photo/genotype_normal.jpg\" width=\"20%\"\u003e\n    \u003cimg src=\"photo/genotype_reduction.jpg\" width=\"70%\"\u003e\n\u003c/p\u003e\n\n### Training on CIFAR-10 (using full-sized model)\n\n**Step1**: Make sure that you already modifed the flag `arch` in [./configs/pcdarts_cifar10.yaml](https://github.com/peteryuX/pcdarts-tf2/tree/master/configs/pcdarts_cifar10.yaml) to match the genotype you want to use in [./modules/genotypes.py](https://github.com/peteryuX/pcdarts-tf2/tree/master/modules/genotypes.py). \n\nNote:\n- The default flag `arch` (`PCDARTS`) is the genotype proposed by official paper. You can train this model by yourself, or use dowload it from [BenchmarkModels](#Models).\n\n**Step2**: Train the full-sized model on CIFAR-10 with specific genotype.\n\n```bash\npython train.py --cfg_path=\"./configs/pcdarts_cifar10.yaml\" --gpu=0\n```\n\n### Testing on CIFAR-10 (using full-sized model)\n\nTo evaluate the full-sized model with the corresponding cfg file on the testing dataset. You can also download my trained model for testing from [Models](#Models) without training it yourself, which default `arch` (`PCDARTS`) is the best cell proposed in paper.\n\n```bash\npython test.py --cfg_path=\"./configs/pcdarts_cifar10.yaml\" --gpu=0\n```\n\n****\n\n## Benchmark\n:coffee:\n\n### Results on CIFAR-10\n| Method | Search Method | Params(M) | Test Error(%)| Search-Cost(GPU-days) |\n| ------ | ------------- | --------- | ------------ | --------------------- |\n| [NASNet-A](https://arxiv.org/abs/1611.01578) | RL | 3.3 | 2.65 | 1800 |\n| [AmoebaNet-B](https://arxiv.org/abs/1802.01548) | Evolution | 2.8 | 2.55 | 3150 |\n| [ENAS](https://arxiv.org/abs/1802.03268) | RL | 4.6 | 2.89 | 0.5 |\n| [DARTSV1](https://arxiv.org/abs/1806.09055) | gradient-based | 3.3 | 3.00 | 0.4 |\n| [DARTSV2](https://arxiv.org/abs/1806.09055) | gradient-based | 3.3 | 2.76 | 1.0 |\n| [SNAS](https://arxiv.org/abs/1812.09926)    | gradient-based | 2.8 | 2.85 | 1.5 |\n| [PC-DARTS](https://github.com/yuhuixu1993/PC-DARTS) (official PyTorch version) | gradient-based | 3.63 | **2.57** | **0.1** |\n| PC-DARTS TF2 (paper architecture) | gradient-based | 3.63 | 2.73 | - |\n| PC-DARTS TF2 (searched by myself) | gradient-based | 3.56 | 2.88 | 0.12 |\n\nNote:\n- Above results are referenced from [official repository](https://github.com/yuhuixu1993/PC-DARTS) and [orignal paper](https://arxiv.org/abs/1907.05737).\n- There still have a slight performance gap between my PC-DARTS TF2 and official version. In both cases, we used Nvidia 1080ti (11G memory). My PC-DARTS TF2 pre-trained model can be found in [Models](#Models).\n- My tensorboard logs can be found from [search_log](https://tensorboard.dev/experiment/SWu96NYrSlyIk7VRmCqjgw/) and [full_train_log](https://tensorboard.dev/experiment/lyI64MKgRAODUqv3V4rzjg/).\n- If you get unsatisfactory results with the archecture searched by yourself, you might try to search it more than one time. (see the discussions [here](https://github.com/yuhuixu1993/PC-DARTS/issues/7))\n\n****\n\n## Models\n:doughnut:\n\nDowload these models bellow, then extract them into `./checkpoints/` for restoring.\n\n| Model Name          | Config File | `arch` | Download Link |\n|---------------------|-------------|--------|---------------|\n| PC-DARTS (CIFAR-10, paper architecture) | [pcdarts_cifar10.yaml](https://github.com/peteryuX/pcdarts-tf2/tree/master//configs/pcdarts_cifar10.yaml) | `PCDARTS` | [GoogleDrive](https://drive.google.com/file/d/1BhLlktX78z90yOaORXvch_GAnIWWkYrX/view?usp=sharing) |\n| PC-DARTS (CIFAR-10, searched by myself) | [pcdarts_cifar10_TF2.yaml](https://github.com/peteryuX/pcdarts-tf2/tree/master//configs/pcdarts_cifar10_TF2.yaml) | `PCDARTS_TF2_SEARCH` | [GoogleDrive](https://drive.google.com/file/d/1UgeZzEnQZ6oeMKpr01rEDflVi9X1dzeq/view?usp=sharing) |\n\nNote:\n- You can find the training settings of the models in the corresponding [./configs/*.yaml](https://github.com/peteryuX/pcdarts-tf2/tree/master/configs) files, and make sure that the `arch` flag in it is matched with the genotypes name in [./modules/genotypes.py](https://github.com/peteryuX/pcdarts-tf2/tree/master/modules/genotypes.py).\n- **Based on the property of the training dataset, all the pre-trained models can only be used for non-commercial applications.**\n\n****\n\n## References\n:hamburger:\n\nThanks for these source codes porviding me with knowledges to complete this repository.\n\n- https://github.com/yuhuixu1993/PC-DARTS (Official)\n    - PC-DARTS:Partial Channel Connections for Memory-Efficient Differentiable Architecture Search\n- https://github.com/quark0/darts\n    - Differentiable architecture search for convolutional and recurrent networks https://arxiv.org/abs/1806.09055\n- https://github.com/zzh8829/yolov3-tf2\n    - YoloV3 Implemented in TensorFlow 2.0\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FpeteryuX%2Fpcdarts-tf2","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FpeteryuX%2Fpcdarts-tf2","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FpeteryuX%2Fpcdarts-tf2/lists"}