{"id":19426935,"url":"https://github.com/kakaobrain/autoclint","last_synced_at":"2025-04-24T17:31:17.919Z","repository":{"id":105098312,"uuid":"195246678","full_name":"kakaobrain/autoclint","owner":"kakaobrain","description":"A specially designed light version of Fast AutoAugment","archived":false,"fork":false,"pushed_at":"2020-05-12T03:01:37.000Z","size":971,"stargazers_count":171,"open_issues_count":0,"forks_count":25,"subscribers_count":9,"default_branch":"master","last_synced_at":"2025-04-03T08:37:41.991Z","etag":null,"topics":["1st-place","autocv-challenge","autocv2-challenge","autodl-challenge","automated-machine-learning","automl","cnn","computer-vision","convolutional-neural-networks","deep-learning","fast-auto-augment","image-classification","pytorch"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kakaobrain.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2019-07-04T13:26:04.000Z","updated_at":"2025-04-02T08:55:14.000Z","dependencies_parsed_at":null,"dependency_job_id":"0b43194e-5c27-4c7d-a9f2-8e243787c547","html_url":"https://github.com/kakaobrain/autoclint","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kakaobrain%2Fautoclint","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kakaobrain%2Fautoclint/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kakaobrain%2Fautoclint/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kakaobrain%2Fautoclint/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kakaobrain","download_url":"https://codeload.github.com/kakaobrain/autoclint/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250674297,"owners_count":21469194,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["1st-place","autocv-challenge","autocv2-challenge","autodl-challenge","automated-machine-learning","automl","cnn","computer-vision","convolutional-neural-networks","deep-learning","fast-auto-augment","image-classification","pytorch"],"created_at":"2024-11-10T14:09:44.694Z","updated_at":"2025-04-24T17:31:17.237Z","avatar_url":"https://github.com/kakaobrain.png","language":"Python","funding_links":[],"categories":["Projects"],"sub_categories":["Distributed Frameworks"],"readme":"# AutoCLINT\n[![KakaoBrain](https://img.shields.io/badge/kakao-brain-ffcd00.svg)](http://kakaobrain.com/)\n[![pytorch](https://img.shields.io/badge/pytorch-1.0.1-%23ee4c2c.svg)](https://pytorch.org/)\n[![tensorflow](https://img.shields.io/badge/tensorflow-1.13.1-ed6c20.svg)](https://www.tensorflow.org/)\n[![autocv 1st place](https://img.shields.io/badge/autocv-1st_place-%235339D3.svg)](#)\n[![autocv2 1st place](https://img.shields.io/badge/autocv2-1st_place-%235339D3.svg)](#)\n[![HitCount](http://hits.dwyl.io/kakaobrain/autoclint.svg)](http://hits.dwyl.io/kakaobrain/autoclint)\n\n\n## Automatic **C**omputationally **LI**ght **N**etwork **T**ransfer\n\nA specially designed light version of **[Fast AutoAugment][]** is implemented to adapt to **various tasks** under **limited resources**.\n\nThis is our solution to [NeurIPS 2019 AutoDL Challenges][].\nWe won the **1st place** in the final learderboards in both [AutoCV][] and [AutoCV2][] Challenges.\n\n\n## [AutoCV][]/[AutoCV2][] Challenge Introduction\n#### Fully Automated Image (and Video) Classification without ANY human intervention\n\u003e Despite recent successes of deep learning and other machine learning techniques, practical experience and expertise is still required to select models and/or choose hyper-parameters when applying techniques to new datasets. This problem is drawing githincreasing interest, yielding progress towards fully automated solutions. In this challenge your machine learning code is trained and tested on this platform, without human intervention whatsoever, on image or video classification tasks you have never seen before, with time and memory limitations. All problems are multi-label classification problems, coming from various domains including medical imaging, satellite imaging, object recognition, character recognition, face recognition, etc. They lend themselves to deep learning solutions, but other methods may be used. Raw data is provided, but formatted in a uniform manner, to encourage you to submit generic algorithms.\n\n\n## Methods\n\nWe employ a network transfer strategy and implement a light version of [Fast AutoAugment][] for the fast adaptation and the efficient search of data augmentation policies.\n\n### Network Transfer\n\nThe [AutoCV][] Challenges are given limited memory and computational resources. Thus, we considered a small size of architecture that could use the pre-trained models that were transferred.\n\nWe have discovered the optimal hyperparameters and architectures to get the best performance in five minutes from five public data sets (Munster, Chuckey, Pedro, Decal and Hammer). In this process, no data augmentation is used.\n\nDue to the variability in image size (median shape 28x28x1 for munster vs. 576x944x3 for decal) the input tensor size of network must be automatically adapted for each dataset to allow for adequate aggregation of spatial information and to keep the aspect ratio of original image.\nWe automatically adapt these parameters to the median size of each dataset, so that the network effectively trains on entire datasets. Due to time constraints, we do not increase the input tensor volume (without channels) beyond 64^2. If the median shape of the dataset is smaller than 64^2 then we use the median shape as original input.\n\n|    | Munster |  Chucky |   Pedro |   Decal |  Hammer | Kreatur | Katze | Kraut | \n|:---|:-------:|:-------:|:-------:|:-------:|:-------:|:-------:|:-----:|:-----:|\n| \u003csub\u003edata type\u003c/sub\u003e | image | image | image | image | image | video | video | video |\n| \u003csub\u003eoriginal median shape\u003c/sub\u003e | 28x28x1 | 32x32x1 | 203x74x3 | 576x944x3 | 300x400x3 | 46x60x80x3 | 46x120x160x1 | 46x120x160x1 |\n| \u003csub\u003einput tensor shape\u003c/sub\u003e | 28x28x1 | 32x32x1 | 128x48x3 | 48x64x3 | 48x64x3 | 8x48x64x3 | 8x48x64x1 | 8x48x64x1 |\n\n### [Fast AutoAugment][]\n\nFast AutoAugment learns augmentation policies using a more efficient search strategy based on density matching. Ideally, Fast AutoAugment should be performed automatically, allowing the training data to adapt to test data.\n\nWe modify the search space and implement a light version of Fast AutoAugment algorithm to surmount the restricted computational resources.\n\nAs Fast AutoAugment, we search the augmentation policies that match the density of train data with density of augmented valid data. We deviate from the original version in that we replace 5-fold with single-fold search and use random search (within subset of searched policy in original) instead of TPE algorithm.\n\n\n## [AutoCV][] Results\n### Public\n\n#### V1.XLARGE\n* experiment environment: [BrainCloud][] V1.XLARGE Type (NVIDIA Tesla V100 1GPU, 14CPU, 122GB)\n\n| metrics | Munster |  Chucky |   Pedro |   Decal |  Hammer |\n|:-------:|--------:|--------:|--------:|--------:|--------:|\n| ALC     |  0.9421 |  0.8082 |  0.7948 |  0.8647 |  0.8147 |\n| \u003csub\u003e2*AUC-1\u003c/sub\u003e |  0.9992 |  0.9297 |  0.9241 |  0.9233 |  0.8863 |\n| curves  | ![](./assets/public_final_result_v1_munster.png) | ![](./assets/public_final_result_v1_Chuckey.png) | ![](./assets/public_final_result_v1_pedro.png) | ![](./assets/public_final_result_v1_Decal.png) | ![](./assets/public_final_result_v1_Hammer.png) |\n\n#### P1.XLARGE\n* experiment environment: [BrainCloud][] P1.XLARGE Type (NVIDIA Tesla P40 1GPU, 6CPU, 61GB)\n\n| metrics | Munster |  Chucky |   Pedro |   Decal |  Hammer |\n|:-------:|--------:|--------:|--------:|--------:|--------:|\n| ALC     |  0.9440 |  0.7835 |  0.7366 |  0.8353 |  0.8286 |\n| \u003csub\u003e2*AUC-1\u003c/sub\u003e |  0.9977 |  0.9353 |  0.9214 |  0.9347 |  0.9142 |\n| curves  | ![](./assets/public_final_result_p1_munster.png) | ![](./assets/public_final_result_p1_Chuckey.png) | ![](./assets/public_final_result_p1_pedro.png) | ![](./assets/public_final_result_p1_Decal.png) | ![](./assets/public_final_result_p1_Hammer.png) |\n\n### Private\n* experiment environment: [CodaLab](https://autodl.lri.fr/) (NVIDIA Tesla P100 1GPU, 4vCPU, 26GB)\n\n| metrics | beatriz | Caucase | Hippoc. |  Saturn | ukulele |\n|:-------:|--------:|--------:|--------:|--------:|--------:|\n| ALC     |  0.6756 |  0.7359 |  0.7744 |  0.8309 |  0.9075 |\n| \u003csub\u003e2*AUC-1\u003c/sub\u003e |  0.8014 |  0.9411 |  0.9534 |  0.9884 |  0.9985 |\n| curves  | ![](./assets/private_final_result_beatriz.png) | ![](./assets/private_final_result_Caucase.png) | ![](./assets/private_final_result_Hippocrate.png) | ![](./assets/private_final_result_Saturn.png) | ![](./assets/private_final_result_ukulele.png) |\n\n\n## [AutoCV2][] Results\n### Public (video only)\n\n#### V1.XLARGE\n* experiment environment: [BrainCloud][] V1.XLARGE Type (NVIDIA Tesla V100 1GPU, 14CPU, 122GB)\n\n| metrics | Kreature | Katze | Kraut |\n|:-------:|---------:|------:|------:|\n| ALC     | 0.8677 | 0.8613 | 0.6678 |\n| \u003csub\u003e2*AUC-1\u003c/sub\u003e | 0.9613 | 0.9588 | 0.7365  |\n| curves  | ![](./assets/public_final_result_v1_kreature.png) | ![](./assets/public_final_result_v1_katze.png) | ![](./assets/public_final_result_v1_kraut.png) |\n\n#### P1.XLARGE\n* experiment environment: [BrainCloud][] P1.XLARGE Type (NVIDIA Tesla P40 1GPU, 6CPU, 61GB)\n\n| metrics | Kreature | Katze | Kraut |\n|:-------:|---------:|------:|------:|\n| ALC     | 0.8675 | 0.8757 | 0.6883 |\n| \u003csub\u003e2*AUC-1\u003c/sub\u003e | 0.9587 | 0.9572 | 0.7559 |\n| curves  | ![](./assets/public_final_result_p1_kreature.png) | ![](./assets/public_final_result_p1_katze.png) | ![](./assets/public_final_result_p1_kraut.png) |\n\n### Private\n* experiment environment: [CodaLab](https://autodl.lri.fr/) (NVIDIA Tesla P100 1GPU, 4vCPU, 26GB)\n\n| metrics | Ideal | freddy | Homer | Isaac2 | Formula |\n|:-------:|------:|-------:|------:|-------:|--------:|\n| ALC     | 0.8229 | 0.7516 | 0.3843 | 0.7064 | 0.7661 |\n| \u003csub\u003e2*AUC-1\u003c/sub\u003e | 0.9605 | 0.9945 | 0.5500 | 0.9845 | 0.9661 |\n| curves  | ![](./assets/private_final_result_ideal.png) | ![](./assets/private_final_result_freddy.png) | ![](./assets/private_final_result_homer.png) | ![](./assets/private_final_result_isaac2.png) | ![](./assets/private_final_result_formula.png) |\n\n\n### Final (blind)\n* experiment environment: [CodaLab](https://autodl.lri.fr/) (NVIDIA Tesla P100 1GPU, 4vCPU, 26GB)\n\n| metrics | Apollon | loukooum | Fiona | Monica1 | Kitsune |\n|:-------:|------:|-------:|------:|-------:|--------:|\n| ALC     | 0.5593 | 0.9256 | 0.4074 | 0.4491 | 0.2132 |\n| \u003csub\u003e2*AUC-1\u003c/sub\u003e | 0.8022 | 0.9978 | 0.5312 | 0.8617 | 0.2467 |\n| curves  | ![](./assets/blind_final_result_apollon.png) | ![](./assets/blind_final_result_loukoum.png) | ![](./assets/blind_final_result_fiona.png) | ![](./assets/blind_final_result_monica1.png) | ![](./assets/blind_final_result_kitsune.png) |\n\n\n## Environment Setup \u0026 Experiments\n* base docker environment: https://hub.docker.com/r/evariste/autodl\n\n* pre requirements\n```bash\n$ apt update\n$ apt install python3-tk\n```\n\n* clone and init. the repository\n```bash\n$ git clone https://github.com/kakaobrain/autoclint.git \u0026\u0026 cd autoclint\n$ # 3rd parties libarary\n$ git submodule init\n$ git submodule update\n$ # download pretrained models\n$ wget https://download.pytorch.org/models/resnet18-5c106cde.pth -O ./models/resnet18-5c106cde.pth\n$ # download public datasets\n$ cd autodl \u0026\u0026 python download_public_datasets.py \u0026\u0026 cd ..\n```\n\n* run public datasets\n```bash\n$ # images\n$ python autodl/run_local_test.py -time_budget=1200 -code_dir='./' -dataset_dir='autodl/AutoDL_public_data/Munster/'; cp autodl/AutoDL_scoring_output/learning-curve-*.png ./results\n$ python autodl/run_local_test.py -time_budget=1200 -code_dir='./' -dataset_dir='autodl/AutoDL_public_data/Chucky/'; cp autodl/AutoDL_scoring_output/learning-curve-*.png ./results\n$ python autodl/run_local_test.py -time_budget=1200 -code_dir='./' -dataset_dir='autodl/AutoDL_public_data/Pedro/'; cp autodl/AutoDL_scoring_output/learning-curve-*.png ./results\n$ python autodl/run_local_test.py -time_budget=1200 -code_dir='./' -dataset_dir='autodl/AutoDL_public_data/Decal/'; cp autodl/AutoDL_scoring_output/learning-curve-*.png ./results\n$ python autodl/run_local_test.py -time_budget=1200 -code_dir='./' -dataset_dir='autodl/AutoDL_public_data/Hammer/'; cp autodl/AutoDL_scoring_output/learning-curve-*.png ./results\n$ # videos\n$ python autodl/run_local_test.py -time_budget=1200 -code_dir='./' -dataset_dir='autodl/AutoDL_public_data/Kreatur/'; cp autodl/AutoDL_scoring_output/learning-curve-*.png ./results\n$ python autodl/run_local_test.py -time_budget=1200 -code_dir='./' -dataset_dir='autodl/AutoDL_public_data/Katze/'; cp autodl/AutoDL_scoring_output/learning-curve-*.png ./results\n$ python autodl/run_local_test.py -time_budget=1200 -code_dir='./' -dataset_dir='autodl/AutoDL_public_data/Kraut/'; cp autodl/AutoDL_scoring_output/learning-curve-*.png ./results\n```\n\n* (optional) display learning curve\n```bash\n$ # item2 utils to visualize learning curve\n$ wget https://www.iterm2.com/utilities/imgcat -O bin/imgcat; chmod 0677 bin/imgcat\n$ bin/imgcat ./results/learning-curve-*.png\n```\n\n## Authors and Licensing\nThis project is developed by [Woonhyuk Baek][], [Ildoo Kim][] and [Sungbin Lim][] at\n[Kakao Brain][]. It is distributed under [Apache License\n2.0](LICENSE).\n\n\n## Citation\n\nIf you apply this library to any project and research, please cite our code:\n\n```\n@article{baek2020autoclint,\n  title         = {AutoCLINT: The Winning Method in AutoCV Challenge 2019}\n  author        = {Woonhyuk Baek and Ildoo Kim and Sungwoong Kim and Sungbin Lim},\n  year          = {2020},\n  eprint        = {2005.04373},\n  archivePrefix = {arXiv}\n}\n```\n\n\n## References \u0026 Open sources\n1. [Fast AutoAugment][]\n    - paper: https://arxiv.org/abs/1905.00397\n    - codes: https://github.com/kakaobrain/fast-autoaugment\n2. Pretrained models for Pytorch\n    - codes: https://github.com/Cadene/pretrained-models.pytorch\n3. TorchVision models\n    - pages: https://pytorch.org/docs/stable/torchvision/models.html\n3. TQDM: Progress Bar for Python and CLI\n    - codes: https://github.com/tqdm/tqdm\n4. AutoCV/AutoDL startking kit\n    - codes: https://github.com/zhengying-liu/autodl_starting_kit_stable\n\n[Kakao Brain]: https://kakaobrain.com/\n[BrainCloud]: https://cloud.kakaobrain.com/\n[Sungbin Lim]: https://github.com/sungbinlim\n[Ildoo Kim]: https://github.com/ildoonet\n[Woonhyuk Baek]: https://github.com/wbaek\n[Fast AutoAugment]: https://arxiv.org/abs/1905.00397\n[AutoCV]: https://autodl.lri.fr/competitions/118#home\n[AutoCV2]: https://autodl.lri.fr/competitions/3#home\n[NeurIPS 2019 AutoDL Challenges]: https://autodl.chalearn.org/\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkakaobrain%2Fautoclint","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkakaobrain%2Fautoclint","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkakaobrain%2Fautoclint/lists"}