{"id":18819502,"url":"https://github.com/shi-labs/semi-supervised-transfer-learning","last_synced_at":"2025-09-12T11:37:27.111Z","repository":{"id":111822322,"uuid":"343317082","full_name":"SHI-Labs/Semi-Supervised-Transfer-Learning","owner":"SHI-Labs","description":"[CVPR 2021] Adaptive Consistency Regularization for Semi-Supervised Transfer Learning","archived":false,"fork":false,"pushed_at":"2021-10-04T21:02:37.000Z","size":7209,"stargazers_count":100,"open_issues_count":2,"forks_count":15,"subscribers_count":4,"default_branch":"main","last_synced_at":"2024-04-23T00:17:33.068Z","etag":null,"topics":["computer-vision","semi-supervised-learning","transfer-learning"],"latest_commit_sha":null,"homepage":"http://arxiv.org/abs/2103.02193","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/SHI-Labs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-03-01T06:46:08.000Z","updated_at":"2024-08-03T03:07:38.964Z","dependencies_parsed_at":"2023-06-04T03:15:13.025Z","dependency_job_id":null,"html_url":"https://github.com/SHI-Labs/Semi-Supervised-Transfer-Learning","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SHI-Labs%2FSemi-Supervised-Transfer-Learning","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SHI-Labs%2FSemi-Supervised-Transfer-Learning/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SHI-Labs%2FSemi-Supervised-Transfer-Learning/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SHI-Labs%2FSemi-Supervised-Transfer-Learning/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/SHI-Labs","download_url":"https://codeload.github.com/SHI-Labs/Semi-Supervised-Transfer-Learning/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248797199,"owners_count":21163099,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["computer-vision","semi-supervised-learning","transfer-learning"],"created_at":"2024-11-08T00:23:22.642Z","updated_at":"2025-04-13T23:33:16.764Z","avatar_url":"https://github.com/SHI-Labs.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Adaptive Consistency Regularization for Semi-Supervised Transfer Learning\n\nThis repository is for Adaptive Knowledge Consistency and Adaptive Representation Consistency introduced in the following paper:\n \nAbulikemu Abuduweili, Xingjian Li, Humphrey Shi, Cheng-Zhong Xu, and Dejing Dou, [Adaptive Consistency Regularization for Semi-Supervised Transfer Learning](https://arxiv.org/abs/2103.02193), CVPR 2021.  \n \n\n## Contents\n1. [Introduction](#Introduction)\n2. [Tasks](#Tasks)\n3. [Citation](#citation)\n\n\n## Introduction\nIn this work, we consider semi-supervised learning and transfer learning jointly, \nleading to a more practical and competitive paradigm that can utilize both powerful pre-trained models from source domain as well as labeled/unlabeled data in the target domain. \nTo better exploit the value of both pre-trained weights and unlabeled target examples, we introduce adaptive consistency regularization that consists of two complementary components: \nAdaptive Knowledge Consistency (AKC) on the (labeled and unlabeled) examples between the source and target model, \nand Adaptive Representation Consistency (ARC) on the target model between labeled and unlabeled examples. \nExamples involved in the consistency regularization are adaptively selected according to their potential contributions (measured by prediction entropy) to\nthe target task.  Moreover, our algorithm is orthogonal to existing methods and thus able to gain additional improvements on top of the existing semi-supervised learning methods.\n\n![model](figs/model.png)\n\n## Tasks\n### Semi-supervised transfer learning \nAKC and ARC regularization terms could be combined with other semi-supervised learning methods, \nlike MixMatch and FixMatch. By utilizing AKC and ARC regularization techniques in MixMatch, the performance increased notably.\n\nResults on CUB-200-2011 dataset:  \n![cub200](figs/cub200.png)\n\nThe actual sample selected ratio in ARC and AKC is shown in Figure 2 on CUB-200-2011 dataset experiment. \nAs can be seen, the sample selected ratio for ARC is gradually increasing. Which can be regarded as a kind of curriculum learning.  \n![sample_ratio](figs/sample_ratio.png)\n\n### Supervised transfer learning \nBoth AKC and ARC improve the performance of standard transfer learning.    \n![supervised](figs/supervised.png)\n\n### Effectiveness of transfer learning in semi-supervised setting\nIn previous works, the effectiveness of transfer learning in semi-supervised settings was underestimated. \nWith the Imprinting technique and proper training strategy, transfer learning could lead to a noticeable improvement, especially when labeled examples are insufficient.\n\nResults of SSL methods with and without transfer learning on CIFAR-10:  \n![tf_ssl](figs/tf_ssl.png)  \n\n\n## Citation\nIf you find the code helpful in your resarch or work, please cite the following papers.\n```BibTex\n@inproceedings{abuduweili2021adaptive,\n  title={Adaptive Consistency Regularization for Semi-Supervised Transfer Learning},\n  author={Abuduweili, Abulikemu and Li, Xingjian and Shi, Humphrey and Xu, Cheng-Zhong and Dou, Dejing},\n  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},\n  pages={6923--6932},\n  year={2021}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshi-labs%2Fsemi-supervised-transfer-learning","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fshi-labs%2Fsemi-supervised-transfer-learning","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshi-labs%2Fsemi-supervised-transfer-learning/lists"}