{"id":28367029,"url":"https://github.com/thuml/debiased-self-training","last_synced_at":"2026-02-06T07:38:10.359Z","repository":{"id":65296973,"uuid":"548722565","full_name":"thuml/Debiased-Self-Training","owner":"thuml","description":"Code release of paper Debiased Self-Training for Semi-Supervised Learning (NeurIPS 2022 Oral)","archived":false,"fork":false,"pushed_at":"2022-12-12T14:39:46.000Z","size":554,"stargazers_count":54,"open_issues_count":0,"forks_count":4,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-07-29T07:50:11.325Z","etag":null,"topics":["semi-supervised-learning"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2202.07136","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/thuml.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-10-10T04:40:06.000Z","updated_at":"2025-07-09T15:57:40.000Z","dependencies_parsed_at":"2023-01-16T09:16:01.841Z","dependency_job_id":null,"html_url":"https://github.com/thuml/Debiased-Self-Training","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/thuml/Debiased-Self-Training","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thuml%2FDebiased-Self-Training","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thuml%2FDebiased-Self-Training/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thuml%2FDebiased-Self-Training/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thuml%2FDebiased-Self-Training/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/thuml","download_url":"https://codeload.github.com/thuml/Debiased-Self-Training/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thuml%2FDebiased-Self-Training/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29154476,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-06T07:18:23.844Z","status":"ssl_error","status_checked_at":"2026-02-06T07:13:32.659Z","response_time":59,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["semi-supervised-learning"],"created_at":"2025-05-29T00:13:55.433Z","updated_at":"2026-02-06T07:38:10.341Z","avatar_url":"https://github.com/thuml.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# Debiased-Self-Training-for-Semi-Supervised-Learning\n\nThis repository contains the implementation code for paper: \u003cbr\u003e\n__Debiased Self-Training for Semi-Supervised Learning__ \u003cbr\u003e\nBaixu Chen, [Junguang Jiang](https://junguangjiang.github.io), [Ximei Wang](https://wxm17.github.io/), Pengfei Wan, [Mingsheng Long](http://ise.thss.tsinghua.edu.cn/~mlong/), [Jianmin Wang](https://www.thss.tsinghua.edu.cn/faculty/wangjianmin.htm)\u003cbr\u003e\n_Thirty-sixth Conference on Neural Information Processing Systems (NeurIPS 2022, Oral)_ \u003cbr\u003e\n[[Project Page](https://github.com/thuml/Debiased-Self-Training)] [[OpenReview](https://openreview.net/forum?id=NI7moUOKtc)] [[Slide](https://neurips.cc/media/neurips-2022/Slides/55076.pdf)] [[Poster](https://neurips.cc/media/PosterPDFs/NeurIPS%202022/08f90c1a417155361a5c4b8d297e0d78.png?t=1666668047.4378536)]\n\n## Table of Contents\n\n\u003cdetails\u003e\n  \u003csummary\u003eDebiased Self-Training\u003c/summary\u003e\n\n- [Updates](#updates)\n- [Introduction](#introduction)\n- [Installation](#installation)\n- [Experiments](#experiments)\n- [Contact](#contact)\n- [Citation](#citation)\n- [Acknowledgments](#acknowledgments)\n\n\u003c/details\u003e\n\n## Updates\n\n#### 2022.11\n\nCode release. For usage, please see [Installation](#installation) part.\n\n| [Code of training-from-scratch experiments (standard SSL benchmarks)](https://github.com/thuml/Transfer-Learning-Library/tree/dev-ssl/projects/dst) |\n| ------------------------------------------------------------ |\n| [**Code of fine-tuning experiments (ImageNet Pre-trained, ResNet50)**](https://github.com/thuml/Transfer-Learning-Library/tree/master/examples/semi_supervised_learning/image_classification) |\n\n## Introduction\n\nDebiased Self-Training (DST) tackles the bias issue in self-training process. DST can be implemented by only several lines of code and enjoys the following advantages.\n\n- DST significantly **boosts the performance** (18.9% over FixMatch on 13 tasks). Meanwhile, it helps **stabilize training** and **improves performance balance**.\n- DST can serve **as a general add-on** to mainstream self-training methods. Besides, the benefits of DST and recent popular techniques (dynamic thresholding, contrastive learning, etc.) are orthogonal.\n\nBelow, we briefly explain the main components of DST. For details, please refer to our paper. \n\n####\tDecoupled Pseudo Labeling\n\n![arch](fig/arch.png)\n\nFirst, we revisit the components of deep neural networks where the bias may accumulate. Our experiments suggest that the backbone has better tolerance to noisy pseudo labels than the classifier head. Based on this observation, we propose to introduce a complete parameter-independent pseudo head to take the duty of training with pseudo labels. As illustrated in (d), we **decouple the generation and utilization of pseudo labels**.\n\n#### Worst Case Estimation\n\n![arch](fig/worst_case.png)\n\nFurther, we further propose a worst-case estimation to decrease bias in the feature space. This is achieved by introducing another worst-case head and a minimax game between the feature extractor and the worst-case head. We illustrate the idea in the above figure.\n\n**(a)** Shift between the hyperplanes learned on limited labeled data and the true hyperplanes.\n\n**(b)** The worst hyperplanes are hyperplanes that correctly distinguish labeled samples while making as many mistakes as possible on unlabeled samples.\n\n**(c)** Feature representations are optimized to improve the performance of the worst hyperplanes.\n\n## Installation\n\nWe release our code as part of [Transfer Learning Library (TLlib)](https://github.com/thuml/Transfer-Learning-Library#introduction). Currently, we maintain our code in two different folders of TLlib.\n\n- [Code of training-from-scratch experiments (standard SSL benchmarks)](https://github.com/thuml/Transfer-Learning-Library/tree/dev-ssl/projects/dst)\n- [Code of fine-tuning experiments](https://github.com/thuml/Transfer-Learning-Library/tree/master/examples/semi_supervised_learning/image_classification)\n\nTo use our code, you should **(1)** install TLliib, and **(2)** follow the instructions in the above links to install addtional dependencies. \n\n## Experiments\n\n**We provide the running scripts as well as optimal hyperparameters in corresponding code.** The main results are shown below.\n\n#### Standard SSL Benchmarks\n\n| Methods       | CIFAR10 | CIFAR100 | SVHN | STL10 |\n| ------------- | :-----: | :------: | :--: | :---: |\n| FixMatch      |  92.9   |   53.7   | 98.0 | 62.5  |\n| DST(FixMatch) |  94.0   |   60.0   | 97.7 | 78.0  |\n\n#### Fine-Tuning Experiments\n\n**Supervised Pre-Training**\n\n| Methods      | Food101 | CIFAR10 | CIFAR100 | CUB200 | Aircraft | Cars | SUN397 | DTD  | Pets | Flowers | Caltech | Avg  |\n| ------------ | :-----: | :-----: | :------: | :----: | :------: | :--: | :----: | :--: | :--: | :-----: | ------- | ---- |\n| ERM          |  33.6   |  59.4   |   47.9   |  48.6  |   29.0   | 37.1 |  40.9  | 50.5 | 82.2 |  87.6   | 82.2    | 54.5 |\n| Pseudo Label |  36.9   |  62.8   |   52.5   |  54.9  |   30.4   | 40.4 |  41.7  | 54.1 | 89.6 |  93.5   | 85.1    | 58.4 |\n| Pi Model     |  34.2   |  66.9   |   48.5   |  47.9  |   26.7   | 37.4 |  40.9  | 51.9 | 83.5 |  92.0   | 82.2    | 55.6 |\n| Mean Teacher |  40.4   |  78.1   |   58.5   |  52.8  |   32.0   | 45.6 |  40.2  | 53.8 | 86.8 |  92.8   | 83.7    | 60.4 |\n| UDA          |  41.9   |  73.0   |   59.8   |  55.4  |   33.5   | 42.7 |  42.1  | 49.7 | 88.0 |  93.4   | 85.3    | 60.4 |\n| FixMatch     |  36.2   |  74.5   |   58.0   |  52.6  |   27.1   | 44.8 |  40.8  | 50.2 | 87.8 |  93.6   | 83.2    | 59.0 |\n| Self Tuning  |  41.4   |  70.9   |   57.2   |  60.5  |   37.0   | 59.8 |  43.5  | 51.7 | 88.4 |  93.5   | 89.1    | 63.0 |\n| FlexMatch    |  48.1   |  94.2   |   69.2   |  65.1  |   38.0   | 55.3 |  50.2  | 55.6 | 91.5 |  94.6   | 89.4    | 68.3 |\n| DebiasMatch  |  57.1   |  92.4   |   69.0   |  66.2  |   41.5   | 65.4 |  48.3  | 54.2 | 90.2 |  95.4   | 89.3    | 69.9 |\n| DST          |  58.1   |  93.5   |   67.8   |  68.6  |   44.9   | 68.6 |  47.0  | 56.3 | 91.5 |  95.1   | 90.3    | 71.1 |\n| Oracle       |  85.5   |  97.5   |   86.3   |  81.1  |   85.1   | 91.1 |  64.1  | 68.8 | 93.2 |  98.1   | 92.6    | 85.8 |\n\n**Unsupervised Pre-Training**\n\n| Methods      | Food101 | CIFAR10 | CIFAR100 | CUB200 | Aircraft | Cars | SUN397 | DTD  | Pets | Flowers | Caltech | Avg  |\n| ------------ | :-----: | :-----: | :------: | :----: | :------: | :--: | :----: | :--: | :--: | :-----: | ------- | ---- |\n| ERM          |  33.5   |  63.0   |   50.8   |  39.4  |   28.1   | 40.3 |  40.7  | 53.7 | 65.4 |  87.5   | 82.8    | 53.2 |\n| Pseudo Label |  33.6   |  71.9   |   53.8   |  42.7  |   30.9   | 51.2 |  41.2  | 55.2 | 69.3 |  94.2   | 86.2    | 57.3 |\n| Pi Model     |  32.7   |  77.9   |   50.9   |  33.6  |   27.2   | 34.4 |  41.1  | 54.9 | 66.7 |  91.4   | 84.1    | 54.1 |\n| Mean Teacher |  36.8   |  79.0   |   56.7   |  43.0  |   33.0   | 53.9 |  39.5  | 54.5 | 67.8 |  92.7   | 83.3    | 58.2 |\n| UDA          |  39.5   |  91.3   |   60.0   |  41.9  |   36.2   | 39.7 |  41.7  | 51.5 | 71.0 |  93.7   | 86.5    | 59.4 |\n| FixMatch     |  44.3   |  86.1   |   58.0   |  42.7  |   38.0   | 55.4 |  42.4  | 53.1 | 67.9 |  95.2   | 83.4    | 60.6 |\n| Self Tuning  |  34.0   |  63.6   |   51.7   |  43.3  |   32.2   | 50.2 |  40.7  | 52.7 | 68.2 |  91.8   | 87.7    | 56.0 |\n| FlexMatch    |  50.2   |  96.6   |   69.2   |  49.4  |   41.3   | 62.5 |  47.2  | 54.5 | 72.4 |  94.8   | 89.4    | 66.1 |\n| DebiasMatch  |  54.2   |  95.5   |   68.1   |  49.1  |   40.9   | 73.0 |  47.6  | 54.4 | 76.6 |  95.5   | 88.7    | 67.6 |\n| DST          |  57.1   |  95.0   |   68.2   |  53.6  |   47.7   | 72.0 |  46.8  | 56.0 | 76.3 |  95.6   | 90.1    | 68.9 |\n| Oracle       |  87.0   |  98.2   |   87.9   |  80.6  |   88.7   | 92.7 |  63.9  | 73.8 | 90.6 |  97.8   | 93.1    | 86.8 |\n\n## Contact\n\nIf you have any questions or want to use the code, please contact\n\n- Baixu Chen (cbx_99_hasta@outlook.com)\n- Junguang Jiang (JiangJunguang1123@outlook.com)\n\n## Citation\n\nIf you find this code useful, please cite our paper.\n\n```\n@@inproceedings{DST,\n  \ttitle={Debiased Self-Training for Semi-Supervised Learning},\n  \tauthor={Chen, Baixu and Jiang, Junguang and Wang, Ximei and Wan, Pengfei and Wang, Jianmin and Long, Mingsheng},\n  \tbooktitle={Advances in Neural Information Processing Systems},\n  \tyear={2022}\n}\n```\n\n## Acknowledgments\n\nWe appreciate the following github repos for their valuable codebase:\n- https://github.com/xternalz/WideResNet-pytorch\n- https://github.com/kekmodel/FixMatch-pytorch\n- https://github.com/TorchSSL/TorchSSL\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthuml%2Fdebiased-self-training","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthuml%2Fdebiased-self-training","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthuml%2Fdebiased-self-training/lists"}