{"id":13546328,"url":"https://github.com/Haochen-Wang409/U2PL","last_synced_at":"2025-04-02T18:30:35.587Z","repository":{"id":37727727,"uuid":"465157505","full_name":"Haochen-Wang409/U2PL","owner":"Haochen-Wang409","description":"[CVPR'22] Semi-Supervised Semantic Segmentation Using Unreliable Pseudo-Labels","archived":false,"fork":false,"pushed_at":"2024-08-20T14:36:25.000Z","size":8980,"stargazers_count":434,"open_issues_count":11,"forks_count":61,"subscribers_count":5,"default_branch":"main","last_synced_at":"2024-11-03T14:35:35.462Z","etag":null,"topics":["cityscapes","computer-vision","contrastive-learning","cvpr2022","deep-learning","mechine-learning","pascal-voc","pytorch","semantic-segmentation","semi-supervised-learning"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Haochen-Wang409.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2022-03-02T04:35:02.000Z","updated_at":"2024-11-03T09:15:21.000Z","dependencies_parsed_at":"2024-01-07T17:11:00.325Z","dependency_job_id":"1fe7dc63-a76f-4f15-a183-305bcf54e5e5","html_url":"https://github.com/Haochen-Wang409/U2PL","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Haochen-Wang409%2FU2PL","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Haochen-Wang409%2FU2PL/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Haochen-Wang409%2FU2PL/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Haochen-Wang409%2FU2PL/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Haochen-Wang409","download_url":"https://codeload.github.com/Haochen-Wang409/U2PL/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246869598,"owners_count":20847156,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cityscapes","computer-vision","contrastive-learning","cvpr2022","deep-learning","mechine-learning","pascal-voc","pytorch","semantic-segmentation","semi-supervised-learning"],"created_at":"2024-08-01T12:00:35.588Z","updated_at":"2025-04-02T18:30:34.558Z","avatar_url":"https://github.com/Haochen-Wang409.png","language":"Python","funding_links":[],"categories":["Papers (with code)","2022","Python"],"sub_categories":[],"readme":"# Using Unreliable Pseudo Labels\n\nOfficial PyTorch implementation of [Semi-Supervised Semantic Segmentation Using Unreliable Pseudo Labels](https://arxiv.org/abs/2203.03884), CVPR 2022.\n\n:bell::bell::bell: The extension of this paper ([U2PL+](https://arxiv.org/abs/2306.02314)) has been accepted by **IJCV**! :bell::bell::bell:\n\nPlease refer to our **[project page](https://haochen-wang409.github.io/U2PL/)** for qualitative results.\n\n[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/semi-supervised-semantic-segmentation-using-2/semi-supervised-semantic-segmentation-on-21)](https://paperswithcode.com/sota/semi-supervised-semantic-segmentation-on-21?p=semi-supervised-semantic-segmentation-using-2)\n[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/semi-supervised-semantic-segmentation-using-2/semi-supervised-semantic-segmentation-on-4)](https://paperswithcode.com/sota/semi-supervised-semantic-segmentation-on-4?p=semi-supervised-semantic-segmentation-using-2)\n[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/semi-supervised-semantic-segmentation-using-2/semi-supervised-semantic-segmentation-on-9)](https://paperswithcode.com/sota/semi-supervised-semantic-segmentation-on-9?p=semi-supervised-semantic-segmentation-using-2)\n[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/semi-supervised-semantic-segmentation-using-2/semi-supervised-semantic-segmentation-on-15)](https://paperswithcode.com/sota/semi-supervised-semantic-segmentation-on-15?p=semi-supervised-semantic-segmentation-using-2)\n\n\n\u003e **Abstract.** \n\u003e The crux of semi-supervised semantic segmentation is to assign adequate pseudo-labels to the pixels of unlabeled images. \n\u003e A common practice is to select the highly confident predictions as the pseudo ground-truth, but it leads to a problem that most pixels may be left unused due to their unreliability. \n\u003e We argue that every pixel matters to the model training, even its prediction is ambiguous. \n\u003e Intuitively, an unreliable prediction may get confused among the top classes (*i.e*., those with the highest probabilities), \n\u003e however, it should be confident about the pixel not belonging to the remaining classes. \n\u003e Hence, such a pixel can be convincingly treated as a negative sample to those most unlikely categories. \n\u003e Based on this insight, we develop an effective pipeline to make sufficient use of unlabeled data. \n\u003e Concretely, we separate reliable and unreliable pixels via the entropy of predictions, push each unreliable pixel to a category-wise queue that consists of negative samples, and manage to train the model with all candidate pixels. \n\u003e Considering the training evolution, where the prediction becomes more and more accurate, we adaptively adjust the threshold for the reliable-unreliable partition. \n\u003e Experimental results on various benchmarks and training settings demonstrate the superiority of our approach over the state-of-the-art alternatives.\n\n![](./img/pipeline.png)\n\n## Results\n### PASCAL VOC 2012\n\nLabeled images are selected from the ```train``` set of original VOC, ```1,464``` images in total. \nAnd the remaining ```9,118``` images are all considered as unlabeled ones.\n\nFor instance, ```1/2 (732)``` represents ```732``` labeled images \nand remaining ```9,850 (9,118 + 732)``` are unlabeled.\n\n| Method                      | 1/16 (92) | 1/8 (183) | 1/4 (366) | 1/2 (732) | Full (1464) |\n| --------------------------- | --------- | --------- | --------- | --------- | ----------- |\n| SupOnly                     | 45.77     | 54.92     | 65.88     | 71.69     | 72.50       |\n| U\u003csup\u003e2\u003c/sup\u003ePL (w/ CutMix) | 67.98     | 69.15     | 73.66     | 76.16     | 79.49       |\n\nLabeled images are selected from the ```train``` set of augmented VOC, ```10,582``` images in total.\n\nFollowing results are all trained under our own splits.\nTraining a model on different splits is recommended to measure the performance of a method.\nYou can train our U\u003csup\u003e2\u003c/sup\u003ePL on splits provided by [CPS](https://github.com/charlesCXK/TorchSemiSeg/tree/main/DATA/pascal_voc/subset_train_aug) or [ST++](https://github.com/LiheYoung/ST-PlusPlus/tree/master/dataset/splits/pascal).\n\n| Method                      | 1/16 (662) | 1/8 (1323) | 1/4 (2646) | 1/2 (5291) |\n| --------------------------- | ---------- | ---------- | ---------- | ---------- |\n| SupOnly                     | 67.87      | 71.55      | 75.80      | 77.13      |\n| U\u003csup\u003e2\u003c/sup\u003ePL (w/ CutMix) | 77.21      | 79.01      | 79.30      | 80.50      |\n\n### Cityscapes\n\nLabeled images are selected from the ```train``` set, ```2,975``` images in total. \n\n| Method                      | 1/16 (186) | 1/8 (372) | 1/4 (744) | 1/2 (1488) |\n| --------------------------- | ---------- | --------- | --------- | ---------- |\n| SupOnly                     | 65.74      | 72.53     | 74.43     | 77.83      |\n| U\u003csup\u003e2\u003c/sup\u003ePL (w/ CutMix) | 70.30      | 74.37     | 76.47     | 79.05      |\n| U\u003csup\u003e2\u003c/sup\u003ePL (w/ AEL)    | 74.90      | 76.48     | 78.51     | 79.12      |\n\n## Checkpoints\n\n- Models on PASCAL VOC 2012 (ResNet101-DeepLabv3+) can be found [here](https://drive.google.com/drive/folders/1_BcixhrEqJEMo3lzKTKPJ_7gCWEjUBWp).\n\n- Models on Cityscapes with AEL (ResNet101-DeepLabv3+)\n\n| 1/16 (186)                                                                                         | 1/8 (372)                                                                                          | 1/4 (744)                                                                                          | 1/2 (1488)                                                                                         |\n| -------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------- |\n| [Google Drive](https://drive.google.com/file/d/1IZb9cAKSHajAnWYFHDq_qDecKcJMXvl5/view?usp=sharing) | [Google Drive](https://drive.google.com/file/d/1QyG-qPdjaUZ2qEkQTr70ioLeNDOI943p/view?usp=sharing) | [Google Drive](https://drive.google.com/file/d/1xgIICfZw_5bf-FFH9GtdO_85kQCnQmsT/view?usp=sharing) | [Google Drive](https://drive.google.com/file/d/1IlN5IIUMO4pZ_b_9Xo6659AbUV0dllfM/view?usp=sharing) |\n| [Baidu Drive](https://pan.baidu.com/s/1DF9WTiV2mLfY2V9SHP14_w) \u003cbr\u003eFetch Code: rrpd                | [Baidu Drive](https://pan.baidu.com/s/1TjtHQM2tXy0H8b-reXMHJw) \u003cbr\u003eFetch Code: welw                | [Baidu Drive](https://pan.baidu.com/s/1LyecRU6rPbrrxgxy2qI65w) \u003cbr\u003eFetch Code: qwcd                | [Baidu Drive](https://pan.baidu.com/s/1ngb7mS0I6UMj1cff_40cYg) \u003cbr\u003eFetch Code: 4p8r                |\n\n\n\n## Installation\n\n```bash\ngit clone https://github.com/Haochen-Wang409/U2PL.git \u0026\u0026 cd U2PL\nconda create -n u2pl python=3.6.9\nconda activate u2pl\npip install -r requirements.txt\npip install pip install torch==1.8.1+cu102 torchvision==0.9.1+cu102 -f https://download.pytorch.org/whl/torch_stable.html\n```\n\n## Usage\n\nU\u003csup\u003e2\u003c/sup\u003ePL is evaluated on both Cityscapes and PASCAL VOC 2012 dataset.\n### Prepare Data\n\n\u003cdetails\u003e\n  \u003csummary\u003eFor Cityscapes\u003c/summary\u003e\n\nDownload \"leftImg8bit_trainvaltest.zip\" from: https://www.cityscapes-dataset.com/downloads/\n\nDownload \"gtFine.zip\" from: https://drive.google.com/file/d/10tdElaTscdhojER_Lf7XlytiyAkk7Wlg/view?usp=sharing\n\nNext, unzip the files to folder ```data``` and make the dictionary structures as follows:\n\n```angular2html\ndata/cityscapes\n├── gtFine\n│   ├── test\n│   ├── train\n│   └── val\n└── leftImg8bit\n    ├── test\n    ├── train\n    └── val\n```\n\n\u003c/details\u003e\n\n\n\u003cdetails\u003e\n  \u003csummary\u003eFor PASCAL VOC 2012\u003c/summary\u003e\n\nRefer to [this link](https://github.com/zhixuanli/segmentation-paper-reading-notes/blob/master/others/Summary%20of%20the%20semantic%20segmentation%20datasets.md) and download ```PASCAL VOC 2012 augmented with SBD``` dataset.\n\nAnd unzip the files to folder ```data``` and make the dictionary structures as follows:\n\n```angular2html\ndata/VOC2012\n├── Annotations\n├── ImageSets\n├── JPEGImages\n├── SegmentationClass\n├── SegmentationClassAug\n└── SegmentationObject\n```\n\u003c/details\u003e\n\nFinally, the structure of dictionary ```data``` should be as follows:\n\n```angular2html\ndata\n├── cityscapes\n│   ├── gtFine\n│   └── leftImg8bit\n├── splits\n│   ├── cityscapes\n│   └── pascal\n└── VOC2012\n    ├── Annotations\n    ├── ImageSets\n    ├── JPEGImages\n    ├── SegmentationClass\n    ├── SegmentationClassAug\n    └── SegmentationObject\n```\n\n### Prepare Pretrained Backbone\n\nBefore training, please download ResNet101 pretrained on ImageNet-1K from one of the following:\n  - [Google Drive](https://drive.google.com/file/d/1nzSX8bX3zoRREn6WnoEeAPbKYPPOa-3Y/view?usp=sharing)\n  - [Baidu Drive](https://pan.baidu.com/s/1FDQGlhjzQENfPp4HTYfbeA) Fetch Code: 3p9h\n\nAfter that, modify ```model_urls``` in ```semseg/models/resnet.py``` to ```\u003c/path/to/resnet101.pth\u003e```\n\n### Train a Fully-Supervised Model\n\nFor instance, we can train a model on PASCAL VOC 2012 with only ```1464``` labeled data for supervision by:\n```bash\ncd experiments/pascal/1464/suponly\n# use torch.distributed.launch\nsh train.sh \u003cnum_gpu\u003e \u003cport\u003e\n\n# or use slurm\n# sh slurm_train.sh \u003cnum_gpu\u003e \u003cport\u003e \u003cpartition\u003e\n```\nOr for Cityscapes, a model supervised by only ```744``` labeled data can be trained by:\n```bash\ncd experiments/cityscapes/744/suponly\n# use torch.distributed.launch\nsh train.sh \u003cnum_gpu\u003e \u003cport\u003e\n\n# or use slurm\n# sh slurm_train.sh \u003cnum_gpu\u003e \u003cport\u003e \u003cpartition\u003e\n```\nAfter training, the model should be evaluated by\n```bash\nsh eval.sh\n```\n### Train a Semi-Supervised Model\n\nWe can train a model on PASCAL VOC 2012 with ```1464``` labeled data and ```9118``` unlabeled data for supervision by:\n```bash\ncd experiments/pascal/1464/ours\n# use torch.distributed.launch\nsh train.sh \u003cnum_gpu\u003e \u003cport\u003e\n\n# or use slurm\n# sh slurm_train.sh \u003cnum_gpu\u003e \u003cport\u003e \u003cpartition\u003e\n```\nOr for Cityscapes, a model supervised by ```744``` labeled data and ```2231``` unlabeled data can be trained by:\n```bash\ncd experiments/cityscapes/744/ours\n# use torch.distributed.launch\nsh train.sh \u003cnum_gpu\u003e \u003cport\u003e\n\n# or use slurm\n# sh slurm_train.sh \u003cnum_gpu\u003e \u003cport\u003e \u003cpartition\u003e\n```\nAfter training, the model should be evaluated by\n```bash\nsh eval.sh\n```\n\n### Train a Semi-Supervised Model on Cityscapes with AEL\n\nFirst, you should switch the branch:\n```bash\ngit checkout with_AEL\n```\nThen, we can train a model supervised by ```744``` labeled data and ```2231``` unlabeled data by:\n```bash\ncd experiments/city_744\n# use torch.distributed.launch\nsh train.sh \u003cnum_gpu\u003e \u003cport\u003e\n\n# or use slurm\n# sh slurm_train.sh \u003cnum_gpu\u003e \u003cport\u003e \u003cpartition\u003e\n```\nAfter training, the model should be evaluated by\n```bash\nsh eval.sh\n```\n\n### Note\n```\u003cnum_gpu\u003e``` means the number of GPUs for training.\n\nTo reproduce our results, we recommend you follow the settings:\n- Cityscapes: ```4 * V100 (32G)``` for SupOnly and ```8 * V100 (32G)``` for Semi-Supervised\n- PASCAL VOC 2012: ```2 * V100 (32G)``` for SupOnly and ```4 * V100 (32G)``` for Semi-Supervised\n\nIf you got ```CUDA Out of Memory``` error, please try training our method in [fp16](https://github.com/NVIDIA/apex) mode.\nOr, change the ```lr``` in ```config.yaml``` in a linear manner (*e.g.*, if you want to train a SupOnly model on Cityscapes with 8 GPUs, \nyou are recommended to change the ```lr``` to ```0.02```).\n\nIf you want to train a model on other split, you need to modify ```data_list``` and ```n_sup``` in ```config.yaml```.\n\nDue to the randomness of function ```torch.nn.functional.interpolate``` when ```mode=\"bilinear\"```, \nthe results of semantic segmentation will not be the same EVEN IF a fixed random seed is set.\n\nTherefore, we recommend you run 3 times and get the average performance.\n\n## License\n\nThis project is released under the [Apache 2.0](LICENSE) license.\n\n## Acknowledgement\n\nThe contrastive learning loss and strong data augmentation (CutMix, CutOut, and ClassMix) \nare borrowed from **ReCo**.\nWe reproduce our U\u003csup\u003e2\u003c/sup\u003ePL based on **AEL** on branch ```with_AEL```.\n- ReCo: https://github.com/lorenmt/reco\n- AEL: https://github.com/hzhupku/SemiSeg-AEL\n\nThanks a lot for their great work!\n\n## Citation\n```bibtex\n@inproceedings{wang2022semi,\n    title={Semi-Supervised Semantic Segmentation Using Unreliable Pseudo Labels},\n    author={Wang, Yuchao and Wang, Haochen and Shen, Yujun and Fei, Jingjing and Li, Wei and Jin, Guoqiang and Wu, Liwei and Zhao, Rui and Le, Xinyi},\n    booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision and Pattern Recognition (CVPR)},\n    year={2022}\n}\n```\n\n## Contact\n\n- Yuchao Wang, 44442222@sjtu.edu.cn\n- Haochen Wang, wanghaochen2022@ia.ac.cn\n- Jingjing Fei, feijingjing1@sensetime.com\n- Wei Li, liwei1@sensetime.com\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FHaochen-Wang409%2FU2PL","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FHaochen-Wang409%2FU2PL","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FHaochen-Wang409%2FU2PL/lists"}