{"id":18076127,"url":"https://github.com/layumi/seg-uncertainty","last_synced_at":"2025-04-05T17:07:09.513Z","repository":{"id":50497712,"uuid":"236876113","full_name":"layumi/Seg-Uncertainty","owner":"layumi","description":"IJCAI2020 \u0026 IJCV2021 :city_sunrise: Unsupervised Scene Adaptation with Memory Regularization in vivo ","archived":false,"fork":false,"pushed_at":"2024-01-19T15:57:35.000Z","size":5272,"stargazers_count":391,"open_issues_count":9,"forks_count":50,"subscribers_count":12,"default_branch":"master","last_synced_at":"2025-03-29T16:06:26.747Z","etag":null,"topics":["cityscapes","domain-adaptation","domainadaptation","gta5","ijcai","ijcai2020","ijcv","mrnet","pytorch","pytorch-implementation","robotcar","self-driving-car","semantic-segmentation","synthia","transfer-learning"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/1912.11164","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/layumi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-01-29T00:48:34.000Z","updated_at":"2025-03-29T08:37:45.000Z","dependencies_parsed_at":"2024-11-24T04:02:49.234Z","dependency_job_id":null,"html_url":"https://github.com/layumi/Seg-Uncertainty","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/layumi%2FSeg-Uncertainty","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/layumi%2FSeg-Uncertainty/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/layumi%2FSeg-Uncertainty/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/layumi%2FSeg-Uncertainty/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/layumi","download_url":"https://codeload.github.com/layumi/Seg-Uncertainty/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247369952,"owners_count":20927928,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cityscapes","domain-adaptation","domainadaptation","gta5","ijcai","ijcai2020","ijcv","mrnet","pytorch","pytorch-implementation","robotcar","self-driving-car","semantic-segmentation","synthia","transfer-learning"],"created_at":"2024-10-31T11:08:57.797Z","updated_at":"2025-04-05T17:07:09.492Z","avatar_url":"https://github.com/layumi.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"## Seg_Uncertainty\n![Python 3.6](https://img.shields.io/badge/python-3.6-green.svg)\n[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](https://opensource.org/licenses/MIT)\n\n![](https://github.com/layumi/Seg_Uncertainty/blob/master/Visual.jpg)\n\n[Zhedong Zheng](zdzheng.xyz), [Yi Yang](https://reler.net)\n\nIn this repo, we provide the code for the two papers, i.e., \n\n- MRNet：[Unsupervised Scene Adaptation with Memory Regularization in vivo](https://arxiv.org/pdf/1912.11164.pdf), IJCAI (2020)\n\n- MRNet+Rectifying: [Rectifying Pseudo Label Learning via Uncertainty Estimation for Domain Adaptive Semantic Segmentation](https://arxiv.org/pdf/2003.03773.pdf), IJCV (2021) [[中文介绍]](https://zhuanlan.zhihu.com/p/130220572) [[Poster]](https://zdzheng.xyz/files/valse_ijcv_poster.pdf)\n\n- [[中文介绍视频]](https://www.bilibili.com/video/BV14p4y1s77p) \n\n## Initial Model\nThe original DeepLab link of ucmerced is failed. Please use the following link.\n\n[Google Drive] https://drive.google.com/file/d/1BMTTMCNkV98pjZh_rU0Pp47zeVqF3MEc/view?usp=share_link \n\n[One Drive] https://1drv.ms/u/s!Avx-MJllNj5b3SqR7yurCxTgIUOK?e=A1dq3m\n\nor use \n```\npip install gdown\npip install --upgrade gdown\ngdown 1BMTTMCNkV98pjZh_rU0Pp47zeVqF3MEc\n```\n\n\n## Table of contents\n* [CommonQ\u0026A](#common-qa)\n* [The Core Code](#the-core-code)\n* [Prerequisites](#prerequisites)\n* [Prepare Data](#prepare-data)\n* [Training](#training)\n* [Testing](#testing)\n* [Trained Model](#trained-model)\n* [Related Works](#related-works)\n* [Citation](#citation)\n\n### News\n- [19 Jan 2024] We further apply the uncertainty to compositional image retrieval. The paper is accepted by ICLR'24 [[code]](https://github.com/Monoxide-Chen/uncertainty_retrieval).\n- [27 Jan 2023] You are welcomed to check our new transformer-based work [PiPa](https://github.com/chen742/PiPa), which achieves **75.6** mIoU on GTA5-\u003eCityscapes. \n- [5 Sep 2021] Zheng etal. apply the Uncertainty to domain adaptive reid, and also achieve good performance. \"Exploiting Sample Uncertainty for Domain Adaptive Person Re-Identification\" Kecheng Zheng, Cuiling Lan, Wenjun Zeng, Zhizheng Zhang, and Zheng-Jun Zha. AAAI 2021\n\n- [13 Aug 2021] We release one new method by Adaptive Boosting (AdaBoost) for Domain Adaptation. You may check the project at https://github.com/layumi/AdaBoost_Seg\n\n### Common Q\u0026A \n1. Why KLDivergence is always non-negative (\u003e=0)?\n\nPlease check the wikipedia at (https://en.wikipedia.org/wiki/Kullback–Leibler_divergence#Properties) . It provides one good demonstration. \n\n2. Why both log_sm and sm are used?\n\nYou may check the pytorch doc at https://pytorch.org/docs/stable/generated/torch.nn.KLDivLoss.html?highlight=nn%20kldivloss#torch.nn.KLDivLoss. \nI follow the discussion at https://discuss.pytorch.org/t/kl-divergence-loss/65393\n\n ### The Core Code\n Core code is relatively simple, and could be directly applied to other works. \n - Memory in vivo:  https://github.com/layumi/Seg_Uncertainty/blob/master/trainer_ms.py#L232\n\n - Recitfying Pseudo label:  https://github.com/layumi/Seg_Uncertainty/blob/master/trainer_ms_variance.py#L166\n \n### Prerequisites\n- Python 3.6\n- GPU Memory \u003e= 11G (e.g., GTX2080Ti or GTX1080Ti)\n- Pytorch or [Paddlepaddle](https://www.paddlepaddle.org.cn/)\n\n\n### Prepare Data\nDownload [GTA5] and [Cityscapes] to run the basic code.\nAlternatively, you could download extra two datasets from [SYNTHIA] and [OxfordRobotCar].\n\n- Download [The GTA5 Dataset]( https://download.visinf.tu-darmstadt.de/data/from_games/ )\n\n- Download [The SYNTHIA Dataset]( http://synthia-dataset.net/download/808/)  SYNTHIA-RAND-CITYSCAPES (CVPR16)\n\n- Download [The Cityscapes Dataset]( https://www.cityscapes-dataset.com/ )\n\n- Download [The Oxford RobotCar Dataset]( http://www.nec-labs.com/~mas/adapt-seg/adapt-seg.html )\n\n The data folder is structured as follows:\n ```\n ├── data/\n │   ├── Cityscapes/  \n |   |   ├── data/\n |   |       ├── gtFine/\n |   |       ├── leftImg8bit/\n │   ├── GTA5/\n |   |   ├── images/\n |   |   ├── labels/\n |   |   ├── ...\n │   ├── synthia/ \n |   |   ├── RGB/\n |   |   ├── GT/\n |   |   ├── Depth/\n |   |   ├── ...\n │   └── Oxford_Robot_ICCV19\n |   |   ├── train/\n |   |   ├── ...\n ```\n\n ### Training \n Stage-I:\n ```bash\n python train_ms.py --snapshot-dir ./snapshots/SE_GN_batchsize2_1024x512_pp_ms_me0_classbalance7_kl0.1_lr2_drop0.1_seg0.5  --drop 0.1 --warm-up 5000 --batch-size 2 --learning-rate 2e-4 --crop-size 1024,512 --lambda-seg 0.5  --lambda-adv-target1 0.0002 --lambda-adv-target2 0.001   --lambda-me-target 0  --lambda-kl-target 0.1  --norm-style gn  --class-balance  --only-hard-label 80  --max-value 7  --gpu-ids 0,1  --often-balance  --use-se  \n ```\n\n Generate Pseudo Label:\n ```bash\n python generate_plabel_cityscapes.py  --restore-from ./snapshots/SE_GN_batchsize2_1024x512_pp_ms_me0_classbalance7_kl0.1_lr2_drop0.1_seg0.5/GTA5_25000.pth\n ```\n\n Stage-II (with recitfying pseudo label):\n ```bash\n python train_ft.py --snapshot-dir ./snapshots/1280x640_restore_ft_GN_batchsize9_512x256_pp_ms_me0_classbalance7_kl0_lr1_drop0.2_seg0.5_BN_80_255_0.8_Noaug --restore-from ./snapshots/SE_GN_batchsize2_1024x512_pp_ms_me0_classbalance7_kl0.1_lr2_drop0.1_seg0.5/GTA5_25000.pth --drop 0.2 --warm-up 5000 --batch-size 9 --learning-rate 1e-4 --crop-size 512,256 --lambda-seg 0.5 --lambda-adv-target1 0 --lambda-adv-target2 0 --lambda-me-target 0 --lambda-kl-target 0 --norm-style gn --class-balance --only-hard-label 80 --max-value 7 --gpu-ids 0,1,2 --often-balance  --use-se  --input-size 1280,640  --train_bn  --autoaug False\n ```\n *** If you want to run the code without rectifying pseudo label, please change [[this line]](https://github.com/layumi/Seg_Uncertainty/blob/master/train_ft.py#L20) to 'from trainer_ms import AD_Trainer', which would apply the conventional pseudo label learning. ***\n\n ### Testing\n ```bash\n python evaluate_cityscapes.py --restore-from ./snapshots/1280x640_restore_ft_GN_batchsize9_512x256_pp_ms_me0_classbalance7_kl0_lr1_drop0.2_seg0.5_BN_80_255_0.8_Noaug/GTA5_25000.pth\n ```\n\n ### Trained Model\n The trained model is available at https://drive.google.com/file/d/1smh1sbOutJwhrfK8dk-tNvonc0HLaSsw/view?usp=sharing\n\n - The folder with `SY` in name is for SYNTHIA-to-Cityscapes\n - The folder with `RB` in name is for Cityscapes-to-Robot Car\n\n ### One Note for SYNTHIA-to-Cityscapes\n Note that the evaluation code I provided for SYNTHIA-to-Cityscapes is still average the IoU by divide 19.\n Actually, you need to re-calculate the value by divide 16. There are only 16 shared classes for SYNTHIA-to-Cityscapes. \n In this way, the result is same as the value reported in paper.\n\n ### Related Works\n We also would like to thank great works as follows:\n - https://github.com/wasidennis/AdaptSegNet\n - https://github.com/RoyalVane/CLAN\n - https://github.com/yzou2/CRST\n\n ### Citation\n ```bibtex\n @inproceedings{zheng2020unsupervised,\n   title={Unsupervised Scene Adaptation with Memory Regularization in vivo},\n   author={Zheng, Zhedong and Yang, Yi},\n   booktitle={IJCAI},\n   year={2020}\n }\n @article{zheng2021rectifying,\n   title={Rectifying Pseudo Label Learning via Uncertainty Estimation for Domain Adaptive Semantic Segmentation },\n   author={Zheng, Zhedong and Yang, Yi},\n   journal={International Journal of Computer Vision (IJCV)},\n   doi={10.1007/s11263-020-01395-y},\n   note={\\mbox{doi}:\\url{10.1007/s11263-020-01395-y}},\n   year={2021}\n }\n ```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flayumi%2Fseg-uncertainty","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flayumi%2Fseg-uncertainty","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flayumi%2Fseg-uncertainty/lists"}