Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/DZhaoXd/SeCo
Semantic Connectivity-Driven Pseudo-labeling for Cross-domain Segmentation
https://github.com/DZhaoXd/SeCo
Last synced: about 1 month ago
JSON representation
Semantic Connectivity-Driven Pseudo-labeling for Cross-domain Segmentation
- Host: GitHub
- URL: https://github.com/DZhaoXd/SeCo
- Owner: DZhaoXd
- Created: 2023-12-08T21:45:55.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2023-12-12T11:17:54.000Z (about 1 year ago)
- Last Synced: 2023-12-13T11:53:26.489Z (about 1 year ago)
- Language: Python
- Size: 1.98 MB
- Stars: 5
- Watchers: 2
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
- Awesome-Segment-Anything - [code
README
# SeCo
Semantic Connectivity-Driven Pseudo-labeling for Cross-domain Segmentation.
:notebook_with_decorative_cover: For more information, please see our paper at **[`ARXIV`](http://arxiv.org/abs/2312.06331)**
## :speech_balloon: Pixel-driven PL. v.s. Connectivity-driven PL
## :speech_balloon: Pipeline
## :speech_balloon: Requirements
```
Python 3.8.0
pytorch 1.10.1
torchvision 0.11.2
einops 0.3.2
```
Please see `requirements.txt` for all the other requirements.
You can use PSA and SCC to obtain high-purity connectivity-based pseudo-labels.
These pseudo-labels can then be exploited and embedded into existing unsupervised domain adaptative semantic segmentation methods.
## :speech_balloon: Pixel Semantic Aggregation
First, you can obtain pixel-level pseudo-labels by pixel thresholding (e.g.[cbst](https://github.com/yzou2/cbst) ) from a UDA method (e.g. [ProDA](https://github.com/microsoft/ProDA) ) or a source-free UDA method (e.g. [DTST](https://github.com/DZhaoXd/DT-ST)), or a UDG method (e.g. [SHADE](https://github.com/HeliosZhao/SHADE) ).
And organize them in the following format.
```
"""
├─image
├─pixel-level pseudo-label
└─list
"""
list (XXX.txt) records the image names (XXX.png) and their corresponding pixel-level pseudo-labels.
```
Then, run the PSA as follows:
```
${exp_name}="HRDA_seco"
CUDA_VISIBLE_DEVICES="1" nohup python seco_sam.py --id-list-path ./splits/cityscapes/${exp_name}/all.txt --class-num ${class_name} > logs/${exp_name} 2>&1 &
```
Afterward, you can find the aggregated pseudo-labels in `root_path/${exp_name}_vit_{B/H}`.
## :speech_balloon: Semantic Connectivity Correction
After PSA, the noise is also amplified, and then you can use SCC to denoise the connected regions.
Refer to ([SCC](https://github.com/DZhaoXd/SeCo/tree/main/SCC) ) part for specific instructions.
## :speech_balloon: Embedded SeCo into UDA semantic segmentation
In the paper, we leverage the pseudo-labels generated by SeCo across multiple codebases.
Due to the diversity in code structures, we provide a unified implementation for utilizing SeCo's pseudo-labels.
This involves partitioning the unlabeled data into two subsets (one as a labeled subset and the other as an unlabeled subset) and employing a semi-supervised method ([Unimatch](https://github.com/LiheYoung/UniMatch) ) for further adaptation, as mentioned in the paper.
##
| Adaptation task | model | Before adaptation | After adaptation | Training logs |
|:-----------------------|:---------------|:-------------------------------------------------| :-------------------|:-------------------|
| GTA → Cityscapes | deeplab-r101 | 55.1 [DTST](https://github.com/DZhaoXd/DT-ST) | 64.6 |[`training_logs/seco_gta`](https://github.com/DZhaoXd/SeCo/blob/main/training_logs/seco_gta.log)|
| Synthia → Cityscapes | deeplab-r101 | 52.3 [DTST](https://github.com/DZhaoXd/DT-ST) | 59.2 |[`training_logs/seco_synthia`](https://github.com/DZhaoXd/SeCo/blob/main/training_logs/seco_synthia.log) |
| GTA → BDD100K | deeplab-r101 | 37.9 [SFOCDA](https://arxiv.org/abs/2106.03422) | 44.3 |[ `training_logs/seco_bdd`](https://github.com/DZhaoXd/SeCo/blob/main/training_logs/seco_bdd.log)|
This table shows the source-free setting, which is more competitive than the report results in Tables 1 and 2 in the paper.
More adaptation scripts will be updated later.
## :speech_balloon: License
Code is released for non-commercial and research purposes **only**. For commercial purposes, please contact the authors.
## Acknowledgement
Many thanks to those wonderful work and the open-source code.
- [Segment Anything](https://segment-anything.com/)
- [HuggingFace](https://huggingface.co/)
- [Segformer](https://arxiv.org/abs/2105.15203)
- [Semantic-Segment-Anything](https://github.com/fudan-zvg/Semantic-Segment-Anything)
- [ProDA](https://github.com/microsoft/ProDA)
- [BETA](https://github.com/xyupeng/BETA)
## :speech_balloon: Citation
If you use this code for your research, please cite our paper:
```bibtex
@misc{zhao2023semantic,
title={Semantic Connectivity-Driven Pseudo-labeling for Cross-domain Segmentation},
author={Dong Zhao and Ruizhi Yang and Shuang Wang and Qi Zang and Yang Hu and Licheng Jiao and Nicu Sebe and Zhun Zhong},
year={2023},
eprint={2312.06331},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
```