{"id":49927849,"url":"https://github.com/brdav/cma","last_synced_at":"2026-05-17T01:06:42.143Z","repository":{"id":187183263,"uuid":"611761588","full_name":"brdav/cma","owner":"brdav","description":"Contrastive Model Adaptation for Cross-Condition Robustness in Semantic Segmentation [ICCV 2023]","archived":false,"fork":false,"pushed_at":"2023-08-30T13:42:52.000Z","size":487,"stargazers_count":41,"open_issues_count":0,"forks_count":5,"subscribers_count":5,"default_branch":"main","last_synced_at":"2024-05-15T09:40:19.564Z","etag":null,"topics":["computer-vision","domain-adaptation","semantic-segmentation"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/brdav.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-03-09T13:47:59.000Z","updated_at":"2024-04-15T08:58:52.000Z","dependencies_parsed_at":"2023-08-09T10:10:03.989Z","dependency_job_id":null,"html_url":"https://github.com/brdav/cma","commit_stats":null,"previous_names":["brdav/cma"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/brdav/cma","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brdav%2Fcma","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brdav%2Fcma/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brdav%2Fcma/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brdav%2Fcma/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/brdav","download_url":"https://codeload.github.com/brdav/cma/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brdav%2Fcma/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33124143,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-16T18:38:32.183Z","status":"ssl_error","status_checked_at":"2026-05-16T18:38:29.903Z","response_time":115,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["computer-vision","domain-adaptation","semantic-segmentation"],"created_at":"2026-05-17T01:06:41.458Z","updated_at":"2026-05-17T01:06:42.119Z","avatar_url":"https://github.com/brdav.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":" ---\n\n\u003cdiv align=\"center\"\u003e    \n \n# Contrastive Model Adaptation for Cross-Condition Robustness in Semantic Segmentation\n\n[![Paper](http://img.shields.io/badge/paper-arxiv.2303.05194-B31B1B.svg)](https://arxiv.org/abs/2303.05194)\n[![Conference](http://img.shields.io/badge/ICCV-2023-4b44ce.svg)](https://iccv2023.thecvf.com/) \n\n\u003c/div\u003e\n\nOfficial code for the ICCV 2023 paper [Contrastive Model Adaptation for Cross-Condition Robustness in Semantic Segmentation](https://arxiv.org/abs/2303.05194). The code is organized using [PyTorch Lightning](https://github.com/Lightning-AI/lightning). \n\n\u003cimg src=\"./docs/method.png\" width=\"900\"/\u003e\n\n## Abstract\n\nStandard unsupervised domain adaptation methods adapt models from a source to a target domain using labeled source data and unlabeled target data jointly. In model adaptation, on the other hand, access to the labeled source data is prohibited, i.e., only the source-trained model and unlabeled target data are available. We investigate normal-to-adverse condition model adaptation for semantic segmentation, whereby image-level correspondences are available in the target domain. The target set consists of unlabeled pairs of adverse- and normal-condition street images taken at GPS-matched locations. Our method--CMA--leverages such image pairs to learn condition-invariant features via contrastive learning. In particular, CMA encourages features in the embedding space to be grouped according to their condition-invariant semantic content and not according to the condition under which respective inputs are captured. To obtain accurate cross-domain semantic correspondences, we warp the normal image to the viewpoint of the adverse image and leverage warp-confidence scores to create robust, aggregated features. With this approach, we achieve state-of-the-art semantic segmentation performance for model adaptation on several normal-to-adverse adaptation benchmarks, such as ACDC and Dark Zurich. We also evaluate CMA on a newly procured adverse-condition generalization benchmark and report favorable results compared to standard unsupervised domain adaptation methods, despite the comparative handicap of CMA due to source data inaccessibility.\n\n\n## Usage\n### Requirements\n\nThe code is run with Python 3.10.4. To install the packages, use:\n```bash\npip install -r requirements.txt\n```\n\n### Optional\n\nLocal correlation is implemented through [this custom CUDA extension](https://github.com/ClementPinard/Pytorch-Correlation-extension). By default, the extension is built just in time using Ninja. In case of problems, the extension can be alternatively pre-installed in the environment (see also the README of the linked repo):\n```bash\npip install spatial-correlation-sampler\n```\n\n\n### Set Data Directory\n\nThe following environment variable must be set:\n```bash\nexport DATA_DIR=/path/to/data/dir\n```\n\n### Download the Data\n\nBefore running the code, download and extract the respective datasets to the directory `$DATA_DIR`.\n\n\u003cdetails\u003e\n  \u003csummary\u003eACDC\u003c/summary\u003e\n  \n  Download rgb_anon_trainvaltest.zip and gt_trainval.zip from [here](https://acdc.vision.ee.ethz.ch/download) and extract them to `$DATA_DIR/ACDC`.\n\n  ```\n  $DATA_DIR\n  ├── ACDC\n  │   ├── rgb_anon\n  │   │   ├── fog\n  │   │   ├── night\n  │   │   ├── rain\n  │   │   ├── snow\n  │   ├── gt\n  │   │   ├── fog\n  │   │   ├── night\n  │   │   ├── rain\n  │   │   ├── snow\n  ├── ...\n  ```\n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003eDark Zurich\u003c/summary\u003e\n  \n  Download Dark_Zurich_train_anon.zip, Dark_Zurich_val_anon.zip, and Dark_Zurich_test_anon_withoutGt.zip from [here](https://www.trace.ethz.ch/publications/2019/GCMA_UIoU/) and extract them to `$DATA_DIR/DarkZurich`.\n\n  ```\n  $DATA_DIR\n  ├── DarkZurich\n  │   ├── rgb_anon\n  │   │   ├── train\n  │   │   ├── val\n  │   │   ├── val_ref\n  │   │   ├── test\n  │   │   ├── test_ref\n  │   ├── gt\n  │   │   ├── val\n  ├── ...\n  ```\n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003eRobotCar\u003c/summary\u003e\n  \n  Download all data from [here](https://data.ciirc.cvut.cz/public/projects/2020VisualLocalization/Cross-Seasons-Correspondence/ROBOTCAR/) and save them to `$DATA_DIR/RobotCar`. As mentioned in the corresponding README.txt, the images must be downloaded from [this link](https://drive.google.com/drive/folders/19yUB49EliCnWThuN2HUukIryX47JWmQp).\n\n  ```\n  $DATA_DIR\n  ├── RobotCar\n  │   ├── images\n  │   │   ├── dawn\n  │   │   ├── dusk\n  │   │   ├── night\n  │   │   ├── night-rain\n  │   │   ├── ...\n  │   ├── correspondence_data\n  │   │   ├── ...\n  │   ├── segmented_images\n  │   │   ├── training\n  │   │   ├── validation\n  │   │   ├── testing\n  ├── ...\n  ```\n\u003c/details\u003e\n\n\n### Download the Pretrained Weights\n\nThe Cityscapes-pretrained SegFormer weights (`segformer.b5.1024x1024.city.160k.pth`) are required for CMA. Download them from the [SegFormer repository](https://github.com/NVlabs/SegFormer) and save them to `./pretrained_models/`.\n\n\n### Model Checkpoints and Results\n\nWe provide the following model checkpoints and validation set predictions:\n\n| Method         | Architecture           | Dataset       | Test mIoU    | Config      |  Checkpoint    |   Predictions  |\n|---------------|----------------|-----------------|-----------------|------------|----------------|------------|\n| CMA | SegFormer | ACDC | 69.1 | [config](https://github.com/brdav/cma/blob/main/configs/cma_segformer_acdc.yaml) | [model](https://www.research-collection.ethz.ch/bitstream/handle/20.500.11850/626144/cma_segformer_acdc.ckpt) | [ACDC val](https://www.research-collection.ethz.ch/bitstream/handle/20.500.11850/626144/colored_preds_val_ACDC_cma_segformer.zip) \n| CMA | DeepLabv2 | ACDC | 50.4 | [config](https://github.com/brdav/cma/blob/main/configs/cma_deeplabv2_acdc.yaml) | [model](https://www.research-collection.ethz.ch/bitstream/handle/20.500.11850/626144/cma_deeplabv2_acdc.ckpt) | [ACDC val](https://www.research-collection.ethz.ch/bitstream/handle/20.500.11850/626144/colored_preds_val_ACDC_cma_deeplabv2.zip) \n|||||||\n| CMA | SegFormer | Dark Zurich | 53.6 | [config](https://github.com/brdav/cma/blob/main/configs/cma_segformer_darkzurich.yaml) | [model](https://www.research-collection.ethz.ch/bitstream/handle/20.500.11850/626144/cma_segformer_darkzurich.ckpt) | [Dark Zurich val](https://www.research-collection.ethz.ch/bitstream/handle/20.500.11850/626144/colored_preds_val_DarkZurich_cma_segformer.zip) \n|||||||\n| CMA | SegFormer | RobotCar | 54.3 | [config](https://github.com/brdav/cma/blob/main/configs/cma_segformer_robotcar.yaml) | [model](https://www.research-collection.ethz.ch/bitstream/handle/20.500.11850/626144/cma_segformer_robotcar.ckpt) | [RobotCar val](https://www.research-collection.ethz.ch/bitstream/handle/20.500.11850/626144/colored_preds_val_RobotCar_cma_segformer.zip) \n\n\n### Create Pseudo-Labels (Optional)\n\nBefore training CMA, optionally create pseudo-labels using the source model. For example for a SegFormer architecture on ACDC:\n```bash\npython -m tools.run generate_pl --config configs/cma_segformer_acdc.yaml --trainer.accelerator gpu\n```\nThis will save the pseudo-labels to `$DATA_DIR/pseudo_labels`. If this step is skipped, precomputed pseudo-labels will be automatically downloaded to `$DATA_DIR/pseudo_labels` on the first training run.\n\n\n### Training\n\nTo train CMA (with AMP), e.g. for a SegFormer architecture on ACDC, use the following command:\n```bash\npython -m tools.run fit --config configs/cma_segformer_acdc.yaml --trainer.accelerator gpu --trainer.precision 16\n```\nNote that a GPU with around 20 GB memory is needed to train CMA. See `configs/` for config files for other datasets and architectures. See the [Lightning CLI Docs](https://pytorch-lightning.readthedocs.io/en/1.9.3/cli/lightning_cli.html) for more information on how to control hyperparameters etc.\n\n\n### Testing\n\nTo evaluate CMA, provide the model checkpoint as argument, e.g. for a SegFormer architecture on RobotCar:\n```bash\npython -m tools.run test --config configs/cma_segformer_robotcar.yaml --trainer.accelerator gpu --ckpt_path /path/to/checkpoint.ckpt\n```\n\nFor ACDC and Dark Zurich, this command would compute the performance on the validation set. To get test set scores, predictions are evaluated on the respective evaluation servers: [ACDC](https://acdc.vision.ee.ethz.ch/submit) and [Dark Zurich](https://codalab.lisn.upsaclay.fr/competitions/3783).\nTo create and save test predictions for e.g. ACDC, use this command:\n```bash\npython -m tools.run predict --config configs/cma_segformer_acdc.yaml --trainer.accelerator gpu --ckpt_path /path/to/checkpoint.ckpt\n```\n\n## ACG Benchmark\n\nTo evaluate a model on the ACG benchmark, first download the filename lists and instructions here: [ACG Benchmark](https://www.research-collection.ethz.ch/bitstream/handle/20.500.11850/626144/ACG.zip)\n\nSee the README file of the downloaded ACG bundle for details on how to retrieve and arrange the necessary datasets. In summary, the file structure should look as follows:\n\n```\n$DATA_DIR\n├── ACG\n│   ├── ...\n├── WildDash2\n│   ├── ...\n├── bdd100k\n│   ├── ...\n├── Foggy_Driving\n│   ├── ...\n├── Foggy_Zurich\n│   ├── ...\n```\n\nBefore running the evaluation, uncomment the respective lines in the config file (in the dataloader and the metrics settings). Then test the model as usual:\n```bash\npython -m tools.run test --config configs/cma_segformer_acdc.yaml --trainer.accelerator gpu --ckpt_path /path/to/checkpoint.ckpt\n```\n\n## Citation\n\nIf you find this code useful in your research, please consider citing the paper:\n```bibtex\n@inproceedings{bruggemann2023contrastive,\n  title={Contrastive Model Adaptation for Cross-Condition Robustness in Semantic Segmentation},\n  author={Bruggemann, David and Sakaridis, Christos and Broedermann, Tim and Van Gool, Luc},\n  booktitle={ICCV},\n  year={2023}\n}\n```\n\n## License\n\nThis repository is released under the MIT license. However, care should be taken to adopt appropriate licensing for third-party code in this repository. Third-party code is marked accordingly.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbrdav%2Fcma","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbrdav%2Fcma","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbrdav%2Fcma/lists"}