{"id":22068756,"url":"https://github.com/yasserben/CLOUDS","last_synced_at":"2025-07-24T06:32:21.680Z","repository":{"id":212697818,"uuid":"732080919","full_name":"yasserben/CLOUDS","owner":"yasserben","description":"[CVPR 2024] Official Implementation of Collaborating Foundation models for Domain Generalized Semantic Segmentation","archived":false,"fork":false,"pushed_at":"2024-07-05T12:01:33.000Z","size":1484,"stargazers_count":55,"open_issues_count":1,"forks_count":2,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-07-31T21:03:44.432Z","etag":null,"topics":["deep-learning","detectron2","domain-adaptation","domain-generalization","foundation-models","mask2former","semantic-segmentation","transformer"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2312.09788","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/yasserben.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2023-12-15T15:40:58.000Z","updated_at":"2024-07-05T12:01:37.000Z","dependencies_parsed_at":"2023-12-17T19:27:36.527Z","dependency_job_id":"437edafa-b802-4dae-ab93-bf6f3f598db5","html_url":"https://github.com/yasserben/CLOUDS","commit_stats":{"total_commits":4,"total_committers":1,"mean_commits":4.0,"dds":0.0,"last_synced_commit":"debfb851bbf27aba5c10b418bfbc1c56d4dd98d2"},"previous_names":["yasserben/clouds"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yasserben%2FCLOUDS","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yasserben%2FCLOUDS/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yasserben%2FCLOUDS/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yasserben%2FCLOUDS/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/yasserben","download_url":"https://codeload.github.com/yasserben/CLOUDS/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":227421325,"owners_count":17775010,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","detectron2","domain-adaptation","domain-generalization","foundation-models","mask2former","semantic-segmentation","transformer"],"created_at":"2024-11-30T20:04:19.816Z","updated_at":"2025-07-24T06:32:21.667Z","avatar_url":"https://github.com/yasserben.png","language":"Python","funding_links":[],"categories":["Paper List"],"sub_categories":["Follow-up Papers"],"readme":"# Collaborating Foundation models for Domain Generalized Semantic Segmentation\n\nThis repository contains the code for the paper: [Collaborating Foundation models for Domain Generalized Semantic Segmentation](https://arxiv.org/abs/2312.09788).\n\n## Overview\n\n**Domain Generalized Semantic Segmentation** (DGSS)\ndeals with training a model on a labeled source domain\nwith the aim of generalizing to unseen domains during inference.\nExisting DGSS methods typically effectuate robust\nfeatures by means of Domain Randomization (DR). Such an\napproach is often limited as it can only account for style\ndiversification and not content. In this work, we take an\northogonal approach to DGSS and propose to use an assembly of\n**C**o**L**laborative F**OU**ndation models for **D**omain\nGeneralized **S**emantic Segmentation (**CLOUDS**). In detail,\n**CLOUDS** is a framework that integrates FMs of various\nkinds: (i) CLIP backbone for its robust feature represen-\ntation, (ii) text-to-image generative models to diversify the\ncontent, thereby covering various modes of the possible target\ndistribution, and (iii) Segment Anything Model (SAM)\nfor iteratively refining the predictions of the segmentation\nmodel. Extensive experiments show that our CLOUDS excels in\nadapting from synthetic to real DGSS benchmarks\nand under varying weather conditions, notably outperforming\nprior methods by 5.6% and 6.7% on averaged mIoU,\nrespectively.\n\n\u003cimg src=\"imgs/main_figure.png\" width=\"1000\"\u003e\n\u003cdiv style=\"text-align: center;\"\u003e\n\u003c/div\u003e\n\n## Installation\n\nSee [installation instructions](INSTALL.md).\n\n## Getting Started\n\nSee [Preparing Datasets for CLOUDS](datasets/README.md).\n\nSee [Getting Started with CLOUDS](GETTING_STARTED.md).\n\n\n## Relevant Files :\n\n[train_net.py](train_net.py) : The training script of CLOUDS\n\n[clouds/clouds.py](clouds/clouds.py) : This file defines the model class and its forward function, which forms the\ncore of our model's architecture and forward pass logic\n\n[generate_txt_im.py](generate_txt_im.py) : The script to generate a dataset using Stable Diffusion\n\n[prompt_llama70b.txt](prompt_llama70b.txt) : The text file containing 100 generated prompts using Llama70b-Chat\n\n## Checkpoints \u0026 Generated dataset\n\nWe provide the following checkpoints for CLOUDS:\n\n* [Checkpoints](https://partage.imt.fr/index.php/s/NpFCf2meKB4MkQT)\n* [Generated dataset](https://partage.imt.fr/index.php/s/Hbazg5FetJjowJ4)\n\n## Citation \n\nIf you find our work useful in your research, please consider citing:\n```\n@InProceedings{Benigmim_2024_CVPR,\n    author    = {Benigmim, Yasser and Roy, Subhankar and Essid, Slim and Kalogeiton, Vicky and Lathuili\\`ere, St\\'ephane},\n    title     = {Collaborating Foundation Models for Domain Generalized Semantic Segmentation},\n    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},\n    month     = {June},\n    year      = {2024},\n    pages     = {3108-3119}\n}\n```\n\n## Acknowledgements\nCLOUDS draws its foundation from the following open-source projects, and we'd like to acknowledge their \nauthors for making their source code available :\n\n[FC-CLIP](https://github.com/bytedance/fc-clip)\n\n[Mask2Former](https://github.com/facebookresearch/Mask2Former)\n\n[HRDA](https://github.com/lhoyer/HRDA)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyasserben%2FCLOUDS","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyasserben%2FCLOUDS","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyasserben%2FCLOUDS/lists"}