{"id":15418803,"url":"https://github.com/robustsam/robustsam","last_synced_at":"2025-10-11T08:31:11.027Z","repository":{"id":244635523,"uuid":"772721093","full_name":"robustsam/RobustSAM","owner":"robustsam","description":"RobustSAM: Segment Anything Robustly on Degraded Images (CVPR 2024 Highlight)","archived":false,"fork":false,"pushed_at":"2024-08-31T15:08:14.000Z","size":17125,"stargazers_count":359,"open_issues_count":16,"forks_count":31,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-09-23T11:50:03.650Z","etag":null,"topics":["artificial-intelligence","computer-vision","cvpr","cvpr2024","deep-learning","eccv","iccv","image-processing","sam","segment-anything","segment-anything-meta","segment-anything-model","segmentation","zero-shot-segmentation"],"latest_commit_sha":null,"homepage":"https://robustsam.github.io/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/robustsam.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-03-15T18:54:49.000Z","updated_at":"2025-09-18T00:55:04.000Z","dependencies_parsed_at":"2024-06-16T09:48:34.020Z","dependency_job_id":"2b9cb22a-ede1-424c-9dc2-a4e48a800189","html_url":"https://github.com/robustsam/RobustSAM","commit_stats":{"total_commits":71,"total_committers":2,"mean_commits":35.5,"dds":"0.028169014084507005","last_synced_commit":"6f0e2637e58f812985cdf9f403a5d7515a66fe38"},"previous_names":["robustsam/robustsam"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/robustsam/RobustSAM","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/robustsam%2FRobustSAM","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/robustsam%2FRobustSAM/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/robustsam%2FRobustSAM/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/robustsam%2FRobustSAM/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/robustsam","download_url":"https://codeload.github.com/robustsam/RobustSAM/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/robustsam%2FRobustSAM/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279006611,"owners_count":26084148,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-11T02:00:06.511Z","response_time":55,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["artificial-intelligence","computer-vision","cvpr","cvpr2024","deep-learning","eccv","iccv","image-processing","sam","segment-anything","segment-anything-meta","segment-anything-model","segmentation","zero-shot-segmentation"],"created_at":"2024-10-01T17:22:38.550Z","updated_at":"2025-10-11T08:31:06.008Z","avatar_url":"https://github.com/robustsam.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# RobustSAM: Segment Anything Robustly on Degraded Images (CVPR 2024 Highlight)\n\n\u003ca href=\"https://colab.research.google.com/drive/1mrOjUNFrfZ2vuTnWrfl9ebAQov3a9S6E?usp=sharing\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"\u003e\u003c/a\u003e\n[![Huggingfaces](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Models-blue)](https://huggingface.co/robustsam/robustsam/tree/main)\n\nOfficial repository for RobustSAM: Segment Anything Robustly on Degraded Images\n\n[Project Page](https://robustsam.github.io/) | [Paper](https://openaccess.thecvf.com/content/CVPR2024/html/Chen_RobustSAM_Segment_Anything_Robustly_on_Degraded_Images_CVPR_2024_paper.html) | [Video](https://www.youtube.com/watch?v=Awukqkbs6zM) | [Dataset](https://huggingface.co/robustsam/robustsam/tree/main/dataset)\n\n## HF Model Cards\n\n[![RobustSAM Base](https://img.shields.io/badge/%F0%9F%A4%97%20Base-blue)](https://huggingface.co/jadechoghari/robustsam-vit-base)\n[![RobustSAM Large](https://img.shields.io/badge/%F0%9F%A4%97%20Large-blue)](https://huggingface.co/jadechoghari/robustsam-vit-large)\n[![RobustSAM Huge](https://img.shields.io/badge/%F0%9F%A4%97%20Huge-blue)](https://huggingface.co/jadechoghari/robustsam-vit-huge)\n\n\n## Updates\n- Aug 2024: ✨ You can refer to the Hugging Face model cards \u0026 demo built by [@jadechoghari](https://github.com/jadechoghari) for easier usage via this [link](https://github.com/robustsam/RobustSAM/issues/10#issuecomment-2294327808).\n- July 2024: ✨ Training code, data and model checkpoints for different ViT backbones are released!\n- June 2024: ✨ Inference code has been released!\n- Feb 2024: ✨ RobustSAM was accepted into CVPR 2024!\n\n\n## Introduction\nSegment Anything Model (SAM) has emerged as a transformative approach in image segmentation, acclaimed for its robust zero-shot segmentation capabilities and flexible prompting system. Nonetheless, its performance is challenged by images with degraded quality. Addressing this limitation, we propose the Robust Segment Anything Model (RobustSAM), which enhances SAM's performance on low-quality images while preserving its promptability and zero-shot generalization.\n\nOur method leverages the pre-trained SAM model with only marginal parameter increments and computational requirements. The additional parameters of RobustSAM can be optimized within 30 hours on eight GPUs, demonstrating its feasibility and practicality for typical research laboratories. We also introduce the Robust-Seg dataset, a collection of 688K image-mask pairs with different degradations designed to train and evaluate our model optimally. Extensive experiments across various segmentation tasks and datasets confirm RobustSAM's superior performance, especially under zero-shot conditions, underscoring its potential for extensive real-world application. Additionally, our method has been shown to effectively improve the performance of SAM-based downstream tasks such as single image dehazing and deblurring.\n\n\u003cimg width=\"1096\" alt=\"image\" src='figures/architecture.jpg'\u003e\n\n## Setup\n1) Create a conda environment and activate it.\n```\nconda create --name robustsam python=3.10 -y\nconda activate robustsam\n```\n2) Clone and enter into repo directory.\n```\ngit clone https://github.com/robustsam/RobustSAM\ncd RobustSAM\n```\n3) Use command below to check your CUDA version.\n```\nnvidia-smi\n```\n4) Replace the CUDA version with yours in command below.\n```\npip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu[$YOUR_CUDA_VERSION]\n# For example: pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu117 # cu117 = CUDA_version_11.7\n```\n5) Install remaining dependencies\n```\npip install -r requirements.txt\n```\n\n6) Download pretrained RobustSAM checkpoints of different sizes and place them into current directory.\n   - [ViT-B RobustSAM checkpoint](https://huggingface.co/robustsam/robustsam/resolve/main/model_checkpoint/robustsam_checkpoint_b.pth)\n   - [ViT-L RobustSAM checkpoint](https://huggingface.co/robustsam/robustsam/resolve/main/model_checkpoint/robustsam_checkpoint_l.pth)\n   - [ViT-H RobustSAM checkpoint](https://huggingface.co/robustsam/robustsam/resolve/main/model_checkpoint/robustsam_checkpoint_h.pth)\n\n## Data Preparation\n1) Change current directory to \"data\" directory.\n```\ncd data\n```\n2) Download train, val, test and extra COCO \u0026 LVIS dataset. (NOTE: Images in train, val and test dataset consist images from [LVIS](https://paperswithcode.com/paper/lvis-a-dataset-for-large-vocabulary-instance-1), [MSRA10K](https://paperswithcode.com/dataset/msra10k), [ThinObject-5k](https://github.com/liewjunhao/thin-object-selection), [NDD20](https://arxiv.org/abs/2005.13359), [STREETS](https://paperswithcode.com/dataset/streets) and [FSS-1000](https://github.com/HKUSTCV/FSS-1000))\n```\nbash download.sh\n```\n3) There are only clear images downloaded in the previous step. Use command below to generate corresponding degraded images. \n```\nbash gen_data.sh\n```\n\n## Training\n1) If you want to train from scratch, use command below.\n```\npython -m torch.distributed.launch train_ddp.py  --multiprocessing-distributed --exp_name [$YOUR_EXP_NAME] --model_size [$MODEL_SIZE]\n# Example usage: python -m torch.distributed.launch train_ddp.py  --multiprocessing-distributed --exp_name test --model_size l\n```\n2) If you want to train from pretrained checkpoint, use command below.\n```\npython -m torch.distributed.launch train_ddp.py  --multiprocessing-distributed --exp_name [$YOUR_EXP_NAME] --model_size [$MODEL_SIZE] --load_model [$CHECKPOINT_PATH]\n# Example usage: python -m torch.distributed.launch train_ddp.py  --multiprocessing-distributed --exp_name test --model_size l --load_model robustsam_checkpoint_l.pth\n```\n\n### Local Gradio App\n\n```\npython gradio_app.py \n```\n\n## Demo\nWe have prepared some images im **demo_images** folder for demo purpose. Besides, two prompting modes are available (box prompts and point prompts).\n- For box prompt:\n```\npython eval.py --bbox --model_size l\n```\n- For point prompt:\n```\npython eval.py --model_size l\n```\nIn default, demo results will be saved to **demo_result/[$PROMPT_TYPE]**.\n\n## Comparison of computational requirements\n\u003cimg width=\"720\" alt=\"image\" src='figures/Computational requirements.PNG'\u003e\n\n## Visual Comparison\n\u003ctable\u003e\n  \u003ctr\u003e\n    \u003ctd\u003e\n      \u003cimg src=\"figures/gif_output/blur_back_n_forth.gif\" width=\"380\"\u003e\n    \u003c/td\u003e\n    \u003ctd\u003e\n      \u003cimg src=\"figures/gif_output/haze_back_n_forth.gif\" width=\"380\"\u003e\n    \u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003e\n      \u003cimg src=\"figures/gif_output/lowlight_back_n_forth.gif\" width=\"380\"\u003e\n    \u003c/td\u003e\n    \u003ctd\u003e\n      \u003cimg src=\"figures/gif_output/rain_back_n_forth.gif\" width=\"380\"\u003e\n    \u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n\u003cimg width=\"1096\" alt=\"image\" src='figures/qualitative_result.PNG'\u003e\n\n## Quantitative Comparison\n### Seen dataset with synthetic degradation\n\u003cimg width=\"720\" alt=\"image\" src='figures/seen_dataset_with_synthetic_degradation.PNG'\u003e\n\n### Unseen dataset with synthetic degradation\n\u003cimg width=\"720\" alt=\"image\" src='figures/unseen_dataset_with_synthetic_degradation.PNG'\u003e\n\n### Unseen dataset with real degradation\n\u003cimg width=\"600\" alt=\"image\" src='figures/unseen_dataset_with_real_degradation.PNG'\u003e\n\n## Reference\nIf you find this work useful, please consider citing us!\n```python\n@inproceedings{chen2024robustsam,\n  title={RobustSAM: Segment Anything Robustly on Degraded Images},\n  author={Chen, Wei-Ting and Vong, Yu-Jiet and Kuo, Sy-Yen and Ma, Sizhou and Wang, Jian},\n  journal={CVPR},\n  year={2024}\n}\n```\n\n\n## Acknowledgements\nWe thank the authors of [SAM](https://github.com/facebookresearch/segment-anything) from which our repo is based off of.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frobustsam%2Frobustsam","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frobustsam%2Frobustsam","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frobustsam%2Frobustsam/lists"}