{"id":13442210,"url":"https://github.com/tianrun-chen/SAM-Adapter-PyTorch","last_synced_at":"2025-03-20T13:32:57.394Z","repository":{"id":153496380,"uuid":"628028132","full_name":"tianrun-chen/SAM-Adapter-PyTorch","owner":"tianrun-chen","description":"Adapting Meta AI's Segment Anything to Downstream Tasks with Adapters and Prompts","archived":false,"fork":false,"pushed_at":"2024-12-25T03:46:44.000Z","size":8037,"stargazers_count":1080,"open_issues_count":39,"forks_count":98,"subscribers_count":8,"default_branch":"main","last_synced_at":"2024-12-25T04:24:54.800Z","etag":null,"topics":["2d-segmentation","adapter","camouflage-images","camouflaged-object-detection","camouflaged-target-detection","fine-tune","fine-tuning","image-segmentation","image-segmentation-pytorch","segment-anything","segment-anything-model"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tianrun-chen.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-04-14T18:33:17.000Z","updated_at":"2024-12-24T02:35:10.000Z","dependencies_parsed_at":"2023-05-19T20:30:45.519Z","dependency_job_id":"425b1bf3-a2f7-4f35-8ad7-078942ac762f","html_url":"https://github.com/tianrun-chen/SAM-Adapter-PyTorch","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tianrun-chen%2FSAM-Adapter-PyTorch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tianrun-chen%2FSAM-Adapter-PyTorch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tianrun-chen%2FSAM-Adapter-PyTorch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tianrun-chen%2FSAM-Adapter-PyTorch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tianrun-chen","download_url":"https://codeload.github.com/tianrun-chen/SAM-Adapter-PyTorch/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244619261,"owners_count":20482388,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["2d-segmentation","adapter","camouflage-images","camouflaged-object-detection","camouflaged-target-detection","fine-tune","fine-tuning","image-segmentation","image-segmentation-pytorch","segment-anything","segment-anything-model"],"created_at":"2024-07-31T03:01:42.990Z","updated_at":"2025-03-20T13:32:57.380Z","avatar_url":"https://github.com/tianrun-chen.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"## SAM-Adapter: Adapting SAM in Underperformed Scenes (!!Now Support SAM2 in \"SAM2-Adapter\" Branch!!)\n\nTianrun Chen, Lanyun Zhu, Chaotao Ding, Runlong Cao, Yan Wang, Shangzhan Zhang, Zejian Li, Lingyun Sun, Papa Mao, Ying Zang\n\n\u003ca href='https://www.kokoni3d.com/'\u003e KOKONI, Moxin Technology (Huzhou) Co., LTD \u003c/a\u003e, Zhejiang University, Singapore University of Technology and Design, Huzhou University, Beihang University.\n\nIn Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 3367-3375).\n\n  \u003ca href='https://tianrun-chen.github.io/SAM-Adaptor/'\u003e\u003cimg src='https://img.shields.io/badge/Project-Page-Green'\u003e\u003c/a\u003e\n## \n\n\u003ca href='https://arxiv.org/abs/2304.09148'\u003e\u003cimg src='https://img.shields.io/badge/ArXiv-2304.09148-red' /\u003e\u003c/a\u003e \n\nUpdate on 8 Aug, 2024: We add support for adapting with SAM2 (Segment Anything 2), a more powerful backbone! Please refer our \u003ca href=\"https://www.researchgate.net/publication/382940773_SAM2-Adapter_Evaluating_Adapting_Segment_Anything_2_in_Downstream_Tasks_Camouflage_Shadow_Medical_Image_Segmentation_and_More\"\u003enew technical report! \u003c/a\u003eand see the code at \"SAM2-Adapter\" Branch!\n\nUpdate on 24 July, 2024: The link of pre-trained model is updated.\n\nUpdate on 30 August 2023: This paper will be prsented at ICCV 2023. \n\nUpdate on 28 April 2023: We tested the performance of polyp segmentation to show our approach can also work on medical datasets.\n\u003cimg src='https://tianrun-chen.github.io/SAM-Adaptor/static/images/polyp.jpg'\u003e\nUpdate on 22 April: We report our SOTA result based on ViT-H version of SAM (use demo.yaml). We have also uploaded the yaml config for ViT-L and ViT-B version of SAM, suitable  GPU with smaller memory (e.g. NVIDIA Tesla V-100), although they may compromise on accuracy.\n\n## Environment\nThis code was implemented with Python 3.8 and PyTorch 1.13.0. You can install all the requirements via:\n```bash\npip install -r requirements.txt\n```\n\n\n## Quick Start\n1. Download the dataset and put it in ./load.\n2. Download the pre-trained [SAM(Segment Anything)](https://github.com/facebookresearch/segment-anything) and put it in ./pretrained.\n3. Training:\n```bash\nCUDA_VISIBLE_DEVICES=0,1,2,3 python -m torch.distributed.launch --nnodes 1 --nproc_per_node 4 loadddptrain.py --config configs/demo.yaml\n```\n!Please note that the SAM model consume much memory. We use 4 x A100 graphics card for training. If you encounter the memory issue, please try to use graphics cards with larger memory!\n\n\n4. Evaluation:\n```bash\npython test.py --config [CONFIG_PATH] --model [MODEL_PATH]\n```\n\n## Train\n```bash\nCUDA_VISIBLE_DEVICES=0,1,2,3 python -m torch.distributed.launch train.py --nnodes 1 --nproc_per_node 4 --config [CONFIG_PATH]\n```\nUpdates on 30 July. As mentioned by @YunyaGaoTree in issue #39\nYou can also try to use the code below to gain (probably) faster training.\n```bash\n!torchrun train.py --config configs/demo.yaml\nCUDA_VISIBLE_DEVICES=0,1,2,3 python -m torch.distributed.launch --nnodes 1 --nproc_per_node 4 loadddptrain.py --config configs/demo.yaml\n```\n\n## Test\n```bash\npython test.py --config [CONFIG_PATH] --model [MODEL_PATH]\n```\n\n## Pre-trained Models\nhttps://drive.google.com/file/d/13JilJT7dhxwMIgcdtnvdzr08vcbREFlR/view?usp=sharing\n\n\n## Dataset\n\n### Camouflaged Object Detection\n- **[COD10K](https://github.com/DengPingFan/SINet/)**\n- **[CAMO](https://drive.google.com/open?id=1h-OqZdwkuPhBvGcVAwmh0f1NGqlH_4B6)**\n- **[CHAMELEON](https://www.polsl.pl/rau6/datasets/)**\n\n### Shadow Detection\n- **[ISTD](https://github.com/DeepInsight-PCALab/ST-CGAN)**\n\n### Polyp Segmentation - Medical Applications\n- **[Kvasir](https://datasets.simula.no/kvasir-seg/)**\n\n## Citation\n\nIf you find our work useful in your research, please consider citing:\n\n```\n\n@inproceedings{chen2023sam,\n  title={Sam-adapter: Adapting segment anything in underperformed scenes},\n  author={Chen, Tianrun and Zhu, Lanyun and Deng, Chaotao and Cao, Runlong and Wang, Yan and Zhang, Shangzhan and Li, Zejian and Sun, Lingyun and Zang, Ying and Mao, Papa},\n  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},\n  pages={3367--3375},\n  year={2023}\n}\n\n@misc{chen2024sam2adapterevaluatingadapting,\n      title={SAM2-Adapter: Evaluating \u0026 Adapting Segment Anything 2 in Downstream Tasks: Camouflage, Shadow, Medical Image Segmentation, and More}, \n      author={Tianrun Chen and Ankang Lu and Lanyun Zhu and Chaotao Ding and Chunan Yu and Deyi Ji and Zejian Li and Lingyun Sun and Papa Mao and Ying Zang},\n      year={2024},\n      eprint={2408.04579},\n      archivePrefix={arXiv},\n      primaryClass={cs.CV},\n      url={https://arxiv.org/abs/2408.04579}, \n}\n\n\n@misc{chen2023sam,\n      title={SAM Fails to Segment Anything? -- SAM-Adapter: Adapting SAM in Underperformed Scenes: Camouflage, Shadow, and More}, \n      author={Tianrun Chen and Lanyun Zhu and Chaotao Ding and Runlong Cao and Shangzhan Zhang and Yan Wang and Zejian Li and Lingyun Sun and Papa Mao and Ying Zang},\n      year={2023},\n      eprint={2304.09148},\n      archivePrefix={arXiv},\n      primaryClass={cs.CV}\n}\n\n\n```\n\n## Acknowledgements\nThe part of the code is derived from Explicit Visual Prompt   \u003ca href='https://nifangbaage.github.io/Explicit-Visual-Prompt/'\u003e\u003cimg src='https://img.shields.io/badge/Project-Page-Green'\u003e\u003c/a\u003e by \nWeihuang Liu, [Xi Shen](https://xishen0220.github.io/), [Chi-Man Pun](https://www.cis.um.edu.mo/~cmpun/), and [Xiaodong Cun](https://vinthony.github.io/) by University of Macau and Tencent AI Lab.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftianrun-chen%2FSAM-Adapter-PyTorch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftianrun-chen%2FSAM-Adapter-PyTorch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftianrun-chen%2FSAM-Adapter-PyTorch/lists"}