{"id":22069284,"url":"https://github.com/SuperMedIntel/Medical-SAM-Adapter","last_synced_at":"2025-07-24T07:32:59.922Z","repository":{"id":156856047,"uuid":"632316603","full_name":"SuperMedIntel/Medical-SAM-Adapter","owner":"SuperMedIntel","description":"Adapting Segment Anything Model for Medical Image Segmentation","archived":false,"fork":false,"pushed_at":"2024-06-21T08:52:27.000Z","size":17337,"stargazers_count":1021,"open_issues_count":52,"forks_count":90,"subscribers_count":10,"default_branch":"main","last_synced_at":"2024-11-26T01:30:40.891Z","etag":null,"topics":["adapter","deep-learning","medical-imaging","segment-anything-model","segmentagtion"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/SuperMedIntel.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-04-25T06:52:16.000Z","updated_at":"2024-11-25T12:52:45.000Z","dependencies_parsed_at":"2023-12-08T06:28:56.980Z","dependency_job_id":"9c643572-4c8a-4ad4-86b0-d7a96129475d","html_url":"https://github.com/SuperMedIntel/Medical-SAM-Adapter","commit_stats":null,"previous_names":["kidswithtokens/medical-sam-adapter","wujunde/medical-sam-adapter","medicinetoken/medical-sam-adapter","supermedintel/medical-sam-adapter"],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SuperMedIntel%2FMedical-SAM-Adapter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SuperMedIntel%2FMedical-SAM-Adapter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SuperMedIntel%2FMedical-SAM-Adapter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SuperMedIntel%2FMedical-SAM-Adapter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/SuperMedIntel","download_url":"https://codeload.github.com/SuperMedIntel/Medical-SAM-Adapter/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":227421466,"owners_count":17775011,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["adapter","deep-learning","medical-imaging","segment-anything-model","segmentagtion"],"created_at":"2024-11-30T20:04:50.580Z","updated_at":"2024-11-30T20:07:58.713Z","avatar_url":"https://github.com/SuperMedIntel.png","language":"Python","funding_links":["https://ko-fi.com/jundewu"],"categories":["Paper List"],"sub_categories":["Follow-up Papers"],"readme":"\u003ch1 align=\"center\"\u003e● Medical SAM Adapter\u003c/h1\u003e\n\n\u003cp align=\"center\"\u003e\n    \u003ca href=\"https://discord.gg/DN4rvk95CC\"\u003e\n        \u003cimg alt=\"Discord\" src=\"https://img.shields.io/discord/1146610656779440188?logo=discord\u0026style=flat\u0026logoColor=white\"/\u003e\u003c/a\u003e\n    \u003cimg src=\"https://img.shields.io/static/v1?label=license\u0026message=GPL\u0026color=white\u0026style=flat\" alt=\"License\"/\u003e\n\u003c/p\u003e\n\nMedical SAM Adapter, or say MSA, is a project to fineturn [SAM](https://github.com/facebookresearch/segment-anything) using [Adaption](https://lightning.ai/pages/community/tutorial/lora-llm/) for the Medical Imaging.\nThis method is elaborated on the paper [Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentation](https://arxiv.org/abs/2304.12620).\n\n## A Quick Overview \n \u003cimg width=\"880\" height=\"380\" src=\"https://github.com/WuJunde/Medical-SAM-Adapter/blob/main/figs/medsamadpt.jpeg\"\u003e\n\n ## News\n - [TOP] Join in our [Discord](https://discord.gg/EqbgSPEX) to ask questions and discuss with others.\n - [TOP] 24-03-02 We have released our pre-trained Adapters in [Medical-Adapter-Zoo](https://huggingface.co/KidsWithTokens/Medical-Adapter-Zoo/tree/main). Try it without painful training 😉 Credit: @shinning0821\n - 23-05-10. This project is still quickly updating 🌝. Check TODO list to see what will be released next.\n - 23-05-11. GitHub Dicussion opened. You guys can now talk, code and make friends on the playground 👨‍❤️‍👨. \n - 23-12-22. Released data loader and example case on [REFUGE](https://refuge.grand-challenge.org/) dataset. Credit: @jiayuanz3\n - 24-01-04. Released the Efficient Med-SAM-Adapter❗️ A new, faster, and more lightweight version incorporates Meta [EfficientSAM](https://yformer.github.io/efficient-sam/)🏇. Full credit goes to @shinning0821. \n - 24-01-07. The image resolution now can be resized by ``-image_size``. Credit: @shinning0821\n - 24-01-11. Added a detailed guide on utilizing the Efficient Med-SAM-Adapter, complete with a comparison of performance and speed. You can find this resource in  [guidance/efficient_sam.ipynb](./guidance/efficient_sam.ipynb). Credit: @shinning0821\n - 24-01-14. We've just launched our first official version, v0.1.0-alpha 🥳. This release includes support for [MobileSAM](https://github.com/ChaoningZhang/MobileSAM), which can be activated by setting ``-net mobile_sam``. Additionally, you now have the flexibility to use ViT, Tiny ViT, and Efficient ViT as encoders. Check the details [here](https://github.com/KidsWithTokens/Medical-SAM-Adapter/releases/tag/v0.1.0-alpha). Credit: @shinning0821\n - 24-01-20. Added a guide on utilizing the mobile sam in Med-SAM-Adapter, with a comparison of performance and speed. You can find it in [guidance/mobile_sam.ipynb](https://github.com/KidsWithTokens/Medical-SAM-Adapter/blob/main/guidance/mobile_sam.ipynb) Credit: @shinning0821\n - 24-01-21. We've added [LoRA](https://huggingface.co/docs/diffusers/training/lora) to our framework🤖. Use it by setting ``-mod`` as ``sam_lora``.\nA guidance can be found in [here](https://github.com/KidsWithTokens/Medical-SAM-Adapter/blob/main/guidance/lora.ipynb). Credit: @shinning0821\n - 24-01-22. We've added dataloader for [LIDC dataset](https://paperswithcode.com/dataset/lidc-idri), a multi-rater(4 raters 👨‍⚕️🧑🏽‍⚕️👩‍⚕️🧑🏽‍⚕️) lesions segmentation from low-dose lung CTs 🩻. You can download the preprocessed LIDC dataset at [here](https://github.com/stefanknegt/Probabilistic-Unet-Pytorch). Also updated environment, and random_click function. Credit: @jiayuanz3\n - 24-03-06. We've supported multi-class segmentation. Use it by setting ``-multimask_output`` to the number of classes favored. Also updated REFUGE example to two classes (optic disc \u0026 cup). Credit: @LJQCN101\n - 24-03-06. We've supported many other datasets and rebuild the code of datasets and dataloaders. Seen in `guidance/Dataset.md` Credit: @shinning0821\n\n## Medical Adapter Zoo 🐘🐊🦍🦒🦨🦜🦥\nWe've released a bunch of pre-trained Adapters for various organs/lesions in [Medical-Adapter-Zoo](https://huggingface.co/KidsWithTokens/Medical-Adapter-Zoo/tree/main). Just pick the adapter that matches your disease and easily adjust SAM to suit your specific needs 😉. \n\nIf you can't find what you're looking for. Please suggest it through any contact method available to us (GitHub issue, HuggingFace community, or [Discord](https://discord.gg/EqbgSPEX)). We'll do our very best to include it.\n \n ## Requirement\n\n Install the environment:\n\n ``conda env create -f environment.yml``\n\n ``conda activate sam_adapt``\n\n Then download [SAM checkpoint](https://dl.fbaipublicfiles.com/segment_anything/sam_vit_b_01ec64.pth), and put it at ./checkpoint/sam/\n\n You can run:\n\n ``wget https://dl.fbaipublicfiles.com/segment_anything/sam_vit_b_01ec64.pth``\n\n ``mv sam_vit_b_01ec64.pth ./checkpoint/sam``\n creat the folder if it does not exist\n\n ## Example Cases\n\n ### Melanoma Segmentation from Skin Images (2D)\n\n 1. Download ISIC dataset part 1 from https://challenge.isic-archive.com/data/. Then put the csv files in \"./data/isic\" under your data path. Your dataset folder under \"your_data_path\" should be like:\nISIC/\n     ISBI2016_ISIC_Part1_Test_Data/...\n     \n     ISBI2016_ISIC_Part1_Training_Data/...\n     \n     ISBI2016_ISIC_Part1_Test_GroundTruth.csv\n     \n      ISBI2016_ISIC_Part1_Training_GroundTruth.csv\n    \n    You can fine the csv files [here](https://github.com/KidsWithTokens/MedSegDiff/tree/master/data/isic_csv)\n\n 3. Begin Adapting! run: ``python train.py -net sam -mod sam_adpt -exp_name *msa_test_isic* -sam_ckpt ./checkpoint/sam/sam_vit_b_01ec64.pth -image_size 1024 -b 32 -dataset isic -data_path *../data*``\n change \"data_path\" and \"exp_name\" for your own useage. you can change \"exp_name\" to anything you want.\n\n You can descrease the ``image size`` or batch size ``b`` if out of memory.\n\n 3. Evaluation: The code can automatically evaluate the model on the test set during traing, set \"--val_freq\" to control how many epoches you want to evaluate once. You can also run val.py for the independent evaluation.\n\n 4. Result Visualization: You can set \"--vis\" parameter to control how many epoches you want to see the results in the training or evaluation process.\n\n In default, everything will be saved at `` ./logs/`` \n\n ### REFUGE: Optic-disc Segmentation from Fundus Images (2D) \n [REFUGE](https://refuge.grand-challenge.org/) dataset contains 1200 fundus images with optic disc/cup segmentations and clinical glaucoma labels. \n\n 1. Dowaload the dataset manually from [here](https://huggingface.co/datasets/realslimman/REFUGE-MultiRater/tree/main), or using command lines:\n\n ``git lfs install``\n\n ``git clone git@hf.co:datasets/realslimman/REFUGE-MultiRater``\n\n unzip and put the dataset to the target folder\n\n ``unzip ./REFUGE-MultiRater.zip``\n\n ``mv REFUGE-MultiRater ./data``\n\n 2. For training the adapter, run: ``python train.py -net sam -mod sam_adpt -exp_name REFUGE-MSAdapt -sam_ckpt ./checkpoint/sam/sam_vit_b_01ec64.pth -image_size 1024 -b 32 -dataset REFUGE -data_path ./data/REFUGE-MultiRater``\n you can change \"exp_name\" to anything you want.\n\n You can descrease the ``image size`` or batch size ``b`` if out of memory.\n\n ### Abdominal Multiple Organs Segmentation (3D)\n\n This tutorial demonstrates how MSA can adapt SAM to 3D multi-organ segmentation task using the BTCV challenge dataset.\nFor BTCV dataset, under Institutional Review Board (IRB) supervision, 50 abdomen CT scans of were randomly selected from a combination of an ongoing colorectal cancer chemotherapy trial, and a retrospective ventral hernia study. The 50 scans were captured during portal venous contrast phase with variable volume sizes (512 x 512 x 85 - 512 x 512 x 198) and field of views (approx. 280 x 280 x 280 mm3 - 500 x 500 x 650 mm3). The in-plane resolution varies from 0.54 x 0.54 mm2 to 0.98 x 0.98 mm2, while the slice thickness ranges from 2.5 mm to 5.0 mm.\nTarget: 13 abdominal organs including\nSpleen\nRight Kidney\nLeft Kidney\nGallbladder\nEsophagus\nLiver\nStomach\nAorta\nIVC\nPortal and Splenic Veins\nPancreas\nRight adrenal gland\nLeft adrenal gland.\nModality: CT\nSize: 30 3D volumes (24 Training + 6 Testing)\nChallenge: BTCV MICCAI Challenge\nThe following figure shows image patches with the organ sub-regions that are annotated in the CT (top left) and the final labels for the whole dataset (right).\n1. Prepare BTCV dataset following [MONAI](https://docs.monai.io/en/stable/index.html) instruction:\nDownload BTCV dataset from: https://www.synapse.org/#!Synapse:syn3193805/wiki/217752. After you open the link, navigate to the \"Files\" tab, then download Abdomen/RawData.zip.\nAfter downloading the zip file, unzip. Then put images from RawData/Training/img in ../data/imagesTr, and put labels from RawData/Training/label in ../data/labelsTr.\nDownload the json file for data splits from this [link](https://drive.google.com/file/d/1qcGh41p-rI3H_sQ0JwOAhNiQSXriQqGi/view). Place the JSON file at ../data/dataset_0.json.\n2. For the Adaptation, run: ``python train.py -net sam -mod sam_adpt -exp_name msa-3d-sam-btcv -sam_ckpt ./checkpoint/sam/sam_vit_b_01ec64.pth -image_size 1024 -b 8 -dataset decathlon -thd True -chunk 96 -data_path ../data -num_sample 4``  \nYou can modify following parameters to save the memory usage: '-b' the batch size, '-chunk' the 3D depth (channel) for each sample, '-num_sample' number of samples for [Monai.RandCropByPosNegLabeld](https://docs.monai.io/en/stable/transforms.html#randcropbyposneglabeld), 'evl_chunk' the 3D channel split step in the evaluation, decrease it if out of memory in the evaluation. \n## Run on  your own dataset\nIt is simple to run MSA on the other datasets. Just write another dataset class following which in `` ./dataset.py``. You only need to make sure you return a dict with \n     {\n                 'image': A tensor saving images with size [C,H,W] for 2D image, size [C, H, W, D] for 3D data.\n                 D is the depth of 3D volume, C is the channel of a scan/frame, which is commonly 1 for CT, MRI, US data. \n                 If processing, say like a colorful surgical video, D could the number of time frames, and C will be 3 for a RGB frame.\n                 'label': The target masks. Same size with the images except the resolutions (H and W).\n                 'p_label': The prompt label to decide positive/negative prompt. To simplify, you can always set 1 if don't need the negative prompt function.\n                 'pt': The prompt. Should be the same as that in SAM, e.g., a click prompt should be [x of click, y of click], one click for each scan/frame if using 3d data.\n                 'image_meta_dict': Optional. if you want save/visulize the result, you should put the name of the image in it with the key ['filename_or_obj'].\n                 ...(others as you want)\n     }\nWelcome to open issues if you meet any problem. It would be appreciated if you could contribute your dataset extensions. Unlike natural images, medical images vary a lot depending on different tasks. Expanding the generalization of a method requires everyone's efforts.\n\n ### TODO LIST\n\n- [ ] Jupyter tutorials.\n- [x] Fix bugs in BTCV. Add BTCV example.\n- [ ] Release REFUGE2, BraTs dataloaders and examples\n- [x] Changable Image Resolution \n- [ ] Fix bugs in Multi-GPU parallel\n- [x] Sample and Vis in training\n- [ ] Release general data pre-processing and post-processing\n- [x] Release evaluation\n- [ ] Deploy on HuggingFace\n- [x] configuration\n- [ ] Release SSL code\n- [ ] Release Medical Adapter Zoo\n\n ## Cite\n ~~~\n@misc{wu2023medical,\n      title={Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentation}, \n      author={Junde Wu and Wei Ji and Yuanpei Liu and Huazhu Fu and Min Xu and Yanwu Xu and Yueming Jin},\n      year={2023},\n      eprint={2304.12620},\n      archivePrefix={arXiv},\n      primaryClass={cs.CV}\n}\n ~~~\n\n## Buy Me A Coffee 🥤😉\nhttps://ko-fi.com/jundewu\n\n\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FSuperMedIntel%2FMedical-SAM-Adapter","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FSuperMedIntel%2FMedical-SAM-Adapter","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FSuperMedIntel%2FMedical-SAM-Adapter/lists"}