Papers/Projects

Basemodel Papers

• img
• arXiv - | Meta | Segment Anything with Concepts: doubles accuracy on promptable concept segmentation in images and videos. |
• img - Decoder/Segment-Everything-Everywhere-All-At-Once) | Microsoft | Semantic Segmentation with various prompt types. |

Derivative Papers

• img - specific prompts in SAM.|
• img - SAM) | - | Segment Anything with specific concepts. |
• img1 - | Code | - | One shot semantic segmentation by integrating an all-purpose feature extraction model and a class-agnostic segmentation model. |
• img
• img1 - |Code | - | SAM-adapter: Adapting SAM in Underperformed Scenes: Camouflage, Shadow, Medical Image Segmentation, and More.|
• img1 - | Code | Sichuan University & Shanghai AI Laboratory | The most comprehensive studies on applying SAM to medical 2D images. |
• img - cell-analytics.github.io/micro-sam/micro_sam.html#installation) | Code | University of Göttingen, Germany | Segment Anything for Microscopy implements automatic and interactive annotation for microscopy data. It is built on top of Segment Anything and specializes it for microscopy and other bio-imaging data. Its core components are: The micro_sam tools for interactive data annotation with napari.The micro_sam library to apply Segment Anything to 2d and 3d data or fine-tune it on your data.The micro_sam models that are fine-tuned on publicly available microscopy data. Our goal is to build fast and interactive annotation tools for microscopy data. |
• img1 - | Project | Code | comet | Grounding DINO + SAM + Stable Diffusion. |
• arXiv - | Code | - | SAM + Camouflaged object detection (COD) task.|
• img - tech.github.io/InterpAny-Clearer/) & Interactive Demo | Code | Shanghai AI Laboratory & Snap Inc. | Editable video frame interpolation with SAM. |
• img - images.githubusercontent.com/74295796/244441627-d947f59d-b0c1-4c22-9967-d8f2bf633879.gif)| arXiv| - | Code| HUST Vision Lab| An interactive natural image matting system with excellent performance for both opaque and transparent objects. |
• img1 - | Code | Portrai Inc. | A SAM application for the analysis of Spatial Transcriptomics.|
• img - | A novel framework to Segment Anything in 3D, named SA3D. |
• Video - | Code | Zhejiang University | A framework called Segment And Track Anything (SAMTrack) that allows users to precisely and effectively segment and track any object in a video.|
• img1 - tuned SAM on 65 biomedical imaging datasets with scribble, click, and bounding box inputs. |
• img - sam.github.io) | Code | UCSD | An open-world 3D native promptable point-cloud segmentation method. |
• safire_image - | Code | KAIST | Extends SAM's point prompting capability to image forensics, enabling precise source-aware segmentation for forgery localization. |
• img - lab/CLIP_Surgery/blob/master/demo.ipynb)| Code | HKUST | This work about SAM based on CLIP's explainability to achieve text to mask without manual points.|
• arXiv - | - | KAIST | The first work of conduct a comprehensive investigation on how to attack SAM with adversarial examples. |
• img - 619/FastSAM) | Code | - | Reformulate the architecture and improve the speed of SAM. |
• img - friendly by replacing the heavyweight image encoder with a lightweight one.|
• img1 - | Code | - | A project to finetune SAM using Adaption for the Medical Imaging. |
• img1 - Labs/Matting-Anything) | SHI Labs | Leverage feature maps from SAM and adopts a Mask-to-Matte module to predict the alpha matte. |
• img1 - | Code | OpenGVLab | A SAM application in the Robotic field.|
• img2 - LAB-CUHK-SZ/SAMPro3D) | CUHKSZ, MSRA | A novel method to segment any 3D indoor scenes by applying the SAM to 2D frames, without need any training, tuning, distillation or 3D pretrained networks. |
• img1 - Any-Point-Cloud) | - | A framework capable of leveraging 2D vision foundation models for self-supervised learning on large-scale 3D point clouds. |
• img - | Code | PLA Information Engineering University | A sample-efficient change detection framework that employs SAM as the visual encoder. |
• Video - | Code | - | An interactive video object tracking and segmentation tool based on SAM and XMem.|
• arXiv - | Code | - | Test-Time Adaptation with SaLIP: A Cascade of SAM and CLIP for Zero-shot Medical Image Segmentation. |
• arXiv - | Code | - | Medical SAM 2: treats all 2D and 3D medical segmentation tasks as video object tracking, enabling universal auto-tracking segmentation. |
• img - research/LISA) | CUHK | Large Language Instructed Segmentation Assistant: inherits LLM language generation while producing segmentation masks from complex implicit text queries. |
• img - research/Sa2VA) | - | Marries SAM 2 with LLaVA for dense grounded understanding of images and videos; unifies text, image, and video into a shared LLM token space. |
• arXiv - | - | - | Fine-tuned SAM with dual-encoder and deformable decoder for semantic segmentation in autonomous driving scenes. |
• arXiv - | - | - | Incorporates SAM into self-training UDA methods for refining pseudo-labels in unsupervised domain adaptive semantic segmentation for intelligent vehicles. |

Derivative Projects

• img - park/segment-anything-with-clip)　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　| Code | - | SAM + CLIP. |
• img - 　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　 | Code | - | SAM + CLIP. |
• img - 　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　 | Code | - | SAM + Zero-shot Instance Segmentation. |
• img - 　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　 | Code | - | An implementation of SAM for generating rotated bounding boxes with MMRotate. |
• img1 - 　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　 | Code | - | Simple static web-based mask drawer, supporting semantic drawing with SAM. |
• - | SAM + Labelme + LabelImg + Auto-labeling. |
• img - 　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　 | Code | - | GPT + SAM. |
• roboflow-sam-optimized-faster - data-segment-anything-model-sam/) | Roboflow | SAM-assisted labeling for training computer vision models. |
• img - 　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　 | Code | - | A class-aware one-stage tool for training fine-tuning models based on SAM. |
• img - 　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　 | Code | NVIDIA | A distilled Segment Anything (SAM) model capable of running real-time with NVIDIA TensorRT. |
• Video - | Code | - | Segment anything with Napari integration of SAM.|
• img - | Code | - | SAM for Medical Imaging.|
• img - | Code | - | SAM + Inpainting/Replacing.|
• img - | Code | - | Edit and Generate Anything in an image.|
• img - | Code | - | Stable Diffusion + SAM.|
• img - | Code | - | SAM + Moving Object Detection. |
• Code - | Evaluating the basic performance of SAM on the Referring Image segmentation task. |
• img - 　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　 | Code | - | A PySide6 based annotation tool for Segment Anything. |
• img - 　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　 | Code | - | A PySide6 based annotation tool for Segment Anything 2. Video tracking with multiple objects. |
• Video - x-yang/Segment-and-Track-Anything) | Zhejiang University | This project, which is based on SAM and DeAOT, focuses on segmenting and tracking objects in videos. |
• img - 　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　 | Code | - | Grounding DINO + SAM to segment any anomaly. |
• img - 　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　 | Code | Fudan | A dense category annotation engine. |
• - Bc) BiliBili Demo　　　　　　　　　　　　　　　　　| Code | - | Labeling tool by SAM, supports SAM, sam-hq, MobileSAM, EdgeSAM etc. |
• img - Anything-Video) | Code | - | SAM + Video. |
• img - park/yolo-world-with-efficientvit-sam)　　　　　　　　　　　　　　　　　　　　　　　　　　　　　| Code | - | Efficient open-vocabulary object detection and segmentation with YOLO-World + EfficientViT SAM. |
• img - | Code | - | SAM + Zero 1-to-3.|

front-end framework

• samjs - | JS SDK for SAM, Support remote sensing data segmentation and vectorization. |

🏅Recognition

• Shypd.Ai - quality AI resources.