{"id":15029598,"url":"https://github.com/syscv/sam-hq","last_synced_at":"2025-05-13T15:09:38.142Z","repository":{"id":172588175,"uuid":"648003298","full_name":"SysCV/sam-hq","owner":"SysCV","description":"Segment Anything in High Quality [NeurIPS 2023]","archived":false,"fork":false,"pushed_at":"2024-12-07T22:07:16.000Z","size":72605,"stargazers_count":3903,"open_issues_count":103,"forks_count":237,"subscribers_count":78,"default_branch":"main","last_synced_at":"2025-05-06T21:41:22.623Z","etag":null,"topics":["high-quality","sam","segment-anything","segment-anything-model","segmentation","zero-shot-segmentation"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2306.01567","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/SysCV.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-06-01T02:04:54.000Z","updated_at":"2025-05-06T03:06:52.000Z","dependencies_parsed_at":"2024-09-21T04:00:51.928Z","dependency_job_id":"64d0fa9e-f42e-4caf-b79a-b1e07843fa20","html_url":"https://github.com/SysCV/sam-hq","commit_stats":null,"previous_names":["syscv/sam-hq"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SysCV%2Fsam-hq","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SysCV%2Fsam-hq/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SysCV%2Fsam-hq/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SysCV%2Fsam-hq/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/SysCV","download_url":"https://codeload.github.com/SysCV/sam-hq/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253969241,"owners_count":21992262,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["high-quality","sam","segment-anything","segment-anything-model","segmentation","zero-shot-segmentation"],"created_at":"2024-09-24T20:11:09.966Z","updated_at":"2025-05-13T15:09:33.113Z","avatar_url":"https://github.com/SysCV.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Segment Anything in High Quality\n\n[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/segment-anything-in-high-quality/zero-shot-segmentation-on-segmentation-in-the)](https://paperswithcode.com/sota/zero-shot-segmentation-on-segmentation-in-the?p=segment-anything-in-high-quality)\n\u003ca href=\"https://colab.research.google.com/drive/1QwAbn5hsdqKOD5niuBzuqQX4eLCbNKFL?usp=sharing\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"\u003e\u003c/a\u003e\n[![Huggingfaces](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Spaces-blue)](https://huggingface.co/spaces/sam-hq-team/sam-hq)\n[![Open in OpenXLab](https://cdn-static.openxlab.org.cn/app-center/openxlab_app.svg)](https://openxlab.org.cn/apps/detail/keleiwhu/sam-hq)\n[![Downloads](https://static.pepy.tech/badge/segment-anything-hq)](https://pepy.tech/project/segment-anything-hq)\n\n\n\u003e [**Segment Anything in High Quality**](https://arxiv.org/abs/2306.01567)           \n\u003e NeurIPS 2023  \n\u003e ETH Zurich \u0026 HKUST \n\nWe propose HQ-SAM to upgrade SAM for high-quality zero-shot segmentation. Refer to our [paper](https://arxiv.org/abs/2306.01567) for more details.\n\n## Latest updates\n\n**2024/11/17 -- HQ-SAM 2 is released**\n\n- A new suite of improved model checkpoints (denoted as **HQ-SAM 2**, beta-version) are released. See [Model Description](sam-hq2/README.md) for details. Change working directory by `cd sam-hq2`\n\n![HQ-SAM2 results comparison](sam-hq2/assets/hq-sam2-results.png?raw=true)\n\nUpdates\n-----------------\n:fire::fire: **SAM for Video Segmentation**: Interested in intersecting SAM and video? HQ-SAM is supported by [DEVA](https://github.com/hkchengrex/Tracking-Anything-with-DEVA) in its text-prompted mode! Also, check the work [MASA](https://github.com/siyuanliii/masa) and [SAM-PT](https://github.com/SysCV/sam-pt) with SAM.\n\n:fire::fire: **SAM in 3D**: Interested in intersecting SAM and 3D Gaussian Splatting? See our new work [Gaussian Grouping](https://github.com/lkeab/gaussian-grouping)! Also, if you are interested in intersecting SAM and NeRF, please see work [SANeRF-HQ](https://github.com/lyclyc52/SANeRF-HQ)!\n\nMore: HQ-SAM is adopted in [Osprey](https://arxiv.org/abs/2312.10032), [CaR](https://torrvision.com/clip_as_rnn/), [SpatialRGPT](https://arxiv.org/abs/2406.01584), [GLaMM](https://arxiv.org/abs/2311.03356), [ENIGMA-51](https://iplab.dmi.unict.it/ENIGMA-51/) to provide fine-grained mask annotations.\n\n\nPlatform integration: HQ-SAM is supported in the [OpenMMLab PlayGround](https://github.com/open-mmlab/playground/blob/main/label_anything/readme.md) for annotation with Label-Studio, in [segment-geospatial](https://github.com/opengeos/segment-geospatial) for segmenting geospatial data, and mask annotation tool [ISAT](https://github.com/yatengLG/ISAT_with_segment_anything), and [Supervisely](https://supervisely.com/blog/segment-anything-in-high-quality-HQ-SAM/)!\n\n2023/08/11: Support [python package](#quick-installation-via-pip) for easier **pip installation**. Light HQ-SAM is in [EfficientSAM series](https://github.com/IDEA-Research/Grounded-Segment-Anything/tree/main/EfficientSAM) combining with [Grounded SAM](https://github.com/IDEA-Research/Grounded-Segment-Anything/)!\n\n\u003c!-- 2023/07/21: HQ-SAM is also in OpenXLab apps, thanks their support! --\u003e\n\n:rocket::rocket: 2023/07/17: We released **Light HQ-SAM** using TinyViT as backbone, for both fast and high-quality zero-shot segmentation, which reaches **41.2 FPS**. Refer to [Light HQ-SAM vs. MobileSAM](#light-hq-sam-vs-mobilesam-on-coco) for more details.\n\n:trophy::1st_place_medal: 2023/07/14: Grounded **HQ-SAM** obtains the **first place**:1st_place_medal: in the [Segmentation in the Wild](https://eval.ai/web/challenges/challenge-page/1931/leaderboard/4567) competition on zero-shot track (hosted in [CVPR 2023 workshop](https://computer-vision-in-the-wild.github.io/cvpr-2023/)), outperforming Grounded SAM. Refer to our [SGinW evaluation](#grounded-hq-sam-vs-grounded-sam-on-seginw) for more details.\n\n2023/07/05: We released [SAM tuning instuctions](#hq-sam-tuning-and-hq-seg44k-data) and [HQSeg-44K data](#hq-sam-tuning-and-hq-seg44k-data).\n\n2023/07/04: HQ-SAM is adopted in [SAM-PT](https://github.com/SysCV/sam-pt) to improve the SAM-based zero-shot video segmentation performance. Also, HQ-SAM is used in [Grounded-SAM](https://github.com/IDEA-Research/Grounded-Segment-Anything), [Inpaint Anything](https://github.com/Uminosachi/sd-webui-inpaint-anything) and [HQTrack](https://github.com/jiawen-zhu/HQTrack) (2nd in VOTS 2023).\n\n2023/06/28: We released the [ONNX export script](#onnx-export) and [colab notebook](https://colab.research.google.com/drive/11U2La49c2IxahzJkAV-EzPqEH3cz_5hq?usp=sharing) for exporting and using ONNX model.\n\n2023/06/23: Play with HQ-SAM demo at [![Huggingfaces](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Spaces-blue)](https://huggingface.co/spaces/sam-hq-team/sam-hq), which supports point, box and text prompts.\n\n2023/06/14: We released the [colab demo](https://colab.research.google.com/drive/1QwAbn5hsdqKOD5niuBzuqQX4eLCbNKFL?usp=sharing) \u003ca href=\"https://colab.research.google.com/drive/1QwAbn5hsdqKOD5niuBzuqQX4eLCbNKFL?usp=sharing\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"\u003e\u003c/a\u003e and [automatic mask generator notebook](https://colab.research.google.com/drive/1dhRq4eR6Fbl-yl1vbQvU9hqyyeOidQaU?usp=sharing).\n\n2023/06/13: We released the [model checkpoints](#model-checkpoints) and [demo visualization codes](#getting-started).\n\nVisual comparison between SAM and HQ-SAM\n-----------------\n**SAM vs. HQ-SAM**\n\u003ctable\u003e\n  \u003ctr\u003e\n    \u003ctd\u003e\u003cimg src=\"visual_demo/1.gif\" width=\"250\"\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cimg src=\"visual_demo/2.gif\" width=\"250\"\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cimg src=\"visual_demo/3.gif\" width=\"250\"\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003e\u003cimg src=\"visual_demo/4.gif\" width=\"250\"\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cimg src=\"visual_demo/5.gif\" width=\"250\"\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cimg src=\"visual_demo/6.gif\" width=\"250\"\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n\u003cimg width=\"900\" alt=\"image\" src='figs/coco_vis_comp.png'\u003e\n\nIntroduction\n-----------------\nThe recent Segment Anything Model (SAM) represents a big leap in scaling up segmentation models, allowing for powerful zero-shot capabilities and flexible prompting. Despite being trained with 1.1 billion masks, SAM's mask prediction quality falls short in many cases, particularly when dealing with objects that have intricate structures. We propose HQ-SAM, equipping SAM with the ability to accurately segment any object, while maintaining SAM's original promptable design, efficiency, and zero-shot generalizability. Our careful design reuses and preserves the pre-trained model weights of SAM, while only introducing minimal additional parameters and computation. We design a learnable High-Quality Output Token, which is injected into SAM's mask decoder and is responsible for predicting the high-quality mask. Instead of only applying it on mask-decoder features, we first fuse them with early and final ViT features for improved mask details. To train our introduced learnable parameters, we compose a dataset of 44K fine-grained masks from several sources. HQ-SAM is only trained on the introduced detaset of 44k masks, which takes only 4 hours on 8 GPUs. We show the efficacy of HQ-SAM in a suite of 9 diverse segmentation datasets across different downstream tasks, where 7 out of them are evaluated in a zero-shot transfer protocol. \n\n\u003cimg width=\"1096\" alt=\"image\" src='figs/sam-hf-framework.png'\u003e\n\nQuantitative comparison between SAM and HQ-SAM\n-----------------\nNote: For box-prompting-based evaluation, we feed SAM, MobileSAM and our HQ-SAM with the same image/video bounding boxes and adopt the single mask output mode of SAM. \n\nWe provide comprehensive performance, model size and speed comparison on SAM variants:\n\u003cimg width=\"1096\" alt=\"image\" src='figs/sam_variants_comp.png'\u003e\n\n### Various ViT backbones on COCO:\n![backbones](figs/sam_vs_hqsam_backbones.png)\nNote: For the COCO dataset, we use a SOTA detector FocalNet-DINO trained on the COCO dataset as our box prompt generator.\n\n### YTVIS and HQ-YTVIS\nNote:Using ViT-L backbone. We adopt the SOTA detector Mask2Former trained on the YouTubeVIS 2019 dataset as our video boxes prompt generator while reusing its object association prediction.\n![ytvis](figs/ytvis.png)\n\n### DAVIS\nNote: Using ViT-L backbone. We adopt the SOTA model XMem as our video boxes prompt generator while reusing its object association prediction.\n![davis](figs/davis.png)\n\n### **Quick Installation via pip**\n```\npip install segment-anything-hq\npython\nfrom segment_anything_hq import sam_model_registry\nmodel_type = \"\u003cmodel_type\u003e\" #\"vit_l/vit_b/vit_h/vit_tiny\"\nsam_checkpoint = \"\u003cpath/to/checkpoint\u003e\"\nsam = sam_model_registry[model_type](checkpoint=sam_checkpoint)\n```\n\nsee specific usage example (such as vit-l) by running belowing command:\n```\nexport PYTHONPATH=$(pwd)\npython demo/demo_hqsam_pip_example.py\n```\n\n\n### **Standard Installation**\nThe code requires `python\u003e=3.8`, as well as `pytorch\u003e=1.7` and `torchvision\u003e=0.8`. Please follow the instructions [here](https://pytorch.org/get-started/locally/) to install both PyTorch and TorchVision dependencies. Installing both PyTorch and TorchVision with CUDA support is strongly recommended.\n\nClone the repository locally and install with\n\n```\ngit clone https://github.com/SysCV/sam-hq.git\ncd sam-hq; pip install -e .\n```\n\nThe following optional dependencies are necessary for mask post-processing, saving masks in COCO format, the example notebooks, and exporting the model in ONNX format. `jupyter` is also required to run the example notebooks.\n\n```\npip install opencv-python pycocotools matplotlib onnxruntime onnx timm\n```\n\n### Example conda environment setup\n```bash\nconda create --name sam_hq python=3.8 -y\nconda activate sam_hq\nconda install pytorch==1.10.0 torchvision==0.11.0 cudatoolkit=11.1 -c pytorch -c nvidia\npip install opencv-python pycocotools matplotlib onnxruntime onnx timm\n\n# under your working directory\ngit clone https://github.com/SysCV/sam-hq.git\ncd sam-hq\npip install -e .\nexport PYTHONPATH=$(pwd)\n```\n\n### **Model Checkpoints**\n\nThree HQ-SAM model versions of the model are available with different backbone sizes. These models can be instantiated by running\n\n```\nfrom segment_anything import sam_model_registry\nsam = sam_model_registry[\"\u003cmodel_type\u003e\"](checkpoint=\"\u003cpath/to/checkpoint\u003e\")\n```\n\nDownload the provided trained model below and put them into the pretrained_checkpoint folder:\n```\nmkdir pretrained_checkpoint\n``` \n\nClick the links below to download the checkpoint for the corresponding model type. We also provide **alternative model downloading links** [here](https://github.com/SysCV/sam-hq/issues/5) or at [hugging face](https://huggingface.co/lkeab/hq-sam/tree/main).\n- `vit_b`: [ViT-B HQ-SAM model.](https://drive.google.com/file/d/11yExZLOve38kRZPfRx_MRxfIAKmfMY47/view?usp=sharing)\n- `vit_l`: [ViT-L HQ-SAM model.](https://drive.google.com/file/d/1Uk17tDKX1YAKas5knI4y9ZJCo0lRVL0G/view?usp=sharing)\n- `vit_h`: [ViT-H HQ-SAM model.](https://drive.google.com/file/d/1qobFYrI4eyIANfBSmYcGuWRaSIXfMOQ8/view?usp=sharing)\n- `vit_tiny` (**Light HQ-SAM** for real-time need): [ViT-Tiny HQ-SAM model.](https://huggingface.co/lkeab/hq-sam/resolve/main/sam_hq_vit_tiny.pth)\n\n### **Getting Started**\n\nFirst download a [model checkpoint](#model-checkpoints). Then the model can be used in just a few lines to get masks from a given prompt:\n\n```\nfrom segment_anything import SamPredictor, sam_model_registry\nsam = sam_model_registry[\"\u003cmodel_type\u003e\"](checkpoint=\"\u003cpath/to/checkpoint\u003e\")\npredictor = SamPredictor(sam)\npredictor.set_image(\u003cyour_image\u003e)\nmasks, _, _ = predictor.predict(\u003cinput_prompts\u003e)\n```\n\nAdditionally, see the usage examples in our [demo](/demo/demo_hqsam.py) , [colab notebook](https://colab.research.google.com/drive/1QwAbn5hsdqKOD5niuBzuqQX4eLCbNKFL?usp=sharing) and [automatic mask generator notebook](https://colab.research.google.com/drive/1dhRq4eR6Fbl-yl1vbQvU9hqyyeOidQaU?usp=sharing).\n\nTo obtain HQ-SAM's visual result:\n```\npython demo/demo_hqsam.py\n```\n\nTo obtain baseline SAM's visual result. Note that you need to download original SAM checkpoint from [baseline-SAM-L model](https://dl.fbaipublicfiles.com/segment_anything/sam_vit_l_0b3195.pth) and put it into the pretrained_checkpoint folder.\n```\npython demo/demo_sam.py\n```\n\nTo obtain Light HQ-SAM's visual result:\n```\npython demo/demo_hqsam_light.py\n```\n\n### **HQ-SAM Tuning and HQ-Seg44k Data**\nWe provide detailed training, evaluation, visualization and data downloading instructions in [HQ-SAM training](train/README.md). You can also replace our training data to obtain your own SAM in specific application domain (like medical, OCR and remote sensing).\n\nPlease change the current folder path to:\n```\ncd train\n```\nand then refer to detailed [readme instruction](train/README.md).\n\n### **Grounded HQ-SAM vs Grounded SAM on [SegInW](https://eval.ai/web/challenges/challenge-page/1931/overview?ref=blog.roboflow.com)**\n\nGrounded HQ-SAM wins the **first place**:1st_place_medal: on SegInW benchmark (consist of 25 public zero-shot in the wild segmentation datasets), and outpuerforming Grounded SAM using the same grounding-dino detector.\n\n\u003ctable\u003e\u003ctbody\u003e\n\u003c!-- START TABLE --\u003e\n\u003c!-- TABLE HEADER --\u003e\n\u003cth valign=\"bottom\"\u003eModel Name\u003c/th\u003e\n\u003cth valign=\"bottom\"\u003eEncoder\u003c/th\u003e\n\u003cth valign=\"bottom\"\u003eGroundingDINO\u003c/th\u003e\n\u003cth valign=\"bottom\"\u003eMean AP\u003c/th\u003e\n\u003cth valign=\"bottom\"\u003eEvaluation Script\u003c/th\u003e\n\u003cth valign=\"bottom\"\u003eLog\u003c/th\u003e\n\u003cth valign=\"bottom\"\u003eOutput Json\u003c/th\u003e\n\u003c!-- TABLE BODY --\u003e\n\u003c!-- ROW: maskformer2_R50_bs16_50ep --\u003e\n \u003ctr\u003e\u003ctd align=\"left\"\u003eGrounded SAM\u003c/td\u003e\n\u003ctd align=\"center\"\u003evit-h\u003c/td\u003e\n\u003ctd align=\"center\"\u003eswin-b\u003c/td\u003e\n\u003ctd align=\"center\"\u003e48.7\u003c/td\u003e\n\u003ctd align=\"center\"\u003e\u003ca href=\"seginw/test_seginw.sh\"\u003escript\u003c/a\u003e\u003c/td\u003e\n\u003ctd align=\"center\"\u003e\u003ca href=\"seginw/logs/grounded_sam.log\"\u003elog\u003c/a\u003e\u003c/td\u003e\n\u003ctd align=\"center\"\u003e\u003ca href=\"https://huggingface.co/sam-hq-team/SegInW/resolve/main/result/grounded_sam.zip\"\u003eresult\u003c/a\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003c!-- ROW: maskformer2_R101_bs16_50ep --\u003e\n \u003ctr\u003e\u003ctd align=\"left\"\u003eGrounded HQ-SAM\u003c/td\u003e\n\u003ctd align=\"center\"\u003evit-h\u003c/td\u003e\n\u003ctd align=\"center\"\u003eswin-b\u003c/td\u003e\n\u003ctd align=\"center\"\u003e\u003cb\u003e49.6\u003c/b\u003e\u003c/td\u003e\n\u003ctd align=\"center\"\u003e\u003ca href=\"seginw/test_seginw_hq.sh\"\u003escript\u003c/a\u003e\u003c/td\u003e\n\u003ctd align=\"center\"\u003e\u003ca href=\"seginw/logs/grounded_hqsam.log\"\u003elog\u003c/a\u003e\u003c/td\u003e\n\u003ctd align=\"center\"\u003e\u003ca href=\"https://huggingface.co/sam-hq-team/SegInW/resolve/main/result/grounded_hqsam.zip\"\u003eresult\u003c/a\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/tbody\u003e\u003c/table\u003e\n\nPlease change the current folder path to:\n```\ncd seginw\n```\nWe provide detailed evaluation instructions and metrics on SegInW in [Grounded-HQ-SAM evaluation](seginw/README.md).\n\n### **Light HQ-SAM vs MobileSAM on COCO**\nWe propose [Light HQ-SAM](#model-checkpoints) based on the tiny vit image encoder provided by MobileSAM. We provide quantitative comparison on zero-shot COCO performance, speed and memory below. Try Light HQ-SAM at [here](#getting-started).\n\n\u003ctable\u003e\u003ctbody\u003e\n\u003c!-- START TABLE --\u003e\n\u003c!-- TABLE HEADER --\u003e\n\u003cth valign=\"bottom\"\u003eModel\u003c/th\u003e\n\u003cth valign=\"bottom\"\u003eEncoder\u003c/th\u003e\n\u003cth valign=\"bottom\"\u003eAP\u003c/th\u003e\n\u003cth valign=\"bottom\"\u003eAP@L\u003c/th\u003e\n\u003cth valign=\"bottom\"\u003eAP@M\u003c/th\u003e\n\u003cth valign=\"bottom\"\u003eAP@S\u003c/th\u003e\n\u003cth valign=\"bottom\"\u003eModel Params (MB)\u003c/th\u003e\n\u003cth valign=\"bottom\"\u003eFPS\u003c/th\u003e\n\u003cth valign=\"bottom\"\u003eMemory (GB)\u003c/th\u003e\n\u003c!-- TABLE BODY --\u003e\n\u003c!-- ROW: maskformer2_R50_bs16_50ep --\u003e\n \u003ctr\u003e\u003ctd align=\"left\"\u003eMobileSAM\u003c/td\u003e\n\u003ctd align=\"center\"\u003eTinyViT\u003c/td\u003e\n\u003ctd align=\"center\"\u003e44.3\u003c/td\u003e\n\u003ctd align=\"center\"\u003e61.8\u003c/td\u003e\n\u003ctd align=\"center\"\u003e48.1\u003c/td\u003e\n\u003ctd align=\"center\"\u003e28.8\u003c/td\u003e\n\u003ctd align=\"center\"\u003e38.6\u003c/td\u003e\n\u003ctd align=\"center\"\u003e44.8\u003c/td\u003e\n\u003ctd align=\"center\"\u003e3.7\u003c/td\u003e\n\u003c/tr\u003e\n\u003c!-- ROW: maskformer2_R101_bs16_50ep --\u003e\n \u003ctr\u003e\u003ctd align=\"left\"\u003e\u003cb\u003eLight HQ-SAM\u003c/b\u003e\u003c/td\u003e\n \u003ctd align=\"center\"\u003eTinyViT\u003c/td\u003e\n\u003ctd align=\"center\"\u003e\u003cb\u003e45.0\u003c/b\u003e\u003c/td\u003e\n\u003ctd align=\"center\"\u003e62.8\u003c/td\u003e\n\u003ctd align=\"center\"\u003e48.8\u003c/td\u003e\n\u003ctd align=\"center\"\u003e29.2\u003c/td\u003e\n\u003ctd align=\"center\"\u003e40.3\u003c/td\u003e\n\u003ctd align=\"center\"\u003e41.2\u003c/td\u003e\n\u003ctd align=\"center\"\u003e3.7\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/tbody\u003e\u003c/table\u003e\n\nNote: For the COCO dataset, we use the same SOTA detector FocalNet-DINO trained on the COCO dataset as our and Mobile sam's box prompt generator.\n\n\n### **ONNX export**\nHQ-SAM's lightweight mask decoder can be exported to ONNX format so that it can be run in any environment that supports ONNX runtime. Export the model with\n```\npython scripts/export_onnx_model.py --checkpoint \u003cpath/to/checkpoint\u003e --model-type \u003cmodel_type\u003e --output \u003cpath/to/output\u003e\n```\nSee the [example notebook](https://colab.research.google.com/drive/11U2La49c2IxahzJkAV-EzPqEH3cz_5hq?usp=sharing) for details on how to combine image preprocessing via HQ-SAM's backbone with mask prediction using the ONNX model. It is recommended to use the latest stable version of PyTorch for ONNX export.\n\n\nCitation\n---------------\nIf you find HQ-SAM useful in your research or refer to the provided baseline results, please star :star: this repository and consider citing :pencil::\n```\n@inproceedings{sam_hq,\n    title={Segment Anything in High Quality},\n    author={Ke, Lei and Ye, Mingqiao and Danelljan, Martin and Liu, Yifan and Tai, Yu-Wing and Tang, Chi-Keung and Yu, Fisher},\n    booktitle={NeurIPS},\n    year={2023}\n}  \n```\nRelated high-quality instance segmentation work:\n```\n@inproceedings{transfiner,\n    title={Mask Transfiner for High-Quality Instance Segmentation},\n    author={Ke, Lei and Danelljan, Martin and Li, Xia and Tai, Yu-Wing and Tang, Chi-Keung and Yu, Fisher},\n    booktitle={CVPR},\n    year={2022}\n}\n```\n\n## Acknowledgments\n- Thanks [SAM](https://github.com/facebookresearch/segment-anything), [Grounded SAM](https://github.com/IDEA-Research/Grounded-Segment-Anything) and [MobileSAM](https://github.com/ChaoningZhang/MobileSAM) for their public code and released models.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsyscv%2Fsam-hq","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsyscv%2Fsam-hq","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsyscv%2Fsam-hq/lists"}