{"id":18936197,"url":"https://github.com/aim-uofa/matcher","last_synced_at":"2025-04-08T09:12:49.755Z","repository":{"id":168008164,"uuid":"643103052","full_name":"aim-uofa/Matcher","owner":"aim-uofa","description":"[ICLR'24] Matcher: Segment Anything with One Shot Using All-Purpose Feature Matching","archived":false,"fork":false,"pushed_at":"2024-12-16T08:46:56.000Z","size":27106,"stargazers_count":481,"open_issues_count":24,"forks_count":31,"subscribers_count":27,"default_branch":"main","last_synced_at":"2025-04-01T08:41:51.515Z","etag":null,"topics":["dinov2","generalist-model","in-context-segmentation","matcher","sam"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2305.13310","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/aim-uofa.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-05-20T05:36:47.000Z","updated_at":"2025-03-30T03:40:36.000Z","dependencies_parsed_at":"2024-12-29T06:12:58.637Z","dependency_job_id":null,"html_url":"https://github.com/aim-uofa/Matcher","commit_stats":{"total_commits":32,"total_committers":3,"mean_commits":"10.666666666666666","dds":0.40625,"last_synced_commit":"9a846e3bf5eaf70d630a90bd59e7e2582f9a6acb"},"previous_names":["aim-uofa/matcher"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aim-uofa%2FMatcher","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aim-uofa%2FMatcher/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aim-uofa%2FMatcher/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aim-uofa%2FMatcher/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/aim-uofa","download_url":"https://codeload.github.com/aim-uofa/Matcher/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247809964,"owners_count":20999816,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dinov2","generalist-model","in-context-segmentation","matcher","sam"],"created_at":"2024-11-08T12:06:20.754Z","updated_at":"2025-04-08T09:12:49.698Z","avatar_url":"https://github.com/aim-uofa.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\n\u003ch1\u003eMatcher: Segment Anything with One Shot Using All-Purpose Feature Matching \u003c/h1\u003e\n\n[Yang Liu](https://scholar.google.com/citations?user=9JcQ2hwAAAAJ\u0026hl=en)\u003csup\u003e1*\u003c/sup\u003e, \u0026nbsp; \n[Muzhi Zhu](https://scholar.google.com/citations?user=064gBH4AAAAJ\u0026hl=en)\u003csup\u003e1*\u003c/sup\u003e, \u0026nbsp; \nHengtao Li\u003csup\u003e1*\u003c/sup\u003e, \u0026nbsp;\n[Hao Chen](https://stan-haochen.github.io/)\u003csup\u003e1\u003c/sup\u003e, \u0026nbsp;\n[Xinlong Wang](https://www.xloong.wang/)\u003csup\u003e2\u003c/sup\u003e, \u0026nbsp;\n[Chunhua Shen](https://cshen.github.io/)\u003csup\u003e1\u003c/sup\u003e\n\n\u003csup\u003e1\u003c/sup\u003e[Zhejiang University](https://www.zju.edu.cn/english/), \u0026nbsp;\n\u003csup\u003e2\u003c/sup\u003e[Beijing Academy of Artificial Intelligence](https://www.baai.ac.cn/english.html)\n\nICLR 2024\n\n\u003c/div\u003e\n\n## 🚀 Overview\n\u003cdiv align=\"center\"\u003e\n\u003cimg width=\"800\" alt=\"image\" src=\"figs/framework.png\"\u003e\n\u003c/div\u003e\n\n## 📖 Description\n\n\nPowered by large-scale pre-training, vision foundation models exhibit significant potential in open-world image understanding. However, unlike large language models that excel at directly \ntackling various language tasks, vision foundation models require a task-specific model structure followed by fine-tuning on specific tasks. In this work, we present **Matcher**, \na novel perception paradigm that utilizes off-the-shelf vision foundation models to address various perception tasks. Matcher can segment anything by using an in-context example without training. \nAdditionally, we design three effective components within the Matcher framework to collaborate with these foundation models and unleash their full potential in diverse perception tasks. \nMatcher demonstrates impressive generalization performance across various segmentation tasks, all without training. Our visualization results further showcase the open-world generality and flexibility of Matcher when applied to images in the wild.\n\n[Paper](https://arxiv.org/abs/2305.13310)\n\n## ℹ️ News\n\n- 2024.1 Matcher has been accepted to ICLR 2024!\n- 2024.1 Matcher supports [Semantic-SAM](https://github.com/UX-Decoder/Semantic-SAM) for better part segmentation.\n- 2024.1 We provide a Gradio Demo.\n- 2024.1 Release code of one-shot semantic segmentation and one-shot part segmentation tasks.\n\n\n## 📖 Recommanded Works\n\n- SINE: A Simple Image Segmentation Framework via In-Context Examples. [GitHub](https://github.com/aim-uofa/SINE).\n- DiffewS: Unleashing the Potential of the Diffusion Model in Few-shot Semantic Segmentation. [GitHub](https://github.com/aim-uofa/DiffewS).\n## 🗓️ TODO\n- [x] Gradio Demo \n- [x] Release code of one-shot semantic segmentation and one-shot part segmentation tasks\n- [ ] Release code and models for VOS\n\n\n## 🏗️ Installation\n\nSee [installation instructions](INSTALL.md).\n\n## 👻 Getting Started\n\nSee [Preparing Datasets for Matcher](datasets/README.md).\n\nSee [Getting Started with Matcher](GETTING_STARTED.md).\n\n\n## 🖼️ Demo\n\n### One-Shot Semantic Segmantation\n\n\u003cdiv align=\"center\"\u003e\n\u003cimg width=\"800\" alt=\"image\" src=\"figs/oss.png\"\u003e\n\u003c/div\u003e\n\n### One-Shot Object Part Segmantation\n\n\u003cdiv align=\"center\"\u003e\n\u003cimg width=\"800\" alt=\"image\" src=\"figs/part.png\"\u003e\n\u003c/div\u003e\n\n### Cross-Style Object and Object Part Segmentation\n\n\u003cdiv align=\"center\"\u003e\n\u003cimg width=\"800\" alt=\"image\" src=\"figs/cross_style.png\"\u003e\n\u003c/div\u003e\n\n### Controllable Mask Output\n\n\u003cdiv align=\"center\"\u003e\n\u003cimg width=\"800\" alt=\"image\" src=\"figs/control.png\"\u003e\n\u003c/div\u003e\n\n\n### Video Object Segmentation\n\nhttps://github.com/aim-uofa/Matcher/assets/119775808/9ff9502d-7d2a-43bc-a8ef-01235097d62b\n\n\n## 🎫 License\n\nFor academic use, this project is licensed under [the 2-clause BSD License](https://opensource.org/license/bsd-2-clause). \nFor commercial use, please contact [Chunhua Shen](mailto:chhshen@gmail.com).\n\n\n## 🖊️ Citation\n\n\nIf you find this project useful in your research, please consider to cite:\n\n\n```BibTeX\n@article{liu2023matcher,\n  title={Matcher: Segment Anything with One Shot Using All-Purpose Feature Matching},\n  author={Liu, Yang and Zhu, Muzhi and Li, Hengtao and Chen, Hao and Wang, Xinlong and Shen, Chunhua},\n  journal={arXiv preprint arXiv:2305.13310},\n  year={2023}\n}\n```\n\n## Acknowledgement\n[SAM](https://github.com/facebookresearch/segment-anything), [DINOv2](https://github.com/facebookresearch/dinov2), [SegGPT](https://github.com/baaivision/Painter/tree/main/SegGPT), [HSNet](https://github.com/juhongm999/hsnet), [Semantic-SAM](https://github.com/UX-Decoder/Semantic-SAM) and [detectron2](https://github.com/facebookresearch/detectron2).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faim-uofa%2Fmatcher","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faim-uofa%2Fmatcher","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faim-uofa%2Fmatcher/lists"}