{"id":13442041,"url":"https://github.com/LiWentomng/Point2Mask","last_synced_at":"2025-03-20T13:32:02.381Z","repository":{"id":185894043,"uuid":"672486037","full_name":"LiWentomng/Point2Mask","owner":"LiWentomng","description":"The code for \"Point2Mask: Point-supervised Panoptic Segmentation via Optimal Transport\", ICCV2023","archived":false,"fork":false,"pushed_at":"2023-09-23T05:33:09.000Z","size":3512,"stargazers_count":65,"open_issues_count":7,"forks_count":6,"subscribers_count":6,"default_branch":"main","last_synced_at":"2024-10-28T05:12:38.206Z","etag":null,"topics":["optimal-transport","panoptic-segmentation","point2mask","points-supervision","single-point-supervision","weakly-supervised-segmentation"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/LiWentomng.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-07-30T08:48:33.000Z","updated_at":"2024-10-11T13:49:46.000Z","dependencies_parsed_at":null,"dependency_job_id":"536485f1-3de8-4a2f-86ee-c2f5a1395db3","html_url":"https://github.com/LiWentomng/Point2Mask","commit_stats":null,"previous_names":["liwentomng/point2mask"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LiWentomng%2FPoint2Mask","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LiWentomng%2FPoint2Mask/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LiWentomng%2FPoint2Mask/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LiWentomng%2FPoint2Mask/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/LiWentomng","download_url":"https://codeload.github.com/LiWentomng/Point2Mask/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244619148,"owners_count":20482369,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["optimal-transport","panoptic-segmentation","point2mask","points-supervision","single-point-supervision","weakly-supervised-segmentation"],"created_at":"2024-07-31T03:01:41.012Z","updated_at":"2025-03-20T13:32:02.374Z","avatar_url":"https://github.com/LiWentomng.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# Point2Mask: Point-supervised Panoptic Segmentation via Optimal Transport\n\u003e [Wentong Li](https://cslwt.github.io/), [Yuqian Yuan](https://yuqianyuan.github.io/), Song Wang, [Jianke Zhu](https://person.zju.edu.cn/jkzhu), Jianshu Li, Jian Liu, and [Lei Zhang](https://www4.comp.polyu.edu.hk/~cslzhang/)\n\u003e\n\u003e **[Paper](https://arxiv.org/pdf/2308.01779.pdf) (arXiv). ICCV2023.**\n\n\n\u003cimg src=\"./data/point2mask.jpg\" width=\"800px\"\u003e\n\n\n\n## Environment Setup\n\n    conda create -n point2mask python=3.8 -y\n    conda activate point2mask\n    pip install torch==1.9.0+cu111 torchvision==0.10.0+cu111 torchaudio==0.9.0 -f https://download.pytorch.org/whl/torch_stable.html\n    pip install openmim\n    mim install mmdet==2.17.0\n    mim install mmcv-full==1.3.9\n    git clone https://github.com/LiWentomng/Point2Mask.git\n    cd Point2Mask\n    pip install -r requirements.txt\n    sh setup.sh\n    \n\n\n## Model Zoo\n\n### 1.Single-point Supervision(P1) \n#### Pascal VOC\n|  Backbone  | Supervision |                                            Models                                            |  PQ  | PQ_th | PQ_st | \n|:----------:|:-----------:|:--------------------------------------------------------------------------------------------:|:----:|:-----:|:-----:|\n| ResNet-50  |     P1      | [model](https://drive.google.com/file/d/1nuVb1Hlgu7uqFw-zE_r2NP-BAAHTBXS-/view?usp=sharing)  | 53.7 | 51.9  | 90.5  | \n| ResNet-101 |     P1      | [model](https://drive.google.com/file/d/1ToLa3dE7Hczz_O8Ej9eYcpjA0z1G4hai/view?usp=sharing)  | 54.8 | 53.0  | 90.4  |\n|   Swin-L   |     P1      | [model](https://drive.google.com/file/d/1WblgH8xKZx0BFKU1qooVH4h7Fw_JxPeD/view?usp=sharing)  | 61.0 | 59.4  | 93.0  |\n\n#### COCO \n|  Backbone  | Supervision |                                            Models                                            |  PQ  | PQ_th | PQ_st | \n|:----------:|:-----------:|:--------------------------------------------------------------------------------------------:|:----:|:-----:|:-----:|\n| ResNet-50  |     P1      | [model](https://drive.google.com/file/d/1Y1GcOKy09y8lkaFqj5EPBMFgRUyKRI7c/view?usp=sharing)  | 32.4 | 32.6  | 32.2  | \n| ResNet-101 |     P1      | [model](https://drive.google.com/file/d/1ckTeH1kU7vMGWciol6zJfrLnWkZwKIrf/view?usp=sharing)  | 34.0 | 34.3  | 33.5  |\n|   Swin-L   |     P1      | [model](https://drive.google.com/file/d/1umLYLgFo9GlKY8QDv5jlss1y2OXmMmYs/view?usp=sharing)  | 37.0 | 37.0  | 36.9  |\n\n\n#### Pascal VOC with COCO P1-pretrained model.\n|  Backbone  | Supervision |                                            Models                                            |  PQ  | PQ_th | PQ_st | \n|:----------:|:-----------:|:--------------------------------------------------------------------------------------------:|:----:|:-----:|:-----:|\n| ResNet-50  |     P1      | [model](https://drive.google.com/file/d/1vd3SY_b4uUJW8WdIBb5zOkZUpd4quqMM/view?usp=sharing)  | 60.7 | 59.1  | 91.8  | \n| ResNet-101 |     P1      | [model](https://drive.google.com/file/d/1lP-Y-l66Bt-d2giIXAwb5JbK0u9RPVdr/view?usp=sharing)  | 63.2 | 61.8  | 92.3  |\n|   Swin-L   |     P1      | [model](https://drive.google.com/file/d/1t4-2RXt6vgNZLCt2SNH-hS_8I0mLz2pE/view?usp=sharing)  | 64.2 | 62.7  | 93.2  |\n\n\n### 2.Ten-point Supervision(P10) \n#### Pascal VOC\n|  Backbone  | Supervision |                                            Models                                            |  PQ  | PQ_th | PQ_st | \n|:----------:|:-----------:|:--------------------------------------------------------------------------------------------:|:----:|:-----:|:-----:|\n| ResNet-50  |     P10     | [model](https://drive.google.com/file/d/1m_KUOvB44IvROluTEU0SlRVA5UZxOlue/view?usp=sharing)  | 59.1 | 57.5  | 91.8  | \n| ResNet-101 |     P10     | [model](https://drive.google.com/file/d/1YkUIEynAn1vqslp22a24MTqZFLPLAU09/view?usp=sharing)  | 60.2 | 58.6  | 92.1  |\n\n\n#### COCO \n|  Backbone  | Supervision |                                            Models                                            |  PQ  | PQ_th | PQ_st | \n|:----------:|:-----------:|:--------------------------------------------------------------------------------------------:|:----:|:-----:|:-----:|\n| ResNet-50  |     P10     | [model](https://drive.google.com/file/d/1WORcd9gYZLMctKup98ooOl1pXOR6ywpH/view?usp=sharing)  | 32.4 | 32.6  | 32.2  | \n| ResNet-101 |     P10     | [model](https://drive.google.com/file/d/1JvMcx2JDX-jcyDhqShrXYwrgjYoLl5q3/view?usp=sharing)  | 36.7 | 37.3  | 35.7  |\n\n\n## Get Started\nWe use Pascal VOC and COCO datasets, please see [Preparing datasets](./data/README.md) for Point2Mask. \n\n\n### Demo\nTo test our model with an input image, please run `demo.py`\n```shell\npython  demo.py --config-file ./configs/point2mask/voc/point2mask_voc_wsup_r50.py --weights /path/to/coco_r50.pth --input image.jpg --out-file prediction.jpg\n```\n\n\n### Training\n\nFor VOC training:\n```shell\nCUDA_VISIBLE_DEVICE=0,1,2,3  tools/dist_train.sh configs/point2mask/voc/point2mask_voc_wsup_r50.py 4\n```\n\nFor COCO training:\n```shell\nCUDA_VISIBLE_DEVICE=0,1,2,3,4,5,6,7  tools/dist_train.sh configs/point2mask/coco/point2mask_coco_wsup_r50.py 8\n```\n\nNote: our models for Pascal VOC are trained with 3090/V100 gpus, and for COCO with A100 GPUs. The  Structured Edge (SE) model used for low-level edege is [here](https://drive.google.com/file/d/1V6DCug0lUC3C9w3WUOFS9PFZ5-1R2ZXV/view?usp=sharing). \n\n### Test\n\nFor PQ evaluation:\n```shell\nCUDA_VISIBLE_DEVICE=0,1,2,3  tools/dist_test.sh configs/point2mask/voc/point2mask_voc_wsup_r50.py work_dirs/xxx.pth 4  --eval pq\n```\n\nFor visual results:\n```shell\nCUDA_VISIBLE_DEVICE=0 tools/dist_test.sh configs/point2mask/voc/point2mask_voc_wsup_r50.py work_dirs/xxx.pth 1 --show-dir xxx\n```\n### Visual Examples\nVisual Results on COCO with ResNet-101.\n\u003cimg src=\"./data/visualization.jpg\" width=\"800px\"\u003e\n\nVisualization of the `learning high-level boundary map`.\n\u003cimg src=\"./data/highlevel_bundary_map.png\" width=\"800px\"\u003e\n\n## Acknowledgement\nCode is largely based on [PSPS](https://github.com/BraveGroup/PSPS), [Panoptic Segformer](https://github.com/zhiqi-li/Panoptic-SegFormer), [MMdetection](https://github.com/open-mmlab/mmdetection).  \nThanks for their great open-source projects! \n\n\n## Citation\n\n```BibTeX\n@inproceedings{point2mask,\n  title={Point2Mask: Point-supervised Panoptic Segmentation via Optimal Transport},\n  author={Wentong Li, Yuqian Yuan, Song Wang, Jianke Zhu, Jianshu Li, Jian Liu and Lei Zhang},\n  booktitle={Proceedinngs of IEEE International Conference on Computer Vision},\n  year={2023}\n}\n```\n\n\n    \n    \n\n\n\n\n\n\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FLiWentomng%2FPoint2Mask","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FLiWentomng%2FPoint2Mask","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FLiWentomng%2FPoint2Mask/lists"}