{"id":19932165,"url":"https://github.com/amazon-science/c2f-seg","last_synced_at":"2025-05-03T11:31:31.996Z","repository":{"id":213506282,"uuid":"734183459","full_name":"amazon-science/c2f-seg","owner":"amazon-science","description":"Official Implementation for ICCV'23 paper Coarse-to-Fine Amodal Segmentation with Shape Prior (C2F-Seg).","archived":false,"fork":false,"pushed_at":"2024-01-07T05:40:23.000Z","size":4426,"stargazers_count":18,"open_issues_count":2,"forks_count":0,"subscribers_count":4,"default_branch":"main","last_synced_at":"2024-01-07T06:50:38.103Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://jianxgao.github.io/C2F-Seg/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/amazon-science.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2023-12-21T04:25:47.000Z","updated_at":"2024-01-03T06:19:04.000Z","dependencies_parsed_at":"2024-01-07T06:46:43.364Z","dependency_job_id":null,"html_url":"https://github.com/amazon-science/c2f-seg","commit_stats":null,"previous_names":["amazon-science/c2f-seg"],"tags_count":0,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/amazon-science%2Fc2f-seg","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/amazon-science%2Fc2f-seg/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/amazon-science%2Fc2f-seg/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/amazon-science%2Fc2f-seg/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/amazon-science","download_url":"https://codeload.github.com/amazon-science/c2f-seg/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224360233,"owners_count":17298319,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-12T23:09:17.399Z","updated_at":"2024-11-12T23:09:18.031Z","avatar_url":"https://github.com/amazon-science.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ccenter\u003e\n\n# Coarse-to-Fine Amodal Segmentation with Shape Prior (C2F-Seg)\n\n[Jianxiong Gao](https://jianxgao.github.io/), [Xuelin Qian†](https://naiq.github.io/), [Yikai Wang](https://yikai-wang.github.io/), [Tianjun Xiao†](https://tianjunxiao.com/), [Tong He](https://hetong007.github.io/), [Zheng Zhang](https://www.amazon.science/author/zheng-zhang), [Yanwei Fu](https://yanweifu.github.io/)\n\n\n[![ArXiv](https://img.shields.io/badge/ArXiv-2308.16825-b31b1b.svg?logo=arXiv)](https://arxiv.org/abs/2308.16825)\n[![HomePage](https://img.shields.io/badge/HomePage-Visit-blue.svg?logo=homeadvisor\u0026logoColor=f5f5f5)](https://jianxgao.github.io/C2F-Seg/)\n[![Dataset](https://img.shields.io/badge/Dataset-MOViD_Amodal-F07B3F.svg)](https://data.dgl.ai/dataset/C2F-Seg/MOViD_A.tar)\n[![License](https://img.shields.io/badge/License-Apache%202.0-green.svg)](https://github.com/amazon-science/c2f-seg/blob/main/LICENSE) \n[![Visitor](https://hits.seeyoufarm.com/api/count/incr/badge.svg?url=https%3A%2F%2Fgithub.com%2Famazon-science%2Fc2f-seg\u0026count_bg=%2352D3D8\u0026title_bg=%23555555\u0026icon=\u0026icon_color=%23E7E7E7\u0026title=Visitors\u0026edge_flat=false)](https://hits.seeyoufarm.com)\n\n\u003c/center\u003e\n\nThis is the Official Implementation for ICCV'23 paper [Coarse-to-Fine Amodal Segmentation with Shape Prior](https://openaccess.thecvf.com/content/ICCV2023/papers/Gao_Coarse-to-Fine_Amodal_Segmentation_with_Shape_Prior_ICCV_2023_paper.pdf).\n\n\n# Introduction\n\n\u003cimg src='./imgs/C2F-Seg.jpg' width=\"100%\"\u003e\nC2F-Seg is a framework designed for amodal segementation. It first generates a coarse mask from the visible mask and visual features via the mask-and-predict procedure with transformers. Then this coarse amodal mask is refined with a convolutional module guided by human-imitated attention on visual features of the amodal object. The learning of visible mask is used as an auxiliary task in training, while in inference we only provide an estimation of amodal mask.\n\n# Environment Setup\n\n```bash\ngit clone https://github.com/amazon-science/c2f-seg.git\ncd c2f-seg\nconda env create -f environment.yml\n```\n\nIf conda is too slow, you can use:\n\n```bash\nconda create --name C2F-Seg python=3.10\nconda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia\npip install -e .[all]\n```\n\n# MOViD-Amodal\n\n\u003cimg src=\"./imgs/example.gif\" width=\"100%\"\u003e\n\n# Dataset and checkpoints\n\n| Dataset      |   $\\text{mIoU}_{full}$ |    $\\text{mIoU}_{occ}$      |  VQ Model        | C2F-Seg  |\n| :---         |   :---:                |   :---:                     |    :---:   |   :---:          | \n| [KINS](https://data.dgl.ai/dataset/C2F-Seg/KINS.tar)         |     82.22             |   53.60        |  [weight](https://data.dgl.ai/dataset/C2F-Seg/vqgan_KINS.pth), [config](https://data.dgl.ai/dataset/C2F-Seg/vqgan_KINS.yml)              | [weight](https://data.dgl.ai/dataset/C2F-Seg/c2f_seg_KINS.pth), [config](https://data.dgl.ai/dataset/C2F-Seg/c2f_seg_KINS.yml)    |\n| [COCOA](https://data.dgl.ai/dataset/C2F-Seg/COCOA.tar)        |     80.28              |    27.71       |  [weight](https://data.dgl.ai/dataset/C2F-Seg/vqgan_COCOA.pth), [config](https://data.dgl.ai/dataset/C2F-Seg/vqgan_COCOA.yml)            | [weight](https://data.dgl.ai/dataset/C2F-Seg/c2f_seg_COCOA.pth), [config](https://data.dgl.ai/dataset/C2F-Seg/c2f_seg_COCOA.yml)    |\n| [MOViD-Amodal](https://data.dgl.ai/dataset/C2F-Seg/MOViD_A.tar) |     71.67              |    36.13       |  [weight](https://data.dgl.ai/dataset/C2F-Seg/vqgan_MOViD_A.pth), [config](https://data.dgl.ai/dataset/C2F-Seg/vqgan_MOViD_A.yml)            | [weight](https://data.dgl.ai/dataset/C2F-Seg/c2f_seg_MOViD_A.pth), [config](https://data.dgl.ai/dataset/C2F-Seg/c2f_seg_MOViD_A.yml)    |\n\nPlease use the following commands to prepare the dataset and checkpoints:\n```bash\n# Example with KINS dataset\nbash download.sh KINS\nwget https://data.dgl.ai/dataset/C2F-Seg/KINS.tar\ntar -xvf KINS.tar\n# Important: Update the root_path in config files!\n```\n\n\n# Running Experiments\n\n### Evaluate model\n\n```bash\n# KINS\nCUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python -m torch.distributed.launch --nproc_per_node=8 \\\ntest_c2f_seg.py --dataset KINS --batch 1 --data_type image --vq_path KINS_vqgan --path KINS_c2f_seg\n\n# MOViD-Amodal\nCUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python -m torch.distributed.launch --nproc_per_node=8 \\\ntest_c2f_seg.py --dataset MOViD_A --batch 1 --data_type video --vq_path MOViD_A_vqgan --path MOViD_A_c2f_seg \n```\n\n### Train VQ model\n\n```bash\n# KINS\nCUDA_VISIBLE_DEVICES=0 python train_vq.py --dataset KINS --path KINS_vqgan --check_point_path ../check_points\n# MOViD-Amodal\nCUDA_VISIBLE_DEVICES=0 python train_vq.py --dataset MOViD_A --path MOViD_A_vqgan --check_point_path ../check_points\n```\n\n### Train C2F-Seg\n\n```bash\n# KINS\nCUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python -m torch.distributed.launch --nproc_per_node=8 \\\ntrain_c2f_seg.py --dataset KINS --batch 16 --data_type image --vq_path KINS_vqgan --path KINS_c2f_seg\n# MOViD-Amodal\nCUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python -m torch.distributed.launch --nproc_per_node=8 \\\ntrain_c2f_seg.py --dataset MOViD_A --batch 1 --data_type video --vq_path MOViD_A_vqgan --path MOViD_A_c2f_seg \n```\n\n\n# Citation\nIf you find our paper useful for your research and applications, please cite using this BibTeX:\n```\n@inproceedings{gao2023coarse,\n  title={Coarse-to-Fine Amodal Segmentation with Shape Prior},\n  author={Gao, Jianxiong and Qian, Xuelin and Wang, Yikai and Xiao, Tianjun and He, Tong and Zhang, Zheng and Fu, Yanwei},\n  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},\n  pages={1262--1271},\n  year={2023}\n}\n```\n\n## Security\n\nSee [CONTRIBUTING](CONTRIBUTING.md#security-issue-notifications) for more information.\n\n## License\n\nThis project is licensed under the Apache-2.0 License.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Famazon-science%2Fc2f-seg","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Famazon-science%2Fc2f-seg","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Famazon-science%2Fc2f-seg/lists"}