{"id":13613530,"url":"https://github.com/ChunmengLiu1/MECPformer","last_synced_at":"2025-04-13T15:33:17.467Z","repository":{"id":169410235,"uuid":"601056001","full_name":"ChunmengLiu1/MECPformer","owner":"ChunmengLiu1","description":"Implementation of \"MECPformer: Multi-estimations Complementary Patch with CNN-Transformers for Weakly Supervised Semantic Segmentation\"","archived":false,"fork":false,"pushed_at":"2023-11-20T09:18:24.000Z","size":2806,"stargazers_count":6,"open_issues_count":1,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-11-07T21:42:27.154Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ChunmengLiu1.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-02-13T09:19:33.000Z","updated_at":"2023-11-27T12:29:09.000Z","dependencies_parsed_at":"2023-11-20T10:32:24.838Z","dependency_job_id":"4ba7b3e2-f2c8-4880-945c-ed67e5eaf75a","html_url":"https://github.com/ChunmengLiu1/MECPformer","commit_stats":null,"previous_names":["chunmengliu1/mecpformer"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ChunmengLiu1%2FMECPformer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ChunmengLiu1%2FMECPformer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ChunmengLiu1%2FMECPformer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ChunmengLiu1%2FMECPformer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ChunmengLiu1","download_url":"https://codeload.github.com/ChunmengLiu1/MECPformer/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248736215,"owners_count":21153552,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T20:00:49.405Z","updated_at":"2025-04-13T15:33:14.742Z","avatar_url":"https://github.com/ChunmengLiu1.png","language":"Python","funding_links":[],"categories":["Papers"],"sub_categories":["2023"],"readme":"# MECPformer\nOfficial Implementation of the paper: [MECPformer: Multi-estimations Complementary Patch with CNN-Transformers for Weakly Supervised Semantic Segmentation](https://arxiv.org/pdf/2303.10689.pdf).\n\nAccepted to Neural Computing and Applications.\n\n\u003cp align=\"left\"\u003e\u003cimg src=\"imgs/framework.png\" alt=\"outline\" width=\"95%\"\u003e\u003c/p\u003e\n\n## Abstract\nThe initial seed based on the convolutional neural network (CNN) for weakly supervised semantic segmentation always highlights the most\ndiscriminative regions but fails to identify the global target information. Methods based on transformers have been proposed successively\nbenefiting from the advantage of capturing long-range feature representations. However, we observe a flaw regardless of the gifts based on\nthe transformer. Given a class, the initial seeds generated based on the transformer may invade regions belonging to other classes. Inspired\nby the mentioned issues, we devise a simple yet effective method with Multi-estimations Complementary Patch (MECP) strategy and Adaptive\nConflict Module (ACM), dubbed MECPformer. Given an image, we manipulate it with the MECP strategy at different epochs, and the network mines and deeply fuses the semantic information at different levels. In addition, ACM adaptively removes conflicting pixels and exploits the network self-training capability to mine potential target information. Without bells and whistles, our MECPformer has reached new state-of-the-art 72.0% mIoU on the PASCAL VOC 2012 and 42.4% on MS COCO 2014 dataset. \n\n## Prerequisite\n\n#### 1. install dependencies \nUbuntu 20.04, with Python 3.6 and the following python dependencies.\n\n```pip install -r requirements.txt```\n\n#### 2. Download dataset \nDownload [the PASCAL VOC 2012 development kit](http://host.robots.ox.ac.uk/pascal/VOC/voc2012).\n\n#### 3. Download pretrained weights\nDownload [Conformer-S pretrained weights](https://drive.google.com/file/d/1qjLDy8MYU_TV2hspyYNCXeWrWho360qa/view?usp=share_link).\n\n#### 4. Download saliency map\nDownload [saliency map](https://drive.google.com/file/d/1n7hVi8U2ylBMjz_bECsl_wSAmlRqnVr8/view?usp=share_link).\n\n## Usage\n\n#### 1. Run the run.sh script for training MECPformer in the initial seeds stage\n\n```bash run.sh```\n\n#### 2. Train semantic segmentation network\nTo train DeepLab-v2, we refer to [deeplab-pytorch](https://github.com/kazuto1011/deeplab-pytorch).\n\n## Testing\n\n#### Download our trained weights\n| Stage       | Backbone   | Google drive | mIoU (%) |\n|--------------|------------|--------------|--------------|\n| Initial seeds | Conformer-S | [Weights](https://drive.google.com/file/d/1pYWyNjPTLVrZ5nlvipmgjRa_bDP6qSRE/view?usp=share_link)  | 66.6 |\n| Final predict | ResNet101 | [Weights](https://drive.google.com/file/d/1thFitdq1MYq_BWKmNIlKZp1BCQzZFJQi/view?usp=share_link)  | 72.0 |\n\n## Acknowledgements\nThis code is borrowed from [TransCAM](https://github.com/liruiwen/TransCAM), [CPN](https://github.com/Ferenas/CPN), and [deeplab-pytorch](https://github.com/kazuto1011/deeplab-pytorch).\n\n## Citing MECPformer\n\nIf you use MECPformer in your research, please use the follwing entry:\n\n```\n@article{liu2023mecpformer,\n  title={MECPformer: Multi-estimations Complementary Patch with CNN-Transformers for Weakly Supervised Semantic Segmentation},\n  author={Liu, Chunmeng and Li, Guangyao and Shen, Yao and Wang, Ruiqi},\n  journal={Neural Comput \u0026 Applic},\n  year={2023}\n}\n\n```\n\nor\n\n```\n\nLiu, C., Li, G., Shen, Y. et al. MECPformer: multi-estimations complementary patch with CNN-transformers for weakly supervised semantic segmentation. Neural Comput \u0026 Applic 35, 23249–23264 (2023). https://doi.org/10.1007/s00521-023-08816-2\n\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FChunmengLiu1%2FMECPformer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FChunmengLiu1%2FMECPformer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FChunmengLiu1%2FMECPformer/lists"}