{"id":28136994,"url":"https://github.com/res2net/res2net-maskrcnn","last_synced_at":"2025-05-14T16:21:36.771Z","repository":{"id":182996375,"uuid":"201641891","full_name":"Res2Net/Res2Net-maskrcnn","owner":"Res2Net","description":"Res2Net for Instance segmentation and Object detection using MaskRCNN","archived":false,"fork":false,"pushed_at":"2020-10-08T12:53:47.000Z","size":22663,"stargazers_count":54,"open_issues_count":2,"forks_count":19,"subscribers_count":5,"default_branch":"master","last_synced_at":"2024-04-23T00:23:22.820Z","etag":null,"topics":["instance-segmentation","mask-rcnn","object-detection","res2net"],"latest_commit_sha":null,"homepage":"https://mmcheng.net/res2net/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Res2Net.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2019-08-10T14:36:10.000Z","updated_at":"2024-03-22T07:38:59.000Z","dependencies_parsed_at":"2023-07-22T10:49:01.805Z","dependency_job_id":null,"html_url":"https://github.com/Res2Net/Res2Net-maskrcnn","commit_stats":null,"previous_names":["res2net/res2net-maskrcnn"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Res2Net%2FRes2Net-maskrcnn","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Res2Net%2FRes2Net-maskrcnn/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Res2Net%2FRes2Net-maskrcnn/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Res2Net%2FRes2Net-maskrcnn/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Res2Net","download_url":"https://codeload.github.com/Res2Net/Res2Net-maskrcnn/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254179889,"owners_count":22027884,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["instance-segmentation","mask-rcnn","object-detection","res2net"],"created_at":"2025-05-14T16:20:52.850Z","updated_at":"2025-05-14T16:21:36.764Z","avatar_url":"https://github.com/Res2Net.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Res2Net for Instance segmentation and Object detection using MaskRCNN\n**Since the MaskRCNN-benchmark of facebook is deprecated, we suggest to use our mmdetection based res2net for object detection and instance segmentation to get the SOTA performance on both two tasks.** https://github.com/Res2Net/mmdetection \n## Update\n- 2020.3.10 The mmdetection based implementation of object detection and instance segmentation using Res2Net_v1b has the SOTA performance. We have released our code on: https://github.com/Res2Net/mmdetection. Our Res2Net_v1b achieves a considerable performance gain on mmdetection compared with existing backbone models.\n## Introduction\nThis repo uses *MaskRCNN* as the baseline method for Instance segmentation and Object detection. We use the [maskrcnn-benchmark](https://github.com/facebookresearch/maskrcnn-benchmark) as the baseline. \n\n[Res2Net](https://github.com/gasvn/Res2Net) is a powerful backbone architecture that can be easily implemented into state-of-the-art models by replacing the bottleneck with Res2Net module.\nMore detail can be found on [ \"Res2Net: A New Multi-scale Backbone Architecture\"](https://arxiv.org/pdf/1904.01169.pdf) and our [project page](https://mmcheng.net/res2net) .\n\n\n## Performance\n\n### Results on Instance segmentation and Object detection using MaskRCNN.\n\n**Performance on Instance segmentation:**\n\n| Backbone     | Setting      | AP      | AP50    | AP75     | APs       |APm        |    APl    |\n|--------------|--------------|---------|---------|----------|-----------|-----------|-----------|\n| ResNet-50    |  64w         | 33.9    | 55.2    | 36.0     | 14.8      | 36.0      | 50.9      |\n| ResNet-50    |  48w×2s      | 34.2    | 55.6    | 36.3     | 14.9      | 36.8      | 50.9      |\n| Res2Net-50   |  26w×4s      | 35.6    | 57.6    | 37.6     | 15.7      | 37.9      | 53.7      |\n| Res2Net-50   |  18w×6s      | 35.7    | 57.5    | 38.1     | 15.4      | 38.1      | 53.7      |\n| Res2Net-50   |  14w×8s      | 35.3    | 57.0    | 37.5     | 15.6      | 37.5      | 53.4      |\n| ResNet-101   |  64w         | 35.5    | 57.0    | 37.9     | 16.0      | 38.2      | 52.9      |\n| Res2Net-101  |  26w×4s      | 37.1    | 59.4    | 39.4     | 16.6      | 40.0      | 55.6      |\n\n**Performance on Object detection:**\n\n| Backbone     | Setting      | AP      | AP50    | AP75     | APs       |APm        |    APl    |\n|--------------|--------------|---------|---------|----------|-----------|-----------|-----------|\n| ResNet-50    |  64w         | 37.5    | 58.4    | 40.3     | 20.6      | 40.1      | 49.7      |\n| ResNet-50    |  48w×2s      | 38.0    | 58.9    | 41.3     | 20.5      | 41.0      | 49.9      |\n| Res2Net-50   |  26w×4s      | 39.6    | 60.9    | 43.1     | 22.0      | 42.3      | 52.8      |\n| Res2Net-50   |  18w×6s      | 39.9    | 60.9    | 43.3     | 21.8      | 42.8      | 53.7      |\n| Res2Net-50   |  14w×8s      | 39.1    | 60.2    | 42.1     | 21.7      | 41.7      | 52.8      |\n| ResNet-101   |  64w         | 39.6    | 60.6    | 43.2     | 22.0      | 43.2      | 52.4      |\n| Res2Net-101  |  26w×4s      | 41.8    | 62.6    | 45.6     | 23.4      | 45.5      | 55.6      |\n\n\n(Noted that pretrained models trained with pytorch usually achieve slightly worse performance than the caffe pretrained models, we took [advice](https://github.com/facebookresearch/maskrcnn-benchmark/issues/504) from the author of MaskRCNN-benchmark to use 2x schedule in all experiments including baseline and our method.)\n\n## Applications\nOther applications such as Classification,  Semantic segmentation, pose estimation, Class activation map can be found on https://mmcheng.net/res2net/ and https://github.com/gasvn/Res2Net .\n\n## Installation\n(**This repo is based on the [mask-rcnn benchmark]((https://github.com/facebookresearch/maskrcnn-benchmark))**, the useage is remain the same with the original repo.)\n\nCheck [INSTALL.md](INSTALL.md) for installation instructions.\n\n\n## Perform training on COCO dataset\n\nFor the following examples to work, you need to first install `maskrcnn_benchmark`.\n\nYou will also need to download the COCO dataset.\nWe recommend to symlink the path to the coco dataset to `datasets/` as follows\n\nWe use `minival` and `valminusminival` sets from [Detectron](https://github.com/facebookresearch/Detectron/blob/master/detectron/datasets/data/README.md#coco-minival-annotations)\n\n```bash\n# symlink the coco dataset\ncd ~/github/maskrcnn-benchmark\nmkdir -p datasets/coco\nln -s /path_to_coco_dataset/annotations datasets/coco/annotations\nln -s /path_to_coco_dataset/train2014 datasets/coco/train2014\nln -s /path_to_coco_dataset/test2014 datasets/coco/test2014\nln -s /path_to_coco_dataset/val2014 datasets/coco/val2014\n# or use COCO 2017 version\nln -s /path_to_coco_dataset/annotations datasets/coco/annotations\nln -s /path_to_coco_dataset/train2017 datasets/coco/train2017\nln -s /path_to_coco_dataset/test2017 datasets/coco/test2017\nln -s /path_to_coco_dataset/val2017 datasets/coco/val2017\n\n# for pascal voc dataset:\nln -s /path_to_VOCdevkit_dir datasets/voc\n```\n\nP.S. `COCO_2017_train` = `COCO_2014_train` + `valminusminival` , `COCO_2017_val` = `minival`\n      \n\nYou can also configure your own paths to the datasets.\nFor that, all you need to do is to modify `maskrcnn_benchmark/config/paths_catalog.py` to\npoint to the location where your dataset is stored.\nYou can also create a new `paths_catalog.py` file which implements the same two classes,\nand pass it as a config argument `PATHS_CATALOG` during training.\n\n### Single GPU training\n\nMost of the configuration files that we provide assume that we are running on 8 GPUs.\nIn order to be able to run it on fewer GPUs, there are a few possibilities:\n\n**1. Run the following without modifications**\n\n```bash\npython /path_to_maskrcnn_benchmark/tools/train_net.py --config-file \"/path/to/config/file.yaml\"\n```\nThis should work out of the box and is very similar to what we should do for multi-GPU training.\nBut the drawback is that it will use much more GPU memory. The reason is that we set in the\nconfiguration files a global batch size that is divided over the number of GPUs. So if we only\nhave a single GPU, this means that the batch size for that GPU will be 8x larger, which might lead\nto out-of-memory errors.\n\nIf you have a lot of memory available, this is the easiest solution.\n\n**2. Modify the cfg parameters**\n\nIf you experience out-of-memory errors, you can reduce the global batch size. But this means that\nyou'll also need to change the learning rate, the number of iterations and the learning rate schedule.\n\nHere is an example for Mask R-CNN Res2Net-50 FPN with the 2x schedule:\n```bash\npython tools/train_net.py --config-file \"configs/pytorch_mask_rcnn_R2_50_s4_FPN_2x.yaml\" SOLVER.IMS_PER_BATCH 2 SOLVER.BASE_LR 0.0025 SOLVER.MAX_ITER 720000 SOLVER.STEPS \"(480000, 640000)\" TEST.IMS_PER_BATCH 1\n```\nThis follows the [scheduling rules from Detectron.](https://github.com/facebookresearch/Detectron/blob/master/configs/getting_started/tutorial_1gpu_e2e_faster_rcnn_R-50-FPN.yaml#L14-L30)\nNote that we have multiplied the number of iterations by 8x (as well as the learning rate schedules),\nand we have divided the learning rate by 8x.\n\nWe also changed the batch size during testing, but that is generally not necessary because testing\nrequires much less memory than training.\n\n\n### Multi-GPU training\nWe use internally `torch.distributed.launch` in order to launch\nmulti-gpu training. This utility function from PyTorch spawns as many\nPython processes as the number of GPUs we want to use, and each Python\nprocess will only use a single GPU.\n\n```bash\nexport NGPUS=8\npython -m torch.distributed.launch --nproc_per_node=$NGPUS /path_to_maskrcnn_benchmark/tools/train_net.py --config-file \"configs/pytorch_mask_rcnn_R2_50_s4_FPN_2x.yaml\"\n```\n\n\n## Inference in a few lines\nWe provide a helper class to simplify writing inference pipelines using pre-trained models.\nHere is how we would do it. Run this from the `demo` folder:\n```python\nfrom maskrcnn_benchmark.config import cfg\nfrom predictor import COCODemo\n\nconfig_file = \"../configs/pytorch_mask_rcnn_R2_50_s4_FPN_2x.yaml\"\n\n# update the config options with the config file\ncfg.merge_from_file(config_file)\n# manual override some options\ncfg.merge_from_list([\"MODEL.DEVICE\", \"cpu\"])\n\ncoco_demo = COCODemo(\n    cfg,\n    min_image_size=800,\n    confidence_threshold=0.7,\n)\n# load image and then run prediction\nimage = ...\npredictions = coco_demo.run_on_opencv_image(image)\n```\n\n## Adding your own dataset\n\nThis implementation adds support for COCO-style datasets.\nBut adding support for training on a new dataset can be done as follows:\n```python\nfrom maskrcnn_benchmark.structures.bounding_box import BoxList\n\nclass MyDataset(object):\n    def __init__(self, ...):\n        # as you would do normally\n\n    def __getitem__(self, idx):\n        # load the image as a PIL Image\n        image = ...\n\n        # load the bounding boxes as a list of list of boxes\n        # in this case, for illustrative purposes, we use\n        # x1, y1, x2, y2 order.\n        boxes = [[0, 0, 10, 10], [10, 20, 50, 50]]\n        # and labels\n        labels = torch.tensor([10, 20])\n\n        # create a BoxList from the boxes\n        boxlist = BoxList(boxes, image.size, mode=\"xyxy\")\n        # add the labels to the boxlist\n        boxlist.add_field(\"labels\", labels)\n\n        if self.transforms:\n            image, boxlist = self.transforms(image, boxlist)\n\n        # return the image, the boxlist and the idx in your dataset\n        return image, boxlist, idx\n\n    def get_img_info(self, idx):\n        # get img_height and img_width. This is used if\n        # we want to split the batches according to the aspect ratio\n        # of the image, as it can be more efficient than loading the\n        # image from disk\n        return {\"height\": img_height, \"width\": img_width}\n```\nThat's it. You can also add extra fields to the boxlist, such as segmentation masks\n(using `structures.segmentation_mask.SegmentationMask`), or even your own instance type.\n\nFor a full example of how the `COCODataset` is implemented, check [`maskrcnn_benchmark/data/datasets/coco.py`](maskrcnn_benchmark/data/datasets/coco.py).\n\n\n\n\n## Citation\nIf you find this work or code is helpful in your research, please cite:\n```\n@article{gao2019res2net,\n  title={Res2Net: A New Multi-scale Backbone Architecture},\n  author={Gao, Shang-Hua and Cheng, Ming-Ming and Zhao, Kai and Zhang, Xin-Yu and Yang, Ming-Hsuan and Torr, Philip},\n  journal={IEEE TPAMI},\n  year={2020},\n  doi={10.1109/TPAMI.2019.2938758}, \n}\n@misc{massa2018mrcnn,\nauthor = {Massa, Francisco and Girshick, Ross},\ntitle = {{maskrnn-benchmark: Fast, modular reference implementation of Instance Segmentation and Object Detection algorithms in PyTorch}},\nyear = {2018},\nhowpublished = {\\url{https://github.com/facebookresearch/maskrcnn-benchmark}},\nnote = {Accessed: [Insert date here]}\n}\n```\n## Acknowledge\nThis code is partly borrowed from [maskrcnn-benchmark](https://github.com/facebookresearch/maskrcnn-benchmark/).\nmaskrcnn-benchmark is released under the MIT license. See [LICENSE](LICENSE) for additional details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fres2net%2Fres2net-maskrcnn","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fres2net%2Fres2net-maskrcnn","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fres2net%2Fres2net-maskrcnn/lists"}