{"id":13737458,"url":"https://github.com/SamsungLabs/ritm_interactive_segmentation","last_synced_at":"2025-05-08T14:31:33.298Z","repository":{"id":39342511,"uuid":"338424600","full_name":"SamsungLabs/ritm_interactive_segmentation","owner":"SamsungLabs","description":"Reviving Iterative Training with Mask Guidance for Interactive Segmentation","archived":false,"fork":false,"pushed_at":"2023-09-25T15:46:51.000Z","size":5907,"stargazers_count":632,"open_issues_count":27,"forks_count":125,"subscribers_count":13,"default_branch":"master","last_synced_at":"2024-11-14T09:07:25.993Z","etag":null,"topics":["hrnets","interactive-segmentation","pretrained-models","pytorch","segmentation"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/SamsungLabs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2021-02-12T20:21:57.000Z","updated_at":"2024-11-08T07:17:30.000Z","dependencies_parsed_at":"2023-09-25T21:13:13.495Z","dependency_job_id":null,"html_url":"https://github.com/SamsungLabs/ritm_interactive_segmentation","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SamsungLabs%2Fritm_interactive_segmentation","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SamsungLabs%2Fritm_interactive_segmentation/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SamsungLabs%2Fritm_interactive_segmentation/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SamsungLabs%2Fritm_interactive_segmentation/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/SamsungLabs","download_url":"https://codeload.github.com/SamsungLabs/ritm_interactive_segmentation/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224737409,"owners_count":17361345,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["hrnets","interactive-segmentation","pretrained-models","pytorch","segmentation"],"created_at":"2024-08-03T03:01:48.595Z","updated_at":"2025-05-08T14:31:33.284Z","avatar_url":"https://github.com/SamsungLabs.png","language":"Python","readme":"## Reviving Iterative Training with Mask Guidance for Interactive Segmentation \n\n\u003cp align=\"center\"\u003e\n    \u003ca href=\"https://paperswithcode.com/sota/interactive-segmentation-on-grabcut?p=reviving-iterative-training-with-mask\"\u003e\n        \u003cimg src=\"https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/reviving-iterative-training-with-mask/interactive-segmentation-on-grabcut\"/\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://paperswithcode.com/sota/interactive-segmentation-on-berkeley?p=reviving-iterative-training-with-mask\"\u003e\n        \u003cimg src=\"https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/reviving-iterative-training-with-mask/interactive-segmentation-on-berkeley\"/\u003e\n    \u003c/a\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./assets/img/teaser.gif\" alt=\"drawing\", width=\"420\"/\u003e\n  \u003cimg src=\"./assets/img/miou_berkeley.png\" alt=\"drawing\", width=\"400\"/\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n    \u003ca href=\"https://arxiv.org/abs/2102.06583\"\u003e\n        \u003cimg src=\"https://img.shields.io/badge/arXiv-2102.06583-b31b1b\"/\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://colab.research.google.com/github/saic-vul/ritm_interactive_segmentation/blob/master/notebooks/colab_test_any_model.ipynb\"\u003e\n        \u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://opensource.org/licenses/MIT\"\u003e\n        \u003cimg src=\"https://img.shields.io/badge/License-MIT-yellow.svg\" alt=\"The MIT License\"/\u003e\n    \u003c/a\u003e\n\u003c/p\u003e\n\nThis repository provides the source code for training and testing state-of-the-art click-based interactive segmentation models with the official PyTorch implementation of the following paper:\n\n\u003e **Reviving Iterative Training with Mask Guidance for Interactive Segmentation**\u003cbr\u003e\n\u003e [Konstantin Sofiiuk](https://github.com/ksofiyuk), [Ilia Petrov](https://github.com/ptrvilya), [Anton Konushin](https://scholar.google.com/citations?user=ZT_k-wMAAAAJ) \u003cbr\u003e\n\u003e Samsung Research\u003cbr\u003e\n\u003e https://arxiv.org/abs/2102.06583\n\u003e\n\u003e **Abstract:** *Recent works on click-based interactive segmentation have demonstrated state-of-the-art results by \n\u003e using various inference-time optimization schemes. These methods are considerably more computationally expensive \n\u003e compared to feedforward approaches, as they require performing backward passes through a network during inference and \n\u003e are hard to deploy on mobile frameworks that usually support only forward passes. In this paper, we extensively \n\u003e evaluate various design choices for interactive segmentation and discover that new state-of-the-art results can be \n\u003e obtained without any additional optimization schemes. Thus, we propose a simple feedforward model for click-based \n\u003e interactive segmentation that employs the segmentation masks from previous steps. It allows not only to segment an \n\u003e entirely new object, but also to start with an external mask and correct it. When analyzing the performance of models\n\u003e trained on different datasets, we observe that the choice of a training dataset greatly impacts the quality of \n\u003e interactive segmentation. We find that the models trained on a combination of COCO and LVIS with diverse and \n\u003e high-quality annotations show performance superior to all existing models.*\n\n\n## Setting up an environment\n\nThis framework is built using Python 3.6 and relies on the PyTorch 1.4.0+. The following command installs all \nnecessary packages:\n\n```.bash\npip3 install -r requirements.txt\n```\n\nYou can also use our [Dockerfile](./Dockerfile) to build a container with the configured environment. \n\nIf you want to run training or testing, you must configure the paths to the datasets in [config.yml](config.yml).\n\n## Interactive Segmentation Demo\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./assets/img/demo_gui.jpg\" alt=\"drawing\" width=\"99%\"/\u003e\n\u003c/p\u003e\n\nThe GUI is based on TkInter library and its Python bindings. You can try our interactive demo with any of the \n[provided models](#pretrained-models). Our scripts automatically detect the architecture of the loaded model, just \nspecify the path to the corresponding checkpoint.\n\nExamples of the script usage:\n\n```.bash\n# This command runs interactive demo with HRNet18 ITER-M model from cfg.INTERACTIVE_MODELS_PATH on GPU with id=0\n# --checkpoint can be relative to cfg.INTERACTIVE_MODELS_PATH or absolute path to the checkpoint\npython3 demo.py --checkpoint=hrnet18_cocolvis_itermask_3p --gpu=0\n\n# This command runs interactive demo with HRNet18 ITER-M model from /home/demo/isegm/weights/\n# If you also do not have a lot of GPU memory, you can reduce --limit-longest-size (default=800)\npython3 demo.py --checkpoint=/home/demo/fBRS/weights/hrnet18_cocolvis_itermask_3p --limit-longest-size=400\n\n# You can try the demo in CPU only mode\npython3 demo.py --checkpoint=hrnet18_cocolvis_itermask_3p --cpu\n```\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cb\u003eRunning demo in docker\u003c/b\u003e\u003c/summary\u003e\n\u003cpre\u003e\u003ccode\u003e# activate xhost\nxhost +\ndocker run -v \"$PWD\":/tmp/ \\\n           -v /tmp/.X11-unix:/tmp/.X11-unix \\\n           -e DISPLAY=$DISPLAY \u0026lt;id-or-tag-docker-built-image\u0026gt; \\\n           python3 demo.py --checkpoint resnet34_dh128_sbd --cpu\n\u003c/code\u003e\u003c/pre\u003e\n\u003c/details\u003e\n\n**Controls**:\n\n| Key                                                           | Description                        |\n| ------------------------------------------------------------- | ---------------------------------- |\n| \u003ckbd\u003eLeft Mouse Button\u003c/kbd\u003e                                  | Place a positive click             |\n| \u003ckbd\u003eRight Mouse Button\u003c/kbd\u003e                                 | Place a negative click             |\n| \u003ckbd\u003eScroll Wheel\u003c/kbd\u003e                                       | Zoom an image in and out           |\n| \u003ckbd\u003eRight Mouse Button\u003c/kbd\u003e + \u003cbr\u003e \u003ckbd\u003eMove Mouse\u003c/kbd\u003e    | Move an image                      |\n| \u003ckbd\u003eSpace\u003c/kbd\u003e                                              | Finish the current object mask     |\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cb\u003eInitializing the ITER-M models with an external segmentation mask\u003c/b\u003e\u003c/summary\u003e\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./assets/img/modifying_external_mask.jpg\" alt=\"drawing\" width=\"80%\"/\u003e\n\u003c/p\u003e\n  \nAccording to our paper, ITER-M models take an image, encoded user input, and a previous step mask as their input. Moreover, a user can initialize the model with an external mask before placing any clicks and correct this mask using the same interface.  As it turns out, our models successfully handle this situation and make it possible to change the mask.\n\n\nTo initialize any ITER-M model with an external mask use the \"Load mask\" button in the menu bar.\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cb\u003eInteractive segmentation options\u003c/b\u003e\u003c/summary\u003e\n\u003cul\u003e\n    \u003cli\u003eZoomIn (can be turned on/off using the checkbox)\u003c/li\u003e\n    \u003cul\u003e\n        \u003cli\u003e\u003ci\u003eSkip clicks\u003c/i\u003e - the number of clicks to skip before using ZoomIn.\u003c/li\u003e\n        \u003cli\u003e\u003ci\u003eTarget size\u003c/i\u003e - ZoomIn crop is resized so its longer side matches this value (increase for large objects).\u003c/li\u003e\n        \u003cli\u003e\u003ci\u003eExpand ratio\u003c/i\u003e - object bbox is rescaled with this ratio before crop.\u003c/li\u003e\n        \u003cli\u003e\u003ci\u003eFixed crop\u003c/i\u003e - ZoomIn crop is resized to (\u003ci\u003eTarget size\u003c/i\u003e, \u003ci\u003eTarget size\u003c/i\u003e).\u003c/li\u003e\n    \u003c/ul\u003e\n    \u003cli\u003eBRS parameters (BRS type can be changed using the dropdown menu)\u003c/li\u003e\n    \u003cul\u003e\n        \u003cli\u003e\u003ci\u003eNetwork clicks\u003c/i\u003e - the number of first clicks that are included in the network's input. Subsequent clicks are processed only using BRS  (NoBRS ignores this option).\u003c/li\u003e\n        \u003cli\u003e\u003ci\u003eL-BFGS-B max iterations\u003c/i\u003e - the maximum number of function evaluation for each step of optimization in BRS (increase for better accuracy and longer computational time for each click).\u003c/li\u003e  \n    \u003c/ul\u003e\n    \u003cli\u003eVisualisation parameters\u003c/li\u003e\n    \u003cul\u003e\n        \u003cli\u003e\u003ci\u003ePrediction threshold\u003c/i\u003e slider adjusts the threshold for binarization of probability map for the current object.\u003c/li\u003e \n        \u003cli\u003e\u003ci\u003eAlpha blending coefficient\u003c/i\u003e slider adjusts the intensity of all predicted masks.\u003c/li\u003e\n        \u003cli\u003e\u003ci\u003eVisualisation click radius\u003c/i\u003e slider adjusts the size of red and green dots depicting clicks.\u003c/li\u003e\n    \u003c/ul\u003e\n\u003c/ul\u003e\n\u003c/details\u003e\n\n## Datasets\n\nWe train all our models on SBD and COCO+LVIS and evaluate them on GrabCut, Berkeley, DAVIS, SBD and PascalVOC. We also provide links to additional datasets: ADE20k and OpenImages, that are used in ablation study.\n\n| Dataset   |                      Description             |           Download Link              |\n|-----------|----------------------------------------------|:------------------------------------:|\n|ADE20k     |  22k images with 434k instances (total)      |  [official site][ADE20k]             |\n|OpenImages |  944k images with 2.6M instances (total)     |  [official site][OpenImages]         |\n|MS COCO    |  118k images with 1.2M instances (train)     |  [official site][MSCOCO]             |\n|LVIS v1.0  |  100k images with 1.2M instances (total)     |  [official site][LVIS]               |\n|COCO+LVIS* |  99k images with 1.5M instances (train)      |  [original LVIS images][LVIS] + \u003cbr\u003e [our combined annotations][COCOLVIS_annotation] |\n|SBD        |  8498 images with 20172 instances for (train)\u003cbr\u003e2857 images with 6671 instances for (test) |[official site][SBD]|\n|Grab Cut   |  50 images with one object each (test)       |  [GrabCut.zip (11 MB)][GrabCut]      |\n|Berkeley   |  96 images with 100 instances (test)         |  [Berkeley.zip (7 MB)][Berkeley]     |\n|DAVIS      |  345 images with one object each (test)      |  [DAVIS.zip (43 MB)][DAVIS]          |\n|Pascal VOC |  1449 images with 3417 instances (validation)|  [official site][PascalVOC]          |\n|COCO_MVal  |  800 images with 800 instances (test)        |  [COCO_MVal.zip (127 MB)][COCO_MVal] |\n\n[ADE20k]: http://sceneparsing.csail.mit.edu/\n[OpenImages]: https://storage.googleapis.com/openimages/web/download.html\n[MSCOCO]: https://cocodataset.org/#download\n[LVIS]: https://www.lvisdataset.org/dataset\n[SBD]: http://home.bharathh.info/pubs/codes/SBD/download.html\n[GrabCut]: https://github.com/saic-vul/fbrs_interactive_segmentation/releases/download/v1.0/GrabCut.zip\n[Berkeley]: https://github.com/saic-vul/fbrs_interactive_segmentation/releases/download/v1.0/Berkeley.zip\n[DAVIS]: https://github.com/saic-vul/fbrs_interactive_segmentation/releases/download/v1.0/DAVIS.zip\n[PascalVOC]: http://host.robots.ox.ac.uk/pascal/VOC/\n[COCOLVIS_annotation]: https://github.com/saic-vul/ritm_interactive_segmentation/releases/download/v1.0/cocolvis_annotation.tar.gz\n[COCO_MVal]: https://github.com/saic-vul/fbrs_interactive_segmentation/releases/download/v1.0/COCO_MVal.zip\n\nDon't forget to change the paths to the datasets in [config.yml](config.yml) after downloading and unpacking.\n\n(*) To prepare COCO+LVIS, you need to download original LVIS v1.0, then download and unpack our \npre-processed annotations that are obtained by combining COCO and LVIS dataset into the folder with LVIS v1.0.\n\n## Testing\n\n### Pretrained models\nWe provide pretrained models with different backbones for interactive segmentation.\n\nYou can find model weights and evaluation results in the tables below:\n\n\u003ctable\u003e\n    \u003cthead align=\"center\"\u003e\n        \u003ctr\u003e\n            \u003cth rowspan=\"2\"\u003e\u003cspan style=\"font-weight:bold\"\u003eTrain\u003c/span\u003e\u003cbr\u003e\u003cspan style=\"font-weight:bold\"\u003eDataset\u003c/span\u003e\u003c/th\u003e\n            \u003cth rowspan=\"2\"\u003eModel\u003c/th\u003e\n            \u003cth colspan=\"2\"\u003eGrabCut\u003c/th\u003e\n            \u003cth\u003eBerkeley\u003c/th\u003e\n            \u003cth colspan=\"2\"\u003eSBD\u003c/th\u003e    \n            \u003cth colspan=\"2\"\u003eDAVIS\u003c/th\u003e\n            \u003cth\u003ePascal\u003cbr\u003eVOC\u003c/th\u003e\n            \u003cth\u003eCOCO\u003cbr\u003eMVal\u003c/th\u003e\n        \u003c/tr\u003e\n        \u003ctr\u003e\n            \u003ctd\u003eNoC\u003cbr\u003e85%\u003c/td\u003e\n            \u003ctd\u003eNoC\u003cbr\u003e90%\u003c/td\u003e\n            \u003ctd\u003eNoC\u003cbr\u003e90%\u003c/td\u003e\n            \u003ctd\u003eNoC\u003cbr\u003e85%\u003c/td\u003e\n            \u003ctd\u003eNoC\u003cbr\u003e90%\u003c/td\u003e\n            \u003ctd\u003eNoC\u003cbr\u003e85%\u003c/td\u003e\n            \u003ctd\u003eNoC\u003cbr\u003e90%\u003c/td\u003e\n            \u003ctd\u003eNoC\u003cbr\u003e85%\u003c/td\u003e\n            \u003ctd\u003eNoC\u003cbr\u003e90%\u003c/td\u003e\n        \u003c/tr\u003e\n    \u003c/thead\u003e\n    \u003ctbody align=\"center\"\u003e\n        \u003ctr\u003e\n            \u003ctd rowspan=\"1\"\u003eSBD\u003c/td\u003e\n            \u003ctd align=\"left\"\u003e\u003ca href=\"https://github.com/saic-vul/ritm_interactive_segmentation/releases/download/v1.0/sbd_h18_itermask.pth\"\u003eHRNet18 IT-M\u003cbr\u003e(38.8 MB)\u003c/a\u003e\u003c/td\u003e\n            \u003ctd\u003e1.76\u003c/td\u003e\n            \u003ctd\u003e2.04\u003c/td\u003e\n            \u003ctd\u003e3.22\u003c/td\u003e\n            \u003ctd\u003e\u003cb\u003e3.39\u003c/b\u003e\u003c/td\u003e\n            \u003ctd\u003e\u003cb\u003e5.43\u003c/b\u003e\u003c/td\u003e\n            \u003ctd\u003e4.94\u003c/td\u003e\n            \u003ctd\u003e6.71\u003c/td\u003e\n            \u003ctd\u003e\u003cins\u003e2.51\u003c/ins\u003e\u003c/td\u003e\n            \u003ctd\u003e4.39\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr\u003e\n            \u003ctd rowspan=\"4\"\u003eCOCO+\u003cbr\u003eLVIS\u003c/td\u003e\n            \u003ctd align=\"left\"\u003e\u003ca href=\"https://github.com/saic-vul/ritm_interactive_segmentation/releases/download/v1.0/coco_lvis_h18_baseline.pth\"\u003eHRNet18\u003cbr\u003e(38.8 MB)\u003c/a\u003e\u003c/td\u003e\n            \u003ctd\u003e1.54\u003c/td\u003e\n            \u003ctd\u003e1.70\u003c/td\u003e\n            \u003ctd\u003e2.48\u003c/td\u003e\n            \u003ctd\u003e4.26\u003c/td\u003e\n            \u003ctd\u003e6.86\u003c/td\u003e\n            \u003ctd\u003e4.79\u003c/td\u003e\n            \u003ctd\u003e6.00\u003c/td\u003e\n            \u003ctd\u003e2.59\u003c/td\u003e\n            \u003ctd\u003e3.58\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr\u003e\n            \u003ctd align=\"left\"\u003e\u003ca href=\"https://github.com/saic-vul/ritm_interactive_segmentation/releases/download/v1.0/coco_lvis_h18s_itermask.pth\"\u003eHRNet18s IT-M\u003cbr\u003e(16.5 MB)\u003c/a\u003e\u003c/td\u003e\n            \u003ctd\u003e1.54\u003c/td\u003e\n            \u003ctd\u003e1.68\u003c/td\u003e\n            \u003ctd\u003e2.60\u003c/td\u003e\n            \u003ctd\u003e4.04\u003c/td\u003e\n            \u003ctd\u003e6.48\u003c/td\u003e\n            \u003ctd\u003e4.70\u003c/td\u003e\n            \u003ctd\u003e5.98\u003c/td\u003e\n            \u003ctd\u003e2.57\u003c/td\u003e\n            \u003ctd\u003e3.33\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr\u003e\n            \u003ctd align=\"left\"\u003e\u003ca href=\"https://github.com/saic-vul/ritm_interactive_segmentation/releases/download/v1.0/coco_lvis_h18_itermask.pth\"\u003eHRNet18 IT-M\u003cbr\u003e(38.8 MB)\u003c/a\u003e\u003c/td\u003e\n            \u003ctd\u003e\u003cb\u003e1.42\u003c/b\u003e\u003c/td\u003e\n            \u003ctd\u003e\u003cb\u003e1.54\u003c/b\u003e\u003c/td\u003e\n            \u003ctd\u003e\u003cins\u003e2.26\u003c/ins\u003e\u003c/td\u003e\n            \u003ctd\u003e3.80\u003c/td\u003e\n            \u003ctd\u003e6.06\u003c/td\u003e\n            \u003ctd\u003e\u003cins\u003e4.36\u003c/ins\u003e\u003c/td\u003e\n            \u003ctd\u003e\u003cins\u003e5.74\u003c/ins\u003e\u003c/td\u003e\n            \u003ctd\u003e\u003cb\u003e2.28\u003c/b\u003e\u003c/td\u003e\n            \u003ctd\u003e\u003cins\u003e2.98\u003c/ins\u003e\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr\u003e\n            \u003ctd align=\"left\"\u003e\u003ca href=\"https://github.com/saic-vul/ritm_interactive_segmentation/releases/download/v1.0/coco_lvis_h32_itermask.pth\"\u003eHRNet32 IT-M\u003cbr\u003e(119 MB)\u003c/a\u003e\u003c/td\u003e\n            \u003ctd\u003e\u003cins\u003e1.46\u003c/ins\u003e\u003c/td\u003e\n            \u003ctd\u003e\u003cins\u003e1.56\u003c/ins\u003e\u003c/td\u003e\n            \u003ctd\u003e\u003cb\u003e2.10\u003c/b\u003e\u003c/td\u003e\n            \u003ctd\u003e\u003cins\u003e3.59\u003c/ins\u003e\u003c/td\u003e\n            \u003ctd\u003e\u003cins\u003e5.71\u003c/ins\u003e\u003c/td\u003e\n            \u003ctd\u003e\u003cb\u003e4.11\u003c/b\u003e\u003c/td\u003e\n            \u003ctd\u003e\u003cb\u003e5.34\u003c/b\u003e\u003c/td\u003e\n            \u003ctd\u003e2.57\u003c/td\u003e\n            \u003ctd\u003e\u003cb\u003e2.97\u003c/b\u003e\u003c/td\u003e\n        \u003c/tr\u003e\n    \u003c/tbody\u003e\n\u003c/table\u003e\n\n\n### Evaluation\n\nWe provide the script to test all the presented models in all possible configurations on GrabCut, Berkeley, DAVIS, \nPascal VOC, and SBD. To test a model, you should download its weights and put them in `./weights` folder (you can \nchange this path in the [config.yml](config.yml), see `INTERACTIVE_MODELS_PATH` variable). To test any of our models, \njust specify the path to the corresponding checkpoint. Our scripts automatically detect the architecture of the loaded model.\n\nThe following command runs the NoC evaluation on all test datasets (other options are displayed using '-h'):\n\n```.bash\npython3 scripts/evaluate_model.py \u003cbrs-mode\u003e --checkpoint=\u003ccheckpoint-name\u003e\n```\n\nExamples of the script usage:\n```.bash\n# This command evaluates HRNetV2-W18-C+OCR ITER-M model in NoBRS mode on all Datasets.\npython3 scripts/evaluate_model.py NoBRS --checkpoint=hrnet18_cocolvis_itermask_3p\n\n# This command evaluates HRNet-W18-C-Small-v2+OCR ITER-M model in f-BRS-B mode on all Datasets.\npython3 scripts/evaluate_model.py f-BRS-B --checkpoint=hrnet18s_cocolvis_itermask_3p\n\n# This command evaluates HRNetV2-W18-C+OCR ITER-M model in NoBRS mode on GrabCut and Berkeley datasets.\npython3 scripts/evaluate_model.py NoBRS --checkpoint=hrnet18_cocolvis_itermask_3p --datasets=GrabCut,Berkeley\n```\n\n### Jupyter notebook\n\nYou can also interactively experiment with our models using [test_any_model.ipynb](./notebooks/test_any_model.ipynb) Jupyter notebook.\n\n## Training\n\nWe provide the scripts for training our models on the SBD dataset. You can start training with the following commands:\n```.bash\n# ResNet-34 non-iterative baseline model\npython3 train.py models/noniterative_baselines/r34_dh128_cocolvis.py --gpus=0 --workers=4 --exp-name=first-try\n\n# HRNet-W18-C-Small-v2+OCR ITER-M model\npython3 train.py models/iter_mask/hrnet18s_cocolvis_itermask_3p.py --gpus=0 --workers=4 --exp-name=first-try\n\n# HRNetV2-W18-C+OCR ITER-M model\npython3 train.py models/iter_mask/hrnet18_cocolvis_itermask_3p.py --gpus=0,1 --workers=6 --exp-name=first-try\n\n# HRNetV2-W32-C+OCR ITER-M model\npython3 train.py models/iter_mask/hrnet32_cocolvis_itermask_3p.py --gpus=0,1,2,3 --workers=12 --exp-name=first-try\n```\n\nFor each experiment, a separate folder is created in the `./experiments` with Tensorboard logs, text logs, \nvisualization and checkpoints. You can specify another path in the [config.yml](config.yml) (see `EXPS_PATH` \nvariable).\n\nPlease note that we trained ResNet-34 and HRNet-18s on 1 GPU, HRNet-18 on 2 GPUs, HRNet-32 on 4 GPUs \n(we used Nvidia Tesla P40 for training). To train on a different GPU you should adjust the batch size using the command\nline argument `--batch-size` or change the default value in a model script.\n\nWe used the pre-trained HRNetV2 models from [the official repository](https://github.com/HRNet/HRNet-Image-Classification). \nIf you want to train interactive segmentation with these models, you need to download the weights and specify the paths to \nthem in [config.yml](config.yml).\n\n## License\n\nThe code is released under the MIT License. It is a short, permissive software license. Basically, you can do whatever you want as long as you include the original copyright and license notice in any copy of the software/source. \n## Citation\n\nIf you find this work is useful for your research, please cite our papers:\n```bibtex\n@inproceedings{ritm2022,\n  title={Reviving iterative training with mask guidance for interactive segmentation},\n  author={Sofiiuk, Konstantin and Petrov, Ilya A and Konushin, Anton},\n  booktitle={2022 IEEE International Conference on Image Processing (ICIP)},\n  pages={3141--3145},\n  year={2022},\n  organization={IEEE}\n}\n\n@inproceedings{fbrs2020,\n   title={f-brs: Rethinking backpropagating refinement for interactive segmentation},\n   author={Sofiiuk, Konstantin and Petrov, Ilia and Barinova, Olga and Konushin, Anton},\n   booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},\n   pages={8623--8632},\n   year={2020}\n}\n```\n","funding_links":[],"categories":["Python"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FSamsungLabs%2Fritm_interactive_segmentation","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FSamsungLabs%2Fritm_interactive_segmentation","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FSamsungLabs%2Fritm_interactive_segmentation/lists"}