{"id":22069084,"url":"https://kumuji.github.io/ugains/","last_synced_at":"2025-07-24T07:32:02.235Z","repository":{"id":187554502,"uuid":"676178786","full_name":"kumuji/ugains","owner":"kumuji","description":"[GCPR 2023] UGainS: Uncertainty Guided Anomaly Instance Segmentation","archived":false,"fork":false,"pushed_at":"2024-07-31T11:51:09.000Z","size":1082,"stargazers_count":9,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-07-31T13:04:39.925Z","etag":null,"topics":["anomaly-detection","anomaly-segmentation","computer-vision","deep-learning","deep-neural-networks","instance-segmentation","pytorch","segment-anything","segment-anything-model"],"latest_commit_sha":null,"homepage":"https://vision.rwth-aachen.de/ugains","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kumuji.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-08-08T15:53:25.000Z","updated_at":"2024-07-31T11:51:12.000Z","dependencies_parsed_at":"2024-08-07T00:05:53.193Z","dependency_job_id":null,"html_url":"https://github.com/kumuji/ugains","commit_stats":null,"previous_names":["kumuji/ugains"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kumuji%2Fugains","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kumuji%2Fugains/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kumuji%2Fugains/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kumuji%2Fugains/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kumuji","download_url":"https://codeload.github.com/kumuji/ugains/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":227421420,"owners_count":17775011,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["anomaly-detection","anomaly-segmentation","computer-vision","deep-learning","deep-neural-networks","instance-segmentation","pytorch","segment-anything","segment-anything-model"],"created_at":"2024-11-30T20:04:39.705Z","updated_at":"2024-11-30T20:07:37.064Z","avatar_url":"https://github.com/kumuji.png","language":"Python","funding_links":[],"categories":["Paper List"],"sub_categories":["Follow-up Papers"],"readme":"# UGainS: Uncertainty Guided Anomaly Instance Segmentation (GCPR 2023)\n\u003cdiv align=\"center\"\u003e\n\u003ca href=\"https://nekrasov.dev/\"\u003eAlexey Nekrasov\u003c/a\u003e\u003csup\u003e1\u003c/sup\u003e, \u003ca href=\"https://www.vision.rwth-aachen.de/person/10/\"\u003eAlexander Hermans\u003c/a\u003e\u003csup\u003e1\u003c/sup\u003e, \u003ca href=\"https://www.linkedin.com/in/lars-kuhnert/\"\u003eLars Kuhnert\u003c/a\u003e\u003csup\u003e2\u003c/sup\u003e, \u003ca href=\"https://www.vision.rwth-aachen.de/person/1/\"\u003eBastian Leibe\u003c/a\u003e\u003csup\u003e1\u003c/sup\u003e\n\n\u003csup\u003e1\u003c/sup\u003eRWTH Aachen University, \u003csup\u003e2\u003c/sup\u003eFord Motor Company\n\nUGainS is an approach that provides anomaly instance segmentation and per-pixel anomaly scores.\n\n\u003ca href=\"https://pytorch.org/get-started/locally/\"\u003e\u003cimg alt=\"PyTorch\" src=\"https://img.shields.io/badge/PyTorch-ee4c2c?logo=pytorch\u0026logoColor=white\"\u003e\u003c/a\u003e\n\u003ca href=\"https://pytorchlightning.ai/\"\u003e\u003cimg alt=\"Lightning\" src=\"https://img.shields.io/badge/-Lightning-792ee5?logo=pytorchlightning\u0026logoColor=white\"\u003e\u003c/a\u003e\n\u003ca href=\"https://hydra.cc/\"\u003e\u003cimg alt=\"Config: Hydra\" src=\"https://img.shields.io/badge/Config-Hydra-89b8cd\"\u003e\u003c/a\u003e\n\u003ca href=\"https://black.readthedocs.io/en/stable/\"\u003e\u003cimg alt=\"Code style: black\" src=\"https://img.shields.io/badge/code%20style-black-black.svg\"\u003e\u003c/a\u003e\n\n[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/ugains-uncertainty-guided-anomaly-instance/object-detection-on-oodis)](https://paperswithcode.com/sota/object-detection-on-oodis?p=ugains-uncertainty-guided-anomaly-instance)\n[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/ugains-uncertainty-guided-anomaly-instance/instance-segmentation-on-oodis)](https://paperswithcode.com/sota/instance-segmentation-on-oodis?p=ugains-uncertainty-guided-anomaly-instance)\n\n![teaser](./docs/github_teaser.jpg)\n\n\u003c/div\u003e\n\u003cbr\u003e\u003cbr\u003e\n\n[[Project Webpage](https://vision.rwth-aachen.de/ugains)] [[arXiv](https://arxiv.org/abs/2308.02046)]\n\n## News\n\n* **2023-07-14**: Paper Accepted at [GCPR 2023](https://www.dagm-gcpr.de/year/2023)\n* **2023-08-03**: Paper on arXiv\n* **2024-04-18**: Code release\n* **2024-06-15**: New benchmark! [OoDIS: Anomaly Instance Segmentation Benchmark](https://vision.rwth-aachen.de/oodis)\n\n## Code\nI took a long time to release the code, mainly because it was a lot of hacking together at the time and didn't really have a concise structure with multiple commented lines instead of having configs and hardwired paths all over the code.\nTo make the code understandable, I reduced the context a bit.\nThis version only contains experiments for a single dataset, but provides a clean overview of the project.\nPlease feel free to open a github issue if you have any questions or need help with the code.\n\n## Installation\n\n### Prerequisites\n- Python 3.11\n- CUDA 11.8\n- cuDNN 8.6.0\n\n### Setting Up Your Environment\nIt is recommended to use a virtual environment for the project to manage dependencies effectively.\n\n1. **Create and activate a virtual environment** (optional but recommended):\n   ````bash\n   python -m venv venv\n   source venv/bin/activate\n   ````\n\n2. **Install dependencies**:\n   Ensure you have Poetry installed. If not, install Poetry using the following command:\n   ````bash\n   pip install poetry\n   ````\n\n   Install Mask2former ops:\n   ```\n   cd ugains/models/mask2former/pixel_decoder/ops/\n   bash make.sh\n   cd ../../../../..\n   ```\n\n   Install FPS:\n   ```\n   cd third_party/pointnet2\n   python setup.py install\n   ```\n\n   Then, install the project dependencies with Poetry:\n   ````bash\n   poetry install\n   ````\n\n## Example Usage\n\nTo run the model with the specified configuration, use the following command:\n```bash\npoetry run test datamodule=cityfishy model=sam_sampling logger=csv experiment=experiment description=description\n```\n\n## Dependencies\nThe project relies on several key Python packages, including but not limited to:\n- PyTorch (`torch\u003e=2.0.1`)\n- TorchVision (`torchvision\u003e=0.15.2`)\n- PyTorch Lightning (`pytorch-lightning\u003e=2.0.2`)\n- Hydra Core (`hydra-core\u003e=1.3.2`)\n- Weights \u0026 Biases (`wandb\u003e=0.15.2`)\n- Albumentations (`albumentations\u003e=1.3.0`)\n- Rich (`rich\u003e=13.3.5`)\n- Python-dotenv (`python-dotenv\u003e=1.0.0`)\n- Fire (`fire\u003e=0.5.0`)\n- Joblib (`joblib\u003e=1.2.0`)\n- Ninja (`ninja\u003e=1.11.1`)\n- GitPython (`gitpython\u003e=3.1.31`)\n- Pandas (`pandas\u003e=2.0.1`)\n- Seaborn (`seaborn\u003e=0.12.2`)\n- Matplotlib (`matplotlib\u003e=3.7.1`)\n- Numpy (`numpy\u003e=1.24.3`)\n- pycocotools\n- einops\n- git+https://github.com/mcordts/cityscapesScripts.git\n\n\n## Data Preparation\n```\ndata\n├── fs_lost_found\n│   ├── leftImg8bit\n│   └── gtCoarse\n│       ├── train\n│       └── test\n│           ├── xx_xxx_000000_000000_gtCoarse_labelTrainIds.png\n│           └── ...\n├── fs_lost_found_instance\n│   └── gtCoarse\n│       ├── train\n│       └── test\n│           ├── xx_xxx_000000_000000_gtCoarse_instanceIds.png\n│           └── ...\n├── ignore_mask.pth\n├── rude0fhk.ckpt\n└── sam_vit_h_4b8939.pth\n```\nInstance labels, Mask2Former model, and ignore mask could be downloaded [here](https://omnomnom.vision.rwth-aachen.de/data/ugains/).\nSam checkpoint could be downloaded from [here](https://github.com/facebookresearch/segment-anything).\nLost and Found images could be found [here](https://wwwlehre.dhbw-stuttgart.de/~sgehrig/lostAndFoundDataset/index.html), and validation images from Fishyscapes Lost and Found [here](https://fishyscapes.com/dataset).\n\n## BibTeX\n```\n@inproceedings{nekrasov2023ugains,\n  title     = {{UGainS: Uncertainty Guided Anomaly Instance Segmentation}},\n  author    = {Nekrasov, Alexey and Hermans, Alexander and Kuhnert, Lars and Leibe, Bastian},\n  booktitle = {GCPR},\n  year      = {2023}\n}\n```\n\n## Thanks\nA big thanks to the authors of Mask2Former, DenseHybrid, SegmentAnything, PEBAL and all the others.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/kumuji.github.io%2Fugains%2F","html_url":"https://awesome.ecosyste.ms/projects/kumuji.github.io%2Fugains%2F","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/kumuji.github.io%2Fugains%2F/lists"}