{"id":13489561,"url":"https://github.com/kkirchheim/pytorch-ood","last_synced_at":"2025-03-28T05:31:02.094Z","repository":{"id":63940845,"uuid":"527624781","full_name":"kkirchheim/pytorch-ood","owner":"kkirchheim","description":"👽 Out-of-Distribution Detection with PyTorch","archived":false,"fork":false,"pushed_at":"2025-02-26T09:37:47.000Z","size":2314,"stargazers_count":283,"open_issues_count":0,"forks_count":27,"subscribers_count":3,"default_branch":"dev","last_synced_at":"2025-03-26T18:03:11.639Z","etag":null,"topics":["anomaly-detection","confidence-estimation","deep-learning","neural-network","novelty-detection","ood-detection","open-set-recognition","pytorch"],"latest_commit_sha":null,"homepage":"https://pytorch-ood.readthedocs.io/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kkirchheim.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-08-22T15:31:36.000Z","updated_at":"2025-03-23T18:48:41.000Z","dependencies_parsed_at":"2023-01-14T15:45:17.045Z","dependency_job_id":"b05a3c6a-0de1-40aa-91f7-f3870014e9c5","html_url":"https://github.com/kkirchheim/pytorch-ood","commit_stats":null,"previous_names":[],"tags_count":17,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kkirchheim%2Fpytorch-ood","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kkirchheim%2Fpytorch-ood/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kkirchheim%2Fpytorch-ood/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kkirchheim%2Fpytorch-ood/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kkirchheim","download_url":"https://codeload.github.com/kkirchheim/pytorch-ood/tar.gz/refs/heads/dev","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245978200,"owners_count":20703675,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["anomaly-detection","confidence-estimation","deep-learning","neural-network","novelty-detection","ood-detection","open-set-recognition","pytorch"],"created_at":"2024-07-31T19:00:30.981Z","updated_at":"2025-03-28T05:31:02.088Z","avatar_url":"https://github.com/kkirchheim.png","language":"Python","funding_links":[],"categories":["Libraries","Python"],"sub_categories":[],"readme":"PyTorch Out-of-Distribution Detection\n****************************************\n\n|docs| |version| |license| |python-version| |downloads|\n\n\n.. |docs| image:: https://img.shields.io/badge/docs-online-blue?style=for-the-badge\n   :target: https://pytorch-ood.readthedocs.io/en/latest/\n   :alt: Documentation\n.. |version| image:: https://img.shields.io/pypi/v/pytorch-ood?color=light\u0026style=for-the-badge\n   :target: https://pypi.org/project/pytorch-ood/\n   :alt: License\n.. |license| image:: https://img.shields.io/pypi/l/pytorch-ood?style=for-the-badge\n   :target: https://gitlab.com/kkirchheim/pytorch-ood/-/blob/master/LICENSE\n   :alt: License\n.. |python-version| image:: https://img.shields.io/badge/-Python 3.8+-blue?logo=python\u0026logoColor=white\u0026style=for-the-badge\n   :target: https://www.python.org/\n   :alt: Python\n.. |downloads| image:: https://img.shields.io/pepy/dt/pytorch-ood?style=for-the-badge\n   :target: https://pepy.tech/project/pytorch-ood\n   :alt: Downloads\n\n.. image:: docs/_static/pytorch-ood-logo.jpg\n   :align: center\n   :width: 100%\n   :alt: pytorch-ood-logo\n\n\nA Python library for Out-of-Distribution (OOD) Detection with Deep Neural Networks based on PyTorch.\n\nThe library provides:\n\n- Out-of-Distribution Detection Methods\n- Loss Functions\n- Datasets\n- Neural Network Architectures, as well as pre-trained weights\n- Data Augmentations\n- Useful Utilities\n\nand is designed to be compatible with frameworks\nlike `pytorch-lightning \u003chttps://www.pytorchlightning.ai\u003e`_ and\n`pytorch-segmentation-models \u003chttps://github.com/qubvel/segmentation_models.pytorch\u003e`_.\nThe library also covers some methods from closely related fields, such as Open-Set Recognition, Novelty Detection,\nConfidence Estimation and Anomaly Detection.\n\n\n📚  Documentation\n^^^^^^^^^^^^^^^^^^^\nThe documentation is available `here \u003chttps://pytorch-ood.readthedocs.io/en/latest/\u003e`_.\n\n**NOTE**: An important convention adopted in ``pytorch-ood`` is that **OOD detectors predict outlier scores**\nthat should be larger for outliers than for inliers.\nIf you notice that the scores predicted by a detector do not match the formulas in the corresponding publication, we may have adjusted the score calculation to comply with this convention.\n\n⏳ Quick Start\n^^^^^^^^^^^^^^^^^\nLoad a WideResNet-40 model (used in major publications), pre-trained on CIFAR-10 with the Energy-Bounded Learning Loss [#EnergyBasedOOD]_ (weights from to original paper), and predict on some dataset ``data_loader`` using\nEnergy-based OOD Detection (EBO) [#EnergyBasedOOD]_, calculating the common metrics.\nOOD data must be marked with labels \u003c 0.\n\n.. code-block:: python\n\n\n    from pytorch_ood.detector import EnergyBased\n    from pytorch_ood.utils import OODMetrics\n    from pytorch_ood.model import WideResNet\n\n    data_loader = ... # your data, OOD with label \u003c 0\n\n    # Create Neural Network\n    model = WideResNet(num_classes=10, pretrained=\"er-cifar10-tune\").eval().cuda()\n    preprocess = WideResNet.transform_for(\"er-cifar10-tune\")\n\n    # Create detector\n    detector = EnergyBased(model)\n\n    # Evaluate\n    metrics = OODMetrics()\n\n    for x, y in data_loader:\n        x = preprocess(x).cuda()\n        metrics.update(detector(x), y)\n\n    print(metrics.compute())\n\n\nYou can find more examples in the `documentation \u003chttps://pytorch-ood.readthedocs.io/en/latest/auto_examples/benchmarks/\u003e`_.\n\nBenchmarks (Beta)\n---------------------------\n\nEvaluate detectors against common benchmarks, for example the OpenOOD ImageNet benchmark\n(including ImageNet-O, OpenImages-O, Textures, SVHN, MNIST).  All datasets (except for ImageNet itself) will be downloaded automatically.\n\n.. code-block:: python\n\n   import pandas as pd\n   from pytorch_ood.benchmark import ImageNet_OpenOOD\n   from pytorch_ood.detector import MaxSoftmax\n   from torchvision.models import resnet50\n   from torchvision.models.resnet import ResNet50_Weights\n\n   model = resnet50(ResNet50_Weights.IMAGENET1K_V1).eval().to(\"cuda:0\")\n   trans = ResNet50_Weights.IMAGENET1K_V1.transforms()\n\n   benchmark = ImageNet_OpenOOD(root=\"data\", image_net_root=\"data/imagenet-2012/\", transform=trans)\n\n   detector = MaxSoftmax(model)\n   results = benchmark.evaluate(detector, loader_kwargs={\"batch_size\": 64}, device=\"cuda:0\")\n   df = pd.DataFrame(results)\n   print(df)\n\n\nThis produces the following table:\n\n+-------------+-------+---------+----------+----------+\n| Dataset     | AUROC | AUPR-IN | AUPR-OUT | FPR95TPR |\n+=============+=======+=========+==========+==========+\n| ImageNetO   | 28.64 | 2.52    | 94.85    | 91.20    |\n+-------------+-------+---------+----------+----------+\n| OpenImagesO | 84.98 | 62.61   | 94.67    | 49.95    |\n+-------------+-------+---------+----------+----------+\n| Textures    | 80.46 | 37.50   | 96.80    | 67.75    |\n+-------------+-------+---------+----------+----------+\n| SVHN        | 97.62 | 95.56   | 98.77    | 11.58    |\n+-------------+-------+---------+----------+----------+\n| MNIST       | 90.04 | 90.45   | 89.88    | 39.03    |\n+-------------+-------+---------+----------+----------+\n\n\n🛠 ️️Installation\n^^^^^^^^^^^^^^^^^\nThe package can be installed via PyPI:\n\n.. code-block:: shell\n\n   pip install pytorch-ood\n\n\n\n**Dependencies**\n\n\n* ``torch``\n* ``torchvision``\n* ``scipy``\n* ``torchmetrics``\n\n\n**Optional Dependencies**\n\n* ``scikit-learn`` for ViM\n* ``gdown`` to download some datasets and model weights\n* ``pandas`` for the `examples \u003chttps://pytorch-ood.readthedocs.io/en/latest/auto_examples/benchmarks/\u003e`_.\n* ``segmentation-models-pytorch`` to run the examples for anomaly segmentation\n\n\n📦 Implemented\n^^^^^^^^^^^^^^^\n\n**Detectors**:\n\n+-----------------------------+------------------------------------------------------------------------------------------------+------+--------------------+\n| Detector                    | Description                                                                                    | Year | Ref                |\n+=============================+================================================================================================+======+====================+\n| OpenMax                     | Implementation of the OpenMax Layer as proposed in the paper *Towards Open Set Deep Networks*. | 2016 | [#OpenMax]_        |\n+-----------------------------+------------------------------------------------------------------------------------------------+------+--------------------+\n| Monte Carlo Dropout         | Implements Monte Carlo Dropout.                                                                | 2016 | [#MonteCarloDrop]_ |\n+-----------------------------+------------------------------------------------------------------------------------------------+------+--------------------+\n| Maximum Softmax Probability | Implements the Softmax Baseline for OOD and Error detection.                                   | 2017 | [#Softmax]_        |\n+-----------------------------+------------------------------------------------------------------------------------------------+------+--------------------+\n| Temperature Scaling         | Implements the Temperature Scaling for Softmax.                                                 | 2017 | [#TempScaling]_    |\n+-----------------------------+------------------------------------------------------------------------------------------------+------+--------------------+\n| ODIN                        | ODIN is a preprocessing method for inputs that aims to increase the discriminability of        | 2018 | [#ODIN]_           |\n|                             | the softmax outputs for In- and Out-of-Distribution data.                                      |      |                    |\n+-----------------------------+------------------------------------------------------------------------------------------------+------+--------------------+\n| Mahalanobis                 | Implements the Mahalanobis Method.                                                             | 2018 | [#Mahalanobis]_    |\n+-----------------------------+------------------------------------------------------------------------------------------------+------+--------------------+\n| GRAM                        | Detects OOD elements via deviations in the gram matrices                                       | 2019 | [#GramBased]_      |\n+-----------------------------+------------------------------------------------------------------------------------------------+------+--------------------+\n| Energy-Based OOD Detection  | Implements the energy score of *Energy-based Out-of-distribution Detection*.                   | 2020 | [#EnergyBasedOOD]_ |\n+-----------------------------+------------------------------------------------------------------------------------------------+------+--------------------+\n| Entropy                     | Uses entropy to detect OOD inputs.                                                             | 2021 | [#MaxEntropy]_     |\n+-----------------------------+------------------------------------------------------------------------------------------------+------+--------------------+\n| ReAct                       | ReAct: Out-of-distribution detection with Rectified Activations.                               | 2021 | [#ReAct]_          |\n+-----------------------------+------------------------------------------------------------------------------------------------+------+--------------------+\n| Maximum Logit               | Implements the MaxLogit method.                                                                | 2022 | [#StreeHaz]_       |\n+-----------------------------+------------------------------------------------------------------------------------------------+------+--------------------+\n| KL-Matching                 | Implements the KL-Matching method for Multi-Class classification.                              | 2022 | [#StreeHaz]_       |\n+-----------------------------+------------------------------------------------------------------------------------------------+------+--------------------+\n| ViM                         | Implements Virtual Logit Matching.                                                             | 2022 | [#ViM]_            |\n+-----------------------------+------------------------------------------------------------------------------------------------+------+--------------------+\n| Weighted Energy-Based       | Implements Weighted Energy-Based for OOD Detection                                             | 2022 | [#WEBO]_           |\n+-----------------------------+------------------------------------------------------------------------------------------------+------+--------------------+\n| Nearest Neighbor            | Implements Depp Nearest Neighbors for OOD Detection                                            | 2022 | [#kNN]_            |\n+-----------------------------+------------------------------------------------------------------------------------------------+------+--------------------+\n| DICE                        | Implements Sparsification for OOD Detection                                                    | 2022 | [#DICE]_           |\n+-----------------------------+------------------------------------------------------------------------------------------------+------+--------------------+\n| ASH                         | Implements Extremely Simple Activation Shaping                                                 | 2023 | [#Ash]_            |\n+-----------------------------+------------------------------------------------------------------------------------------------+------+--------------------+\n| SHE                         | Implements Simplified Hopfield Networks                                                        | 2023 | [#She]_            |\n+-----------------------------+------------------------------------------------------------------------------------------------+------+--------------------+\n\n**Objective Functions**:\n\n+----------------------------+--------------------------------------------------------------------------------------------------+------+--------------------+\n| Objective Function         | Description                                                                                      | Year | Ref                |\n+============================+==================================================================================================+======+====================+\n| Objectosphere              | Implementation of the paper *Reducing Network Agnostophobia*.                                    | 2016 | [#Objectosphere]_  |\n+----------------------------+--------------------------------------------------------------------------------------------------+------+--------------------+\n| Center Loss                | Generalized version of the *Center Loss* from the Paper *A Discriminative Feature Learning       | 2016 | [#CenterLoss]_     |\n|                            | Approach for Deep Face Recognition*.                                                             |      |                    |\n+----------------------------+--------------------------------------------------------------------------------------------------+------+--------------------+\n| Outlier Exposure           | Implementation of the paper *Deep Anomaly Detection With Outlier Exposure*.                      | 2018 | [#OE]_             |\n+----------------------------+--------------------------------------------------------------------------------------------------+------+--------------------+\n| Confidence Loss            | Model learn confidence additional to class membership prediction.                                | 2018 | [#ConfidenceLoss]_ |\n+----------------------------+--------------------------------------------------------------------------------------------------+------+--------------------+\n| Deep SVDD                  | Implementation of the Deep Support Vector Data Description from the paper *Deep One-Class        | 2018 | [#SVDD]_           |\n|                            | Classification*.                                                                                 |      |                    |\n+----------------------------+--------------------------------------------------------------------------------------------------+------+--------------------+\n| Energy-Bounded Loss        | Adds a regularization term to the cross-entropy that aims to increase the energy gap between IN  | 2020 | [#EnergyBasedOOD]_ |\n|                            | and OOD samples.                                                                                 |      |                    |\n+----------------------------+--------------------------------------------------------------------------------------------------+------+--------------------+\n| CAC Loss                   | Class Anchor Clustering Loss from *Class Anchor Clustering: a Distance-based Loss for Training   | 2021 | [#CACLoss]_        |\n|                            | Open Set Classifiers*                                                                            |      |                    |\n+----------------------------+--------------------------------------------------------------------------------------------------+------+--------------------+\n| Entropic Open-Set Loss     | Entropy maximization and meta classification for OOD in semantic segmentation                    | 2021 | [#MaxEntropy]_     |\n+----------------------------+--------------------------------------------------------------------------------------------------+------+--------------------+\n| II Loss                    | Implementation of II Loss function from *Learning a neural network-based representation for      | 2022 | [#IILoss]_         |\n|                            | open set recognition*.                                                                           |      |                    |\n+----------------------------+--------------------------------------------------------------------------------------------------+------+--------------------+\n| MCHAD Loss                 | Implementation of the MCHAD Loss from the paper *Multi Class Hypersphere Anomaly Detection*.     | 2022 | [#MCHAD]_          |\n+----------------------------+--------------------------------------------------------------------------------------------------+------+--------------------+\n| VOS Energy-Based Loss      | Implementation of the paper *VOS: Learning what you don’t know by virtual outlier synthesis*.    | 2022 | [#WEBO]_           |\n+----------------------------+--------------------------------------------------------------------------------------------------+------+--------------------+\n\n\n**Image Datasets**:\n\n+-----------------------+-----------------------------------------------------------------------------------------------------------------+------+---------------+\n| Dataset               | Description                                                                                                     | Year | Ref           |\n+=======================+=================================================================================================================+======+===============+\n| Chars74k              | The Chars74K dataset contains 74,000 images across 64 classes, comprising English letters and Arabic numerals.  | 2012 | [#Chars74k]_  |\n+-----------------------+-----------------------------------------------------------------------------------------------------------------+------+---------------+\n| TinyImages            | The TinyImages dataset is often used as auxiliary OOD training data. However, use is discouraged.               | 2012 | [#TinyImgs]_  |\n+-----------------------+-----------------------------------------------------------------------------------------------------------------+------+---------------+\n| Textures              | Textures dataset, also known as DTD, often used as OOD Examples.                                                | 2013 | [#Textures]_  |\n+-----------------------+-----------------------------------------------------------------------------------------------------------------+------+---------------+\n| FoolingImages         | OOD Images Generated to fool certain Deep Neural Networks.                                                      | 2015 | [#FImages]_   |\n+-----------------------+-----------------------------------------------------------------------------------------------------------------+------+---------------+\n| Tiny ImageNet         | A derived version of ImageNet with 64x64-sized images.                                                          | 2015 | [#TinyIN]_    |\n+-----------------------+-----------------------------------------------------------------------------------------------------------------+------+---------------+\n| TinyImages300k        | A cleaned version of the TinyImages Dataset with 300.000 images, often used as auxiliary OOD training data.     | 2018 | [#OE]_        |\n+-----------------------+-----------------------------------------------------------------------------------------------------------------+------+---------------+\n| LSUN                  | A version of the Large-scale Scene UNderstanding Dataset with 10.000 images, often used as auxiliary            | 2018 | [#ODIN]_      |\n|                       | OOD training data.                                                                                              |      |               |\n+-----------------------+-----------------------------------------------------------------------------------------------------------------+------+---------------+\n| MNIST-C               | Corrupted version of the MNIST.                                                                                 | 2019 | [#MnistC]_    |\n+-----------------------+-----------------------------------------------------------------------------------------------------------------+------+---------------+\n| CIFAR10-C             | Corrupted version of the CIFAR 10.                                                                              | 2019 | [#Cifar10]_   |\n+-----------------------+-----------------------------------------------------------------------------------------------------------------+------+---------------+\n| CIFAR100-C            | Corrupted version of the CIFAR 100.                                                                             | 2019 | [#Cifar10]_   |\n+-----------------------+-----------------------------------------------------------------------------------------------------------------+------+---------------+\n| ImageNet-C            | Corrupted version of the ImageNet.                                                                              | 2019 | [#Cifar10]_   |\n+-----------------------+-----------------------------------------------------------------------------------------------------------------+------+---------------+\n| ImageNet - A, O, R    | Different Outlier Variants for the ImageNet.                                                                    | 2019 | [#ImageNets]_ |\n+-----------------------+-----------------------------------------------------------------------------------------------------------------+------+---------------+\n| ImageNet - V2         | A new test set for the ImageNet.                                                                                | 2019 | [#ImageNV2]_  |\n+-----------------------+-----------------------------------------------------------------------------------------------------------------+------+---------------+\n| ImageNet - ES         | Event stream (ES) version of the ImageNet.                                                                      | 2021 | [#ImageNES]_  |\n+-----------------------+-----------------------------------------------------------------------------------------------------------------+------+---------------+\n| iNaturalist           | A Subset of iNaturalist, with 10.000 images.                                                                    | 2021 | [#INatural]_  |\n+-----------------------+-----------------------------------------------------------------------------------------------------------------+------+---------------+\n| Fractals              | A dataset with Fractals from *PIXMIX: Dreamlike Pictures Comprehensively Improve Safety Measures*               | 2022 | [#PixMix]_    |\n+-----------------------+-----------------------------------------------------------------------------------------------------------------+------+---------------+\n| Feature               | A dataset with Feature visualizations from *PIXMIX: Dreamlike Pictures Comprehensively Improve Safety Measures* | 2022 | [#PixMix]_    |\n| Visualizations        |                                                                                                                 |      |               |\n+-----------------------+-----------------------------------------------------------------------------------------------------------------+------+---------------+\n| FS Static             | The FishyScapes (FS) Static dataset contains real world OOD images from the CityScapes dataset.                 | 2021 | [#FS]_        |\n+-----------------------+-----------------------------------------------------------------------------------------------------------------+------+---------------+\n| FS LostAndFound       | The FishyScapes dataset contains images from the CityScapes dataset blended with unknown objects scraped from   | 2021 | [#FS]_        |\n|                       | the web.                                                                                                        |      |               |\n+-----------------------+-----------------------------------------------------------------------------------------------------------------+------+---------------+\n| MVTech-AD             | The MVTec AD is a dataset for benchmarking anomaly detection methods with a focus on industrial inspection.     | 2021 | [#MVTech]_    |\n+-----------------------+-----------------------------------------------------------------------------------------------------------------+------+---------------+\n| StreetHazards         | Anomaly Segmentation Dataset                                                                                    | 2022 | [#StreeHaz]_  |\n+-----------------------+-----------------------------------------------------------------------------------------------------------------+------+---------------+\n| CIFAR100-GAN          | Images sampled from low likelihood regions of a BigGAN trained on CIFAR 100 from the paper *On Outlier Exposure | 2022 | [#CifarGAN]_  |\n|                       | with Generative Models.*                                                                                        |      |               |\n+-----------------------+-----------------------------------------------------------------------------------------------------------------+------+---------------+\n| SSB - hard            | The hard split of the Semantic Shift Benchmark, which contains 49.00 images.                                    | 2022 | [#SSB]_       |\n+-----------------------+-----------------------------------------------------------------------------------------------------------------+------+---------------+\n| NINCO                 | The NINCO (No ImageNet Class Objects) dataset which contains 5.879 images of 64 OOD classes.                    | 2023 | [#NINCO]_     |\n+-----------------------+-----------------------------------------------------------------------------------------------------------------+------+---------------+\n| SuMNIST               | The SuMNIST dataset is based on MNIST but each image display four numbers instead of one.                       | 2023 | [#SuMNIST]_   |\n+-----------------------+-----------------------------------------------------------------------------------------------------------------+------+---------------+\n| Gaussian Noise        | Dataset with samples drawn from a normal distribution.                                                          |      |               |\n+-----------------------+-----------------------------------------------------------------------------------------------------------------+------+---------------+\n| Uniform Noise         | Dataset with samples drawn from a uniform distribution.                                                         |      |               |\n+-----------------------+-----------------------------------------------------------------------------------------------------------------+------+---------------+\n\n\n\n**Text Datasets**:\n\n+-------------+---------------------------------------------------------------------------------------------------------------------------+------+-----------------+\n| Dataset     | Description                                                                                                               | Year | Ref             |\n+=============+===========================================================================================================================+======+=================+\n| Multi30k    | Multi-30k dataset, as used by Hendrycks et al. in the OOD baseline paper.                                                 | 2016 | [#Multi30k]_    |\n+-------------+---------------------------------------------------------------------------------------------------------------------------+------+-----------------+\n| WikiText2   | Texts from the wikipedia often used as auxiliary OOD training data.                                                       | 2016 | [#WikiText2]_   |\n+-------------+---------------------------------------------------------------------------------------------------------------------------+------+-----------------+\n| WikiText103 | Texts from the wikipedia often used as auxiliary OOD training data.                                                       | 2016 | [#WikiText2]_   |\n+-------------+---------------------------------------------------------------------------------------------------------------------------+------+-----------------+\n| NewsGroup20 | Textx from different newsgroups, as used by Hendrycks et al. in the OOD baseline paper.                                   |      |                 |\n+-------------+---------------------------------------------------------------------------------------------------------------------------+------+-----------------+\n\n\n**Augmentation Methods**:\n\n+-----------------------+-----------------------------------------------------------------------------------------------------------------+------+----------------+\n| Augmentation          | Description                                                                                                     | Year | Ref            |\n+=======================+=================================================================================================================+======+================+\n| PixMix                | PixMix image augmentation method                                                                                | 2022 | [#PixMix]_     |\n+-----------------------+-----------------------------------------------------------------------------------------------------------------+------+----------------+\n| COCO Outlier Pasting  | From \"Entropy maximization and meta classification for OOD in semantic segmentation\"                            | 2021 | [#MaxEntropy]_ |\n+-----------------------+-----------------------------------------------------------------------------------------------------------------+------+----------------+\n\n\n🤝  Contributing\n^^^^^^^^^^^^^^^^^\nWe encourage everyone to contribute to this project by adding implementations of OOD Detection methods, datasets etc,\nor check the existing implementations for bugs.\n\n\n📝 Citing\n^^^^^^^^^^\n\n``pytorch-ood`` was presented at a CVPR Workshop in 2022.\nIf you use it in a scientific publication, please consider citing::\n\n    @InProceedings{kirchheim2022pytorch,\n        author    = {Kirchheim, Konstantin and Filax, Marco and Ortmeier, Frank},\n        title     = {PyTorch-OOD: A Library for Out-of-Distribution Detection Based on PyTorch},\n        booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},\n        month     = {June},\n        year      = {2022},\n        pages     = {4351-4360}\n    }\n\n🛡️ ️License\n^^^^^^^^^^^\n\nThe code is licensed under Apache 2.0. We have taken care to make sure any third party code included or adapted has compatible (permissive) licenses such as MIT, BSD, etc.\nThe legal implications of using pre-trained models in commercial services are, to our knowledge, not fully understood.\n\n----\n\n🔗 References\n^^^^^^^^^^^^^^\n\n.. [#OpenMax]  Bendale, A., \u0026 Boult, T. E. (2016). Towards open set deep networks. CVPR.\n\n.. [#ODIN] Liang, S., et al. (2017). Enhancing the reliability of out-of-distribution image detection in neural networks. ICLR.\n\n.. [#Mahalanobis] Lee, K., et al. (2018). A simple unified framework for detecting out-of-distribution samples and adversarial attacks. NeurIPS.\n\n.. [#MonteCarloDrop] Gal, Y., \u0026 Ghahramani, Z. (2016). Dropout as a bayesian approximation: Representing model uncertainty in deep learning. ICML.\n\n.. [#Softmax] Hendrycks, D., \u0026 Gimpel, K. (2016). A baseline for detecting misclassified and out-of-distribution examples in neural networks. ICLR.\n\n.. [#TempScaling] Guo, C., et al. (2017). On calibration of modern neural networks. ICML.\n\n.. [#ConfidenceLoss] DeVries, T., \u0026 Taylor, G. W. (2018). Learning confidence for out-of-distribution detection in neural networks. `ArXiv \u003chttps://arxiv.org/pdf/1802.04865\u003e`__.\n\n.. [#EnergyBasedOOD] Liu, W., et al. (2020). Energy-based out-of-distribution detection. NeurIPS.\n\n.. [#Objectosphere] Dhamija, A. R., et al. (2018). Reducing network agnostophobia. NeurIPS.\n\n.. [#OE] Hendrycks, D., Mazeika, M., \u0026 Dietterich, T. (2018). Deep anomaly detection with outlier exposure. ICLR.\n\n.. [#SVDD] Ruff, L., et al. (2018). Deep one-class classification. ICML.\n\n.. [#IILoss] Hassen, M., \u0026 Chan, P. K. (2020). Learning a neural-network-based representation for open set recognition. SDM.\n\n.. [#CACLoss] Miller, D., et al. (2021). Class anchor clustering: A loss for distance-based open set recognition. WACV.\n\n.. [#CenterLoss] Wen, Y., et al. (2016). A discriminative feature learning approach for deep face recognition. ECCV.\n\n.. [#Cifar10] Hendrycks, D., \u0026 Dietterich, T. (2019). Benchmarking neural network robustness to common corruptions and perturbations. ICLR.\n\n.. [#FImages] Nguyen, A., et al. (2015). Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. CVPR.\n\n.. [#TinyIN] Le, Y., et al. (2015). Tiny ImageNet Visual Recognition Challenge. `Stanford \u003chttps://cs231n.stanford.edu/reports/2015/pdfs/yle_project.pdf\u003e`_.\n\n.. [#ImageNets] Hendrycks, D., et al. (2021). Natural adversarial examples. CVPR.\n\n.. [#ImageNV2] Recht, B., et al. (2019).  Do imagenet classifiers generalize to imagenet?. PMLR.\n\n.. [#ImageNES] Lin, Y., et al. (2021).  ES-ImageNet: A Million Event-Stream Classification Dataset for Spiking Neural Networks. `Front Neurosci \u003chttps://pubmed.ncbi.nlm.nih.gov/34899154/\u003e`_.\n\n.. [#MnistC] Mu, N., \u0026 Gilmer, J. (2019). MNIST-C: A robustness benchmark for computer vision. ICLR Workshop.\n\n.. [#FS] Blum, H. et al (2021) The Fishyscapes Benchmark: Measuring Blind Spots in Semantic Segmentation. IJCV.\n\n.. [#MVTech] Bergmann, P. et al (2021) The MVTec Anomaly Detection Dataset: A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection. IJCV\n\n.. [#StreeHaz] Hendrycks, D., et al. (2022). Scaling out-of-distribution detection for real-world settings. ICML.\n\n.. [#CifarGAN] Kirchheim, K., \u0026 Ortmeier, F. (2022) On Outlier Exposure with Generative Models. NeurIPS.\n\n.. [#SSB] Vaze, S., et al. (2022)  Open-set recognition: A good closed-set classifier is all you need. ICLR.\n\n.. [#NINCO] Bitterwolf, J., et al. (2023) In or Out? Fixing ImageNet Out-of-Distribution Detection Evaluation. ICML.\n\n.. [#SuMNIST] Kirchheim, K. (2023) Towards Deep Anomaly Detection with Structured Knowledge Representations. SAFECOMP.\n\n.. [#Textures] Cimpoi, M., et al. (2014). Describing textures in the wild. CVPR.\n\n.. [#TinyImgs] Torralba, A., et al. (2007). 80 million tiny images: a large dataset for non-parametric object and scene recognition. IEEE Transactions on Pattern Analysis and Machine Learning.\n\n.. [#Chars74k] de Campos, T. E., et al. (2009). Character recognition in natural images. In Proceedings of the International Conference on Computer Vision Theory and Applications (VISAPP).\n\n.. [#Multi30k] Elliott, D., et al. (2016). Multi30k: Multilingual english-german image descriptions. Proceedings of the 5th Workshop on Vision and Language.\n\n.. [#WikiText2] Merity, S., et al. (2016). Pointer sentinel mixture models. `ArXiv \u003chttps://arxiv.org/abs/1609.07843\u003e`__\n\n.. [#INatural] Huang, R., \u0026 Li, Y. (2021) MOS: Towards Scaling Out-of-distribution Detection for Large Semantic Space. CVPR.\n\n.. [#MCHAD] Kirchheim, K., et al. (2022) Multi Class Hypersphere Anomaly Detection. ICPR.\n\n.. [#ViM] Wang, H., et al. (2022) ViM: Out-Of-Distribution with Virtual-logit Matching. CVPR.\n\n.. [#WEBO] Du, X., et al. (2022) VOS: Learning What You Don't Know by Virtual Outlier Synthesis. ICLR.\n\n.. [#kNN] Sun, Y., et al. (2022) Out-of-Distribution Detection with Deep Nearest Neighbors. ICML.\n\n.. [#PixMix] Hendrycks, D, et al. (2022) PixMix: Dreamlike Pictures Comprehensively Improve Safety Measures. CVPR.\n\n.. [#MaxEntropy] Chan R,  et al. (2021) Entropy maximization and meta classification for out-of-distribution detection in semantic segmentation. CVPR.\n\n.. [#DICE] Sun, et al. (2022) DICE: Leveraging Sparsification for Out-of-Distribution Detection. ECCV.\n\n.. [#ASH] Djurisic,  et al. (2023) Extremely Simple Activation Shaping for Out-of-Distribution Detection, ICLR.\n\n.. [#She] Zhang,  et al. (2023) Out-of-Distribution Detection Based on In-Distribution Data Patterns Memorization with Modern Hopfield Energy, ICLR.\n\n.. [#ReAct] Sun,  et al. (2023) ReAct: Out-of-distribution Detection With Rectified Activations, NeurIPS\n\n.. [#GramBased] Shama,  et al. (2019) Detecting Out-of-Distribution Examples with In-distribution Examples and Gram Matrices, NeurIPS\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkkirchheim%2Fpytorch-ood","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkkirchheim%2Fpytorch-ood","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkkirchheim%2Fpytorch-ood/lists"}