{"id":15648210,"url":"https://github.com/sayakpaul/generalized-odin-tf","last_synced_at":"2025-04-30T14:34:17.198Z","repository":{"id":106648749,"uuid":"355863898","full_name":"sayakpaul/Generalized-ODIN-TF","owner":"sayakpaul","description":"TensorFlow 2 implementation of the paper Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Data (https://arxiv.org/abs/2002.11297).","archived":false,"fork":false,"pushed_at":"2021-09-07T07:00:22.000Z","size":523,"stargazers_count":45,"open_issues_count":1,"forks_count":5,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-03-30T17:02:16.487Z","etag":null,"topics":["keras","ood-detection","robustness","tensorflow2","vision"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sayakpaul.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2021-04-08T10:35:34.000Z","updated_at":"2024-08-20T08:47:38.000Z","dependencies_parsed_at":null,"dependency_job_id":"d5dd33e0-f333-47a6-af73-851e649c22f3","html_url":"https://github.com/sayakpaul/Generalized-ODIN-TF","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sayakpaul%2FGeneralized-ODIN-TF","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sayakpaul%2FGeneralized-ODIN-TF/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sayakpaul%2FGeneralized-ODIN-TF/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sayakpaul%2FGeneralized-ODIN-TF/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sayakpaul","download_url":"https://codeload.github.com/sayakpaul/Generalized-ODIN-TF/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251721381,"owners_count":21632827,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["keras","ood-detection","robustness","tensorflow2","vision"],"created_at":"2024-10-03T12:24:05.572Z","updated_at":"2025-04-30T14:34:17.145Z","avatar_url":"https://github.com/sayakpaul.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Generalized-ODIN-TF\nTensorFlow 2 implementation of the paper [Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Data](https://arxiv.org/abs/2002.11297).\n\nDetecting out-of-distribution (OOD) data is a challenging problem for deep neural networks to tackle especially when they weren't exposed to OOD data. One way to solve this is to [expose networks](https://arxiv.org/abs/1812.04606) to OOD data _during_ its training. But this can become a brittle approach when the space of the OOD data gets larger for a network. What if we present a network with OOD data it _hasn't_ been exposed to during its training? \n\nTherefore, we need a better way to deal with the problem. Generalized ODIN (**O**ut-of-**DI**stribution detector for **N**eural networks) is a good first step toward that. \n\n## Organization of the files\n\n```shell\n├── Baseline.ipynb: Trains a ResNet20 model on the CIFAR-10 dataset (in-distribution dataset). We will consider this to be our baseline model.\n├── Calculate_Epsilon.ipynb: Searches for the best epsilon (perturbation magnitude) as proposed in the paper. \n├── Evaluation_OOD.ipynb: Evaluates the baseline model as well as the Generalized ODIN model.\n├── Generalized_ODIN.ipynb: Trains the Generalized ODIN model on the CIFAR-10 dataset. \n└── scripts\n    ├── metrics.py: Utilities for evalutation metrics (AUROC and TNR@TPR95)\n    ├── resnet20.py: ResNet20 model utilities. \n    └── resnet20_odin.py: ResNet20 with Generalized ODIN utilities. \n```\n\n_TNR: True Negative Rate, TPR: True Positive Rate_\n\n## Task of interest\n\nTrain a model on the CIFAR-10 dataset (in-distribution dataset) in a way that maximizes its capability to detect OOD samples. This project uses the [SVHN dataset](http://ufldl.stanford.edu/housenumbers/) for the OOD samples.\n\n## Main differences in the implementation\n\n* The authors use ResNet34. In this project, ResNet20 has been used. \n* The learning rate schedule goes like following: Decay by a factor of 0.1 at 25%, 50%, and 75% of the total training epochs. \n* DeConf-I's been used to calculate `h(x)`. Refer to the paper for more details (Section 3.1.1).\n\n## Results\n\n\u003cp align=\"center\"\u003e\n\n|                  \t| Train Top-1 \t| Test Top-1 \t| AUROC \t| TNR@TPR95 \t|\n|------------------\t|:-----------:\t|:----------:\t|:-----:\t|:---------:\t|\n| Generalized ODIN \t|    99.46    \t|    91.42   \t| **92.15** \t|   **54.18**   \t|\n|     Baseline     \t|    99.58    \t|    90.7    \t| 91.14 \t|   40.53   \t|\n\n\u003c/p\u003e\n\n## How to use these models to detect OOD samples?\n\nTake the output of the ODIN and see if it crosses a threshold. If it does then the corresponding samples IID otherwise OOD.\n\nHere's an advice I got from Yen-Chang Hsu (first author of the paper):\n\n\u003e The selection of thresholds is application-dependent. It will still rely on having a validation set (which includes in-distribution and optionally OOD data) for an application. One way is to select the threshold is pick one that makes TPR=95%.\n\n## Pre-trained models\n\nAvailable [here](https://github.com/sayakpaul/Generalized-ODIN-TF/releases/download/v1.0.0/models.tar.gz).\n\n## Acknowledgements\n\n* Thanks to Yen-Chang Hsu for providing constant guidance. \n* Thanks to the [ML-GDE program](https://developers.google.com/programs/experts/) for providing GCP support. \n\n## Paper citation\n\n```\n@INPROCEEDINGS{9156473,\n  author={Y. -C. {Hsu} and Y. {Shen} and H. {Jin} and Z. {Kira}},\n  booktitle={2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)}, \n  title={Generalized ODIN: Detecting Out-of-Distribution Image Without Learning From Out-of-Distribution Data}, \n  year={2020},\n  volume={},\n  number={},\n  pages={10948-10957},\n  doi={10.1109/CVPR42600.2020.01096}}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsayakpaul%2Fgeneralized-odin-tf","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsayakpaul%2Fgeneralized-odin-tf","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsayakpaul%2Fgeneralized-odin-tf/lists"}