{"id":17719845,"url":"https://github.com/zjysteven/mixoe","last_synced_at":"2025-07-05T20:02:20.902Z","repository":{"id":156562226,"uuid":"467358076","full_name":"zjysteven/MixOE","owner":"zjysteven","description":"Mixture Outlier Exposure for Out-of-Distribution Detection in Fine-grained Environments","archived":false,"fork":false,"pushed_at":"2023-04-12T18:17:26.000Z","size":5257,"stargazers_count":20,"open_issues_count":0,"forks_count":4,"subscribers_count":1,"default_branch":"main","last_synced_at":"2023-04-26T19:24:03.468Z","etag":null,"topics":["fine-grained-classification","fine-grained-recognition","novelty-detection","openset-recognition","out-of-distribution-detection","outlier-exposure"],"latest_commit_sha":null,"homepage":"https://openaccess.thecvf.com/content/WACV2023/html/Zhang_Mixture_Outlier_Exposure_Towards_Out-of-Distribution_Detection_in_Fine-Grained_Environments_WACV_2023_paper.html","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zjysteven.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-03-08T04:05:30.000Z","updated_at":"2023-04-21T03:01:39.000Z","dependencies_parsed_at":null,"dependency_job_id":"7b70c7f8-2986-4eff-a1f4-efd5de2d1fa9","html_url":"https://github.com/zjysteven/MixOE","commit_stats":null,"previous_names":[],"tags_count":0,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zjysteven%2FMixOE","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zjysteven%2FMixOE/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zjysteven%2FMixOE/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zjysteven%2FMixOE/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zjysteven","download_url":"https://codeload.github.com/zjysteven/MixOE/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246604612,"owners_count":20804100,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["fine-grained-classification","fine-grained-recognition","novelty-detection","openset-recognition","out-of-distribution-detection","outlier-exposure"],"created_at":"2024-10-25T15:09:49.426Z","updated_at":"2025-04-01T14:32:07.121Z","avatar_url":"https://github.com/zjysteven.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Towards OOD detection in fine-grained environments\n\nThis repository contains the code necessary to replicate the results of our WACV'23 paper:\n\n***Mixture Outlier Exposure: Towards Out-of-Distribution Detection in Fine-grained Environments***\n\n[Jingyang Zhang](https://zjysteven.github.io/), Nathan Inkawhich, Randolph Linderman, Yiran Chen, Hai Li @ [Duke CEI Lab](https://cei.pratt.duke.edu/)\n\nCheck out our paper [here](https://openaccess.thecvf.com/content/WACV2023/html/Zhang_Mixture_Outlier_Exposure_Towards_Out-of-Distribution_Detection_in_Fine-Grained_Environments_WACV_2023_paper.html)!!!\n\n## Overview\n\n### Motivation\n\n\u003cp align=\"center\"\u003e\n    \u003cimg src='/figures/overview.png' width='870'\u003e\n\u003c/p\u003e\n\u003cp\u003e\n    \u003cem\u003eLeft: A comparison of OOD detection in coarse- and fine-grained environments. Intuitively, fine-grained detection is much more challenging. Right: A *conceptual* illustration of MixOE. A Standard model with no OOD considerations tends to be over-confident on OOD samples. OE is able to calibrate the prediction confidence on coarse-grained OOD, but the outputs on fine-grained OOD are uncontrolled (marked by ``?''). MixOE aims for a smooth decay of the confidence as the inputs transition from ID to OOD, and thus enables detection of both coarse/fine-grained OOD.\u003c/em\u003e\n\u003c/p\u003e\n\n\nThe capability of detecting Out-of-distribution (OOD) samples that do not belong to one of the known classes of DNNs during inference time is crucial for reliable operations in the wild.\nExisting works typically use *coarse-grained* benchmarks (e.g., CIFAR-10 v.s. SVHN/LSUN) to perform evaluation, which fail to approximate many real-world scenarios which inherently have *fine-grained* attributes (e.g., bird species recognition, medical image classification).\nIn such fine-grained environments, one may expect OOD samples to be highly granular w.r.t. in-distribution (ID) data, which intuitively can be very difficult to identify.\n**Unfortunately, OOD detection in fine-grained environments remains largely underexplored.**\n\n### Our contributions\n1. **We construct four large-scale, fine-grained test environments for OOD detection.** The test benches are generated using a holdout-class method on public fine-grained classification datasets. Later we have detailed instructions for you to prepare the datasets and easily reproduce our test environments.\n\n\u003cp align=\"center\"\u003e\n    \u003cimg src='/figures/initial_eval.png' width='650'\u003e\n\u003c/p\u003e\n\u003cp\u003e\n    \u003cem\u003eTNR95 of existing methods against coarse-grained (first row) and fine-grained OOD data (second row). The gray dashed line is the baseline performance (MSP). Fine-grained novelties are significantly harder to detect in all datasets for all methods. Also note how the methods that utilize outlier data (coral ones) help with coarse-grained OOD but barely improve fine-grained detection rates.\u003c/em\u003e\n\u003c/p\u003e\n\n2. Through initial evaluation, **we find that existing methods struggle to detect fine-grained novelties** (underperforming a simple baseline [MSP](https://arxiv.org/abs/1610.02136)). Even the methods that explicitly incorporate auxiliary outlier data (e.g., [Outlier Exposure](https://arxiv.org/abs/1812.04606)) does *not* help much. We further conduct analysis to explain why this is the case.\n\n\u003cp align=\"center\"\u003e\n    \u003cimg src='/figures/method_vis.png' width='870'\u003e\n\u003c/p\u003e\n\u003cp\u003e\n    \u003cem\u003eVisualization of the data samples (second row) and their representations in the DNN's feature space (first row). The color lightness in (d)/(e) indicates the prediction confidence encoded in the soft target of each corresponding outlier sample. Note, (b) and (c) are test OOD samples and are never seen during the training. The empirical training outliers (d) enclose the region where coarse OOD data (b) locate but does not cover the much broader area the fine OOD samples (c) span. MixOE mixes the ID (a) and training outliers (d) to induce larger coverage which accounts for both coarse- and fine-grained novelties. Moreover, the soft targets of the mixed data will calibrate the model's prediction confidence to smoothly decay from ID to OOD.\u003c/em\u003e\n\u003c/p\u003e\n\n3. **We propose Mixture Outlier Exposure (MixOE)**, a novel method for OOD detection in fine-grained environments. MixOE consistently improves the detection rates against both fine- and coarse-grained OOD samples across all the four test benches.\n\nSee our paper for details!\n\n## Get started\n### Environment\nFollow the commands below to set up the environment.\n\n1. Clone the repo: `git clone https://github.com/zjysteven/MixOE.git`\n\n2. Create a conda environment\n```\nconda create -n mixoe python=3.8\nconda activate mixoe\npython -m pip install -r requirements.txt\n```\n\n### Dataset\nPlease see detailed instructions [here](/data/README.md).\n\n## Reproducing experiments\nWe provide sample scripts `train.sh` and `eval.sh` in `scripts/` folder showcasing how to train/evaluate our method and other considered methods. Note that all OE-based methods (including MixOE) fine-tune the trained baseline model, so to run OE/MixOE you first need to do standard training with `train_baseline.py`. For everyone's convenience, we have uploaded the trained baseline weights [here](https://drive.google.com/drive/folders/1XdGWCgYR1lBiJZIuhlO0e3VHisJ8MDlH?usp=sharing).\n\n## Reference\nIf you find our work/code helpful, please consider citing our work.\n```\n@InProceedings{Zhang_2023_WACV,\n    author    = {Zhang, Jingyang and Inkawhich, Nathan and Linderman, Randolph and Chen, Yiran and Li, Hai},\n    title     = {Mixture Outlier Exposure: Towards Out-of-Distribution Detection in Fine-Grained Environments},\n    booktitle = {Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},\n    month     = {January},\n    year      = {2023},\n    pages     = {5531-5540}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzjysteven%2Fmixoe","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzjysteven%2Fmixoe","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzjysteven%2Fmixoe/lists"}