Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/deeplearning-wisc/multi-label-ood
https://github.com/deeplearning-wisc/multi-label-ood
Last synced: 7 days ago
JSON representation
- Host: GitHub
- URL: https://github.com/deeplearning-wisc/multi-label-ood
- Owner: deeplearning-wisc
- Created: 2021-09-16T04:23:41.000Z (about 3 years ago)
- Default Branch: master
- Last Pushed: 2021-09-22T05:54:34.000Z (about 3 years ago)
- Last Synced: 2024-08-02T15:34:26.178Z (3 months ago)
- Language: Python
- Size: 3.65 MB
- Stars: 42
- Watchers: 1
- Forks: 5
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Can multi-label classification networks know what they don’t know?
This is a [PyTorch](http://pytorch.org) implementation of [Can multi-label classification networks know what they don't know?](......) by Haoran Wang, Weitang Liu, Alex Bocchieri, Sharon Li.
Code is modified from
[multilabel-ood](https://github.com/xksteven/multilabel-ood),
[ODIN](https://github.com/facebookresearch/odin),
[Outlier Exposure](https://github.com/hendrycks/outlier-exposure), and
[deep Mahalanobis
detector](https://github.com/pokaxpoka/deep_Mahalanobis_detector)![teaser](demo_figs/teaser.png)
## Datasets### In-distribution dataset
PASCAL-VOC: please download the dataset from
[this mirror](https://pjreddie.com/projects/pascal-voc-dataset-mirror/). Parsed labels for PASCAL-VOC are under ./dataset/pascal folder. Create the symlink to the location of Pascal dataset.```
ln -s path/to/PASCALdataset Pascal
```COCO: please download the MS-COCO 2014 dataset from [here](http://cocodataset.org/#download). Install the pycocotools to preprocess the dataset
```
pip3 install git+https://github.com/waleedka/coco.git#egg=pycocotools&subdirectory=PythonAPI
```Preprocess the COCO dataset.
```
python3 utils/coco-preprocessing.py path/to/coco-dataset
```NUS-WIDE: please download the dataset from [here](https://lms.comp.nus.edu.sg/wp-content/uploads/2019/research/nuswide/NUS-WIDE.html). Parsed labels for NUS-WIDE are under ./dataset/nus-wide folder.
### Out-of-distribution dataset
OOD dataset can be downloaded
[here](https://drive.google.com/drive/folders/1BGMRQz3eB_npaGD46HC6K_uzt105HPRy?usp=sharing)## Pre-trained models
Pre-trained models can be downloaded from
[here](https://drive.google.com/drive/folders/1ZfWB6vSYTK004j0bmfj6W0Xs6kwDTFX0?usp=sharing).## Training the models
### Below are the examples on COCO dataset.
Train the densenet model for COCO dataset
```
python3 train.py --arch densenet --dataset coco --save_dir ./saved_models/
```Evaluate the trained model
```
python3 validate.py --arch densenet --dataset coco --load_path ./saved_models/
```## OOD dection
To reproduce the JointEnergy score for COCO dataset, please run:
```
python3 eval.py --arch densenet --dataset coco --ood_data imagenet --ood energy
--method sum
```To reproduce the scores for logit/msp/prob/lof/isol, please run:
```
python3 eval.py --arch densenet --dataset coco --ood_data imagenet --ood
logit/msp/prob/lof/isol/ --method max
```To finetune the parameters for Odin and Mahalanobis, please run:
```
python3 fine_tune.py --arch densenet --dataset coco --ood odin/M --method max
```After getting the best_T and best_noise, please run the evaluation:
```
python3 eval.py --arch densenet --dataset coco --ood_data imagenet --ood odin/M
--method max --T best_T --noise --best_noise
```## OOD Detection Result
OOD detection performance comparison using JointEnergy vs. competitive
baselines.
![result](demo_figs/result_screenshot.png)## Citation
@article{wang2021canmulti,
title={Can multi-label classification networks know what they don't know?},
author={Wang, Haoran and Liu, Weitang and Bocchieri, Alex and Li, Yixuan},
journal={Advances in Neural Information Processing Systems},
year={2021}
}