{"id":13578935,"url":"https://github.com/abhijitbendale/OSDN","last_synced_at":"2025-04-05T19:34:20.838Z","repository":{"id":66924062,"uuid":"63182969","full_name":"abhijitbendale/OSDN","owner":"abhijitbendale","description":"Code and data for the research paper \"Towards Open Set Deep Networks\" A Bendale, T Boult, CVPR 2016","archived":false,"fork":false,"pushed_at":"2018-04-30T00:08:34.000Z","size":195,"stargazers_count":243,"open_issues_count":7,"forks_count":57,"subscribers_count":14,"default_branch":"master","last_synced_at":"2024-11-05T16:50:38.407Z","etag":null,"topics":["deep-learning","extract-features","open-set","openmax","softmax","softmax-regression"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/abhijitbendale.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2016-07-12T18:31:17.000Z","updated_at":"2024-10-13T16:16:48.000Z","dependencies_parsed_at":"2023-02-22T21:30:29.211Z","dependency_job_id":null,"html_url":"https://github.com/abhijitbendale/OSDN","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/abhijitbendale%2FOSDN","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/abhijitbendale%2FOSDN/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/abhijitbendale%2FOSDN/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/abhijitbendale%2FOSDN/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/abhijitbendale","download_url":"https://codeload.github.com/abhijitbendale/OSDN/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247393198,"owners_count":20931805,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","extract-features","open-set","openmax","softmax","softmax-regression"],"created_at":"2024-08-01T15:01:35.146Z","updated_at":"2025-04-05T19:34:20.831Z","avatar_url":"https://github.com/abhijitbendale.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"This software package contains code used for conducting experiments in\nfollowing work:\n\nA. Bendale, T. Boult “[Towards Open Set Deep Networks](http://vast.uccs.edu/~abendale/papers/0348.pdf)” IEEE Conference on \nComputer Vision and Pattern Recognition (CVPR), 2016 [pdf](http://vast.uccs.edu/~abendale/papers/0348.pdf)\n\nAuthors: Abhijit Bendale (abendale@vast.uccs.edu)\nTerrance Boult (tboult@vast.uccs.edu)\nVision and Security Technology Lab\nUniversity of Colorado at Colorado Springs\n\nCiting the paper\n\n```\n@InProceedings{bendale-boult-cvpr2016,\ntitle={Towards Open Set Deep Networks},\nauthor={Bendale, Abhijit and Boult, Terrance},\nbooktitle={Computer Vision and Pattern Recognition (CVPR), 2016 IEEE Conference on},\nyear={2016},\norganization={IEEE}\n}\n```\n\nThe code is provided \"as is\", without any guarantees. Please refer\nCOPYRIGHT.txt and libMR/COPYRIGHT_Libmr.txt for more details\nabout license and usage restrictions.\n\n\nThe package is divided into two parts: LibMR and OpenMax. \n\n1) LibMR contains code for Extreme Value Theory based Weibull fitting (and other\ncode used in the work for Meta-Recognition). Weibull fitting performed by \nLibMR is slightly different than that done by other popular packages such as\nMATLAB's wblfit() function. We have added a short tutorial to illustrate\nthese differences.\n\n2) OpenMax contains code used in the CVPR 2016 paper on \"Towards Open Set Deep \nNetworks\". The software packages for OpenMax code calls LibMR functions for \nperforming Weibull fitting. In this code, we are providing code to compute\nOpenMax probabilities from pre-computed Mean Activation Vectors and Softmax\nprobabilities. Mean Activation Vectors and Softmax probabilities are obtained\nby passing images through Caffe's architecture. \n\n\n\n## Usage:\n\n\n#### 1) Compiling LibMR\n\nCompile LibMR and python interface to LibMR using following commands.\nFor pythong interfaces to work, you would require Cython to be pre-installed\non your machine\n```bash\ncd libMR/\nchmod +x compile.sh\n./compile.sh\n```\n\n#### 2) Precomputed Features\n##### 2.a) Extract features using a pre-trained AlexNet network. The extracted features\nare saved in a mat file. We save fc7, fc8, prob and scores from Caffe framework.\nExample of saved files can be found in data/train_features/\n\nE.g. loading the file data/train_features/n01440764/n01440764_9981.JPEG.mat in ipython\nwill lead to \n\n```python\n\u003e\u003e\u003e from scipy.io import loadmat\n\u003e\u003e\u003e import scipy as sp\n\u003e\u003e\u003e features = loadmat('data/train_features/n01440764/n01440764_9981.JPEG.mat')\n\u003e\u003e\u003e print features.keys()\n['fc7', 'fc8', '__header__', '__globals__', 'scores', 'IMG_NAME', '__version__', 'prob']\n\u003e\u003e\u003e print features['fc8'].shape\n(10, 1000)\n\u003e\u003e\u003e print sp.mean(features['prob'], axis=0)[0]\n0.999071\n\u003e\u003e\u003e print features['scores'][0][0]\n0.9990708\n```\n\nwhere fc7, fc8, prob, scores are outputs of respective layers provided by Caffe Library.\nFurther, fc8 layer contains 10 channels for 1000 classes of ImageNet. These channels are\nreferred to as crops in Caffe library [classifier.py](https://github.com/BVLC/caffe/blob/master/python/caffe/classifier.py), check the function predict() ). The average of prob layer (i.e. SoftMax layer) is the\nfinal probability value reported by AlexNet as architected in Caffe Library. In the paper (and\ncode), each of these 10 crops is referred to as channel. Hence, average of features['prob'],\nis essentially features['scores']. \n\nYou will have to install Caffe library on your machine with python interface activated. If you do\nhave it configured, then features for entire ImageNet data can be extracted using the script,\npython preprocessing/imageNet_Features.py\n\n##### 2.b) Compute Mean Activation Vector\nMean Activation Vector is defined as the Mean Vector in the pen-ultimate layer. In our paper,\nwe considered fc8 as the pen-ultimate layer. Hence, MAV is the mean of features in fc8. \nThe Mean is computed for each channel. The script to compute mean can be found in \n\npython preprocessing/MAV_Compute.py\n\nSince this process is very time consuming, we are proving pre-computed mean activation vectors\non ILSVRC 2012 dataset with this code repository. You can download the mean activation vectors\nusing following command\n\n```bash\nwget http://vast.uccs.edu/OSDN/data.tar\ntar -xvf data.tar\n```\nThe structure of Mean Files is very simple. For e.g.\n\n```python\n\u003e\u003e\u003e from scipy.io import loadmat\n\u003e\u003e\u003e mav = loadmat('data/mean_files/n01440764.mat')\n\u003e\u003e\u003e print mav['n01440764'].shape\n(10, 1000)\n```\n\nThe above is the MAV for object category n01440764. MAV is computed in similar manner for each\nof the 1000 ImageNet Categories. Note, for computing MAV, only those images are considered that \nclassified correctly by the network.\n\n##### 2.c) Compute Distances from Mean Activation Vector\n\nOnce the mean activation vector for each category is computed, the next step is to \ncompute category specific distance distribution. \n\n```python\n\u003e\u003e\u003e from scipy.io import loadmat\n\u003e\u003e\u003e distance_distribution = loadmat('data/mean_distance_files/n01440764_distances.mat')\n\u003e\u003e\u003e print distance_distribution.keys()\n['__header__', 'eucos', '__globals__', 'euclidean', 'cosine', '__version__']\n\u003e\u003e\u003e print distance_distribution['euclidean'].shape\n(10, 1224)\n```\n\nDistances (e.g. euclidean or cosine or eucos) are computed from MAV to correctly classified \ntraining examples. In the above example, n01440764 category has 1300 training images.\nDuring training process, only 1224 images were classified correctly by the network. Thus, we build\ndistance distribution using these correctly classified images. These distances are computed \nfor each channel (for defn of channel see point 2.a). This process forms a distance \ndistribution from the MAV. This distance distribution is used for Weibull fitting. In our \nexperiments, we consider tail size for Weibull fitting as 20. This means, that 20 distances\nfarthest away from MAV were used to estimate Weibull parameters. Again, since the distance\n computation process is very time comsuming, we are providing pre-computed distances for\neach class, so that you have access to the distance distribution from MAV for each\ncategory. It can be downloaded from\n\n```bash\nwget http://vast.uccs.edu/OSDN/data.tar\n```\n\nThere are 1000 distance distributions, one for each category. The script used for computing\ndistance distributions can be found in\n\n```python\npython preprocessing/compute_distances.py\n```\n\n##### 2.d) Details of Weibull tail fitting on distance distributions can be found in the\nfunction weibull_tailfitting() in file evt_fitting.py . However, compute_openmax.py performs\nWeibull tailfitting for each category while computing probability for OpenMax\n\t\n#### 3) Computing Probability values using OpenMax algorithm\n\nOpenMax probability for given image can be computed using following command.\n\n```python\npython compute_openmax.py --image_arrname data/train_features/n01440764/n01440764_14280.JPEG.mat\n```\n\nThe script accepts image feature files (features extracted from caffe as mentioned above). It computes openmax probability for the said image using default weibull tail sizes and other parameters. For more details, check the paper or get in touch with authors.\n\n#### 4) Fooling Images\nFooling images are generously provided by Anh Nguyen and Prof. Jeff Clune from University of Wyoming. We will upload the fooling images and features extracted for fooling images in few days. When you use fooling images, please be sure to cite the following paper\n\n```\n@InProceedings{nguyen2015deep,\ntitle={Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images},\nauthor={Nguyen, Anh and Yosinski, Jason and Clune, Jeff},\nbooktitle={Computer Vision and Pattern Recognition (CVPR), 2015 IEEE Conference on},\nyear={2015},\norganization={IEEE}\n}\n```\n\nPlease download the images from\n\n```\nwget http://vast.uccs.edu/OSDN/fooling_images.tar\n```\n\nThe images are made available under Creative Commons liecense. In case any questions about fooling images, please get in touch with\nProf. Jeff Clune or Anh Nguyen from [University of Wyoming](http://www.evolvingai.org/members). You can also find their fantanstic work\non fooling images on their [webpage](http://www.evolvingai.org/fooling)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fabhijitbendale%2FOSDN","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fabhijitbendale%2FOSDN","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fabhijitbendale%2FOSDN/lists"}