{"id":8403922,"url":"https://github.com/ramprs/grad-cam","last_synced_at":"2025-05-16T01:05:08.424Z","repository":{"id":41207486,"uuid":"59858921","full_name":"ramprs/grad-cam","owner":"ramprs","description":"[ICCV 2017] Torch code for Grad-CAM","archived":false,"fork":false,"pushed_at":"2022-09-17T02:42:00.000Z","size":1616,"stargazers_count":1552,"open_issues_count":11,"forks_count":228,"subscribers_count":17,"default_branch":"master","last_synced_at":"2025-04-08T11:15:22.710Z","etag":null,"topics":["convolutional-neural-networks","deep-learning","grad-cam","heatmap","iccv17","interpretability","visual-explanation"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/1610.02391","language":"Lua","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ramprs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-05-27T19:37:36.000Z","updated_at":"2025-04-08T03:19:43.000Z","dependencies_parsed_at":"2023-01-18T10:34:38.984Z","dependency_job_id":null,"html_url":"https://github.com/ramprs/grad-cam","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ramprs%2Fgrad-cam","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ramprs%2Fgrad-cam/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ramprs%2Fgrad-cam/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ramprs%2Fgrad-cam/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ramprs","download_url":"https://codeload.github.com/ramprs/grad-cam/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254448579,"owners_count":22072764,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["convolutional-neural-networks","deep-learning","grad-cam","heatmap","iccv17","interpretability","visual-explanation"],"created_at":"2024-04-22T23:40:17.870Z","updated_at":"2025-05-16T01:05:08.372Z","avatar_url":"https://github.com/ramprs.png","language":"Lua","funding_links":[],"categories":["Lua"],"sub_categories":[],"readme":"\n## Grad-CAM: Gradient-weighted Class Activation Mapping\n\nCode for the paper\n\n**[Grad-CAM: Why did you say that? Visual Explanations from Deep Networks via Gradient-based Localization][7]**  \nRamprasaath R. Selvaraju, Abhishek Das, Ramakrishna Vedantam, Michael Cogswell, Devi Parikh, Dhruv Batra  \n[https://arxiv.org/abs/1610.02391][7]\n\n\nDemo: [gradcam.cloudcv.org][8]\n\n![Overview](http://i.imgur.com/JaGbdZ5.png)\n\n### Usage\n\nDownload Caffe model(s) and prototxt for VGG-16/VGG-19/AlexNet using `sh models/download_models.sh`.\n\n#### Classification\n\n```\nth classification.lua -input_image_path images/cat_dog.jpg -label 243 -gpuid 0\nth classification.lua -input_image_path images/cat_dog.jpg -label 283 -gpuid 0\n```\n\n##### Options\n\n- `proto_file`: Path to the `deploy.prototxt` file for the CNN Caffe model. Default is `models/VGG_ILSVRC_16_layers_deploy.prototxt`\n- `model_file`: Path to the `.caffemodel` file for the CNN Caffe model. Default is `models/VGG_ILSVRC_16_layers.caffemodel`\n- `input_image_path`: Path to the input image. Default is `images/cat_dog.jpg`\n- `input_sz`: Input image size. Default is 224 (Change to 227 if using AlexNet)\n- `layer_name`: Layer to use for Grad-CAM. Default is `relu5_3` (use `relu5_4` for VGG-19 and `relu5` for AlexNet)\n- `label`: Class label to generate grad-CAM for (-1 = use predicted class, 283 = Tiger cat, 243 = Boxer). Default is -1. These correspond to ILSVRC synset IDs\n- `out_path`: Path to save images in. Default is `output/`\n- `gpuid`: 0-indexed id of GPU to use. Default is -1 = CPU\n- `backend`: Backend to use with [loadcaffe][3]. Default is `nn`\n- `save_as_heatmap`: Whether to save heatmap or raw Grad-CAM. 1 = save heatmap, 0 = save raw Grad-CAM. Default is 1\n\n##### Examples\n\n'border collie' (233)\n\n![](http://i.imgur.com/nTaVH57.png)\n![](http://i.imgur.com/fjZ8E3Z.png)\n![](http://i.imgur.com/RzPhOYo.png)\n\n'tabby cat' (282)\n\n![](http://i.imgur.com/nTaVH57.png)\n![](http://i.imgur.com/94ZMSNI.png)\n![](http://i.imgur.com/wmtUTgj.png)\n\n'boxer' (243)\n\n![](http://i.imgur.com/OAoSQYT.png)\n![](http://i.imgur.com/iZuijZy.png)\n![](http://i.imgur.com/o7RStQm.png)\n\n'tiger cat' (283)\n\n![](http://i.imgur.com/OAoSQYT.png)\n![](http://i.imgur.com/NzXRy5E.png)\n![](http://i.imgur.com/fP0Dd87.png)\n\n#### Visual Question Answering\n\nClone the [VQA][5] ([http://arxiv.org/abs/1505.00468][4]) sub-repository (`git submodule init \u0026\u0026 git submodule update`), and download and unzip the provided extracted features and pretrained model.\n\n```\nth visual_question_answering.lua -input_image_path images/cat_dog.jpg -question 'What animal?' -answer 'dog' -gpuid 0\nth visual_question_answering.lua -input_image_path images/cat_dog.jpg -question 'What animal?' -answer 'cat' -gpuid 0\n\n```\n\n##### Options\n\n- `proto_file`: Path to the `deploy.prototxt` file for the CNN Caffe model. Default is `models/VGG_ILSVRC_19_layers_deploy.prototxt`\n- `model_file`: Path to the `.caffemodel` file for the CNN Caffe model. Default is `models/VGG_ILSVRC_19_layers.caffemodel`\n- `input_image_path`: Path to the input image. Default is `images/cat_dog.jpg`\n- `input_sz`: Input image size. Default is 224 (Change to 227 if using AlexNet)\n- `layer_name`: Layer to use for Grad-CAM. Default is `relu5_4` (use `relu5_3` for VGG-16 and `relu5` for AlexNet)\n- `question`: Input question. Default is `What animal?`\n- `answer`: Optional answer (For eg. \"cat\") to generate Grad-CAM for ('' = use predicted answer). Default is ''\n- `out_path`: Path to save images in. Default is `output/`\n- `model_path`: Path to VQA model checkpoint. Default is `VQA_LSTM_CNN/lstm.t7`\n- `gpuid`: 0-indexed id of GPU to use. Default is -1 = CPU\n- `backend`: Backend to use with [loadcaffe][3]. Default is `cudnn`\n- `save_as_heatmap`: Whether to save heatmap or raw Grad-CAM. 1 = save heatmap, 0 = save raw Grad-CAM. Default is 1\n\n##### Examples\n\nWhat animal? Dog\n\n![](http://i.imgur.com/OAoSQYT.png)\n![](http://i.imgur.com/QBTstax.png)\n![](http://i.imgur.com/NRyhfdL.png)\n\nWhat animal? Cat\n\n![](http://i.imgur.com/OAoSQYT.png)\n![](http://i.imgur.com/hqBWRAm.png)\n![](http://i.imgur.com/lwj5oAX.png)\n\nWhat color is the fire hydrant? Green\n\n![](http://i.imgur.com/Zak2NZW.png)\n![](http://i.imgur.com/GbhRhkg.png)\n![](http://i.imgur.com/lrAgGj0.png)\n\nWhat color is the fire hydrant? Yellow\n\n![](http://i.imgur.com/Zak2NZW.png)\n![](http://i.imgur.com/cHzOo7k.png)\n![](http://i.imgur.com/CJ6QiGD.png)\n\nWhat color is the fire hydrant? Green and Yellow\n\n![](http://i.imgur.com/Zak2NZW.png)\n![](http://i.imgur.com/i7AwHXx.png)\n![](http://i.imgur.com/7N6BVgq.png)\n\nWhat color is the fire hydrant? Red and Yellow\n\n![](http://i.imgur.com/Zak2NZW.png)\n![](http://i.imgur.com/uISYeOR.png)\n![](http://i.imgur.com/ebZVlTI.png)\n\n#### Image Captioning\n\nClone the [neuraltalk2][6] sub-repository. Running `sh models/download_models.sh` will download the pretrained model and place it in the neuraltalk2 folder.\n\nChange lines 2-4 of `neuraltalk2/misc/LanguageModel.lua` to the following:\n\n```\nlocal utils = require 'neuraltalk2.misc.utils'\nlocal net_utils = require 'neuraltalk2.misc.net_utils'\nlocal LSTM = require 'neuraltalk2.misc.LSTM'\n```\n\n\n```\nth captioning.lua -input_image_path images/cat_dog.jpg -caption 'a dog and cat posing for a picture' -gpuid 0\nth captioning.lua -input_image_path images/cat_dog.jpg -caption '' -gpuid 0\n\n```\n##### Options\n\n- `input_image_path`: Path to the input image. Default is `images/cat_dog.jpg`\n- `input_sz`: Input image size. Default is 224 (Change to 227 if using AlexNet)\n- `layer`: Layer to use for Grad-CAM. Default is 30 (relu5_3 for vgg16)\n- `caption`: Optional input caption. No input will use the generated caption as default\n- `out_path`: Path to save images in. Default is `output/`\n- `model_path`: Path to captioning model checkpoint. Default is `neuraltalk2/model_id1-501-1448236541.t7`\n- `gpuid`: 0-indexed id of GPU to use. Default is -1 = CPU\n- `backend`: Backend to use with [loadcaffe][3]. Default is `cudnn`\n- `save_as_heatmap`: Whether to save heatmap or raw Grad-CAM. 1 = save heatmap, 0 = save raw Grad-CAM. Default is 1\n\n##### Examples\n\na dog and cat posing for a picture\n\n![](http://i.imgur.com/OAoSQYT.png)\n![](http://i.imgur.com/TiKdMMw.png)\n![](http://i.imgur.com/GSQeR2M.png)\n\na bathroom with a toilet and a sink\n\n![](http://i.imgur.com/gE6VXql.png)\n![](http://i.imgur.com/K3E9TWS.png)\n![](http://i.imgur.com/em2oHRy.png)\n\n### License\n\nBSD\n\n#### 3rd-party\n\n- [VQA_LSTM_CNN][5], BSD\n- [neuraltalk2][6], BSD\n\n[3]: https://github.com/szagoruyko/loadcaffe\n[4]: http://arxiv.org/abs/1505.00468\n[5]: https://github.com/VT-vision-lab/VQA_LSTM_CNN\n[6]: https://github.com/karpathy/neuraltalk2 \n[7]: https://arxiv.org/abs/1610.02391\n[8]: http://gradcam.cloudcv.org/\n[9]: https://ramprs.github.io/\n[10]: http://abhishekdas.com/\n[11]: http://vrama91.github.io/\n[12]: http://mcogswell.io/\n[13]: https://computing.ece.vt.edu/~parikh/\n[14]: https://computing.ece.vt.edu/~dbatra/\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Framprs%2Fgrad-cam","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Framprs%2Fgrad-cam","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Framprs%2Fgrad-cam/lists"}