{"id":13689293,"url":"https://github.com/utkuozbulak/pytorch-cnn-visualizations","last_synced_at":"2025-05-12T15:36:52.134Z","repository":{"id":37549489,"uuid":"107760466","full_name":"utkuozbulak/pytorch-cnn-visualizations","owner":"utkuozbulak","description":"Pytorch implementation of convolutional neural network visualization techniques","archived":false,"fork":false,"pushed_at":"2025-01-01T10:31:10.000Z","size":8035,"stargazers_count":8009,"open_issues_count":3,"forks_count":1494,"subscribers_count":114,"default_branch":"master","last_synced_at":"2025-04-23T17:46:57.602Z","etag":null,"topics":["cam","cnn-visualization","deep-dream","grad-cam","gradient","gradient-visualization","guided-backpropagation","guided-grad-cam","pytorch","saliency","segmentation","smooth-grad"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/utkuozbulak.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-10-21T07:25:07.000Z","updated_at":"2025-04-23T17:13:02.000Z","dependencies_parsed_at":"2025-01-09T04:11:11.205Z","dependency_job_id":"a432edd2-44ec-4dba-b9e5-1d46f130e853","html_url":"https://github.com/utkuozbulak/pytorch-cnn-visualizations","commit_stats":{"total_commits":165,"total_committers":6,"mean_commits":27.5,"dds":"0.19393939393939397","last_synced_commit":"ba22fa07c9f31964e6cfa669e80f3461e11c4ebc"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/utkuozbulak%2Fpytorch-cnn-visualizations","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/utkuozbulak%2Fpytorch-cnn-visualizations/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/utkuozbulak%2Fpytorch-cnn-visualizations/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/utkuozbulak%2Fpytorch-cnn-visualizations/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/utkuozbulak","download_url":"https://codeload.github.com/utkuozbulak/pytorch-cnn-visualizations/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253766651,"owners_count":21960962,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cam","cnn-visualization","deep-dream","grad-cam","gradient","gradient-visualization","guided-backpropagation","guided-grad-cam","pytorch","saliency","segmentation","smooth-grad"],"created_at":"2024-08-02T15:01:41.633Z","updated_at":"2025-05-12T15:36:52.071Z","avatar_url":"https://github.com/utkuozbulak.png","language":"Python","funding_links":[],"categories":["XAI Libraries for Vision/Vision-Language","Python","Repos","对象检测_分割","Python Libraries(sort in alphabeta order)","📚 Project Purpose"],"sub_categories":["资源传输下载","Evaluation methods","Machine Learning (Intermediate-Level"],"readme":"# Convolutional Neural Network Visualizations \n\nThis repository contains a number of convolutional neural network visualization techniques implemented in PyTorch.\n\n**Note**: I removed cv2 dependencies and moved the repository towards PIL. A few things might be broken (although I tested all methods), I would appreciate if you could create an issue if something does not work.\n\n**Note**: The code in this repository was tested with torch version 0.4.1 and some of the functions may not work as intended in later versions. Although it shouldn't be too much of an effort to make it work, I have no plans at the moment to make the code in this repository compatible with the latest version because I'm still using 0.4.1.\n\n## Implemented Techniques\n\n* [Gradient visualization with vanilla backpropagation](#gradient-visualization)\n* [Gradient visualization with guided backpropagation](#gradient-visualization) [1]\n* [Gradient visualization with saliency maps](#gradient-visualization) [4]\n* [Gradient-weighted class activation mapping](#gradient-visualization) [3] (Generalization of [2]) \n* [Guided, gradient-weighted class activation mapping](#gradient-visualization) [3]\n* [Score-weighted class activation mapping](#gradient-visualization) [15] (Gradient-free generalization of [2])\n* [Element-wise gradient-weighted class activation mapping](#hierarchical-gradient-visualization) [16]\n* [Smooth grad](#smooth-grad) [8]\n* [CNN filter visualization](#convolutional-neural-network-filter-visualization) [9]\n* [Inverted image representations](#inverted-image-representations) [5]\n* [Deep dream](#deep-dream) [10]\n* [Class specific image generation](#class-specific-image-generation) [4] [14]\n* [Grad times image](#grad-times-image) [12]\n* [Integrated gradients](#gradient-visualization) [13]\n* [Layerwise relevance propagation](#gradient-visualization) [17]\n\n## General Information\n\nDepending on the technique, the code uses pretrained **AlexNet** or **VGG** from the model zoo. Some of the code also assumes that the layers in the model are separated into two sections; **features**, which contains the convolutional layers and **classifier**, that contains the fully connected layer (after flatting out convolutions). If you want to port this code to use it on your model that does not have such separation, you just need to do some editing on parts where it calls *model.features* and *model.classifier*.\n\nEvery technique has its own python file (e.g. *gradcam.py*) which I hope will make things easier to understand. *misc_functions.py* contains functions like image processing and image recreation which is shared by the implemented techniques.\n\nAll images are pre-processed with mean and std of the ImageNet dataset before being fed to the model. None of the code uses GPU as these operations are quite fast for a single image (except for deep dream because of the example image that is used for it is huge). You can make use of gpu with very little effort. The example pictures below include numbers in the brackets after the description, like *Mastiff (243)*, this number represents the class id in the ImageNet dataset.\n\nI tried to comment on the code as much as possible, if you have any issues understanding it or porting it, don't hesitate to send an email or create an issue.\n\nBelow, are some sample results for each operation.\n\n## Gradient Visualization\n\u003ctable border=0 align=center\u003e\n\t\u003ctbody\u003e\n    \u003ctr\u003e\n\t\t\t\u003ctd\u003e  \u003c/td\u003e\n\t\t\t\u003ctd align=\"center\"\u003e Target class: King Snake (56) \u003c/td\u003e\n\t\t\t\u003ctd align=\"center\"\u003e Target class: Mastiff (243) \u003c/td\u003e\n\t\t\t\u003ctd align=\"center\"\u003e Target class: Spider (72)\u003c/td\u003e\n\t\t\u003c/tr\u003e\n\t\t\u003ctr\u003e\n\t\t\t\u003ctd width=\"19%\" align=\"center\"\u003e Original Image \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" \u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/input_images/snake.png\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/input_images/cat_dog.png\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/input_images/spider.png\"\u003e \u003c/td\u003e\n\t\t\u003c/tr\u003e\n\t\t\u003ctr\u003e\n\t\t\t\u003ctd width=\"19%\" align=\"center\"\u003e Colored Vanilla Backpropagation \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" \u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/snake_Vanilla_BP_color.jpg\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/cat_dog_Vanilla_BP_color.jpg\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/spider_Vanilla_BP_color.jpg\"\u003e \u003c/td\u003e\n\t\t\u003c/tr\u003e\n\t\t\t\u003ctd width=\"19%\" align=\"center\"\u003e Vanilla Backpropagation Saliency \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" \u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/snake_Vanilla_BP_gray.jpg\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/cat_dog_Vanilla_BP_gray.jpg\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/spider_Vanilla_BP_gray.jpg\"\u003e \u003c/td\u003e\n\t\t\u003c/tr\u003e\n    \u003ctr\u003e\n\t\t\t\u003ctd width=\"19%\" align=\"center\"\u003e Colored Guided Backpropagation \u003cbr /\u003e  \u003cbr /\u003e  (GB)\u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" \u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/snake_Guided_BP_color.jpg\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/cat_dog_Guided_BP_color.jpg\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/spider_Guided_BP_color.jpg\"\u003e \u003c/td\u003e\n\t\t\u003c/tr\u003e\n    \u003ctr\u003e\n\t\t\t\u003ctd width=\"19%\" align=\"center\"\u003eGuided Backpropagation Saliency\u003cbr /\u003e  \u003cbr /\u003e (GB)\u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" \u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/snake_Guided_BP_gray.jpg\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/cat_dog_Guided_BP_gray.jpg\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/spider_Guided_BP_gray.jpg\"\u003e \u003c/td\u003e\n\t\t\u003c/tr\u003e\n    \u003ctr\u003e\n\t\t\t\u003ctd width=\"19%\" align=\"center\"\u003eGuided Backpropagation Negative Saliency\u003cbr /\u003e  \u003cbr /\u003e (GB)\u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" \u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/snake_neg_sal.jpg\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/cat_dog_neg_sal.jpg\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/spider_neg_sal.jpg\"\u003e \u003c/td\u003e\n\t\t\u003c/tr\u003e\n    \u003ctr\u003e\n\t\t\t\u003ctd width=\"19%\" align=\"center\"\u003eGuided Backpropagation Positive Saliency\u003cbr /\u003e  \u003cbr /\u003e (GB)\u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" \u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/snake_pos_sal.jpg\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/cat_dog_pos_sal.jpg\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/spider_pos_sal.jpg\"\u003e \u003c/td\u003e\n\t\t\u003c/tr\u003e\n    \u003ctr\u003e\n\t\t\t\u003ctd width=\"19%\" align=\"center\"\u003e Gradient-weighted Class Activation Map \u003cbr /\u003e  \u003cbr /\u003e (Grad-CAM)\u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" \u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/snake_Cam_Grayscale.jpg\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/cat_dog_Cam_Grayscale.jpg\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/spider_Cam_Grayscale.jpg\"\u003e \u003c/td\u003e\n\t\t\u003c/tr\u003e\n    \u003ctr\u003e\n\t\t\t\u003ctd width=\"19%\" align=\"center\"\u003e Gradient-weighted Class Activation Heatmap \u003cbr /\u003e  \u003cbr /\u003e (Grad-CAM)\u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" \u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/snake_Cam_Heatmap.jpg\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/cat_dog_Cam_Heatmap.jpg\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/spider_Cam_Heatmap.jpg\"\u003e \u003c/td\u003e\n\t\t\u003c/tr\u003e\n    \u003ctr\u003e\n\t\t\t\u003ctd width=\"19%\" align=\"center\"\u003e Gradient-weighted Class Activation Heatmap on Image \u003cbr /\u003e  \u003cbr /\u003e (Grad-CAM)\u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" \u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/snake_Cam_On_Image.jpg\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/cat_dog_Cam_On_Image.jpg\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/spider_Cam_On_Image.jpg\"\u003e \u003c/td\u003e\n\t\t\u003c/tr\u003e\n    \u003ctr\u003e\n\t\t\t\u003ctd width=\"19%\" align=\"center\"\u003e Score-weighted Class Activation Map \u003cbr /\u003e  \u003cbr /\u003e (Score-CAM)\u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" \u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/snake_ScoreCAM_Grayscale.png\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/cat_dog_ScoreCAM_Grayscale.png\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/spider_ScoreCAM_Grayscale.png\"\u003e \u003c/td\u003e\n\t\t\u003c/tr\u003e\n    \u003ctr\u003e\n\t\t\t\u003ctd width=\"19%\" align=\"center\"\u003e Score-weighted Class Activation Heatmap \u003cbr /\u003e  \u003cbr /\u003e (Score-CAM)\u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" \u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/snake_ScoreCAM_Heatmap.png\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/cat_dog_ScoreCAM_Heatmap.png\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/spider_ScoreCAM_Heatmap.png\"\u003e \u003c/td\u003e\n\t\t\u003c/tr\u003e\n    \u003ctr\u003e\n\t\t\t\u003ctd width=\"19%\" align=\"center\"\u003e Score-weighted Class Activation Heatmap on Image \u003cbr /\u003e  \u003cbr /\u003e (Score-CAM)\u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" \u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/snake_ScoreCAM_On_Image.png\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/cat_dog_ScoreCAM_On_Image.png\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/spider_ScoreCAM_On_Image.png\"\u003e \u003c/td\u003e\n\t\t\u003c/tr\u003e\n    \u003ctr\u003e\n\t\t\t\u003ctd width=\"19%\" align=\"center\"\u003e Colored Guided Gradient-weighted Class Activation Map \u003cbr /\u003e  \u003cbr /\u003e (Guided-Grad-CAM)\u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" \u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/snake_GGrad_Cam.jpg\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/cat_dog_GGrad_Cam.jpg\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/spider_GGrad_Cam.jpg\"\u003e \u003c/td\u003e\n\t\t\u003c/tr\u003e\n    \u003ctr\u003e\n\t\t\t\u003ctd width=\"19%\" align=\"center\"\u003e Guided Gradient-weighted Class Activation Map Saliency \u003cbr /\u003e  \u003cbr /\u003e (Guided-Grad-CAM)\u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" \u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/snake_GGrad_Cam_gray.jpg\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/cat_dog_GGrad_Cam_gray.jpg\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/spider_GGrad_Cam_gray.jpg\"\u003e \u003c/td\u003e\n\t\t\u003c/tr\u003e\n    \u003ctr\u003e\n\t\t\t\u003ctd width=\"19%\" align=\"center\"\u003e Integrated Gradients \u003cbr /\u003e (without image multiplication)  \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" \u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/snake_Integrated_G_gray.jpg\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/cat_dog_Integrated_G_gray.jpg\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/spider_Integrated_G_gray.jpg\"\u003e \u003c/td\u003e\n\t\t\u003c/tr\u003e\n    \u003ctr\u003e\n\t\t\t\u003ctd width=\"19%\" align=\"center\"\u003e Layerwise Relevance \u003cbr /\u003e (LRP) - Layer 7  \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" \u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/LRP_out_snake_7.png\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/LRP_out_dog_7.png\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/LRP_out_spider_7.png\"\u003e \u003c/td\u003e\n\t\t\u003c/tr\u003e\n    \u003ctr\u003e\n\t\t\t\u003ctd width=\"19%\" align=\"center\"\u003e Layerwise Relevance \u003cbr /\u003e (LRP) - Layer 1  \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" \u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/LRP_out_snake.png\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/LRP_out_dog.png\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/LRP_out_spider.png\"\u003e \u003c/td\u003e\n\t\t\u003c/tr\u003e\n\t\u003c/tbody\u003e\n\u003c/table\u003e\n\n## Hierarchical Gradient Visualization\nLayerCAM [16] is a simple modification of Grad-CAM [3], which can generate reliable class activation maps from different layers. For the examples provided below, a pre-trained **VGG16** was used.\n\n\u003ctable border=0 align=center\u003e\n\t\u003ctbody\u003e \n    \u003ctr\u003e\n\t\t\t\u003ctd\u003e  \u003c/td\u003e\n\t\t\t\u003ctd align=\"center\"\u003e Class Activation Map \u003c/td\u003e\n\t\t\t\u003ctd align=\"center\"\u003e Class Activation HeatMap \u003c/td\u003e\n\t\t\t\u003ctd align=\"center\"\u003e Class Activation HeatMap on Image\u003c/td\u003e\n\t\t\u003c/tr\u003e\n\t\t\u003ctr\u003e\n\t\t\t\u003ctd width=\"19%\" align=\"center\"\u003e LayerCAM \u003cbr /\u003e (Layer 9)\u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"results/hierarchical_gradient_visualization/snake_LayerCam_pool2_Grayscale.png\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"results/hierarchical_gradient_visualization/snake_LayerCam_pool2_Heatmap.png\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"results/hierarchical_gradient_visualization/snake_LayerCam_pool2_On_Image.png\"\u003e \u003c/td\u003e\n\t\t\u003c/tr\u003e\n\t\t\u003ctr\u003e\n\t\t\t\u003ctd width=\"19%\" align=\"center\"\u003e LayerCAM \u003cbr /\u003e (Layer 16)\u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"results/hierarchical_gradient_visualization/snake_LayerCam_pool3_Grayscale.png\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"results/hierarchical_gradient_visualization/snake_LayerCam_pool3_Heatmap.png\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"results/hierarchical_gradient_visualization/snake_LayerCam_pool3_On_Image.png\"\u003e \u003c/td\u003e\n\t\t\u003c/tr\u003e\n\t\t\u003ctr\u003e\n\t\t\t\u003ctd width=\"19%\" align=\"center\"\u003e LayerCAM \u003cbr /\u003e (Layer 23)\u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"results/hierarchical_gradient_visualization/snake_LayerCam_pool4_Grayscale.png\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"results/hierarchical_gradient_visualization/snake_LayerCam_pool4_Heatmap.png\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"results/hierarchical_gradient_visualization/snake_LayerCam_pool4_On_Image.png\"\u003e \u003c/td\u003e\n\t\t\u003c/tr\u003e\n\t\t\u003ctr\u003e\n\t\t\t\u003ctd width=\"19%\" align=\"center\"\u003e LayerCAM \u003cbr /\u003e (Layer 30)\u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"results/hierarchical_gradient_visualization/snake_LayerCam_pool5_Grayscale.png\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"results/hierarchical_gradient_visualization/snake_LayerCam_pool5_Heatmap.png\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"results/hierarchical_gradient_visualization/snake_LayerCam_pool5_On_Image.png\"\u003e \u003c/td\u003e\n\t\t\u003c/tr\u003e\n\t\u003c/tbody\u003e\n\u003c/table\u003e\n\n\n## Grad Times Image\nAnother technique that is proposed is simply multiplying the gradients with the image itself. Results obtained with the usage of multiple gradient techniques are below.\n\n\u003ctable border=0  align=center\u003e\n\t\u003ctbody\u003e \n\t\t\u003ctr\u003e\n\t\t\t\u003ctd width=\"19%\" align=\"center\"\u003e Vanilla Grad \u003cbr /\u003e \u003ci\u003eX\u003c/i\u003e \u003cbr /\u003e Image\u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/snake_Vanilla_grad_times_image_gray.jpg\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/cat_dog_Vanilla_grad_times_image_gray.jpg\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/spider_Vanilla_grad_times_image_gray.jpg\"\u003e \u003c/td\u003e\n\t\t\u003c/tr\u003e\n\t\t\u003ctr\u003e\n\t\t\t\u003ctd width=\"19%\" align=\"center\"\u003e Guided Grad \u003cbr /\u003e \u003ci\u003eX\u003c/i\u003e \u003cbr /\u003e Image\u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/snake_Guided_grad_times_image_gray.jpg\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/cat_dog_Guided_grad_times_image_gray.jpg\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/spider_Guided_grad_times_image_gray.jpg\"\u003e \u003c/td\u003e\n\t\t\u003c/tr\u003e\n\t\t\u003ctr\u003e\n\t\t\t\u003ctd width=\"19%\" align=\"center\"\u003e Integrated Grad \u003cbr /\u003e \u003ci\u003eX\u003c/i\u003e \u003cbr /\u003e Image\u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/snake_Integrated_grad_times_image_gray.jpg\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/cat_dog_Integrated_grad_times_image_gray.jpg\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/gradient_visualizations/spider_Integrated_grad_times_image_gray.jpg\"\u003e \u003c/td\u003e\n\t\t\u003c/tr\u003e\n\t\u003c/tbody\u003e\n\u003c/table\u003e\n\n\n## Smooth Grad\nSmooth grad is adding some Gaussian noise to the original image and calculating gradients multiple times and averaging the results [8]. There are two examples at the bottom which use _vanilla_ and _guided_ backpropagation to calculate the gradients. Number of images (_n_) to average over is selected as 50. _σ_ is shown at the bottom of the images.\n\n\u003ctable border=0 align=center\u003e\n\t\u003ctbody\u003e \n    \u003ctr\u003e\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cstrong\u003eVanilla Backprop\u003c/strong\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003c/td\u003e\n\t\t\u003c/tr\u003e\n\u003ctr\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/cnn-gifs/master/vanilla/snake_.gif\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/cnn-gifs/master/vanilla/dog_.gif\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/cnn-gifs/master/vanilla/spider_.gif\"\u003e \u003c/td\u003e\n\t\t\u003c/tr\u003e\n\t\u003c/tbody\u003e\n\u003c/table\u003e\n\n\n\u003ctable border=0 align=center\u003e\n\t\u003ctbody\u003e \n    \u003ctr\u003e\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cstrong\u003eGuided Backprop\u003c/strong\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003c/td\u003e\n\t\t\u003c/tr\u003e\n\u003ctr\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/cnn-gifs/master/gbp/snake_.gif\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/cnn-gifs/master/gbp/dog_.gif\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/cnn-gifs/master/gbp/spider_.gif\"\u003e \u003c/td\u003e\n\t\t\u003c/tr\u003e\n\t\u003c/tbody\u003e\n\u003c/table\u003e\n\n## Convolutional Neural Network Filter Visualization\nCNN filters can be visualized when we optimize the input image with respect to output of the specific convolution operation. For this example I used a pre-trained **VGG16**. Visualizations of layers start with basic color and direction filters at lower levels. As we approach towards the final layer the complexity of the filters also increase. If you employ external techniques like blurring, gradient clipping etc. you will probably produce better images.\n\n\u003ctable border=0 align=center\u003e\n\t\u003ctbody\u003e \n\t\t\u003ctr\u003e\n\t\t\t\u003ctd width=\"19%\" align=\"center\"\u003e Layer 2 \u003cbr /\u003e (Conv 1-2)\u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/layer_visualizations/layer_vis_l2_f1.jpg\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/layer_visualizations/layer_vis_l2_f21.jpg\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/layer_visualizations/layer_vis_l2_f54.jpg\"\u003e \u003c/td\u003e\n\t\t\u003c/tr\u003e\n\t\t\u003ctr\u003e\n\t\t\t\u003ctd width=\"19%\" align=\"center\"\u003e Layer 10 \u003cbr /\u003e (Conv 2-1)\u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/layer_visualizations/layer_vis_l10_f7.jpg\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/layer_visualizations/layer_vis_l10_f10.jpg\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/layer_visualizations/layer_vis_l10_f69.jpg\"\u003e \u003c/td\u003e\n\t\t\u003c/tr\u003e\n\t\t\u003ctr\u003e\n\t\t\t\u003ctd width=\"19%\" align=\"center\"\u003e Layer 17 \u003cbr /\u003e (Conv 3-1)\u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/layer_visualizations/layer_vis_l17_f4.jpg\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/layer_visualizations/layer_vis_l17_f8.jpg\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/layer_visualizations/layer_vis_l17_f9.jpg\"\u003e \u003c/td\u003e\n\t\t\u003c/tr\u003e\n\t\t\u003ctr\u003e\n\t\t\t\u003ctd width=\"19%\" align=\"center\"\u003e Layer 24 \u003cbr /\u003e (Conv 4-1)\u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/layer_visualizations/layer_vis_l24_f4.jpg\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/layer_visualizations/layer_vis_l24_f17.jpg\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/layer_visualizations/layer_vis_l24_f22.jpg\"\u003e \u003c/td\u003e\n\t\t\u003c/tr\u003e\n\t\u003c/tbody\u003e\n\u003c/table\u003e\n\nAnother way to visualize CNN layers is to to visualize activations for a specific input on a specific layer and filter. This was done in [1] Figure 3. Below example is obtained from layers/filters of VGG16 for the first image using guided backpropagation. The code for this opeations is in *layer_activation_with_guided_backprop.py*. The method is quite similar to guided backpropagation but instead of guiding the signal from the last layer and a specific target, it guides the signal from a specific layer and filter. \n\n\u003ctable border=0 align=center\u003e\n\t\u003ctbody\u003e \n    \u003ctr\u003e\t\t\u003ctd width=\"27%\" align=\"center\"\u003e Input Image \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e Layer Vis. (Filter=0)\u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e Filter Vis. (Layer=29)\u003c/td\u003e\n\t\t\u003c/tr\u003e\n\u003ctr\u003e\n\t\t\t\u003ctd width=\"27%\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/input_images/spider.png\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/cnn-gifs/master/spider_layer_graph.gif\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/cnn-gifs/master/spider_filter_graph.gif\"\u003e \u003c/td\u003e\n\t\t\u003c/tr\u003e\n\t\u003c/tbody\u003e\n\u003c/table\u003e\n\n\n## Inverted Image Representations\nI think this technique is the most complex technique in this repository in terms of understanding what the code does. It is mainly because of complex regularization. If you truly want to understand how this is implemented I suggest you read the second and third page of the paper [5], specifically, the regularization part. Here, the aim is to generate original image after nth layer. The further we go into the model, the harder it becomes. The results in the paper are incredibly good (see Figure 6) but here, the result quickly becomes messy as we iterate through the layers. This is because the authors of the paper tuned the parameters for each layer individually. You can tune the parameters just like the to ones that are given in the paper to optimize results for each layer. The inverted examples from several layers of **AlexNet** with the previous *Snake* picture are below.\n\n\n\u003ctable border=0 align=center\u003e\n\t\u003ctbody\u003e \n    \u003ctr\u003e\t\t\u003ctd width=\"27%\" align=\"center\"\u003e Layer 0: \u003cstrong\u003eConv2d\u003c/strong\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e Layer 2: \u003cstrong\u003eMaxPool2d\u003c/strong\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e Layer 4: \u003cstrong\u003eReLU\u003c/strong\u003e \u003c/td\u003e\n\t\t\u003c/tr\u003e\n\t\t\u003ctr\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/inverted_images/Layer_0_Inverted.jpg\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/inverted_images/Layer_2_Inverted.jpg\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/inverted_images/Layer_4_Inverted.jpg\"\u003e \u003c/td\u003e\n\t\t\u003c/tr\u003e\n\t\u003c/tbody\u003e\n\u003c/table\u003e\n\u003ctable border=0 align=center\u003e\n\t\u003ctbody\u003e \n    \u003ctr\u003e\t\t\u003ctd width=\"27%\" align=\"center\"\u003e Layer 7: \u003cstrong\u003eReLU\u003c/strong\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e Layer 9: \u003cstrong\u003eReLU\u003c/strong\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e Layer 12: \u003cstrong\u003eMaxPool2d\u003c/strong\u003e \u003c/td\u003e\n\t\t\u003c/tr\u003e\n\t\t\u003ctr\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/inverted_images/Layer_7_Inverted.jpg\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/inverted_images/Layer_9_Inverted.jpg\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/inverted_images/Layer_12_Inverted.jpg\"\u003e \u003c/td\u003e\n\t\t\u003c/tr\u003e\n\t\u003c/tbody\u003e\n\u003c/table\u003e\n\n\n\n## Deep Dream\nDeep dream is technically the same operation as layer visualization the only difference is that you don't start with a random image but use a real picture. The samples below were created with **VGG19**, the produced result is entirely up to the filter so it is kind of hit or miss. The more complex models produce mode high level features. If you replace **VGG19** with an **Inception** variant you will get more noticable shapes when you target higher conv layers. Like layer visualization, if you employ additional techniques like gradient clipping, blurring etc. you might get better visualizations.\n\n\u003ctable border=0 align=center\u003e\n\t\u003ctbody\u003e\n\t\t\u003ctr\u003e\n\t\t\t\u003ctd width=\"19%\" align=\"center\"\u003eOriginal Image\u003c/td\u003e\n\t\t\t\u003ctd width=\"70%\" align=\"center\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/input_images/dd_tree.png\"\u003e \u003c/td\u003e\n\t\t\u003c/tr\u003e\n\t\t\u003ctr\u003e\n\t\t\t\u003ctd width=\"19%\" align=\"center\"\u003eVGG19 \u003cbr /\u003e Layer: 34  \u003cbr /\u003e (Final Conv. Layer) Filter: 94\u003c/td\u003e\n\t\t\t\u003ctd width=\"70%\" align=\"center\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/dd_l34_f94_iter250.jpg\"\u003e \u003c/td\u003e\n\t\t\u003c/tr\u003e\n\t\t\u003ctr\u003e\n\t\t\t\u003ctd width=\"19%\" align=\"center\"\u003eVGG19 \u003cbr /\u003e Layer: 34  \u003cbr /\u003e (Final Conv. Layer) Filter: 103\u003c/td\u003e\n\t\t\t\u003ctd width=\"70%\" align=\"center\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/pytorch-cnn-visualizations/master/results/dd_l34_f103_iter250.jpg\"\u003e \u003c/td\u003e\n\t\t\u003c/tr\u003e\n\t\u003c/tbody\u003e\n\u003c/table\u003e\n\n\n## Class Specific Image Generation\nThis operation produces different outputs based on the model and the applied regularization method. Below, are some samples produced with **VGG19** incorporated with Gaussian blur every other iteration (see [14] for details). The quality of generated images also depend on the model, **AlexNet** generally has green(ish) artifacts but VGGs produce (kind of) better images. Note that these images are generated with regular CNNs with optimizing the input and **not with GANs**.\n\n\u003ctable border=0 align=center\u003e\n\t\u003ctbody\u003e\n    \u003ctr\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e Target class: Worm Snake (52) - (VGG19) \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e Target class: Spider (72) - (VGG19) \u003c/td\u003e\n\t\t\u003c/tr\u003e\n\t\t\u003ctr\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/cnn-gifs/master/snake.gif\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/cnn-gifs/master/spider.gif\"\u003e \u003c/td\u003e\n\t\t\u003c/tr\u003e\n\t\u003c/tbody\u003e\n\u003c/table\u003e\n\nThe samples below show the produced image with no regularization, l1 and l2 regularizations on target class: **flamingo** (130) to show the differences between regularization methods. These images are generated with a pretrained AlexNet. \n\n\u003ctable border=0 align=\"center\"\u003e\n\t\u003ctbody\u003e \n    \u003ctr\u003e\t\t\u003ctd width=\"27%\" align=\"center\"\u003e No Regularization \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e L1 Regularization \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e L2 Regularization \u003c/td\u003e\n\t\t\u003c/tr\u003e\n\t\t\u003ctr\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/cnn-gifs/master/flamingo_no_norm.gif\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/cnn-gifs/master/flamingo_l1_norm.gif\"\u003e \u003c/td\u003e\n\t\t\t\u003ctd width=\"27%\" align=\"center\"\u003e \u003cimg src=\"https://raw.githubusercontent.com/utkuozbulak/cnn-gifs/master/flamingo_l2_norm.gif\"\u003e \u003c/td\u003e\n\t\t\u003c/tr\u003e\n\t\u003c/tbody\u003e\n\u003c/table\u003e\n\n\nProduced samples can further be optimized to resemble the desired target class, some of the operations you can incorporate to improve quality are; blurring, clipping gradients that are below a certain treshold, random color swaps on some parts, random cropping the image, forcing generated image to follow a path to force continuity.\n\nSome of these techniques are implemented in *generate_regularized_class_specific_samples.py* (courtesy of [alexstoken](https://github.com/alexstoken)).\n\n## Requirements:\n```\ntorch == 0.4.1\ntorchvision \u003e= 0.1.9\nnumpy \u003e= 1.13.0\nmatplotlib \u003e= 1.5\nPIL \u003e= 1.1.7\n```\n\n## Citation\n\nIf you find the code in this repository useful for your research consider citing it.\n\n\t@misc{uozbulak_pytorch_vis_2022,\n\t  author = {Utku Ozbulak},\n\t  title = {PyTorch CNN Visualizations},\n\t  year = {2019},\n\t  publisher = {GitHub},\n\t  journal = {GitHub repository},\n\t  howpublished = {\\url{https://github.com/utkuozbulak/pytorch-cnn-visualizations}},\n\t  commit = {b7e60adaf64c9be97b480509285718603d1e9ba4}\n\t}\n\t\n## References:\n\n[1] J. T. Springenberg, A. Dosovitskiy, T. Brox, and M. Riedmiller. *Striving for Simplicity: The All Convolutional Net*, https://arxiv.org/abs/1412.6806\n\n[2] B. Zhou, A. Khosla, A. Lapedriza, A. Oliva, A. Torralba. *Learning Deep Features for Discriminative Localization*, https://arxiv.org/abs/1512.04150\n\n[3] R. R. Selvaraju, A. Das, R. Vedantam, M. Cogswell, D. Parikh, and D. Batra. *Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization*, https://arxiv.org/abs/1610.02391\n\n[4] K. Simonyan, A. Vedaldi, A. Zisserman. *Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps*, https://arxiv.org/abs/1312.6034\n\n[5] A. Mahendran, A. Vedaldi. *Understanding Deep Image Representations by Inverting Them*, https://arxiv.org/abs/1412.0035\n\n[6] H. Noh, S. Hong, B. Han,  *Learning Deconvolution Network for Semantic Segmentation* https://www.cv-foundation.org/openaccess/content_iccv_2015/papers/Noh_Learning_Deconvolution_Network_ICCV_2015_paper.pdf\n\n[7] A. Nguyen, J. Yosinski, J. Clune.  *Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable  Images* https://arxiv.org/abs/1412.1897\n\n[8] D. Smilkov, N. Thorat, N. Kim, F. Viégas, M. Wattenberg. *SmoothGrad: removing noise by adding noise* https://arxiv.org/abs/1706.03825\n\n[9] D. Erhan, Y. Bengio, A. Courville, P. *Vincent. Visualizing Higher-Layer Features of a Deep Network* https://www.researchgate.net/publication/265022827_Visualizing_Higher-Layer_Features_of_a_Deep_Network\n\n[10] A. Mordvintsev, C. Olah, M. Tyka. *Inceptionism: Going Deeper into Neural Networks* https://research.googleblog.com/2015/06/inceptionism-going-deeper-into-neural.html\n\n[11] I. J. Goodfellow, J. Shlens, C. Szegedy. *Explaining and Harnessing Adversarial Examples* https://arxiv.org/abs/1412.6572\n\n[12] A. Shrikumar, P. Greenside, A. Shcherbina, A. Kundaje. *Not Just a Black Box: Learning Important Features Through Propagating Activation Differences* https://arxiv.org/abs/1605.01713\n\n[13] M. Sundararajan, A. Taly, Q. Yan. *Axiomatic Attribution for Deep Networks* https://arxiv.org/abs/1703.01365\n\n[14] J. Yosinski, J. Clune, A. Nguyen, T. Fuchs, Hod Lipson, *Understanding Neural Networks Through Deep Visualization* https://arxiv.org/abs/1506.06579\n\n[15] H. Wang, Z. Wang, M. Du, F. Yang, Z. Zhang, S. Ding, P. Mardziel, X. Hu. *Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks* https://arxiv.org/abs/1910.01279\n\n[16] P. Jiang, C. Zhang, Q. Hou, M. Cheng, Y. Wei. LayerCAM: *Exploring Hierarchical Class Activation Maps for Localization* http://mmcheng.net/mftp/Papers/21TIP_LayerCAM.pdf\n\n[17] G. Montavon1, A. Binder, S. Lapuschkin, W. Samek, and K. Muller. *Layer-Wise Relevance Propagation: An Overview* https://www.researchgate.net/publication/335708351_Layer-Wise_Relevance_Propagation_An_Overview\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Futkuozbulak%2Fpytorch-cnn-visualizations","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Futkuozbulak%2Fpytorch-cnn-visualizations","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Futkuozbulak%2Fpytorch-cnn-visualizations/lists"}