{"id":29061678,"url":"https://github.com/skt1803/ai-explanation-tool-grid-explainer","last_synced_at":"2026-04-13T12:02:08.485Z","repository":{"id":296591426,"uuid":"993887417","full_name":"SKT1803/ai-explanation-tool-grid-explainer","owner":"SKT1803","description":"Visual AI explanation tool using grid-based occlusion for CNN interpretability.","archived":false,"fork":false,"pushed_at":"2025-05-31T18:44:41.000Z","size":5565,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-06-01T06:57:04.306Z","etag":null,"topics":["cnn","explainable-ai","grid-explainer","grid-layout","keras","lime","masking","occlusion-model","resnet-50","shap","tensorflow","visualization"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/SKT1803.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-05-31T18:28:50.000Z","updated_at":"2025-05-31T18:48:51.000Z","dependencies_parsed_at":"2025-06-01T07:09:25.467Z","dependency_job_id":null,"html_url":"https://github.com/SKT1803/ai-explanation-tool-grid-explainer","commit_stats":null,"previous_names":["skt1803/ai-explanation-tool-grid-explainer"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/SKT1803/ai-explanation-tool-grid-explainer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SKT1803%2Fai-explanation-tool-grid-explainer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SKT1803%2Fai-explanation-tool-grid-explainer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SKT1803%2Fai-explanation-tool-grid-explainer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SKT1803%2Fai-explanation-tool-grid-explainer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/SKT1803","download_url":"https://codeload.github.com/SKT1803/ai-explanation-tool-grid-explainer/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SKT1803%2Fai-explanation-tool-grid-explainer/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262219802,"owners_count":23276888,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cnn","explainable-ai","grid-explainer","grid-layout","keras","lime","masking","occlusion-model","resnet-50","shap","tensorflow","visualization"],"created_at":"2025-06-27T08:03:50.052Z","updated_at":"2026-04-13T12:02:08.461Z","avatar_url":"https://github.com/SKT1803.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\" id=\"gridexp\"\u003e\n\n\u003cimg src=\"example-images/logo.png\" width=\"125px\"\u003e\n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"https://img.shields.io/badge/TensorFlow-FF6F00?style=for-the-badge\u0026logo=tensorflow\u0026logoColor=white\" /\u003e\n  \u003cimg src=\"https://img.shields.io/badge/GridExplainer-3B82F6?style=for-the-badge\u0026logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAC0AAAAtCAMAAAANxBKoAAAAaVBMVEX59/VimMCSlZiOkZT//PkzfLL9+vdZk707gbXP0dL49vTD1uNdlb5mnMNuocaxyt2JjJDR1tmDrcx1pcieoKKoqquZm51Sj7wqd6+70eD39fOur7DW3N/l6euQttFJirqgoqT39vS1trf7RgueAAABRElEQVRIx+2U3ZKDIAyFRYqogH9Yka3Wte//kEtQx7Uijhc7e9NvnBwvMngSE4Lgg5c0Tdd3f2p0A4LI8iZ7KiSEQOxZAZ2Nz+4OvPbfGVVLCGnzTFFKueYmqiwpMMZF87U/m5I4jkkuOUJIMGEilwkOwxB/X8qux0tnO7IX35RzijQCmX3XDic6B3qmmXkWSYDS0UExKKUGXWVSZpJJkKps6rpu7n4nHDHE/U6uVenpoONsRWIC2TsnzmwxALoSmYFBEKZKwFUl00DfMaAvgW6WwO8blSEGw8e+N1X+YbZxEq7ZrondbEOCC1iD422Qm0172BWbN230zKBtpB2+5HgGhxZYnRiMEytnc1I2v6tMr/yds+y3s8fzDhZTBwt3Bys7dqaDT0P3AF6zOJzYa/AWTWG6FKNZHB0cLUE6i2WRD//DD2PKKGImtJtKAAAAAElFTkSuQmCC\" /\u003e\n\u003c/div\u003e\n\n\n\n\u003c/div\u003e\n\n\n\n\n# AI Explanation Tool, GridExplainer: Visual Model Explanation via Grid-Based Occlusion\n\nGridExplainer is a simple yet powerful interpretability tool for deep learning image models. It provides intuitive, visual explanations by measuring how hiding different image segments affects model confidence.\n\n---\n\n## Purpose\nWhile advanced tools like SHAP, LIME, and Grad-CAM provide rich, gradient or perturbation-based explanations, they often rely on:\n\n- Complex mathematical abstractions\n\n- Internal model access (e.g., gradients, layers)\n\n- Outputs that are hard to explain to non-technical users\n\n\nGridExplainer was developed with this key idea:\n\u003cpre\u003e\n❓ “Why not visually show which regions matter most — by simply hiding them and observing the effect?”\n\u003c/pre\u003e\n\nThis approach results in transparent, grid-based visualizations that mimic human reasoning and are easy to interpret.\n\n---\n\n## Why GridExplainer Makes Sense\n\n1. Direct Visual Feedback\nInstead of analyzing internal activations, GridExplainer asks:\n\n- “If I hide this part of the image, does the model get less confident?”\n\nThis produces concrete, verifiable evidence — no guesswork.\n\n2. Alignment with Human Intuition\nHumans tend to say:\n\n- “It’s a dog because of the ears and the snout.”\n\nGridExplainer systematically occludes parts of the image, simulating how people visually diagnose features.\n\n3. Transparency over Complexity\nWhile it may not offer theoretical optimality, it offers trust, clarity, and usability, which are essential in:\n\n- Medical AI\n\n- Legal tech\n\n- Business intelligence\n\n- Public-facing models\n\n\n---\n\n## How It Works\n\n1. Grid segmentation: The input image is split into an N×N grid.\n\n2. Occlusion-based perturbation: Each cell is masked (by mean value) one at a time.\n\n3. Importance scoring: We record how much the prediction confidence drops.\n\n4. Visualization: A heatmap grid is overlaid showing the impact of each cell.\n\n\n---\n\n## Technologies Used\n\n- TensorFlow / Keras\n- ResNet50 (pre-trained ImageNet model)\n- Matplotlib / OpenCV / NumPy\n- SHAP dataset (for test images)\n- Google Colab-compatible\n\n---\n\n## Installation\n\n```bash\n pip install shap opencv-python tensorflow matplotlib\n\n```\n\n---\n\n## Quick Start\n\n```bash\nfrom tensorflow.keras.applications.resnet50 import ResNet50\nfrom grid_explainer import GridExplainer\n\n# Load model and image\nmodel = ResNet50(weights=\"imagenet\")\nX, y = shap.datasets.imagenet50()\nimg = X[49]\n\n# Explain\nexplainer = GridExplainer(model, grid_size=8)\nexplainer.explain(img)\nexplainer.visualize()\n\n```\n\nYou can use any image sample between 0-50, and you can also try different samples that are not in the dataset.\n\n---\n\n\n## Notes\n\n- Works on any image classification model (even black-box ones).\n\n- The cell masking strategy is currently \"mean replacement\" but can be extended.\n\n- Designed for interpretability and education, not production-scale attribution.\n\n\n---\n\n## Sample Outputs\n\nHere are examples of the GridExplainer in action. Each grid cell shows the relative importance of that segment for the model’s prediction:\n\n- You can interpret the numbers on each cell as \"how important this region was\" — the higher the number, the bigger the drop in confidence when occluded.\n\n\u003ctable\u003e\n    \u003ctr\u003e\n    \u003ctd\u003eOriginal Image\u003c/td\u003e\n    \u003ctd\u003eGrid Explanation\u003c/td\u003e\n  \u003c/tr\u003e\n  \n  \u003ctr\u003e\n    \u003ctd\u003e\u003cimg src=\"example-images/i_1.png\" width=\"500\"\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cimg src=\"example-outputs/o_1.png\" width=\"500\"\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n\n \u003ctr\u003e\n    \u003ctd\u003e\u003cimg src=\"example-images/i_2.png\" width=\"500\"\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cimg src=\"example-outputs/o_2.png\" width=\"500\"\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n\n   \u003ctr\u003e\n    \u003ctd\u003e\u003cimg src=\"example-images/i_4.png\" width=\"500\"\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cimg src=\"example-outputs/o_4.png\" width=\"500\"\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n\n   \u003ctr\u003e\n    \u003ctd\u003e\u003cimg src=\"example-images/i_8.png\" width=\"500\"\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cimg src=\"example-outputs/o_8.png\" width=\"500\"\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n\n   \u003ctr\u003e\n    \u003ctd\u003e\u003cimg src=\"example-images/i_23.png\" width=\"500\"\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cimg src=\"example-outputs/o_23.png\" width=\"500\"\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n\n   \u003ctr\u003e\n    \u003ctd\u003e\u003cimg src=\"example-images/i_26.png\" width=\"500\"\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cimg src=\"example-outputs/o_26.png\" width=\"500\"\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n\n   \u003ctr\u003e\n    \u003ctd\u003e\u003cimg src=\"example-images/i_28.png\" width=\"500\"\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cimg src=\"example-outputs/o_28.png\" width=\"500\"\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n\n   \u003ctr\u003e\n    \u003ctd\u003e\u003cimg src=\"example-images/i_1.png\" width=\"500\"\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cimg src=\"example-outputs/o_1.png\" width=\"500\"\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n  \n   \u003ctr\u003e\n    \u003ctd\u003e\u003cimg src=\"example-images/i_31.png\" width=\"500\"\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cimg src=\"example-outputs/o_31.png\" width=\"500\"\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n\n   \u003ctr\u003e\n    \u003ctd\u003e\u003cimg src=\"example-images/i_35.png\" width=\"500\"\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cimg src=\"example-outputs/o_35.png\" width=\"500\"\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n\n   \u003ctr\u003e\n    \u003ctd\u003e\u003cimg src=\"example-images/i_39.png\" width=\"500\"\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cimg src=\"example-outputs/o_39.png\" width=\"500\"\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n\n   \u003ctr\u003e\n    \u003ctd\u003e\u003cimg src=\"example-images/i_41.png\" width=\"500\"\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cimg src=\"example-outputs/o_41.png\" width=\"500\"\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n\n   \u003ctr\u003e\n    \u003ctd\u003e\u003cimg src=\"example-images/i_49.png\" width=\"500\"\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cimg src=\"example-outputs/o_49.png\" width=\"500\"\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n\n\u003c/table\u003e\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fskt1803%2Fai-explanation-tool-grid-explainer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fskt1803%2Fai-explanation-tool-grid-explainer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fskt1803%2Fai-explanation-tool-grid-explainer/lists"}