{"id":28100734,"url":"https://github.com/poloclub/interactive-classification","last_synced_at":"2025-05-13T18:38:34.871Z","repository":{"id":42112787,"uuid":"121327895","full_name":"poloclub/interactive-classification","owner":"poloclub","description":"Interactive Classification for Deep Learning Interpretation","archived":false,"fork":false,"pushed_at":"2023-01-25T07:47:14.000Z","size":12298,"stargazers_count":77,"open_issues_count":8,"forks_count":17,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-05-06T18:14:45.288Z","etag":null,"topics":["deep-learning","inpainting","neural-networks","visualization"],"latest_commit_sha":null,"homepage":"https://cabreraalex.com/#/paper/interactive-classification","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/poloclub.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-02-13T02:12:51.000Z","updated_at":"2025-03-11T00:51:06.000Z","dependencies_parsed_at":"2023-02-14T05:47:03.024Z","dependency_job_id":null,"html_url":"https://github.com/poloclub/interactive-classification","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/poloclub%2Finteractive-classification","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/poloclub%2Finteractive-classification/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/poloclub%2Finteractive-classification/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/poloclub%2Finteractive-classification/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/poloclub","download_url":"https://codeload.github.com/poloclub/interactive-classification/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254004900,"owners_count":21998142,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","inpainting","neural-networks","visualization"],"created_at":"2025-05-13T18:38:34.284Z","updated_at":"2025-05-13T18:38:34.853Z","avatar_url":"https://github.com/poloclub.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Interactive Classification for Deep Learning Interpretation\n\nWe have designed and developed an interactive system that allows users to experiment with deep learning image classifiers and explore their robustness and sensitivity.\nSelected areas of an image can be removed in real time with classical computer vision inpainting algorithms, allowing users to ask a variety of \"what if\" questions by experimentally modifying images and seeing how the deep learning model reacts.\nThe system also computes class activation maps for any selected class, which highlight the important semantic regions of an image the model uses for classification.\nThe system runs fully in browser using Tensorflow.js, React, and SqueezeNet.\nAn advanced inpainting version is also available using a server running the PatchMatch algorithm from the [GIMP Resynthesizer plugin](https://github.com/bootchk/resynthesizer).\n\n[![YouTube video demo](images/thumbnail.png)](https://youtu.be/llub5GcOF6w)\n\nThis is the code repository for the accepted [CVPR 2018][cvpr18] Demo: **Interactive Classification for Deep Learning Interpretation**.\nVisit our research group homepage [Polo Club of Data Science][poloclub] at [Georgia Tech](http://www.gatech.edu) for more related research!\n\n## Example Scenario: Interpreting \"Failed\" Classification\n\nThe modified image (left), originally classified as *dock* is misclassified as *ocean liner* when the masts of a couple boats are removed from the original image (right).\nThe top five classification scores are tabulated underneath each image.\n\n![Failed classification](images/failure.png)\n\n## Installation\n\nDownload or clone this repository:\n\n```bash\ngit clone https://github.com/poloclub/interactive-classification.git\n```\n\nWithin the cloned repo, install the required packages with yarn:\n\n```bash\nyarn\n\n```\n\n## Usage\n\nTo run, type:\n\n```bash\nyarn start\n\n```\n\n## Advanced Inpainting\n\nThe following steps are needed to set up PatchMatch inpainting, which currently only works on Linux:\n\n1. Clone the [Resynthesizer](https://github.com/bootchk/resynthesizer) repository and follow the instructions for building the project (stop after running `make`)\n2. Find the `libresynthesizer.a` shared library in the generated `lib` folder and copy it to the `inpaint` folder in this repository\n3. Run `gcc resynth.c -L. -lresynthesizer -lm -lglib-2.0 -o prog` (may have to install glib2.0 first) to generate the `prog` executable\n4. You can now run `python3 inpaint_server.py` and PatchMatch will be used as the inpainting algorithm when running the React application with `yarn start`.\n\n## Citation\n\n**[Interactive Classification for Deep Learning Interpretation][site]**  \n[Angel Cabrera][angel], [Fred Hohman][fred], [Jason Lin][jason], [Duen Horng (Polo) Chau][polo]  \n*Demo, Conference on Computer Vision and Pattern Recognition (CVPR). June 18, 2018. Salt Lake City, USA.*\n\n```\n@article{cabrera2018interactive,\n  title={Interactive Classification for Deep Learning Interpretation},\n  author={Cabrera, Angel and Hohman, Fred and Lin, Jason and Chau, Duen Horng},\n  journal={Demo, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},\n  year={2018},\n  organization={IEEE}\n}\n```\n\n## Researchers\n\n|  Name                 | Affiliation                     |\n|-----------------------|---------------------------------|\n| [Angel Cabrera][angel]           | Georgia Tech |\n| [Fred Hohman][fred]    | Georgia Tech |\n| [Jason Lin][jason]        | Georgia Tech |\n| [Duen Horng (Polo) Chau][polo]             | Georgia Tech |\n\n## License\n\nMIT License. See [`LICENSE.md`](LICENSE.md).\n\n## Contact\n\nFor questions or support [open an issue][issues].\n\n[site]: http://fredhohman.com/papers/interactive-classification\n[paper]: http://fredhohman.com/papers/18-interactive-cvpr.pdf\n[video]: https://youtu.be/llub5GcOF6w \"Video\"\n[fred]: http://fredhohman.com \"Fred Hohman\"\n[angel]: http://cabreraalex.com/ \"Alex Cabrera\"\n[jason]: http://jlin.xyz/ \"Jason Lin\"\n[polo]: http://www.cc.gatech.edu/~dchau/ \"Polo Chau\"\n[issues]: https://github.com/poloclub/interactive-classification/issues \"Issues\"\n[cvpr18]: http://cvpr2018.thecvf.com\n[poloclub]: https://poloclub.github.io\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpoloclub%2Finteractive-classification","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpoloclub%2Finteractive-classification","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpoloclub%2Finteractive-classification/lists"}