{"id":28961577,"url":"https://github.com/zer0int/clip-resnet-classic-deepdream","last_synced_at":"2025-10-09T12:13:13.538Z","repository":{"id":297984688,"uuid":"998492565","full_name":"zer0int/CLIP-ResNet-classic-DeepDream","owner":"zer0int","description":"Classic original Inception style DeepDream, but with CLIP ResNet. And CLIP ViT for comparison.","archived":false,"fork":false,"pushed_at":"2025-06-08T18:20:31.000Z","size":1090,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"CLIP-vision","last_synced_at":"2025-06-24T02:04:12.055Z","etag":null,"topics":["clip","cnn","deepdream","gradient-ascent","resnet","visualization","vit","xai"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zer0int.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-06-08T18:12:35.000Z","updated_at":"2025-06-09T15:07:19.000Z","dependencies_parsed_at":"2025-06-08T19:23:45.584Z","dependency_job_id":"0731ca95-a386-417f-9b55-e6fca26d502b","html_url":"https://github.com/zer0int/CLIP-ResNet-classic-DeepDream","commit_stats":null,"previous_names":["zer0int/clip-resnet-classic-deepdream"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/zer0int/CLIP-ResNet-classic-DeepDream","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zer0int%2FCLIP-ResNet-classic-DeepDream","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zer0int%2FCLIP-ResNet-classic-DeepDream/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zer0int%2FCLIP-ResNet-classic-DeepDream/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zer0int%2FCLIP-ResNet-classic-DeepDream/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zer0int","download_url":"https://codeload.github.com/zer0int/CLIP-ResNet-classic-DeepDream/tar.gz/refs/heads/CLIP-vision","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zer0int%2FCLIP-ResNet-classic-DeepDream/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279001441,"owners_count":26083078,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-09T02:00:07.460Z","response_time":59,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["clip","cnn","deepdream","gradient-ascent","resnet","visualization","vit","xai"],"created_at":"2025-06-24T02:04:12.113Z","updated_at":"2025-10-09T12:13:13.508Z","avatar_url":"https://github.com/zer0int.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🌌 Classic DeepDream with CLIP ResNet 🤖💭💫\n\n### Recommended Quickstart options:\n\n- Single, most salient channel [0]:\n ```\npython clip_deepdream_rn.py --img img/lab.jpg --layers layer3 --mode s_channel --lr 0.008\n ```\n- Single, most salient channel [1], filtered out 'generalizing' activations:\n ```\npython clip_deepdream_rn.py --img img/lab.jpg --layers layer3 --mode s_channel --filter_acts --lr 0.008\n ```\n- 🌟 All channels of a layer3 [2], but nudged by mean/std and norm:\n ```\npython clip_deepdream_rn.py --img img/lab.jpg --layers layer3 --mode m_ch_norm --lr 0.005\n ```\n- 💫 Neuron mode, two layers [3]: Noisier image (you can try: decrease to --lr 0.01):\n ```\npython clip_deepdream_rn.py --img img/lab.jpg --layers layer2 layer3 --mode s_neuron --lr 0.03\n ```\n![ResNet-DeepDream](https://github.com/user-attachments/assets/e81d6b1d-821e-4b34-a54b-cccdb7e3d73e)\n----------\n\n- Single (\"s_\") most salient channel:\n- Optional: filter generalizing 'salient to everything' activations\n ```\n--mode s_channel --filter_acts\n ```\n\n- ALL channels (\"m_\" is for \"multi\") of target layer:\n- For fun, also select a different model. Default is RN50x4.\n ```\n--mode m_ch_norm --model RN101\n ```\n\n- Single most salient neuron:\n- Uses Gradient Ascent on Text + GradCAM for attention-informed selection\n- Batch size is 10 by default; if OOM for large models, reduce it:\n ```\n--mode s_neuron --batch_size_ga 6 --k 6\n ```\n\n- Default for octaves is -3 to +3 and scale 1.5; for lower resolution (much faster), try:\n ```\n--octaves -2 -1 0 1 2 --octave_scale 1.3\n ```\n\n- For ALL available arguments, do:\n ```\npython clip_deepdream_rn.py --help\n ```\n\n- CLIP 'seeing tiger' and 'making tiger' (s_neuron mode):\n\n![resnet-ga-neuron](https://github.com/user-attachments/assets/b3db2eb8-ee05-41df-a0e7-c382c5a1ac4b)\n\n- Channel mode; use `--save_steps` to save every 20th step:\n\nhttps://github.com/user-attachments/assets/b4ad8ce1-bc04-4b08-9a63-25aadcd189b5\n\n----------\n\n### Novel DeepDream with CLIP ViT\n\n- Very different technique, but good for comparison.\n- Use a square image for ViT input\n- See `clip_deepdream_vit.py --help` for all args\n\n ```\npython clip_deepdream_vit.py --img img/lab.png\n ```\n![vit-dream](https://github.com/user-attachments/assets/38310c26-6dc6-4d3d-bb89-bb5c6923e8c6)\n\n- See also (for more ViT): [zer0int/CLIP-DeepDream](https://github.com/zer0int/CLIP-DeepDream)\n\n----------\n\n- Image source: 'dog: labrador' from original DeepDream, [www.tensorflow.org/tutorials/generative/deepdream](https://www.tensorflow.org/tutorials/generative/deepdream)\n- Rats: [self] (quality: very old!)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzer0int%2Fclip-resnet-classic-deepdream","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzer0int%2Fclip-resnet-classic-deepdream","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzer0int%2Fclip-resnet-classic-deepdream/lists"}