{"id":27957641,"url":"https://github.com/laion-ai/deep-image-diffusion-prior","last_synced_at":"2026-02-26T01:51:53.461Z","repository":{"id":47437463,"uuid":"509904403","full_name":"LAION-AI/deep-image-diffusion-prior","owner":"LAION-AI","description":"Inverts CLIP text embeds to image embeds and visualizes with deep-image-prior. ","archived":false,"fork":false,"pushed_at":"2022-07-03T02:45:06.000Z","size":1341,"stargazers_count":35,"open_issues_count":0,"forks_count":2,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-05-07T18:13:50.781Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/LAION-AI.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-07-03T02:29:13.000Z","updated_at":"2024-07-29T03:51:23.000Z","dependencies_parsed_at":"2022-09-05T13:01:01.341Z","dependency_job_id":null,"html_url":"https://github.com/LAION-AI/deep-image-diffusion-prior","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LAION-AI%2Fdeep-image-diffusion-prior","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LAION-AI%2Fdeep-image-diffusion-prior/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LAION-AI%2Fdeep-image-diffusion-prior/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LAION-AI%2Fdeep-image-diffusion-prior/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/LAION-AI","download_url":"https://codeload.github.com/LAION-AI/deep-image-diffusion-prior/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252931550,"owners_count":21827112,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-05-07T18:13:56.343Z","updated_at":"2026-02-26T01:51:48.425Z","avatar_url":"https://github.com/LAION-AI.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Deep Image Diffusion Prior\n\nby [@nousr](https://twitter.com/nousr_)\n\nInvert CLIP text embeds to image embeds and visualize them with `Deep Image Prior`.\n\n\u003ca href=\"https://replicate.com/laion-ai/deep-image-diffusion-prior\" target=\"_blank\"\u003e\u003cimg src=\"https://img.shields.io/static/v1?label=Replicate\u0026message=Demo\u0026color=blue\"\u003e\u003c/a\u003e\n\n\u003cimg src=\"/example.png\" width=\"512\"\u003e\u003c/img\u003e\n\n\u003e An oil painting of mountains, in the style of monet\n\n## Quick start (docker required)\n\n* Install [docker](https://docs.docker.com/get-docker/)\n* Install [cog](https://github.com/replicate/cog/)\n\nThe following command will download all weights and run a prediction with your inputs inside a proper docker container.\n\n```sh\ncog predict r8.im/laion-ai/deep-image-diffusion-prior \\\n  -i prompt=... \\\n  -i offset_type=... \\\n  -i num_scales=... \\\n  -i input_noise_strength=... \\\n  -i lr=... \\\n  -i offset_lr_fac=... \\\n  -i lr_decay=... \\\n  -i param_noise_strength=... \\\n  -i display_freq=... \\\n  -i iterations=... \\\n  -i num_samples_per_batch=... \\\n  -i num_cutouts=... \\\n  -i guidance_scale=... \\\n  -i seed=... \n```\n\nOr you can use the [jupyter notebook](/deep_image_diffusion_prior.ipynb)\n\n## Special Thanks\n\n* [LAION](https://discord.gg/uPMftTmrvS) for support, resources, and community\n\n* [@RiversHaveWings](https://twitter.com/RiversHaveWings) for making me aware of this technique\n\n* [Stability AI](https://stability.ai/) for compute which makes these models possible\n\n* [lucidrains](https://github.com/lucidrains) for spearheading the open-source replication of DALLE 2\n\n## Intended use\n\nSee the world \"through CLIP's eyes\" by taking advantage of the `diffusion prior` as replicated by Laion to invert CLIP \"ViT-L/14\" text embeds to image embeds (as in unCLIP/DALLE2). After, a process known as `deep-image-prior` developed by Katherine Crowson is run to visualize the features in CLIP's weights corresponding to activations from your prompt.  \n\n## Ethical considerations\n\nJust to avoid any confusion, this research is a recreation of (one part of) OpenAI's DALLE2 paper. It is _not_, \"DALLE2\", the product/service from OpenAI you may have seen on the web.\n\n## Caveats and recommendations\n\nThese visualizations can be quite abstract compared to other text-2-image models. However, you can often find a sort of dream like quality due to this. Many outputs are artistically _fantastic_ because of this, but whether or not the visual matches your prompt as often is another matter.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flaion-ai%2Fdeep-image-diffusion-prior","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flaion-ai%2Fdeep-image-diffusion-prior","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flaion-ai%2Fdeep-image-diffusion-prior/lists"}