{"id":19320898,"url":"https://github.com/eliahuhorwitz/conffusion","last_synced_at":"2025-08-01T13:35:51.356Z","repository":{"id":64007948,"uuid":"567179083","full_name":"eliahuhorwitz/Conffusion","owner":"eliahuhorwitz","description":"Official Implementation for the \"Conffusion: Confidence Intervals for Diffusion Models\" paper.","archived":false,"fork":false,"pushed_at":"2022-11-27T19:11:26.000Z","size":2240,"stargazers_count":119,"open_issues_count":0,"forks_count":3,"subscribers_count":5,"default_branch":"main","last_synced_at":"2023-11-07T17:26:09.244Z","etag":null,"topics":["computer-vision","conformal-prediction","deep-learning","diffusion","diffusion-models","generative-model","image-generation","image-to-image","inpainting","prediction-intervals","pytorch","quantile-regression","super-resolution","uncertainty-estimation","uncertainty-quantification"],"latest_commit_sha":null,"homepage":"https://www.vision.huji.ac.il/conffusion/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/eliahuhorwitz.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-11-17T08:38:22.000Z","updated_at":"2023-10-24T08:59:00.000Z","dependencies_parsed_at":"2023-01-14T18:45:46.744Z","dependency_job_id":null,"html_url":"https://github.com/eliahuhorwitz/Conffusion","commit_stats":null,"previous_names":[],"tags_count":0,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eliahuhorwitz%2FConffusion","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eliahuhorwitz%2FConffusion/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eliahuhorwitz%2FConffusion/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eliahuhorwitz%2FConffusion/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/eliahuhorwitz","download_url":"https://codeload.github.com/eliahuhorwitz/Conffusion/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223903603,"owners_count":17222541,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["computer-vision","conformal-prediction","deep-learning","diffusion","diffusion-models","generative-model","image-generation","image-to-image","inpainting","prediction-intervals","pytorch","quantile-regression","super-resolution","uncertainty-estimation","uncertainty-quantification"],"created_at":"2024-11-10T01:33:41.179Z","updated_at":"2024-11-10T01:33:41.780Z","avatar_url":"https://github.com/eliahuhorwitz.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Con*ffusion*: Confidence Intervals for Diffusion Models\n### \u003ca href=\"https://www.vision.huji.ac.il/conffusion\" target=\"_blank\"\u003eProject\u003c/a\u003e | \u003ca href=\"http://arxiv.org/abs/2211.09795\" target=\"_blank\"\u003ePaper\u003c/a\u003e \u003cbr\u003e\nOfficial PyTorch Implementation for the \"Con*ffusion*: Confidence Intervals for Diffusion Models\" paper.  \n\n![](imgs/header.gif)\n\n\u003cstrong\u003eCon*ffusion*:\u003c/strong\u003e Given a corrupted input image, our method Con*ffusion*, repurposes a pretrained diffusion model to generate lower and upper bounds around each reconstructed pixel. The true pixel value is guaranteed to fall within these bounds with probability p. We present the bounds for inpainting (the context is dimmed for visualization) and super-resolution. Tighter intervals provide more information; we visualize the normalized interval size, darker values are tighter intervals\n___\n\n\u003e **Con*ffusion*: Confidence Intervals for Diffusion Models**\u003cbr\u003e\n\u003e Eliahu Horwitz, Yedid Hoshen\u003cbr\u003e\n\u003e \u003ca href=\"http://arxiv.org/abs/2211.09795\" target=\"_blank\"\u003ehttp://arxiv.org/abs/2211.09795 \u003c/a\u003e \u003cbr\u003e\n\u003e\n\u003e**Abstract:** Diffusion models have become the go-to method for many generative tasks, particularly for image-to-image generation tasks such as super-resolution and inpainting. \n\u003e Current diffusion-based methods do not provide statistical guarantees regarding the generated results, often preventing their use in high-stakes situations. \n\u003e To bridge this gap, we construct a confidence interval around each generated pixel such that the true value of the pixel is guaranteed to fall within the interval with a probability set by the user. \n\u003e Since diffusion models parametrize the data distribution, a straightforward way of constructing such intervals is by drawing multiple samples and calculating their bounds. \n\u003e However, this method has several drawbacks: i) slow sampling speeds ii) suboptimal bounds iii) requires training a diffusion model per task. \n\u003e To mitigate these shortcomings we propose Con*ffusion*, wherein we fine-tune a pre-trained diffusion model to predict interval bounds in a single forward pass. \n\u003e We show that Con*ffusion* outperforms the baseline method while being three orders of magnitude faster.\n\n\n\n\n## Getting Started\n\n### Setup \n1.  Clone the repo:\n```bash\ngit clone https://github.com/eliahuhorwitz/Conffusion.git\ncd Conffusion\n```\n2. Create a new environment and install the libraries:\n```bash\npython3 -m venv conffusion_venv\nsource conffusion_venv/bin/activate\npip install -r requirements.txt\n```\n\n3. Download the data from the \u003ca href=\"https://drive.google.com/file/d/10nP41Z-cHmLuHrIOhvxe-IZrP-gnK4vm/view?usp=share_link\" target=\"_blank\"\u003ehere\u003c/a\u003e and extract it into a dir named `datasets` \n\n4. Run the desired model by following the links below:\n\n### N-Con*ffusion*\n\n\u003e [**Super-resolution**](https://github.com/eliahuhorwitz/Conffusion/tree/main/super_resolution_n_conffusion): N-Con*ffusion* code for super-resolution (also includes DM\u003csub\u003eSB\u003c/sub\u003e and DM\u003csub\u003eSBA\u003c/sub\u003e code)\n\n\n\u003e [**Inpainting**](https://github.com/eliahuhorwitz/Conffusion/tree/main/inpainting_n_conffusion): N-Con*ffusion* code for inpainting (also includes DM\u003csub\u003eSB\u003c/sub\u003e and DM\u003csub\u003eSBA\u003c/sub\u003e code)\n\n### G-Con*ffusion*\n\u003e [**Super-resolution + Inpainting**](https://github.com/eliahuhorwitz/Conffusion/tree/main/g_conffusion): G-Con*ffusion* code for super-resolution and inpainting\n\n\n\n## Citation\nIf you find this useful for your research, please use the following.\n\n```\n@article{horwitz2022conffusion,\n  title={Conffusion: Confidence Intervals for Diffusion Models},\n  author={Horwitz, Eliahu and Hoshen, Yedid},\n  journal={arXiv preprint arXiv:2211.09795},\n  year={2022}\n}\n```\n\n\n\n## Acknowledgments\n- The SR3 implementation is based on \u003ca href=\"https://github.com/Janspiry/Image-Super-Resolution-via-Iterative-Refinement\" target=\"_blank\"\u003ethis\u003c/a\u003e unofficial implementation of SR3. \n- The Palette implementation is based on \u003ca href=\"https://github.com/Janspiry/Palette-Image-to-Image-Diffusion-Models\" target=\"_blank\"\u003ethis\u003c/a\u003e unofficial implementation of Palette.\n- The Guided-diffusion implementation is based on \u003ca href=\"https://github.com/openai/guided-diffusion\" target=\"_blank\"\u003ethis\u003c/a\u003e official implementation.\n- The implementation of the calibration and evaluation metrics is based on \u003ca href=\"https://github.com/aangelopoulos/im2im-uq\" target=\"_blank\"\u003ethis\u003c/a\u003e official implementation of im2im-uq.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feliahuhorwitz%2Fconffusion","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Feliahuhorwitz%2Fconffusion","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feliahuhorwitz%2Fconffusion/lists"}