{"id":13488363,"url":"https://github.com/bytedance/res-adapter","last_synced_at":"2025-05-15T09:07:28.277Z","repository":{"id":225780227,"uuid":"764135570","full_name":"bytedance/res-adapter","owner":"bytedance","description":"[AAAI 2025] Official codes of \"ResAdapter: Domain Consistent Resolution Adapter for Diffusion Models\".","archived":false,"fork":false,"pushed_at":"2025-03-09T09:40:32.000Z","size":212793,"stargazers_count":742,"open_issues_count":9,"forks_count":25,"subscribers_count":18,"default_branch":"main","last_synced_at":"2025-04-11T19:55:49.197Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://res-adapter.github.io/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bytedance.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-02-27T14:43:08.000Z","updated_at":"2025-04-08T16:30:29.000Z","dependencies_parsed_at":"2025-03-28T04:10:37.172Z","dependency_job_id":null,"html_url":"https://github.com/bytedance/res-adapter","commit_stats":null,"previous_names":["bytedance/res-adapter"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bytedance%2Fres-adapter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bytedance%2Fres-adapter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bytedance%2Fres-adapter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bytedance%2Fres-adapter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bytedance","download_url":"https://codeload.github.com/bytedance/res-adapter/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254310515,"owners_count":22049469,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T18:01:14.413Z","updated_at":"2025-05-15T09:07:23.258Z","avatar_url":"https://github.com/bytedance.png","language":"Python","funding_links":[],"categories":["T2I Diffusion Model augmentation","Projekte"],"sub_categories":["🌄 Image"],"readme":"\u003cdiv align=\"center\"\u003e\n\n\u003ch1\u003e ResAdapter: Domain Consistent Resolution Adapter for Diffusion Models  \u003c/h1\u003e\n\nJiaxiang Cheng, Pan Xie*, Xin Xia, Jiashi Li, Jie Wu, Yuxi Ren, Huixia Li, Xuefeng Xiao, Min Zheng, Lean Fu (*Corresponding author)\n\nByteDance Inc.\n\n⭐ If ResAdapter is helpful to your images or projects, please help star this repo. Thanks! 🤗\n\n\n\u003ca href='https://res-adapter.github.io/'\u003e\u003cimg src='https://img.shields.io/badge/Project-Page-green'\u003e\u003c/a\u003e \n\u003ca href='https://arxiv.org/abs/2403.02084'\u003e\u003cimg src='https://img.shields.io/badge/ Paper-Arxiv-red'\u003e\u003c/a\u003e \n\u003ca href='https://huggingface.co/papers/2403.02084'\u003e\u003cimg src='https://img.shields.io/badge/ Paper-Huggingface-blue'\u003e\u003c/a\u003e \n![GitHub Org's stars](https://img.shields.io/github/stars/bytedance%2Fres-adapter)\n\n[![Hugging Face](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Space-green)](https://huggingface.co/spaces/jiaxiangc/res-adapter)\n[![Replicate](https://img.shields.io/badge/Replicate-Gradio-red)](https://replicate.com/bytedance/res-adapter)\n[![ComfyUI](https://img.shields.io/badge/ComfyUI-ResAdapter-blue)](https://github.com/jiaxiangc/ComfyUI-ResAdapter)\n![visitors](https://visitor-badge.laobi.icu/badge?page_id=bytedance.res-adapter) \n\n**We propose ResAdapter, a plug-and-play resolution adapter for enabling any diffusion model generate resolution-free images: no additional training, no additional inference and no style transfer.**\n\n\u003cimg src=\"assets/misc/dreamlike1.png\" width=\"49.9%\"\u003e\u003cimg src=\"assets/misc/dreamlike2.png\" width=\"50%\"\u003e\nComparison examples between resadapter and [dreamlike-diffusion-1.0](https://civitai.com/models/1274/dreamlike-diffusion-10).\n\n\u003c/div\u003e\n\n\n## Release\n- `[2024/12/10]` 🎉 ResAdapter is accepted by AAAI 2025.\n- `[2024/04/07]` 🔥 We release the official [gradio space](https://huggingface.co/spaces/jiaxiangc/res-adapter) in Huggingface.\n- `[2024/04/05]` 🔥 We release the [resadapter_v2 weights](https://huggingface.co/jiaxiangc/res-adapter).\n- `[2024/03/30]` 🔥 We release the [ComfyUI-ResAdapter](https://github.com/jiaxiangc/ComfyUI-ResAdapter).\n- `[2024/03/28]` 🔥 We release the [resadapter_v1 weights](https://huggingface.co/jiaxiangc/res-adapter).\n- `[2024/03/04]` 🔥 We release the [arxiv paper](https://arxiv.org/abs/2403.02084).\n\u003c!-- - `[2024/03/12]` Code: 🔥 we release the [inference code](https://github.com/bytedance/res-adapter/blob/main/main.py). --\u003e\n\n\n## Quicktour\n\nWe provide a standalone [example code](quicktour.py) to help you quickly use resadapter with diffusion models.\n\n\u003cdiv align=center\u003e\n\n\u003cimg src=\"assets/misc/dreamshaper_resadapter.png\" width=\"100%\"\u003e\n\u003cimg src=\"assets/misc/dreamshaper_baseline.png\" width=\"100%\"\u003e\n\nComparison examples (640x384) between resadapter and [dreamshaper-xl-1.0](https://huggingface.co/Lykon/dreamshaper-xl-1-0). Top: with resadapter. Bottom: without resadapter.\n\n\u003c/div\u003e\n\n```python\n# pip install diffusers, transformers, accelerate, safetensors, huggingface_hub\nimport torch\nfrom torchvision.utils import save_image\nfrom safetensors.torch import load_file\nfrom huggingface_hub import hf_hub_download\nfrom diffusers import AutoPipelineForText2Image, DPMSolverMultistepScheduler\n\ngenerator = torch.manual_seed(0)\nprompt = \"portrait photo of muscular bearded guy in a worn mech suit, light bokeh, intricate, steel metal, elegant, sharp focus, soft lighting, vibrant colors\"\nwidth, height = 640, 384\n\n# Load baseline pipe\nmodel_name = \"lykon-models/dreamshaper-xl-1-0\"\npipe = AutoPipelineForText2Image.from_pretrained(model_name, torch_dtype=torch.float16, variant=\"fp16\").to(\"cuda\")\npipe.scheduler = DPMSolverMultistepScheduler.from_config(pipe.scheduler.config, use_karras_sigmas=True, algorithm_type=\"sde-dpmsolver++\")\n\n# Inference baseline pipe\nimage = pipe(prompt, width=width, height=height, num_inference_steps=25, num_images_per_prompt=4, output_type=\"pt\").images\nsave_image(image, f\"image_baseline.png\", normalize=True, padding=0)\n\n# Load resadapter for baseline\nresadapter_model_name = \"resadapter_v1_sdxl\"\npipe.load_lora_weights(\n    hf_hub_download(repo_id=\"jiaxiangc/res-adapter\", subfolder=resadapter_model_name, filename=\"pytorch_lora_weights.safetensors\"), \n    adapter_name=\"res_adapter\",\n    ) # load lora weights\npipe.set_adapters([\"res_adapter\"], adapter_weights=[1.0])\npipe.unet.load_state_dict(\n    load_file(hf_hub_download(repo_id=\"jiaxiangc/res-adapter\", subfolder=resadapter_model_name, filename=\"diffusion_pytorch_model.safetensors\")),\n    strict=False,\n    ) # load norm weights\n\n# Inference resadapter pipe\nimage = pipe(prompt, width=width, height=height, num_inference_steps=25, num_images_per_prompt=4, output_type=\"pt\").images\nsave_image(image, f\"image_resadapter.png\", normalize=True, padding=0)\n```\n\n## Download\n\n### Models\n\nWe have released all resadapter weights, you can download resadapter models from [Huggingface](https://huggingface.co/jiaxiangc/res-adapter/tree/main). The following is our resadapter model card:\n\n|Models  | Parameters | Resolution Range | Ratio Range | Links |\n| --- | --- |--- | --- | --- |\n|resadapter_v2_sd1.5| 0.9M | 128 \u003c= x \u003c= 1024 | 0.28 \u003c= r \u003c= 3.5 | [Download](https://huggingface.co/jiaxiangc/res-adapter/tree/main/resadapter_v2_sd1.5)|\n|resadapter_v2_sdxl| 0.5M | 256 \u003c= x \u003c= 1536 | 0.28 \u003c= r \u003c= 3.5 | [Download](https://huggingface.co/jiaxiangc/res-adapter/tree/main/resadapter_v2_sdxl)|\n|resadapter_v1_sd1.5| 0.9M | 128 \u003c= x \u003c= 1024 | 0.5 \u003c= r \u003c= 2 | [Download](https://huggingface.co/jiaxiangc/res-adapter/tree/main/resadapter_v1_sd1.5)|\n|resadapter_v1_sd1.5_extrapolation| 0.9M | 512 \u003c= x \u003c= 1024 | 0.5 \u003c= r \u003c= 2  | [Download](https://huggingface.co/jiaxiangc/res-adapter/tree/main/resadapter_v1_sd1.5_extrapolation)|\n|resadapter_v1_sd1.5_interpolation| 0.9M | 128 \u003c= x \u003c= 512 | 0.5 \u003c= r \u003c= 2  | [Download](https://huggingface.co/jiaxiangc/res-adapter/tree/main/resadapter_v1_sd1.5_interpolation)|\n|resadapter_v1_sdxl| 0.5M | 256 \u003c= x \u003c= 1536 | 0.5 \u003c= r \u003c= 2  | [Download](https://huggingface.co/jiaxiangc/res-adapter/tree/main/resadapter_v1_sdxl) |\n|resadapter_v1_sdxl_extrapolation| 0.5M | 1024 \u003c= x \u003c= 1536 | 0.5 \u003c= r \u003c= 2  | [Download](https://huggingface.co/jiaxiangc/res-adapter/tree/main/resadapter_v1_sdxl_extrapolation) |\n|resadapter_v1_sdxl_interpolation| 0.5M | 256 \u003c= x \u003c= 1024 | 0.5 \u003c= r \u003c= 2  | [Download](https://huggingface.co/jiaxiangc/res-adapter/tree/main/resadapter_v1_sdxl_interpolation) |\n\nHint1: We update the resadapter name format according to [controlnet](https://github.com/lllyasviel/ControlNet-v1-1-nightly).\n\nHint2: If you want use resadapter with personalized diffusion models, you should download them from [CivitAI](https://civitai.com/).\n\nHint3: If you want use resadapter with ip-adapter, controlnet and lcm-lora, you should download them from [Huggingface](https://huggingface.co/jiaxiangc/res-adapter).\n\nHint4: Here is an [installation guidance](models/README.md) for preparing environment and downloading models.\n\n## Inference\n\nIf you want generate images in our inference script, you should install dependency libraries and download related models according to [installation guidance](models/README.md). After filling in [example configs](configs), you can directly run this script.\n\n```bash\npython main.py --config /path/to/file\n```\n\n### ResAdapter with Personalized Models for Text to Image\n\n\u003cdiv align=center\u003e\n\n\u003cimg src=\"assets/misc/dreamshaper-1024/resadapter1.jpg\" width=\"25%\"\u003e\u003cimg src=\"assets/misc/dreamshaper-1024/resadapter2.jpg\" width=\"25%\"\u003e\u003cimg src=\"assets/misc/dreamshaper-1024/resadapter3.jpg\" width=\"25%\"\u003e\u003cimg src=\"assets/misc/dreamshaper-1024/resadapter4.jpg\" width=\"25%\"\u003e\n\u003cimg src=\"assets/misc/dreamshaper-1024/baseline1.jpg\" width=\"25%\"\u003e\u003cimg src=\"assets/misc/dreamshaper-1024/baseline2.jpg\" width=\"25%\"\u003e\u003cimg src=\"assets/misc/dreamshaper-1024/baseline3.jpg\" width=\"25%\"\u003e\u003cimg src=\"assets/misc/dreamshaper-1024/baseline4.jpg\" width=\"25%\"\u003e\n\nComparison examples (960x1104) between resadapter and [dreamshaper-7](https://civitai.com/models/1274/dreamlike-diffusion-10). Top: with resadapter. Bottom: without resadapter.\n\n\u003c/div\u003e\n\n### ResAdapter with ControlNet for Image to Image\n\n\u003cdiv align=center\u003e\n\n\u003cimg src=\"assets/misc/controlnet/condition_bird.jpg\" width=\"20%\"\u003e\u003cimg src=\"assets/misc/controlnet/bird_1_ResAdapter.jpg\" width=\"20%\"\u003e\u003cimg src=\"assets/misc/controlnet/bird_2_ResAdapter.jpg\" width=\"20%\"\u003e\u003cimg src=\"assets/misc/controlnet/bird_3_ResAdapter.jpg\" width=\"20%\"\u003e\u003cimg src=\"assets/misc/controlnet/bird_4_ResAdapter.jpg\" width=\"20%\"\u003e\n\u003cimg src=\"assets/misc/controlnet/condition_bird.jpg\" width=\"20%\"\u003e\u003cimg src=\"assets/misc/controlnet/bird_1_Baseline.jpg\" width=\"20%\"\u003e\u003cimg src=\"assets/misc/controlnet/bird_5_Baseline.jpg\" width=\"20%\"\u003e\u003cimg src=\"assets/misc/controlnet/bird_3_Baseline.jpg\" width=\"20%\"\u003e\u003cimg src=\"assets/misc/controlnet/bird_4_Baseline.jpg\" width=\"20%\"\u003e\n\nComparison examples (840x1264) between resadapter and [lllyasviel/sd-controlnet-canny](https://huggingface.co/lllyasviel/sd-controlnet-canny). Top: with resadapter, bottom: without resadapter.\n\n\u003c/div\u003e\n\n### ResAdapter with ControlNet-XL for Image to Image\n\n\u003cdiv align=center\u003e\n\n\u003cimg src=\"assets/misc/controlnet-xl/condition_man.jpg\" width=\"20%\"\u003e\u003cimg src=\"assets/misc/controlnet-xl/man_0_ResAdapter.jpg\" width=\"20%\"\u003e\u003cimg src=\"assets/misc/controlnet-xl/man_1_ResAdapter.jpg\" width=\"20%\"\u003e\u003cimg src=\"assets/misc/controlnet-xl/man_2_ResAdapter.jpg\" width=\"20%\"\u003e\u003cimg src=\"assets/misc/controlnet-xl/man_3_ResAdapter.jpg\" width=\"20%\"\u003e\n\u003cimg src=\"assets/misc/controlnet-xl/condition_man.jpg\" width=\"20%\"\u003e\u003cimg src=\"assets/misc/controlnet-xl/man_0_Baseline.jpg\" width=\"20%\"\u003e\u003cimg src=\"assets/misc/controlnet-xl/man_1_Baseline.jpg\" width=\"20%\"\u003e\u003cimg src=\"assets/misc/controlnet-xl/man_2_Baseline.jpg\" width=\"20%\"\u003e\u003cimg src=\"assets/misc/controlnet-xl/man_3_Baseline.jpg\" width=\"20%\"\u003e\n\nComparison examples (336x504) between resadapter and [diffusers/controlnet-canny-sdxl-1.0](https://huggingface.co/diffusers/controlnet-canny-sdxl-1.0). Top: with resadapter, bottom: without resadapter.\n\n\u003c/div\u003e\n\n### ResAdapter with IP-Adapter for Face Variance\n\n\u003cdiv align=center\u003e\n\u003cimg src=\"assets/ip_adapter/ai_face2.png\" width=\"20%\"\u003e\u003cimg src=\"assets/misc/ip-adapter/resadapter3.jpg\" width=\"20%\"\u003e\u003cimg src=\"assets/misc/ip-adapter/resadapter4.jpg\" width=\"20%\"\u003e\u003cimg src=\"assets/misc/ip-adapter/resadapter5.jpg\" width=\"20%\"\u003e\u003cimg src=\"assets/misc/ip-adapter/resadapter7.jpg\" width=\"20%\"\u003e\n\u003cimg src=\"assets/ip_adapter/ai_face2.png\" width=\"20%\"\u003e\u003cimg src=\"assets/misc/ip-adapter/baseline3.jpg\" width=\"20%\"\u003e\u003cimg src=\"assets/misc/ip-adapter/baseline4.jpg\" width=\"20%\"\u003e\u003cimg src=\"assets/misc/ip-adapter/baseline5.jpg\" width=\"20%\"\u003e\u003cimg src=\"assets/misc/ip-adapter/baseline7.jpg\" width=\"20%\"\u003e\n\nComparison examples (864x1024) between resadapter and [h94/IP-Adapter](https://huggingface.co/h94/IP-Adapter). Top: with resadapter, bottom: without resadapter.\n\n\n\u003c/div\u003e\n\n\n### ResAdapter with LCM-LoRA for Speeding up\n\n\u003cdiv align=center\u003e\n\n\u003cimg src=\"assets/misc/lcm-lora/resadapter5.jpg\" width=\"20%\"\u003e\u003cimg src=\"assets/misc/lcm-lora/resadapter3.jpg\" width=\"20%\"\u003e\u003cimg src=\"assets/misc/lcm-lora/resadapter2.jpg\" width=\"20%\"\u003e\u003cimg src=\"assets/misc/lcm-lora/resadapter4.jpg\" width=\"20%\"\u003e\u003cimg src=\"assets/misc/lcm-lora/resadapter1.jpg\" width=\"20%\"\u003e\n\u003cimg src=\"assets/misc/lcm-lora/baseline5.jpg\" width=\"20%\"\u003e\u003cimg src=\"assets/misc/lcm-lora/baseline3.jpg\" width=\"20%\"\u003e\u003cimg src=\"assets/misc/lcm-lora/baseline2.jpg\" width=\"20%\"\u003e\u003cimg src=\"assets/misc/lcm-lora/baseline4.jpg\" width=\"20%\"\u003e\u003cimg src=\"assets/misc/lcm-lora/baseline1.jpg\" width=\"20%\"\u003e\n\nComparison examples (512x512) between resadapter and [dreamshaper-xl-1.0](https://huggingface.co/Lykon/dreamshaper-xl-1-0) with [lcm-sdxl-lora](https://huggingface.co/latent-consistency/lcm-lora-sdxl). Top: with resadapter, bottom: without resadapter.\n\n\n\u003c/div\u003e\n\n## Community Resource\n\n### Gradio\n- Replicate website: [bytedance/res-adapter](https://replicate.com/bytedance/res-adapter) by ([@Chenxi](https://github.com/chenxwh))\n- Huggingface space: \n  - [jiaxiangc/res-adapter](https://huggingface.co/spaces/jiaxiangc/res-adapter) (official space)\n  - [ameerazam08/Res-Adapter-GPU-Demo](https://huggingface.co/spaces/ameerazam08/Res-Adapter-GPU-Demo) by ([@Ameer Azam](https://github.com/AMEERAZAM08))\n\nAn text-to-image example about res-adapter in huggingface space. More information in [jiaxiangc/res-adapter](https://huggingface.co/spaces/jiaxiangc/res-adapter).\n\n\u003cimg src=\"assets/misc/huggingface_space_demo.png\"\u003e\n  \n### ComfyUI\n- [jiaxiangc/ComfyUI-ResAdapter](https://github.com/jiaxiangc/ComfyUI-ResAdapter) (official comfyui node)\n- [blepping/ComfyUI-ApplyResAdapterUnet](https://github.com/blepping/ComfyUI-ApplyResAdapterUnet) by ([@blepping](https://github.com/blepping))\n\nAn text-to image example about ComfyUI-ResAdapter. More examples about lcm-lora, controlnet and ipadapter can be found in [ComfyUI-ResAdapter](https://github.com/jiaxiangc/ComfyUI-ResAdapter/tree/main).\n\nhttps://github.com/jiaxiangc/ComfyUI-ResAdapter/assets/162297627/82453931-23de-4f72-8a9c-1053c4c8d81a\n\n### WebUI\n\nI am learning how to make webui extension.\n\n## Local Gradio Demo\n\nRun the following script:\n\n```bash\n# pip install peft, gradio, httpx==0.23.3\npython app.py\n```\n\n\n\n## Usage Tips\n\n1. If you are not satisfied with interpolation images, try to increase the alpha of resadapter to 1.0.\n2. If you are not satisfied with extrapolate images, try to choose the alpha of resadapter in 0.3 ~ 0.7.\n3. If you find the images with style conflicts, try to decrease the alpha of resadapter.\n4. If you find resadapter is not compatible with other accelerate lora, try to decrease the alpha of resadapter to 0.5 ~ 0.7.\n\n## Acknowledgements\n\n- ResAdapter is developed by AutoML Team at ByteDance Inc, all copyright reserved.\n- Thanks to the [HuggingFace](https://huggingface.co/) gradio team for their free GPU support!\n- Thanks to the [IP-Adapter](https://huggingface.co/h94/IP-Adapter), [ControlNet](https://github.com/lllyasviel/ControlNet-v1-1-nightly), [LCM-LoRA](https://huggingface.co/latent-consistency/lcm-lora-sdxl) for their nice work.\n- Thank [@Chenxi](https://github.com/chenxwh) and [@AMEERAZAM08](https://github.com/AMEERAZAM08) to provide gradio demos.\n- Thank [@fengyuzzz](https://github.com/fengyuzzz) to support video demos in [ComfyUI-ResAdapter](https://github.com/jiaxiangc/ComfyUI-ResAdapter/tree/main).\n\n## Star History\n[![Star History Chart](https://api.star-history.com/svg?repos=bytedance/res-adapter\u0026type=Date)](https://star-history.com/#bytedance/res-adapter\u0026Date)\n\n## Citation\nIf you find ResAdapter useful for your research and applications, please cite us using this BibTeX:\n```\n@article{cheng2024resadapter,\n  title={ResAdapter: Domain Consistent Resolution Adapter for Diffusion Models},\n  author={Cheng, Jiaxiang and Xie, Pan and Xia, Xin and Li, Jiashi and Wu, Jie and Ren, Yuxi and Li, Huixia and Xiao, Xuefeng and Zheng, Min and Fu, Lean},\n  booktitle={arXiv preprint arxiv:2403.02084},\n  year={2024}\n}\n```\nFor any question, please feel free to contact us via chengjiaxiang@bytedance.com or xiepan.01@bytedance.com.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbytedance%2Fres-adapter","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbytedance%2Fres-adapter","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbytedance%2Fres-adapter/lists"}