{"id":13488682,"url":"https://github.com/showlab/X-Adapter","last_synced_at":"2025-03-28T01:37:15.541Z","repository":{"id":210507265,"uuid":"726733674","full_name":"showlab/X-Adapter","owner":"showlab","description":"[CVPR 2024] X-Adapter: Adding Universal Compatibility of Plugins for Upgraded Diffusion Model","archived":false,"fork":false,"pushed_at":"2024-04-23T12:19:42.000Z","size":131327,"stargazers_count":704,"open_issues_count":20,"forks_count":43,"subscribers_count":44,"default_branch":"main","last_synced_at":"2024-08-01T18:39:13.567Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://showlab.github.io/X-Adapter/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/showlab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2023-12-03T08:20:03.000Z","updated_at":"2024-08-01T16:34:24.000Z","dependencies_parsed_at":"2024-01-16T09:05:19.628Z","dependency_job_id":"2d722c78-ace8-4d1f-91af-914d42605e53","html_url":"https://github.com/showlab/X-Adapter","commit_stats":null,"previous_names":["showlab/x-adapter"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/showlab%2FX-Adapter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/showlab%2FX-Adapter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/showlab%2FX-Adapter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/showlab%2FX-Adapter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/showlab","download_url":"https://codeload.github.com/showlab/X-Adapter/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":222333976,"owners_count":16968058,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T18:01:20.011Z","updated_at":"2025-03-28T01:37:15.536Z","avatar_url":"https://github.com/showlab.png","language":"Python","funding_links":[],"categories":["Additional conditions"],"sub_categories":[],"readme":"# X-Adapter\n\nThis repository is the official implementation of [X-Adapter](https://arxiv.org/abs/2312.02238).\n\n**[X-Adapter: Adding Universal Compatibility of Plugins for Upgraded Diffusion Model](https://arxiv.org/abs/2312.02238)**\n\u003cbr/\u003e\n[Lingmin Ran](),\n[Xiaodong Cun](https://vinthony.github.io/academic/),\n[Jia-Wei Liu](https://jia-wei-liu.github.io/), \n[Rui Zhao](https://ruizhaocv.github.io/), \n[Song Zijie](), \n[Xintao Wang](https://xinntao.github.io/),\n[Jussi Keppo](https://www.jussikeppo.com/), \n[Mike Zheng Shou](https://sites.google.com/view/showlab)\n\u003cbr/\u003e\n\n[![Project Website](https://img.shields.io/badge/Project-Website-orange)](https://showlab.github.io/X-Adapter/)\n[![arXiv](https://img.shields.io/badge/arXiv-2312.02238-b31b1b.svg)](https://arxiv.org/abs/2312.02238)\n\n![Overview_v7](https://github.com/showlab/X-Adapter/assets/152716091/eb41c508-826c-404f-8223-09765765823b)\n\n\u003cem\u003e X-Adapter enables plugins pretrained on the old version (e.g. SD1.5) directly work with the upgraded Model (e.g., SDXL) without further retraining.\u003c/em\u003e\n\n[//]: # (\u003cp align=\"center\"\u003e)\n\n[//]: # (\u003cimg src=\"https://tuneavideo.github.io/assets/teaser.gif\" width=\"1080px\"/\u003e  )\n\n[//]: # (\u003cbr\u003e)\n\n[//]: # (\u003cem\u003eGiven a video-text pair as input, our method, Tune-A-Video, fine-tunes a pre-trained text-to-image diffusion model for text-to-video generation.\u003c/em\u003e)\n\n[//]: # (\u003c/p\u003e)\n\n### Thank @[kijai](https://github.com/kijai) for CumfyUI implementation [here](https://github.com/kijai/ComfyUI-Diffusers-X-Adapter)! Please refer to this [tutorial](https://www.reddit.com/r/StableDiffusion/comments/1asuyiw/xadapter/) for hyperparameter setting.\n\n## News\n\n- [17/02/2024] Inference code released\n\n## Setup\n\n### Requirements\n\n```shell\nconda create -n xadapter python=3.10\nconda activate xadapter\n\npip install torch==1.13.1+cu116 torchvision==0.14.1+cu116 --extra-index-url https://download.pytorch.org/whl/cu116\npip install -r requirements.txt\n```\n\nInstalling [xformers](https://github.com/facebookresearch/xformers) is highly recommended for high efficiency and low GPU cost.\n\n### Weights\n\n**[Stable Diffusion]** [Stable Diffusion](https://arxiv.org/abs/2112.10752) is a latent text-to-image diffusion model capable of generating photo-realistic images given any text input. The pre-trained Stable Diffusion models can be downloaded from Hugging Face (e.g., [Stable Diffusion v1-5](https://huggingface.co/runwayml/stable-diffusion-v1-5)). You can also use fine-tuned Stable Diffusion models trained on different styles (e.g., [Anything V4.0](https://huggingface.co/andite/anything-v4.0), [Redshift](https://huggingface.co/nitrosocke/redshift-diffusion), etc.).\n\n**[ControlNet]** [Controlnet](https://github.com/lllyasviel/ControlNet) is a method to control diffusion models with spatial conditions. You can download the ControlNet family [here](https://huggingface.co/lllyasviel/ControlNet).\n\n**[LoRA]** [LoRA](https://arxiv.org/abs/2106.09685) is a lightweight adapter to fine-tune large-scale pretrained model. It is widely used for style or identity customization in diffusion models. You can download LoRA from the diffusion community (e.g., [civitai](https://civitai.com/)).\n\n### Checkpoint\n\nModels can be downloaded from our [Hugging Face page](https://huggingface.co/Lingmin-Ran/X-Adapter). Put the checkpoint in folder `./checkpoint/X-Adapter`.\n\n## Usage\n\nAfter preparing all checkpoints, we can run inference code using different plugins. You can refer to this [tutorial](https://www.reddit.com/r/StableDiffusion/comments/1asuyiw/xadapter/) to quickly get started with X-Adapter. \n\n### Controlnet Inference\n\nSet `--controlnet_canny_path` or `--controlnet_depth_path` to ControlNet's path in the bash script. The default value is its Hugging Face model card. \n\n    sh ./bash_scripts/canny_controlnet_inference.sh\n    sh ./bash_scripts/depth_controlnet_inference.sh\n\n### LoRA Inference\n\nSet `--lora_model_path` to LoRA's checkpoint in the bash script. In this example we use [MoXin](https://civitai.com/models/12597/moxin), and we put it in folder `./checkpoint/lora`.\n\n    sh ./bash_scripts/lora_inference.sh\n\n### Controlnet-Tile Inference\n\nSet `--controlnet_tile_path` to ControlNet-tile's path in the bash script. The default value is its Hugging Face model card. \n\n    sh ./bash_scripts/controlnet_tile_inference.sh\n\n## Cite\nIf you find X-Adapter useful for your research and applications, please cite us using this BibTeX:\n\n```bibtex\n@article{ran2023xadapter,\n  title={X-Adapter: Adding Universal Compatibility of Plugins for Upgraded Diffusion Model},\n  author={Lingmin Ran and Xiaodong Cun and Jia-Wei Liu and Rui Zhao and Song Zijie and Xintao Wang and Jussi Keppo and Mike Zheng Shou},\n  journal={arXiv preprint arXiv:2312.02238},\n  year={2023}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshowlab%2FX-Adapter","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fshowlab%2FX-Adapter","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshowlab%2FX-Adapter/lists"}