{"id":13488714,"url":"https://compvis.github.io/LoRAdapter/","last_synced_at":"2025-03-28T01:37:29.138Z","repository":{"id":239624997,"uuid":"800044669","full_name":"CompVis/LoRAdapter","owner":"CompVis","description":null,"archived":false,"fork":false,"pushed_at":"2024-10-14T15:08:35.000Z","size":21697,"stargazers_count":113,"open_issues_count":2,"forks_count":4,"subscribers_count":16,"default_branch":"main","last_synced_at":"2025-03-22T10:51:18.294Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/CompVis.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-05-13T15:40:16.000Z","updated_at":"2025-03-16T13:10:21.000Z","dependencies_parsed_at":"2024-10-31T00:32:41.468Z","dependency_job_id":"ea25cb81-c811-4161-b909-0d072badae47","html_url":"https://github.com/CompVis/LoRAdapter","commit_stats":null,"previous_names":["compvis/loradapter"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CompVis%2FLoRAdapter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CompVis%2FLoRAdapter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CompVis%2FLoRAdapter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CompVis%2FLoRAdapter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/CompVis","download_url":"https://codeload.github.com/CompVis/LoRAdapter/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245952919,"owners_count":20699561,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T18:01:20.534Z","updated_at":"2025-03-28T01:37:29.133Z","avatar_url":"https://github.com/CompVis.png","language":"Python","readme":"# \n\n\u003cp align=\"center\"\u003e\n \u003ch2 align=\"center\"\u003eConditional LoRAdapter for Efficient 0-Shot Control \u0026 Altering of T2I Models\u003c/h2\u003e\n \u003cp align=\"center\"\u003e \n Nick Stracke\u003csup\u003e1\u003c/sup\u003e · Stefan A. Baumann\u003csup\u003e1\u003c/sup\u003e · Josh Susskind\u003csup\u003e2\u003c/sup\u003e · Miguel A. Bautista\u003csup\u003e2\u003c/sup\u003e · Björn Ommer\u003csup\u003e1\u003c/sup\u003e\n \u003c/p\u003e\n \u003cp align=\"center\"\u003e \n    \u003csup\u003e1\u003c/sup\u003e \u003cb\u003eCompVis Group @ LMU Munich\u003c/b\u003e \u003cbr/\u003e\n    \u003csup\u003e2\u003c/sup\u003e \u003cb\u003eApple\u003c/b\u003e\n \u003c/p\u003e\n\u003c/p\u003e\n\u003cp align=\"center\"\u003e\n \u003cb\u003eECCV 2024\u003c/b\u003e\n\u003c/p\u003e\n\n\n[![Project Page](https://img.shields.io/badge/Project-Page-blue)](https://compvis.github.io/LoRAdapter/)\n[![Paper](https://img.shields.io/badge/arXiv-PDF-b31b1b)](https://arxiv.org/abs/2405.07913)\n\nThis repository contains an implementation of the paper \"CTRLorALTer: Conditional LoRAdapter for Efficient 0-Shot Control \u0026 Altering of T2I Models\".\n\nWe present LoRAdapter, an approach that unifies both style and structure conditioning under the same formulation using a novel conditional LoRA block that enables zero-shot control.\nLoRAdapter is an efficient, powerful, and architecture-agnostic approach to condition text-to-image diffusion models, which enables fine-grained control conditioning during generation and outperforms recent state-of-the-art approaches.\n\n![teaser](./docs/static/images/main.jpg)\n\n## 🔥 Updates\n- Implemented [B-LoRA](https://b-lora.github.io/B-LoRA/) implicit content and style disentangle using LoRAdapter\n- Released Code and Weights for inference\n\n## 💪 TODO\n- [x] Add training Code\n- [ ] Add more structure conditioning checkpoints (including SDXL)\n- [ ] Experiment with SD3\n\n\n## Setup\n\nCreate the conda environment\n\n`conda env create -f environment.yaml `\n\nActivate the conda environment\n\n`conda activate loradapter`\n\n## Weights\n\nAll weights are available on [HuggingFace](https://huggingface.co/kliyer/LoRAdapter/tree/main).\n\nFor ease of you, you can also use the provided bash script `download_weights.sh` to automatically download all available weights and place them in the the right directory.\n\n\n## Usage\nSampling works according to the following schema:\n```\npython sample.py experiment=\u003ccheck ./config/experiments\u003e\n```\nAll currently available experiments are listed in `/config/experiments`. Feel free to adjust the configs according to you own needs.\n\n### B-LoRA\nSampling using the [B-LoRA](https://b-lora.github.io/B-LoRA/) LoRAdapter is possible using the config `sample_b-lora_sdxl.yaml`. By default this will condition on both content and style of the image. For conditioning on _only_ content or _only_ style, change the `adaption_mode` to either `b-lora_content` or `b-lora_style`. Also set `ignore_check` to true as we are only loading the checkpoint partially.\n\nFor best results provide information about the missing modality via the text prompt or using another LoRAdapter.\n\n\n\n## 🎓 Citation\n\nIf you use this codebase or otherwise found our work valuable, please cite our paper:\n\n```bibtex\n@misc{stracke2024loradapter,\n  title={CTRLorALTer: Conditional LoRAdapter for Efficient 0-Shot Control \u0026 Altering of T2I Models},\n  author={Nick Stracke and Stefan Andreas Baumann and Joshua Susskind and Miguel Angel Bautista and Björn Ommer},\n  year={2024},\n  eprint={2405.07913},\n  archivePrefix={arXiv},\n  primaryClass={cs.CV}\n}\n```\n","funding_links":[],"categories":["Additional conditions"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/compvis.github.io%2FLoRAdapter%2F","html_url":"https://awesome.ecosyste.ms/projects/compvis.github.io%2FLoRAdapter%2F","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/compvis.github.io%2FLoRAdapter%2F/lists"}