{"id":13562507,"url":"https://github.com/PRIV-Creation/Awesome-Controllable-T2I-Diffusion-Models","last_synced_at":"2025-04-03T18:33:47.675Z","repository":{"id":153285821,"uuid":"627274798","full_name":"PRIV-Creation/Awesome-Controllable-T2I-Diffusion-Models","owner":"PRIV-Creation","description":"A collection of resources on controllable generation with text-to-image diffusion models.","archived":false,"fork":false,"pushed_at":"2024-05-21T22:28:11.000Z","size":915,"stargazers_count":683,"open_issues_count":0,"forks_count":19,"subscribers_count":43,"default_branch":"main","last_synced_at":"2024-05-22T21:47:42.528Z","etag":null,"topics":["awesome","awesome-list","controllable-generation","diffusion-models","multi-concept","personalization","spatial-controls","text-to-image"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/PRIV-Creation.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-04-13T06:14:11.000Z","updated_at":"2024-05-28T04:28:35.263Z","dependencies_parsed_at":"2024-05-02T21:00:52.687Z","dependency_job_id":"c3bb3110-0057-4b47-83c7-c4c02553f0a8","html_url":"https://github.com/PRIV-Creation/Awesome-Controllable-T2I-Diffusion-Models","commit_stats":null,"previous_names":["priv-creation/awesome-controllable-t2i-diffusion-models"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PRIV-Creation%2FAwesome-Controllable-T2I-Diffusion-Models","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PRIV-Creation%2FAwesome-Controllable-T2I-Diffusion-Models/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PRIV-Creation%2FAwesome-Controllable-T2I-Diffusion-Models/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PRIV-Creation%2FAwesome-Controllable-T2I-Diffusion-Models/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/PRIV-Creation","download_url":"https://codeload.github.com/PRIV-Creation/Awesome-Controllable-T2I-Diffusion-Models/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247057097,"owners_count":20876512,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["awesome","awesome-list","controllable-generation","diffusion-models","multi-concept","personalization","spatial-controls","text-to-image"],"created_at":"2024-08-01T13:01:09.378Z","updated_at":"2025-04-03T18:33:42.663Z","avatar_url":"https://github.com/PRIV-Creation.png","language":null,"funding_links":[],"categories":["Others","2023","Other Lists"],"sub_categories":["TeX Lists"],"readme":" \u003c!-- # \u003cp align=center\u003e`awesome gan-inversion`\u003c/p\u003e --\u003e\n[![Awesome](https://cdn.rawgit.com/sindresorhus/awesome/d7305f38d29fed78fa85652e3a63e154dd8e8829/media/badge.svg)](https://github.com/sindresorhus/awesome)\n[![Maintenance](https://img.shields.io/badge/Maintained%3F-yes-green.svg)](https://GitHub.com/Naereen/StrapDown.js/graphs/commit-activity)\n[![PR's Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=flat)](http://makeapullrequest.com) \n[![Survey Paper](https://img.shields.io/badge/Paper-arXiv-blue.svg?style=flat)](https://arxiv.org/abs/2403.04279) \n\n\u003cbr /\u003e\n\u003cp align=\"center\"\u003e\n  \u003ch1 align=\"center\"\u003eAwesome Controllable T2I Diffusion Models\u003c/h1\u003e\n\u003c/p\u003e\n\u003cbr /\u003e\n\nWe are focusing on how to **Control** text-to-image diffusion models with **Novel Conditions**. \n\nFor more detailed information, please refer to our survey paper: [Controllable Generation with Text-to-Image Diffusion Models: A Survey](https://arxiv.org/abs/2403.04279)\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"assets/count.png\" alt=\"img\" width=\"49%\" /\u003e\n  \u003cimg src=\"assets/controllable_generation.png\" alt=\"img\" width=\"49%\" /\u003e\n\u003c/p\u003e\n\n\n\n\n## 💖 Citation\n**If you find value in our survey paper or curated collection, please consider citing our work and starring our repo to support us.**\n\n```text\n@article{cao2024controllable,\n  title={Controllable Generation with Text-to-Image Diffusion Models: A Survey},\n  author={Pu Cao and Feng Zhou and Qing Song and Lu Yang},\n  journal={arXiv preprint arXiv:2403.04279},\n  year={2024}\n}\n```\n\n## 🎁 How to contribute to this repository?\nSince the following content is generated based on our database, please provide the following information in the **issue** to help us fill in the database to add new papers (please do not submit a PR directly).\n```text\n1. Paper title\n2. arXiv ID (if any)\n3. Publication status (if any)\n```\n\n## 🌈 Contents\n\n- [Generation with Specific Condition](#Generation-with-Specific-Condition)\n  - [Personalization](#Personalization)\n    - [Subject-Driven Generation](#Subject-Driven-Generation)\n    - [Person-Driven Generation](#Person-Driven-Generation)\n    - [Style-Driven Generation](#Style-Driven-Generation)\n    - [Interaction-Driven Generation](#Interaction-Driven-Generation)\n    - [Image-Driven Generation](#Image-Driven-Generation)\n    - [Distribution-Driven Generation](#Distribution-Driven-Generation)\n  - [Spatial Control](#Spatial-Control)\n  - [Advanced Text-Conditioned Generation](#Advanced-Text-Conditioned-Generation)\n  - [In-Context Generation](#In-Context-Generation)\n  - [Brain-Guided Generation](#Brain-Guided-Generation)\n  - [Sound-Guided Generation](#Sound-Guided-Generation)\n  - [Text Rendering](#Text-Rendering)\n- [Generation with Multiple Conditions](#Generation-with-Multiple-Conditions)\n  - [Joint Training](#Joint-Training)\n  - [Continual Learning](#Continual-Learning)\n  - [Weight Fusion](#Weight-Fusion)\n  - [Attention-based Integration](#Attention-based-Integration)\n  - [Guidance Composition](#Guidance-Composition)\n- [Universal Controllable Generation](#Universal-Controllable-Generation)\n  - [Universal Conditional Score Prediction](#Universal-Conditional-Score-Prediction)\n  - [Universal Condition-Guided Score Estimation](#Universal-Condition-Guided-Score-Estimation)\n\n\n\u003c!-- start --\u003e\n\n## 🚀Generation with Specific Condition\n### 🍇Personalization\n#### 🍉Subject-Driven Generation\n**PartCraft: Crafting Creative Objects by Parts.**\u003cbr\u003e\n*Kam Woh Ng, Xiatian Zhu, Yi-Zhe Song, Tao Xiang.*\u003cbr\u003e\nECCV 2024. [[PDF](https://arxiv.org/abs/2407.04604)]\n\n**RefDrop: Controllable Consistency in Image or Video Generation via Reference Feature Guidance.**\u003cbr\u003e\n*Jiaojiao Fan, Haotian Xue, Qinsheng Zhang, Yongxin Chen.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2405.17661)]\n\n**ClassDiffusion: More Aligned Personalization Tuning with Explicit Class Guidance.**\u003cbr\u003e\n*Jiannan Huang, Jun Hao Liew, Hanshu Yan, Yuyang Yin, Yao Zhao, Yunchao Wei.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2405.17532)]\n\n**User-Friendly Customized Generation with Multi-Modal Prompts.**\u003cbr\u003e\n*A, b, s, t, r, a, c, t,  , n, o, t,  , f, o, u, n, d.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2405.16501)]\n\n**Personalized Residuals for Concept-Driven Text-to-Image Generation.**\u003cbr\u003e\n*Cusuh Ham, Matthew Fisher, James Hays, Nicholas Kolkin, Yuchen Liu, Richard Zhang, Tobias Hinz.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2405.12978)]\n\n**Improving Subject-Driven Image Synthesis with Subject-Agnostic Guidance.**\u003cbr\u003e\n*Kelvin C. K. Chan, Yang Zhao, Xuhui Jia, Ming-Hsuan Yang, Huisheng Wang.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2405.01356)]\n\n**MMTryon: Multi-Modal Multi-Reference Control for High-Quality Fashion  Generation.**\u003cbr\u003e\n*Xujie Zhang, Ente Lin, Xiu Li, Yuxuan Luo, Michael Kampffmeyer, Xin Dong, Xiaodan Liang.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2405.00448)]\n\n**Infusion: Preventing Customized Text-to-Image Diffusion from Overfitting.**\u003cbr\u003e\n*Weili Zeng, Yichao Yan, Qi Zhu, Zhuo Chen, Pengzhi Chu, Weiming Zhao, Xiaokang Yang.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2404.14007)]\n\n**CAT: Contrastive Adapter Training for Personalized Image Generation.**\u003cbr\u003e\n*Jae Wan Park, Sang Hyun Park, Jun Young Koh, Junha Lee, Min Song.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2404.07554)]\n\n**MoMA: Multimodal LLM Adapter for Fast Personalized Image Generation.**\u003cbr\u003e\n*Kunpeng Song, Yizhe Zhu, Bingchen Liu, Qing Yan, Ahmed Elgammal, Xiao Yang.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2404.05674)]\n\n**U-VAP: User-specified Visual Appearance Personalization via Decoupled Self Augmentation.**\u003cbr\u003e\n*You Wu, Kean Liu, Xiaoyue Mi, Fan Tang, Juan Cao, Jintao Li.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2403.20231)]\n\n**Automated Black-box Prompt Engineering for Personalized Text-to-Image Generation.**\u003cbr\u003e\n*Yutong He, Alexander Robey, Naoki Murata, Yiding Jiang, Joshua Williams, George J. Pappas, Hamed Hassani, Yuki Mitsufuji, Ruslan Salakhutdinov, J. Zico Kolter.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2403.19103)]\n\n**Attention Calibration for Disentangled Text-to-Image Personalization.**\u003cbr\u003e\n*Yanbing Zhang, Mengping Yang, Qin Zhou, Zhe Wang.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2403.18551)]\n\n**Selectively Informative Description can Reduce Undesired Embedding Entanglements in Text-to-Image Personalization.**\u003cbr\u003e\n*Jimyeong Kim, Jungwon Park, Wonjong Rhee.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2403.15330)]\n\n**MM-Diff: High-Fidelity Image Personalization via Multi-Modal Condition Integration.**\u003cbr\u003e\n*Zhichao Wei, Qingkun Su, Long Qin, Weizhi Wang.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2403.15059)]\n\n**Generative Active Learning for Image Synthesis Personalization.**\u003cbr\u003e\n*Xulu Zhang, Wengyu Zhang, Xiao-Yong Wei, Jinlin Wu, Zhaoxiang Zhang, Zhen Lei, Qing Li.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2403.14987)]\n\n**Harmonizing Visual and Textual Embeddings for Zero-Shot Text-to-Image Customization.**\u003cbr\u003e\n*Yeji Song, Jimyeong Kim, Wonhark Park, Wonsik Shin, Wonjong Rhee, Nojun Kwak.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2403.14155)]\n\n**Tuning-Free Image Customization with Image and Text Guidance.**\u003cbr\u003e\n*Pengzhi Li, Qiang Nie, Ying Chen, Xi Jiang, Kai Wu, Yuhuan Lin, Yong Liu, Jinlong Peng, Chengjie Wang, Feng Zheng.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2403.12658)]\n\n**Fast Personalized Text-to-Image Syntheses With Attention Injection.**\u003cbr\u003e\n*Yuxuan Zhang, Yiren Song, Jinpeng Yu, Han Pan, Zhongliang Jing.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2403.11284)]\n\n**OMG: Occlusion-friendly Personalized Multi-concept Generation in Diffusion Models.**\u003cbr\u003e\n*Zhe Kong, Yong Zhang, Tianyu Yang, Tao Wang, Kaihao Zhang, Bizhu Wu, Guanying Chen, Wei Liu, Wenhan Luo.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2403.10983)]\n\n**StableGarment: Garment-Centric Generation via Stable Diffusion.**\u003cbr\u003e\n*Rui Wang, Hailong Guo, Jiaming Liu, Huaxia Li, Haibo Zhao, Xu Tang, Yao Hu, Hao Tang, Peipei Li.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2403.10783)]\n\n**Block-wise LoRA: Revisiting Fine-grained LoRA for Effective Personalization and Stylization in Text-to-Image Generation.**\u003cbr\u003e\n*Likun Li, Haoqi Zeng, Changpeng Yang, Haozhe Jia, Di Xu.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2403.07500)]\n\n**FaceChain-SuDe: Building Derived Class to Inherit Category Attributes for One-shot Subject-Driven Generation.**\u003cbr\u003e\n*Pengchong Qiao, Lei Shang, Chang Liu, Baigui Sun, Xiangyang Ji, Jie Chen.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2403.06775)]\n\n**RealCustom: Narrowing Real Text Word for Real-Time Open-Domain Text-to-Image Customization.**\u003cbr\u003e\n*Mengqi Huang, Zhendong Mao, Mingcong Liu, Qian He, Yongdong Zhang.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2403.00483)]\n\n**DiffuseKronA: A Parameter Efficient Fine-tuning Method for Personalized Diffusion Models.**\u003cbr\u003e\n*Shyam Marjit, Harshit Singh, Nityanand Mathur, Sayak Paul, Chia-Mu Yu, Pin-Yu Chen.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2402.17412)]\n\n**Direct Consistency Optimization for Compositional Text-to-Image Personalization.**\u003cbr\u003e\n*Kyungmin Lee, Sangkyung Kwak, Kihyuk Sohn, Jinwoo Shin.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2402.12004)]\n\n**ComFusion: Personalized Subject Generation in Multiple Specific Scenes From Single Image.**\u003cbr\u003e\n*Yan Hong, Jianfu Zhang.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2402.11849)]\n\n**Visual Concept-driven Image Generation with Text-to-Image Diffusion Model.**\u003cbr\u003e\n*Tanzila Rahman, Shweta Mahajan, Hsin-Ying Lee, Jian Ren, Sergey Tulyakov, Leonid Sigal.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2402.11487)]\n\n**Textual Localization: Decomposing Multi-concept Images for Subject-Driven Text-to-Image Generation.**\u003cbr\u003e\n*Junjie Shentu, Matthew Watson, Noura Al Moubayed.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2402.09966)]\n\n**DreamMatcher: Appearance Matching Self-Attention for Semantically-Consistent Text-to-Image Personalization.**\u003cbr\u003e\n*Jisu Nam, Heesu Kim, DongJae Lee, Siyoon Jin, Seungryong Kim, Seunggyu Chang.*\u003cbr\u003e\nCVPR 2024. [[PDF](https://arxiv.org/abs/2402.09812)]\n\n**SeFi-IDE: Semantic-Fidelity Identity Embedding for Personalized Diffusion-Based Generation.**\u003cbr\u003e\n*Yang Li, Songlin Yang, Wei Wang, Jing Dong.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2402.00631)]\n\n**Pick-and-Draw: Training-free Semantic Guidance for Text-to-Image Personalization.**\u003cbr\u003e\n*Henglei Lv, Jiayu Xiao, Liang Li, Qingming Huang.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2401.16762)]\n\n**Object-Driven One-Shot Fine-tuning of Text-to-Image Diffusion with Prototypical Embedding.**\u003cbr\u003e\n*Jianxiang Lu, Cong Xie, Hui Guo.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2401.15708)]\n\n**BootPIG: Bootstrapping Zero-shot Personalized Image Generation Capabilities in Pretrained Diffusion Models.**\u003cbr\u003e\n*Senthil Purushwalkam, Akash Gokul, Shafiq Joty, Nikhil Naik.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2401.13974)]\n\n**PALP: Prompt Aligned Personalization of Text-to-Image Models.**\u003cbr\u003e\n*Moab Arar, Andrey Voynov, Amir Hertz, Omri Avrahami, Shlomi Fruchter, Yael Pritch, Daniel Cohen-Or, Ariel Shamir.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2401.06105)]\n\n**Cross Initialization for Personalized Text-to-Image Generation.**\u003cbr\u003e\n*Lianyu Pang, Jian Yin, Haoran Xie, Qiping Wang, Qing Li, Xudong Mao.*\u003cbr\u003e\nCVPR 2024. [[PDF](https://arxiv.org/abs/2312.15905)]\n\n**DreamTuner: Single Image is Enough for Subject-Driven Generation.**\u003cbr\u003e\n*Miao Hua, Jiawei Liu, Fei Ding, Wei Liu, Jie Wu, Qian He.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2312.13691)]\n\n**Decoupled Textual Embeddings for Customized Image Generation.**\u003cbr\u003e\n*Yufei Cai, Yuxiang Wei, Zhilong Ji, Jinfeng Bai, Hu Han, Wangmeng Zuo.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2312.11826)]\n\n**Compositional Inversion for Stable Diffusion Models.**\u003cbr\u003e\n*Xulu Zhang, Xiao-Yong Wei, Jinlin Wu, Tianyi Zhang, Zhaoxiang Zhang, Zhen Lei, Qing Li.*\u003cbr\u003e\nAAAI 2024. [[PDF](https://arxiv.org/abs/2312.08048v3)]\n\n**Customization Assistant for Text-to-image Generation.**\u003cbr\u003e\n*Yufan Zhou, Ruiyi Zhang, Jiuxiang Gu, Tong Sun.*\u003cbr\u003e\nCVPR 2024. [[PDF](https://arxiv.org/abs/2312.03045)]\n\n**VideoBooth: Diffusion-based Video Generation with Image Prompts.**\u003cbr\u003e\n*Yuming Jiang, Tianxing Wu, Shuai Yang, Chenyang Si, Dahua Lin, Yu Qiao, Chen Change Loy, Ziwei Liu.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2312.00777)]\n\n**HiFi Tuner: High-Fidelity Subject-Driven Fine-Tuning for Diffusion Models.**\u003cbr\u003e\n*Zhonghao Wang, Wei Wei, Yang Zhao, Zhisheng Xiao, Mark Hasegawa-Johnson, Humphrey Shi, Tingbo Hou.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2312.00079)]\n\n**VideoAssembler: Identity-Consistent Video Generation with Reference Entities using Diffusion Model.**\u003cbr\u003e\n*Haoyu Zhao, Tianyi Lu, Jiaxi Gu, Xing Zhang, Zuxuan Wu, Hang Xu, Yu-Gang Jiang.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2311.17338)]\n\n**CLiC: Concept Learning in Context.**\u003cbr\u003e\n*Mehdi Safaee, Aryan Mikaeili, Or Patashnik, Daniel Cohen-Or, Ali Mahdavi-Amiri.*\u003cbr\u003e\nCVPR 2024. [[PDF](https://arxiv.org/abs/2311.17083)]\n\n**CatVersion- Concatenating Embeddings for Diffusion-Based Text-to-Image Personalization.**\u003cbr\u003e\n*Ruoyu Zhao, Mingrui Zhu, Shiyin Dong, Nannan Wang, Xinbo Gao.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2311.14631)]\n\n**Lego- Learning to Disentangle and Invert Concepts Beyond Object Appearance in Text-to-Image Diffusion Models.**\u003cbr\u003e\n*Saman Motamed, Danda Pani Paudel, Luc Van Gool.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2311.13833)]\n\n**An Image is Worth Multiple Words- Multi-attribute Inversion for Constrained Text-to-Image Synthesis.**\u003cbr\u003e\n*Aishwarya Agarwal, Srikrishna Karanam, Tripti Shukla, Balaji Vasan Srinivasan.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2311.11919)]\n\n**DIFFNAT- Improving Diffusion Image Quality Using Natural Image  Statistics.**\u003cbr\u003e\n*Aniket Roy, Maiterya Suin, Anshul Shah, Ketul Shah, Jiang Liu, Rama Chellappa.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2311.09753)]\n\n**A Data Perspective on Enhanced Identity Preservation for Diffusion  Personalization.**\u003cbr\u003e\n*Xingzhe He, Zhiwen Cao, Nicholas Kolkin, Lantao Yu, Helge Rhodin, Ratheesh Kalarot.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2311.04315)]\n\n**AN IMAGE IS WORTH MULTIPLE WORDS : LEARNING OBJECT LEVEL CONCEPTS USING MULTI-CONCEPT PROMPT LEARNING.**\u003cbr\u003e\n*Chen Jin, Ryutaro Tanno, Amrutha Saseendran, Tom Diethe, Philip Teare.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2310.12274)]\n\n**Kosmos-G: Generating Images in Context with Multimodal Large Language Models.**\u003cbr\u003e\n*Xichen Pan, Li Dong, Shaohan Huang, Zhiliang Peng, Wenhu Chen, Furu Wei.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2310.02992)]\n\n**Navigating Text-To-Image Customization: From LyCORIS Fine-Tuning to Model Evaluation.**\u003cbr\u003e\n*Shin-Ying Yeh, Yu-Guan Hsieh, Zhidong Gao, Bernard B W Yang, Giyeong Oh, Yanmin Gong.*\u003cbr\u003e\nICLR 2024. [[PDF](https://arxiv.org/abs/2309.14859)]\n\n**Subject-Diffusion- Open Domain Personalized  Text-to-Image Generation without Test-time  Fine-tuning.**\u003cbr\u003e\n*Jian Ma, Junhao Liang, Chen Chen, Haonan Lu.*\u003cbr\u003e\nICLR2024 (3566). [[PDF](https://arxiv.org/abs/2307.11410)]\n\n**Domain-Agnostic Tuning-Encoder for Fast Personalization of  Text-To-Image Models .**\u003cbr\u003e\n*Moab Arar, Rinon Gal, Yuval Atzmon, Gal Chechik, Daniel Cohen-Or, Ariel Shamir, Amit H. Bermano.*\u003cbr\u003e\nSIGGRAPH ASIA 2023. [[PDF](https://arxiv.org/abs/2307.06925)]\n\n**Generate Anything Anywhere in Any Scene.**\u003cbr\u003e\n*Yuheng Li, Haotian Liu, Yangming Wen, Yong Jae Lee.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2306.17154)]\n\n**Controlling Text-to-Image Diffusion by Orthogonal Finetuning .**\u003cbr\u003e\n*Zeju Qiu, Weiyang Liu, Haiwen Feng, Yuxuan Xue, Yao Feng, Zhen Liu, Dan Zhang, Adrian Weller, Bernhard Schölkopf.*\u003cbr\u003e\nNeurIPS 2023. [[PDF](https://arxiv.org/abs/2306.07280)]\n\n**COMCAT: Towards Efficient Compression and Customization of\nAttention-Based Vision Models.**\u003cbr\u003e\n*Jinqi Xiao, Miao Yin, Yu Gong, Xiao Zang, Jian Ren, Bo Yuan.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2305.17235)]\n\n**Break-A-Scene: Extracting Multiple Concepts from a Single Image.**\u003cbr\u003e\n*Omri Avrahami, Kfir Aberman, Ohad Fried, Daniel Cohen-Or, Dani Lischinski.*\u003cbr\u003e\nSIGGRAPH ASIA 2023. [[PDF](https://arxiv.org/abs/2305.16311)]\n\n**ProSpect- Prompt Spectrum for Attribute-Aware Personalization of  Diffusion Models.**\u003cbr\u003e\n*Yuxin Zhang, Weiming Dong, Fan Tang, Nisha Huang, Haibin Huang, Chongyang Ma, Tong-Yee Lee, Oliver Deussen, Changsheng Xu.*\u003cbr\u003e\nTOG 2023. [[PDF](https://arxiv.org/abs/2305.16225)]\n\n**BLIP-Diffusion- Pre-trained Subject Representation for  Controllable Text-to-Image Generation and Editing.**\u003cbr\u003e\n*Dongxu Li, Junnan Li, Steven C. H. Hoi.*\u003cbr\u003e\nNeurIPS 2023. [[PDF](https://arxiv.org/abs/2305.14720)]\n\n**DisenBooth- Disentangled Parameter-Efficient Tuning for Subject-Driven  Text-to-Image Generation.**\u003cbr\u003e\n*Hong Chen, Yipeng Zhang, Simin Wu, Xin Wang, Xuguang Duan, Yuwei Zhou, Wenwu Zhu.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2305.03374)]\n\n**Key-Locked Rank One Editing for Text-to-Image Personalization.**\u003cbr\u003e\n*Yoad Tewel, Rinon Gal, Gal Chechik, Yuval Atzmon.*\u003cbr\u003e\nSIGGRAPH 2023. [[PDF](https://arxiv.org/abs/2305.01644)]\n\n**Gradient-Free Textual Inversion.**\u003cbr\u003e\n*Zhengcong Fei, Mingyuan Fan, Junshi Huang.*\u003cbr\u003e\nACM MM 2023. [[PDF](https://arxiv.org/abs/2304.05818)]\n\n**Controllable Textual Inversion for Personalized Text-to-Image Generation.**\u003cbr\u003e\n*Jianan Yang, Haobo Wang, Yanming Zhang, Ruixuan Xiao, Sai Wu, Gang Chen, Junbo Zhao.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2304.05265)]\n\n**InstantBooth- Personalized Text-to-Image Generation without Test-Time  Finetuning.**\u003cbr\u003e\n*Jing Shi, Wei Xiong, Zhe Lin, Hyun Joon Jung.*\u003cbr\u003e\nCVPR 2024. [[PDF](https://arxiv.org/abs/2304.03411)]\n\n**Taming Encoder for Zero Fine-tuning Image Customization  with Text-to-Image Diffusion Models.**\u003cbr\u003e\n*Xuhui Jia, Yang Zhao, Kelvin C. K. Chan, Yandong Li, Han Zhang, Boqing Gong, Tingbo Hou, Huisheng Wang, Yu-Chuan Su.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2304.02642)]\n\n**Subject-driven Text-to-Image Generation via  Apprenticeship Learning.**\u003cbr\u003e\n*Wenhu Chen, Hexiang Hu, Yandong Li, Nataniel Ruiz, Xuhui Jia, Ming-Wei Chang, William W. Cohen.*\u003cbr\u003e\nNeurIPS 2023. [[PDF](https://arxiv.org/abs/2304.00186)]\n\n**A Closer Look at Parameter-Efficient Tuning in Diffusion Models .**\u003cbr\u003e\n*Chendong Xiang, Fan Bao, Chongxuan Li, Hang Su, Jun Zhu.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2303.18181)]\n\n**SVDiff- Compact Parameter Space for Diffusion Fine-Tuning.**\u003cbr\u003e\n*Ligong Han, Yinxiao Li, Han Zhang, Peyman Milanfar, Dimitris Metaxas, Feng Yang.*\u003cbr\u003e\nICCV 2023. [[PDF](https://arxiv.org/abs/2303.11305)]\n\n**P+- Extended Textual Conditioning in Text-to-Image Generation.**\u003cbr\u003e\n*Andrey Voynov, Qinghao Chu, Daniel Cohen-Or, Kfir Aberman.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2303.09522)]\n\n**Unified Multi-Modal Latent Diffusion for\nJoint Subject and Text Conditional Image Generation.**\u003cbr\u003e\n*Yiyang Ma, Huan Yang, Wenjing Wang, Jianlong Fu, Jiaying Liu.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2303.09319)]\n\n**ELITE- Encoding Visual Concepts into Textual Embeddings  for Customized Text-to-Image Generation.**\u003cbr\u003e\n*Yuxiang Wei, Yabo Zhang, Zhilong Ji, Jinfeng Bai, Lei Zhang, Wangmeng Zuo.*\u003cbr\u003e\nICCV 2023. [[PDF](https://arxiv.org/abs/2302.13848)]\n\n**Designing an Encoder for Fast Personalization of Text-to-Image  Models.**\u003cbr\u003e\n*Rinon Gal, Moab Arar, Yuval Atzmon, Amit H. Bermano, Gal Chechik, Daniel Cohen-Or.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2302.12228)]\n\n**Is This Loss Informative? Faster Text-to-Image Customization by Tracking Objective Dynamics.**\u003cbr\u003e\n*Anton Voronov, Mikhail Khoroshikh, Artem Babenko, Max Ryabinin.*\u003cbr\u003e\nNeurIPS 2023. [[PDF](https://arxiv.org/abs/2302.04841)]\n\n**Multi-Concept Customization of Text-to-Image Diffusion.**\u003cbr\u003e\n*Nupur Kumari, Bingliang Zhang, Richard Zhang, Eli Shechtman, Jun-Yan Zhu.*\u003cbr\u003e\nCVPR 2023. [[PDF](https://arxiv.org/abs/2212.04488)]\n\n**DreamArtist: Towards Controllable One-Shot Text-to-Image Generation via Positive-Negative Prompt-Tuning.**\u003cbr\u003e\n*Ziyi Dong, Pengxu Wei, Liang Lin.*\u003cbr\u003e\narXiv 2022. [[PDF](https://arxiv.org/abs/2211.11337)]\n\n**Re-Imagen: Retrieval-Augmented Text-to-Image Generator.**\u003cbr\u003e\n*Wenhu Chen, Hexiang Hu, Chitwan Saharia, William W. Cohen.*\u003cbr\u003e\nICLR 2023. [[PDF](https://arxiv.org/abs/2209.14491)]\n\n**DreamBooth- Fine Tuning Text-to-Image Diffusion Models  for Subject-Driven Generation.**\u003cbr\u003e\n*Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, Kfir Aberman.*\u003cbr\u003e\nCVPR 2023. [[PDF](https://arxiv.org/abs/2208.12242)]\n\n**An Image is Worth One Word- Personalizing  Text-to-Image Generation using Textual Inversion.**\u003cbr\u003e\n*Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit H. Bermano, Gal Chechik, Daniel Cohen-Or.*\u003cbr\u003e\nICLR 2023. [[PDF](https://arxiv.org/abs/2208.01618)]\n\n\n#### 🍊Person-Driven Generation\n**Face Adapter for Pre-Trained Diffusion Models with Fine-Grained ID and Attribute Control.**\u003cbr\u003e\n*Yue Han, Junwei Zhu, Keke He, Xu Chen, Yanhao Ge, Wei Li, Xiangtai Li, Jiangning Zhang, Chengjie Wang, Yong Liu.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2405.12970)]\n\n**MasterWeaver: Taming Editability and Identity for Personalized Text-to-Image Generation.**\u003cbr\u003e\n*Yuxiang Wei, Zhilong Ji, Jinfeng Bai, Hongzhi Zhang, Lei Zhang, Wangmeng Zuo.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2405.05806)]\n\n**InstantFamily: Masked Attention for Zero-shot Multi-ID Image Generation.**\u003cbr\u003e\n*Chanran Kim, Jeongin Lee, Shichang Joung, Bongmo Kim, Yeul-Min Baek.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2404.19427)]\n\n**ConsistentID: Portrait Generation with Multimodal Fine-Grained Identity Preserving.**\u003cbr\u003e\n*Jiehui Huang, Xiao Dong, Wenhui Song, Hanhui Li, Jun Zhou, Yuhao Cheng, Shutao Liao, Long Chen, Yiqiang Yan, Shengcai Liao, Xiaodan Liang.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2404.16771)]\n\n**PuLID: Pure and Lightning ID Customization via Contrastive Alignment.**\u003cbr\u003e\n*Zinan Guo, Yanze Wu, Zhuowei Chen, Lang Chen, Qian He.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2404.16022)]\n\n**ID-Aligner: Enhancing Identity-Preserving Text-to-Image Generation with Reward Feedback Learning.**\u003cbr\u003e\n*Weifeng Chen, Jiacheng Zhang, Jie Wu, Hefeng Wu, Xuefeng Xiao, Liang Lin.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2404.15449)]\n\n**ID-Animator: Zero-Shot Identity-Preserving Human Video Generation.**\u003cbr\u003e\n*Xuanhua He, Quande Liu, Shengju Qian, Xin Wang, Tao Hu, Ke Cao, Keyu Yan, Jie Zhang.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2404.15275)]\n\n**From Parts to Whole: A Unified Reference Framework for Controllable Human Image Generation.**\u003cbr\u003e\n*Zehuan Huang, Hongxing Fan, Lipeng Wang, Lu Sheng.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2404.15267)]\n\n**MoA: Mixture-of-Attention for Subject-Context Disentanglement in Personalized Image Generation.**\u003cbr\u003e\n*Kuan-Chieh Wang, Daniil Ostashev, Yuwei Fang, Sergey Tulyakov, Kfir Aberman.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2404.11565)]\n\n**LCM-Lookahead for Encoder-based Text-to-Image Personalization.**\u003cbr\u003e\n*Rinon Gal, Or Lichter, Elad Richardson, Or Patashnik, Amit H. Bermano, Gal Chechik, Daniel Cohen-Or.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2404.03620)]\n\n**FlashFace: Human Image Personalization with High-fidelity Identity Preservation.**\u003cbr\u003e\n*Shilong Zhang, Lianghua Huang, Xi Chen, Yifei Zhang, Zhi-Fan Wu, Yutong Feng, Wei Wang, Yujun Shen, Yu Liu, Ping Luo.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2403.17008)]\n\n**IDAdapter: Learning Mixed Features for Tuning-Free Personalization of Text-to-Image Models.**\u003cbr\u003e\n*Siying Cui, Jia Guo, Xiang An, Jiankang Deng, Yongle Zhao, Xinyu Wei, Ziyong Feng.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2403.13535)]\n\n**Infinite-ID: Identity-preserved Personalization via ID-semantics Decoupling Paradigm.**\u003cbr\u003e\n*Yi Wu, Ziqiang Li, Heliang Zheng, Chaoyue Wang, Bin Li.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2403.11781)]\n\n**Face2Diffusion for Fast and Editable Face Personalization.**\u003cbr\u003e\n*Kaede Shiohara, Toshihiko Yamasaki.*\u003cbr\u003e\nCVPR 2024. [[PDF](https://arxiv.org/abs/2403.05094)]\n\n**CapHuman: Capture Your Moments in Parallel Universes.**\u003cbr\u003e\n*Chao Liang, Fan Ma, Linchao Zhu, Yingying Deng, Yi Yang.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2402.00627)]\n\n**StableIdentity: Inserting Anybody into Anywhere at First Sight.**\u003cbr\u003e\n*Qinghe Wang, Xu Jia, Xiaomin Li, Taiqing Li, Liqian Ma, Yunzhi Zhuge, Huchuan Lu.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2401.15975)]\n\n**InstantID: Zero-shot Identity-Preserving Generation in Seconds.**\u003cbr\u003e\n*Qixun Wang, Xu Bai, Haofan Wang, Zekui Qin, Anthony Chen, Huaxia Li, Xu Tang, Yao Hu.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2401.07519)]\n\n**Towards a Simultaneous and Granular Identity-Expression Control in Personalized Face Generation.**\u003cbr\u003e\n*Renshuai Liu, Bowen Ma, Wei Zhang, Zhipeng Hu, Changjie Fan, Tangjie Lv, Yu Ding, Xuan Cheng.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2401.01207)]\n\n**PortraitBooth: A Versatile Portrait Model for Fast Identity-preserved Personalization.**\u003cbr\u003e\n*Xu Peng, Junwei Zhu, Boyuan Jiang, Ying Tai, Donghao Luo, Jiangning Zhang, Wei Lin, Taisong Jin, Chengjie Wang, Rongrong Ji.*\u003cbr\u003e\nCVPR 2024. [[PDF](https://arxiv.org/abs/2312.06354)]\n\n**Stellar: Systematic Evaluation of Human-Centric Personalized Text-to-Image Methods.**\u003cbr\u003e\n*Panos Achlioptas, Alexandros Benetatos, Iordanis Fostiropoulos, Dimitris Skourtis.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2312.06116)]\n\n**PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embedding.**\u003cbr\u003e\n*Zhen Li, Mingdeng Cao, Xintao Wang, Zhongang Qi, Ming-Ming Cheng, Ying Shan.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2312.04461)]\n\n**DemoCaricature: Democratising Caricature Generation with a Rough Sketch.**\u003cbr\u003e\n*Dar-Yen Chen, Subhadeep Koley, Aneeshan Sain, Pinaki Nath Chowdhury, Tao Xiang, Ayan Kumar Bhunia, Yi-Zhe Song.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2312.04364)]\n\n**ViscoNet: Bridging and Harmonizing Visual and Textual Conditioning for ControlNet.**\u003cbr\u003e\n*Soon Yau Cheong, Armin Mustafa, Andrew Gilbert.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2312.03154)]\n\n**FaceStudio: Put Your Face Everywhere in Seconds.**\u003cbr\u003e\n*Yuxuan Yan, Chi Zhang, Rui Wang, Yichao Zhou, Gege Zhang, Pei Cheng, Gang Yu, Bin Fu.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2312.02663)]\n\n**Retrieving Conditions from Reference Images for Diffusion Models.**\u003cbr\u003e\n*Haoran Tang, Xin Zhou, Jieren Deng, Zhihong Pan, Hao Tian, Pratik Chaudhari.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2312.02521)]\n\n**When StyleGAN Meets Stable Diffusion- a W + Adapter for Personalized Image Generation.**\u003cbr\u003e\n*Xiaoming Li, Xinyu Hou, Chen Change Loy.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2311.17461)]\n\n**High-fidelity Person-centric Subject-to-Image Synthesis.**\u003cbr\u003e\n*Yibin Wang, Weizhong Zhang, Jianwei Zheng, Cheng Jin.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2311.10329)]\n\n**MagiCapture- High-Resolution Multi-Concept Portrait Customization.**\u003cbr\u003e\n*Junha Hyung, Jaeyo Shin, Jaegul Choo.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2309.06895)]\n\n**PhotoVerse- Tuning-Free Image Customization with Text-to-Image Diffusion Models.**\u003cbr\u003e\n*Li Chen, Mengyi Zhao, Yiheng Liu, Mingxu Ding, Yangyang Song, Shizun Wang, Xu Wang, Hao Yang, Jing Liu, Kang Du, Min Zheng.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2309.05793)]\n\n**HyperDreamBooth- HyperNetworks for Fast  Personalization of Text-to-Image Models .**\u003cbr\u003e\n*Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Wei Wei, Tingbo Hou, Yael Pritch, Neal Wadhwa, Michael Rubinstein, Kfir Aberman.*\u003cbr\u003e\nCVPR 2024. [[PDF](https://arxiv.org/abs/2307.06949)]\n\n**DreamIdentity- Improved Editability for Efficient  Face-identity Preserved Image Generation .**\u003cbr\u003e\n*Zhuowei Chen, Shancheng Fang, Wei Liu, Qian He, Mengqi Huang, Yongdong Zhang, Zhendong Mao.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2307.00300)]\n\n**Face0- Instantaneously Conditioning a Text-to-Image Model on a Face.**\u003cbr\u003e\n*Dani Valevski, Danny Wasserman, Yossi Matias, Yaniv Leviathan.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2306.06638)]\n\n**Conditioning Diffusion Models via Attributes and Semantic Masks for Face Generation.**\u003cbr\u003e\n*Nico Giambi, Giuseppe Lisanti.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2306.00914)]\n\n**FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attention.**\u003cbr\u003e\n*Guangxuan Xiao, Tianwei Yin, William T. Freeman, Frédo Durand, Song Han.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2305.10431)]\n\n\n#### 🍋Style-Driven Generation\n**RB-Modulation: Training-Free Personalization of Diffusion Models using Stochastic Optimal Control.**\u003cbr\u003e\n*Litu Rout, Yujia Chen, Nataniel Ruiz, Abhishek Kumar, Constantine Caramanis, Sanjay Shakkottai, Wen-Sheng Chu.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2405.17401)]\n\n**StyleMaster: Towards Flexible Stylized Image Generation with Diffusion Models.**\u003cbr\u003e\n*A, b, s, t, r, a, c, t,  , n, o, t,  , f, o, u, n, d.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2405.15287)]\n\n**VectorPainter: A Novel Approach to Stylized Vector Graphics Synthesis  with Vectorized Strokes.**\u003cbr\u003e\n*Juncheng Hu, Ximing Xing, Zhengqi Zhang, Jing Zhang, Qian Yu.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2405.02962)]\n\n**Customizing Text-to-Image Models with a Single Image Pair.**\u003cbr\u003e\n*Maxwell Jones, Sheng-Yu Wang, Nupur Kumari, David Bau, Jun-Yan Zhu.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2405.01536)]\n\n**MuseumMaker: Continual Style Customization without Catastrophic Forgetting.**\u003cbr\u003e\n*Chenxi Liu, Gan Sun, Wenqi Liang, Jiahua Dong, Can Qin, Yang Cong.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2404.16612)]\n\n**StyleBooth: Image Style Editing with Multimodal Instruction.**\u003cbr\u003e\n*Zhen Han, Chaojie Mao, Zeyinzi Jiang, Yulin Pan, Jingfeng Zhang.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2404.12154)]\n\n**Towards Highly Realistic Artistic Style Transfer via Stable Diffusion with Step-aware and Layer-aware Prompt.**\u003cbr\u003e\n*Zhanjie Zhang, Quanwei Zhang, Huaizhong Lin, Wei Xing, Juncheng Mo, Shuaicheng Huang, Jinheng Xie, Guangyuan Li, Junsheng Luan, Lei Zhao, Dalong Zhang, Lixia Chen.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2404.11474)]\n\n**Text-to-Image Synthesis for Any Artistic Styles: Advancements in Personalized Artistic Image Generation via Subdivision and Dual Binding.**\u003cbr\u003e\n*Junseo Park, Beomseok Ko, Hyeryung Jang.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2404.05256)]\n\n**InstantStyle: Free Lunch towards Style-Preserving in Text-to-Image Generation.**\u003cbr\u003e\n*Haofan Wang, Matteo Spinelli, Qixun Wang, Xu Bai, Zekui Qin, Anthony Chen.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2404.02733)]\n\n**Implicit Style-Content Separation using B-LoRA.**\u003cbr\u003e\n*Yarden Frenkel, Yael Vinker, Ariel Shamir, Daniel Cohen-Or.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2403.14572)]\n\n**DEADiff: An Efficient Stylization Diffusion Model with Disentangled Representations.**\u003cbr\u003e\n*Tianhao Qi, Shancheng Fang, Yanze Wu, Hongtao Xie, Jiawei Liu, Lang Chen, Qian He, Yongdong Zhang.*\u003cbr\u003e\nCVPR 2024. [[PDF](https://arxiv.org/abs/2403.06951)]\n\n**Visual Style Prompting with Swapping Self-Attention.**\u003cbr\u003e\n*Jaeseok Jeong, Junho Kim, Yunjey Choi, Gayoung Lee, Youngjung Uh.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2402.12974)]\n\n**Towards Accurate Guided Diffusion Sampling through Symplectic Adjoint Method.**\u003cbr\u003e\n*Jiachun Pan, Hanshu Yan, Jun Hao Liew, Jiashi Feng, Vincent Y. F. Tan.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2312.12030)]\n\n**Style Aligned Image Generation via Shared Attention.**\u003cbr\u003e\n*Amir Hertz, Andrey Voynov, Shlomi Fruchter, Daniel Cohen-Or.*\u003cbr\u003e\nCVPR 2024. [[PDF](https://arxiv.org/abs/2312.02133)]\n\n**ArtAdapter: Text-to-Image Style Transfer using Multi-Level Style Encoder and Explicit Adaptation.**\u003cbr\u003e\n*Dar-Yen Chen, Hamish Tennent, Ching-Wen Hsu.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2312.02109)]\n\n**StyleCrafter- Enhancing Stylized Text-to-Video Generation with Style Adapter.**\u003cbr\u003e\n*Gongye Liu, Menghan Xia, Yong Zhang, Haoxin Chen, Jinbo Xing, Xintao Wang, Yujiu Yang, Ying Shan.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2312.00330)]\n\n**StyleDrop: Text-to-Image Generation in Any Style.**\u003cbr\u003e\n*Kihyuk Sohn, Nataniel Ruiz, Kimin Lee, Daniel Castro Chin, Irina Blok, Huiwen Chang, Jarred Barber, Lu Jiang, Glenn Entis, Yuanzhen Li, Yuan Hao, Irfan Essa, Michael Rubinstein, Dilip Krishnan.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2306.00983)]\n\n\n#### 🍍Interaction-Driven Generation \n**SG-Adapter: Enhancing Text-to-Image Generation with Scene Graph Guidance.**\u003cbr\u003e\n*Guibao Shen, Luozhou Wang, Jiantao Lin, Wenhang Ge, Chaozhe Zhang, Xin Tao, Yuan Zhang, Pengfei Wan, Zhongyuan Wang, Guangyong Chen, Yijun Li, Ying-Cong Chen.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2405.15321)]\n\n**VirtualModel: Generating Object-ID-retentive Human-object Interaction Image by Diffusion Model for E-commerce Marketing.**\u003cbr\u003e\n*Binghui Chen, Chongyang Zhong, Wangmeng Xiang, Yifeng Geng, Xuansong Xie.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2405.09985)]\n\n**InteractDiffusion: Interaction Control in Text-to-Image Diffusion Models.**\u003cbr\u003e\n*Jiun Tian Hoe, Xudong Jiang, Chee Seng Chan, Yap-Peng Tan, Weipeng Hu.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2312.05849)]\n\n**MotionCrafter: One-Shot Motion Customization of Diffusion Models.**\u003cbr\u003e\n*Yuxin Zhang, Fan Tang, Nisha Huang, Haibin Huang, Chongyang Ma, Weiming Dong, Changsheng Xu.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2312.05288)]\n\n**DreaMoving: A Human Dance Video Generation Framework based on Diffusion Models.**\u003cbr\u003e\n*Mengyang Feng, Jinlin Liu, Kai Yu, Yuan Yao, Zheng Hui, Xiefan Guo, Xianhui Lin, Haolan Xue, Chen Shi, Xiaowen Li, Aojie Li, Xiaoyang Kang, Biwen Lei, Miaomiao Cui, Peiran Ren, Xuansong Xie.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2312.05107)]\n\n**Customizing Motion in Text-to-Video Diffusion Models.**\u003cbr\u003e\n*Joanna Materzynska, Josef Sivic, Eli Shechtman, Antonio Torralba, Richard Zhang, Bryan Russell.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2312.04966)]\n\n**SAVE: Protagonist Diversification with Structure Agnostic Video Editing.**\u003cbr\u003e\n*Yeji Song, Wonsik Shin, Junsoo Lee, Jeesoo Kim, Nojun Kwak.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2312.02503)]\n\n**LAMP: Learn A Motion Pattern for Few-Shot-Based Video Generation.**\u003cbr\u003e\n*Ruiqi Wu, Liangyu Chen, Tong Yang, Chunle Guo, Chongyi Li, Xiangyu Zhang.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2310.10769)]\n\n**MotionDirector- Motion Customization of Text-to-Video Diffusion Models.**\u003cbr\u003e\n*Rui Zhao, Yuchao Gu, Jay Zhangjie Wu, David Junhao Zhang, Jiawei Liu, Weijia Wu, Jussi Keppo, Mike Zheng Shou.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2310.08465)]\n\n**AnimateDiff- Animate Your Personalized Text-to-Image Diffusion Models  without Specific Tuning.**\u003cbr\u003e\n*Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, Bo Dai.*\u003cbr\u003e\nICLR2024 (6688). [[PDF](https://arxiv.org/abs/2307.04725)]\n\n**ReVersion- Diffusion-Based Relation Inversion from Images .**\u003cbr\u003e\n*Ziqi Huang, Tianxing Wu, Yuming Jiang, Kelvin C. K. Chan, Ziwei Liu.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2303.13495)]\n\n\n#### 🍎Image-Driven Generation\n**FreeControl: Training-Free Spatial Control of Any Text-to-Image Diffusion Model with Any Condition.**\u003cbr\u003e\n*Sicheng Mo, Fangzhou Mu, Kuan Heng Lin, Yanli Liu, Bochen Guan, Yin Li, Bolei Zhou.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2312.07536)]\n\n**Context Diffusion: In-Context Aware Image Generation.**\u003cbr\u003e\n*Ivona Najdenkoska, Animesh Sinha, Abhimanyu Dubey, Dhruv Mahajan, Vignesh Ramanathan, Filip Radenovic.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2312.03584)]\n\n**ViscoNet: Bridging and Harmonizing Visual and Textual Conditioning for ControlNet.**\u003cbr\u003e\n*Soon Yau Cheong, Armin Mustafa, Andrew Gilbert.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2312.03154)]\n\n**IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models.**\u003cbr\u003e\n*Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, Wei Yang.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2308.06721)]\n\n**Uni-ControlNet: All-in-One Control to Text-to-Image Diffusion Models.**\u003cbr\u003e\n*Shihao Zhao, Dongdong Chen, Yen-Chun Chen, Jianmin Bao, Shaozhe Hao, Lu Yuan, Kwan-Yee K. Wong.*\u003cbr\u003e\nNeurIPS 2023. [[PDF](https://arxiv.org/abs/2305.16322)]\n\n**Prompt-Free Diffusion: Taking “Text” out of Text-to-Image Diffusion Models.**\u003cbr\u003e\n*Xingqian Xu, Jiayi Guo, Zhangyang Wang, Gao Huang, Irfan Essa, Humphrey Shi.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2305.16223)]\n\n**Versatile Diffusion: Text, Images and Variations All in One Diffusion Model.**\u003cbr\u003e\n*Xingqian Xu, Zhangyang Wang, Eric Zhang, Kai Wang, Humphrey Shi.*\u003cbr\u003e\nICCV 2023. [[PDF](https://arxiv.org/abs/2211.08332)]\n\n**Hierarchical Text-Conditional Image Generation with CLIP Latents.**\u003cbr\u003e\n*Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, Mark Chen.*\u003cbr\u003e\narXiv 2022. [[PDF](https://arxiv.org/abs/2204.06125)]\n\n\n#### 🍐Distribution-Driven Generation\n**DreamDistribution: Prompt Distribution Learning for Text-to-Image Diffusion Models.**\u003cbr\u003e\n*Brian Nlong Zhao, Yuhang Xiao, Jiashu Xu, Xinyang Jiang, Yifan Yang, Dongsheng Li, Laurent Itti, Vibhav Vineet, Yunhao Ge.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2312.14216)]\n\n**Concept-centric Personalization with Large-scale Diffusion Priors.**\u003cbr\u003e\n*Pu Cao, Lu Yang, Feng Zhou, Tianrui Huang, Qing Song.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2312.08195)]\n\n\n### 🍔Spatial Control\n**Enhancing Image Layout Control with Loss-Guided Diffusion Models.**\u003cbr\u003e\n*Zakaria Patel, Kirill Serkh.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2405.14101)]\n\n**Compositional Text-to-Image Generation with Dense Blob Representations.**\u003cbr\u003e\n*Weili Nie, Sifei Liu, Morteza Mardani, Chao Liu, Benjamin Eckart, Arash Vahdat.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2405.08246)]\n\n**FlexEControl: Flexible and Efficient Multimodal Control for Text-to-Image Generation.**\u003cbr\u003e\n*Xuehai He, Jian Zheng, Jacob Zhiyuan Fang, Robinson Piramuthu, Mohit Bansal, Vicente Ordonez, Gunnar A Sigurdsson, Nanyun Peng, Xin Eric Wang.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2405.04834)]\n\n**Sketch2Human: Deep Human Generation with Disentangled Geometry and Appearance Control.**\u003cbr\u003e\n*Linzi Qu, Jiaxiang Shang, Hui Ye, Xiaoguang Han, Hongbo Fu.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2404.15889)]\n\n**GLoD: Composing Global Contexts and Local Details in Image Generation.**\u003cbr\u003e\n*Moyuru Yamada.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2404.15447)]\n\n**LTOS: Layout-controllable Text-Object Synthesis via Adaptive Cross-attention Fusions.**\u003cbr\u003e\n*Xiaoran Zhao, Tianhao Wu, Yu Lai, Zhiliang Tian, Zhen Huang, Yahui Liu, Zejiang He, Dongsheng Li.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2404.13579)]\n\n**Ctrl-Adapter: An Efficient and Versatile Framework for Adapting Diverse Controls to Any Diffusion Model.**\u003cbr\u003e\n*Han Lin, Jaemin Cho, Abhay Zala, Mohit Bansal.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2404.09967)]\n\n**ControlNet++: Improving Conditional Controls with Efficient Consistency Feedback.**\u003cbr\u003e\n*Ming Li, Taojiannan Yang, Huafeng Kuang, Jie Wu, Zhaoning Wang, Xuefeng Xiao, Chen Chen.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2404.07987)]\n\n**ObjBlur: A Curriculum Learning Approach With Progressive Object-Level Blurring for Improved Layout-to-Image Generation.**\u003cbr\u003e\n*Stanislav Frolov, Brian B. Moser, Sebastian Palacio, Andreas Dengel.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2404.07564)]\n\n**SmartControl: Enhancing ControlNet for Handling Rough Visual Conditions.**\u003cbr\u003e\n*Xiaoyu Liu, Yuxiang Wei, Ming Liu, Xianhui Lin, Peiran Ren, Xuansong Xie, Wangmeng Zuo.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2404.06451)]\n\n**U-Sketch: An Efficient Approach for Sketch to Image Diffusion Models.**\u003cbr\u003e\n*Ilias Mitsouras, Eleftherios Tsonis, Paraskevi Tzouveli, Athanasios Voulodimos.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2403.18425)]\n\n**ECNet: Effective Controllable Text-to-Image Diffusion Models.**\u003cbr\u003e\n*Sicheng Li, Keqiang Sun, Zhixin Lai, Xiaoshi Wu, Feng Qiu, Haoran Xie, Kazunori Miyata, Hongsheng Li.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2403.18417)]\n\n**Be Yourself: Bounded Attention for Multi-Subject Text-to-Image Generation.**\u003cbr\u003e\n*Omer Dahary, Or Patashnik, Kfir Aberman, Daniel Cohen-Or.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2403.16990)]\n\n**Diffusion-based Aesthetic QR Code Generation via Scanning-Robust Perceptual Guidance.**\u003cbr\u003e\n*Jia-Wei Liao, Winston Wang, Tzu-Sian Wang, Li-Xuan Peng, Cheng-Fu Chou, Jun-Cheng Chen.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2403.15878)]\n\n**ReGround: Improving Textual and Spatial Grounding at No Cost.**\u003cbr\u003e\n*Yuseung Lee, Minhyuk Sung.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2403.13589)]\n\n**DetDiffusion: Synergizing Generative and Perceptive Models for Enhanced Data Generation and Perception.**\u003cbr\u003e\n*Yibo Wang, Ruiyuan Gao, Kai Chen, Kaiqiang Zhou, Yingjie Cai, Lanqing Hong, Zhenguo Li, Lihui Jiang, Dit-Yan Yeung, Qiang Xu, Kai Zhang.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2403.13304)]\n\n**Consistency Models Improve Diffusion Inverse Solvers.**\u003cbr\u003e\n*Tongda Xu, Ziran Zhu, Dailan He, Yuanyuan Wang, Ming Sun, Ning Li, Hongwei Qin, Yan Wang, Jingjing Liu, Ya-Qin Zhang.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2403.12063)]\n\n**One-Step Image Translation with Text-to-Image Models.**\u003cbr\u003e\n*Gaurav Parmar, Taesung Park, Srinivasa Narasimhan, Jun-Yan Zhu.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2403.12036)]\n\n**It's All About Your Sketch: Democratising Sketch Control in Diffusion Models.**\u003cbr\u003e\n*Subhadeep Koley, Ayan Kumar Bhunia, Deeptanshu Sekhri, Aneeshan Sain, Pinaki Nath Chowdhury, Tao Xiang, Yi-Zhe Song.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2403.07234)]\n\n**NoiseCollage: A Layout-Aware Text-to-Image Diffusion Model Based on Noise Cropping and Merging.**\u003cbr\u003e\n*Takahiro Shirakawa, Seiichi Uchida.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2403.03485)]\n\n**PLACE: Adaptive Layout-Semantic Fusion for Semantic Image Synthesis.**\u003cbr\u003e\n*Zhengyao Lv, Yuxiang Wei, Wangmeng Zuo, Kwan-Yee K. Wong.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2403.01852)]\n\n**Box It to Bind It: Unified Layout Control and Attribute Binding in T2I Diffusion Models.**\u003cbr\u003e\n*Ashkan Taghipour, Morteza Ghahremani, Mohammed Bennamoun, Aref Miri Rekavandi, Hamid Laga, Farid Boussaid.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2402.17910)]\n\n**CustomSketching: Sketch Concept Extraction for Sketch-based Image Synthesis and Editing.**\u003cbr\u003e\n*Chufeng Xiao, Hongbo Fu.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2402.17624)]\n\n**Layout-to-Image Generation with Localized Descriptions using ControlNet with Cross-Attention Control.**\u003cbr\u003e\n*Denis Lukovnikov, Asja Fischer.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2402.13404)]\n\n**MIGC: Multi-Instance Generation Controller for Text-to-Image Synthesis.**\u003cbr\u003e\n*Dewei Zhou, You Li, Fan Ma, Xiaoting Zhang, Yi Yang.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2402.05408)]\n\n**InstanceDiffusion: Instance-level Control for Image Generation.**\u003cbr\u003e\n*Xudong Wang, Trevor Darrell, Sai Saketh Rambhatla, Rohit Girdhar, Ishan Misra.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2402.03290)]\n\n**Adversarial Supervision Makes Layout-to-Image Diffusion Models Thrive.**\u003cbr\u003e\n*Yumeng Li, Margret Keuper, Dan Zhang, Anna Khoreva.*\u003cbr\u003e\nICLR 2024. [[PDF](https://arxiv.org/abs/2401.08815)]\n\n**Towards Flexible, Scalable, and Adaptive Multi-Modal Conditioned Face Synthesis.**\u003cbr\u003e\n*Jingjing Ren, Cheng Xu, Haoyu Chen, Xinran Qin, Chongyi Li, Lei Zhu.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2312.16274)]\n\n**SCEdit: Efficient and Controllable Image Diffusion Generation via Skip Connection Editing.**\u003cbr\u003e\n*Zeyinzi Jiang, Chaojie Mao, Yulin Pan, Zhen Han, Jingfeng Zhang.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2312.11392)]\n\n**Local Conditional Controlling for Text-to-Image Diffusion Models.**\u003cbr\u003e\n*Yibo Zhao, Liang Peng, Yang Yang, Zekai Luo, Hengjia Li, Yao Chen, Wei Zhao, qinglin lu, Boxi Wu, Wei Liu.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2312.08768)]\n\n**FreeControl: Training-Free Spatial Control of Any Text-to-Image Diffusion Model with Any Condition.**\u003cbr\u003e\n*Sicheng Mo, Fangzhou Mu, Kuan Heng Lin, Yanli Liu, Bochen Guan, Yin Li, Bolei Zhou.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2312.07536)]\n\n**InteractDiffusion: Interaction Control in Text-to-Image Diffusion Models.**\u003cbr\u003e\n*Jiun Tian Hoe, Xudong Jiang, Chee Seng Chan, Yap-Peng Tan, Weipeng Hu.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2312.05849)]\n\n**DemoCaricature: Democratising Caricature Generation with a Rough Sketch.**\u003cbr\u003e\n*Dar-Yen Chen, Subhadeep Koley, Aneeshan Sain, Pinaki Nath Chowdhury, Tao Xiang, Ayan Kumar Bhunia, Yi-Zhe Song.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2312.04364)]\n\n**LOOSE CONTROL: Lifting ControlNet for Generalized Depth Conditioning.**\u003cbr\u003e\n*Shariq Farooq Bhat, Niloy J. Mitra, Peter Wonka.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2312.03079)]\n\n**Layered Rendering Diffusion Model for Zero-Shot Guided Image Synthesis.**\u003cbr\u003e\n*Zipeng Qi, Guoxi Huang, Zebin Huang, Qin Guo, Jinwen Chen, Junyu Han, Jian Wang, Gang Zhang, Lufei Liu, Errui Ding, Jingdong Wang.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2311.18435)]\n\n**AnyLens: A Generative Diffusion Model with Any Rendering Lens.**\u003cbr\u003e\n*Andrey Voynov, Amir Hertz, Moab Arar, Shlomi Fruchter, Daniel Cohen-Or.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2311.17609)]\n\n**LoCo: Locally Constrained Training-Free Layout-to-Image Synthesis.**\u003cbr\u003e\n*Peiang Zhao, Han Li, Ruiyang Jin, S. Kevin Zhou.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2311.12342)]\n\n**An Image is Worth Multiple Words- Multi-attribute Inversion for Constrained Text-to-Image Synthesis.**\u003cbr\u003e\n*Aishwarya Agarwal, Srikrishna Karanam, Tripti Shukla, Balaji Vasan Srinivasan.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2311.11919)]\n\n**Enhancing Object Coherence in Layout-to-Image Synthesis.**\u003cbr\u003e\n*Yibin Wang, Weizhong Zhang, Jianwei Zheng, Cheng Jin.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2311.10522)]\n\n**R\u0026B: REGION AND BOUNDARY AWARE ZERO-SHOT\nGROUNDED TEXT-TO-IMAGE GENERATION.**\u003cbr\u003e\n*Jiayu Xiao, Henglei Lv, Liang Li, Shuhui Wang, Qingming Huang.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2310.08872)]\n\n**HyperHuman- Hyper-Realistic Human Generation with Latent Structural  Diffusion.**\u003cbr\u003e\n*Xian Liu, Jian Ren, Aliaksandr Siarohin, Ivan Skorokhodov, Yanyu Li, Dahua Lin, Xihui Liu, Ziwei Liu, Sergey Tulyakov.*\u003cbr\u003e\nICLR 2024. [[PDF](https://arxiv.org/abs/2310.08579)]\n\n**JointNet: Extending Text-to-Image Diffusion for Dense Distribution Modeling.**\u003cbr\u003e\n*Jingyang Zhang, Shiwei Li, Yuanxun Lu, Tian Fang, David McKinnon, Yanghai Tsin, Long Quan, Yao Yao.*\u003cbr\u003e\nICLR 2024. [[PDF](https://arxiv.org/abs/2310.06347)]\n\n**Dense Text-to-Image Generation with Attention Modulation.**\u003cbr\u003e\n*Yunji Kim, Jiyoung Lee, Jin-Hwa Kim, Jung-Woo Ha, Jun-Yan Zhu.*\u003cbr\u003e\nICCV 2023. [[PDF](https://arxiv.org/abs/2308.12964)]\n\n**SSMG: Spatial-Semantic Map Guided Diffusion Model for Free-form Layout-to-Image Generation.**\u003cbr\u003e\n*Chengyou Jia, Minnan Luo, Zhuohang Dang, Guang Dai, Xiaojun Chang, Mengmeng Wang, Jingdong Wang.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2308.10156)]\n\n**Localized Text-to-Image Generation For Free via Cross Attention Control.**\u003cbr\u003e\n*Yutong He, Ruslan Salakhutdinov, J. Zico Kolter.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2306.14636)]\n\n**Zero-shot spatial layout conditioning for text-to-image diffusion models.**\u003cbr\u003e\n*Guillaume Couairon, Marlène Careil, Matthieu Cord, Stéphane Lathuilière, Jakob Verbeek.*\u003cbr\u003e\nICCV 2023. [[PDF](https://arxiv.org/abs/2306.13754)]\n\n**Grounded Text-to-Image Synthesis with Attention Refocusing.**\u003cbr\u003e\n*Quynh Phung, Songwei Ge, Jia-Bin Huang.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2306.05427)]\n\n**GeoDiffusion: Text-Prompted Geometric Control for Object Detection Data Generation.**\u003cbr\u003e\n*Kai Chen, Enze Xie, Zhe Chen, Yibo Wang, Lanqing Hong, Zhenguo Li, Dit-Yan Yeung.*\u003cbr\u003e\nICLR 2024. [[PDF](https://arxiv.org/abs/2306.04607)]\n\n**Conditioning Diffusion Models via Attributes and Semantic Masks for Face Generation.**\u003cbr\u003e\n*Nico Giambi, Giuseppe Lisanti.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2306.00914)]\n\n**Late-Constraint Diffusion Guidance for Controllable Image Synthesis.**\u003cbr\u003e\n*Chang Liu, Dong Liu.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2305.11520)]\n\n**HumanSD: A Native Skeleton-Guided Diffusion Model for Human Image Generation.**\u003cbr\u003e\n*Xuan Ju, Ailing Zeng, Chenchen Zhao, Jianan Wang, Lei Zhang, Qiang Xu.*\u003cbr\u003e\nICCV 2023. [[PDF](https://arxiv.org/abs/2304.04269)]\n\n**LayoutDiffusion: Controllable Diffusion Model for\nLayout-to-image Generation.**\u003cbr\u003e\n*Guangcong Zheng, Xianpan Zhou, Xuewei Li, Zhongang Qi, Ying Shan, Xi Li.*\u003cbr\u003e\nCVPR 2023. [[PDF](https://arxiv.org/abs/2303.17189)]\n\n**Freestyle Layout-to-Image Synthesis.**\u003cbr\u003e\n*Han Xue, Zhiwu Huang, Qianru Sun, Li Song, Wenjun Zhang.*\u003cbr\u003e\nCVPR 2023. [[PDF](https://arxiv.org/abs/2303.14412)]\n\n**FreeDoM: Training-Free Energy-Guided Conditional Diffusion Model.**\u003cbr\u003e\n*Jiwen Yu, Yinhuai Wang, Chen Zhao, Bernard Ghanem, Jian Zhang.*\u003cbr\u003e\nICCV 2023. [[PDF](https://arxiv.org/abs/2303.09833)]\n\n**Modulating Pretrained Diffusion Models for Multimodal Image Synthesis.**\u003cbr\u003e\n*Cusuh Ham, James Hays, Jingwan Lu, Krishna Kumar Singh, Zhifei Zhang, Tobias Hinz.*\u003cbr\u003e\nSIGGRAPH 2023. [[PDF](https://arxiv.org/abs/2302.12764)]\n\n**LayoutDiffuse: Adapting Foundational Diffusion Models for Layout-to-Image Generation.**\u003cbr\u003e\n*Jiaxin Cheng, Xiao Liang, Xingjian Shi, Tong He, Tianjun Xiao, Mu Li.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2302.08908)]\n\n**Universal Guidance for Diffusion Models.**\u003cbr\u003e\n*Arpit Bansal, Hong-Min Chu, Avi Schwarzschild, Soumyadip Sengupta, Micah Goldblum, Jonas Geiping, Tom Goldstein.*\u003cbr\u003e\nCVPRW 2023. [[PDF](https://arxiv.org/abs/2302.07121)]\n\n**GLIGEN: Open-Set Grounded Text-to-Image Generation.**\u003cbr\u003e\n*Yuheng Li, Haotian Liu, Qingyang Wu, Fangzhou Mu, Jianwei Yang, Jianfeng Gao, Chunyuan Li, Yong Jae Lee.*\u003cbr\u003e\nCVPR 2023. [[PDF](https://arxiv.org/abs/2301.07093)]\n\n**SpaText: Spatio-Textual Representation for Controllable Image Generation.**\u003cbr\u003e\n*Omri Avrahami, Thomas Hayes, Oran Gafni, Sonal Gupta, Yaniv Taigman, Devi Parikh, Dani Lischinski, Ohad Fried, Xi Yin.*\u003cbr\u003e\nCVPR 2023. [[PDF](https://arxiv.org/abs/2211.14305)]\n\n**Sketch-Guided Text-to-Image Diffusion Models.**\u003cbr\u003e\n*Andrey Voynov, Kfir Aberman, Daniel Cohen-Or.*\u003cbr\u003e\nSIGGRAPH 2023. [[PDF](https://arxiv.org/abs/2211.13752)]\n\n**eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers.**\u003cbr\u003e\n*Yogesh Balaji, Seungjun Nah, Xun Huang, Arash Vahdat, Jiaming Song, Qinsheng Zhang, Karsten Kreis, Miika Aittala, Timo Aila, Samuli Laine, Bryan Catanzaro, Tero Karras, Ming-Yu Liu.*\u003cbr\u003e\narXiv 2022. [[PDF](https://arxiv.org/abs/2211.01324)]\n\n\n### 🍿Advanced Text-Conditioned Generation\n**An Empirical Study and Analysis of Text-to-Image Generation Using Large Language Model-Powered Textual Representation.**\u003cbr\u003e\n*Zhiyu Tan, Mengping Yang, Luozheng Qin, Hao Yang, Ye Qian, Qiang Zhou, Cheng Zhang, Hao Li.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2405.12914)]\n\n**On Mechanistic Knowledge Localization in Text-to-Image Generative Models.**\u003cbr\u003e\n*Samyadeep Basu, Keivan Rezaei, Priyatham Kattakinda, Ryan Rossi, Cherry Zhao, Vlad Morariu, Varun Manjunatha, Soheil Feizi.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2405.01008)]\n\n**Object-Attribute Binding in Text-to-Image Generation: Evaluation and Control.**\u003cbr\u003e\n*Maria Mihaela Trusca, Wolf Nuyts, Jonathan Thomm, Robert Honig, Thomas Hofmann, Tinne Tuytelaars, Marie-Francine Moens.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2404.13766)]\n\n**Prompt Optimizer of Text-to-Image Diffusion Models for Abstract Concept Understanding.**\u003cbr\u003e\n*Zezhong Fan, Xiaohan Li, Chenhao Fang, Topojoy Biswas, Kaushiki Nag, Jianpeng Xu, Kannan Achan.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2404.11589)]\n\n**Object-Conditioned Energy-Based Attention Map Alignment in Text-to-Image Diffusion Models.**\u003cbr\u003e\n*Yasi Zhang, Peiyu Yu, Ying Nian Wu.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2404.07389)]\n\n**InitNO: Boosting Text-to-Image Diffusion Models via Initial Noise Optimization.**\u003cbr\u003e\n*Xiefan Guo, Jinlin Liu, Miaomiao Cui, Jiankai Li, Hongyu Yang, Di Huang.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2404.04650)]\n\n**CoMat: Aligning Text-to-Image Diffusion Model with Image-to-Text Concept Matching.**\u003cbr\u003e\n*Dongzhi Jiang, Guanglu Song, Xiaoshi Wu, Renrui Zhang, Dazhong Shen, Zhuofan Zong, Yu Liu, Hongsheng Li.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2404.03653)]\n\n**Getting it Right: Improving Spatial Consistency in Text-to-Image Models.**\u003cbr\u003e\n*Agneet Chatterjee, Gabriela Ben Melech Stan, Estelle Aflalo, Sayak Paul, Dhruba Ghosh, Tejas Gokhale, Ludwig Schmidt, Hannaneh Hajishirzi, Vasudev Lal, Chitta Baral, Yezhou Yang.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2404.01197)]\n\n**Relation Rectification in Diffusion Model.**\u003cbr\u003e\n*Yinwei Wu, Xingyi Yang, Xinchao Wang.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2403.20249)]\n\n**VersaT2I: Improving Text-to-Image Models with Versatile Reward.**\u003cbr\u003e\n*Jianshu Guo, Wenhao Chai, Jie Deng, Hsiang-Wei Huang, Tian Ye, Yichen Xu, Jiawei Zhang, Jenq-Neng Hwang, Gaoang Wang.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2403.18493)]\n\n**Improving Text-to-Image Consistency via Automatic Prompt Optimization.**\u003cbr\u003e\n*Oscar Mañas, Pietro Astolfi, Melissa Hall, Candace Ross, Jack Urbanek, Adina Williams, Aishwarya Agrawal, Adriana Romero-Soriano, Michal Drozdzal.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2403.17804)]\n\n**Isolated Diffusion: Optimizing Multi-Concept Text-to-Image Generation Training-Freely with Isolated Diffusion Guidance.**\u003cbr\u003e\n*Jingyuan Zhu, Huimin Ma, Jiansheng Chen, Jian Yuan.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2403.16954)]\n\n**Glyph-ByT5: A Customized Text Encoder for Accurate Visual Text Rendering.**\u003cbr\u003e\n*Zeyu Liu, Weicong Liang, Zhanhao Liang, Chong Luo, Ji Li, Gao Huang, Yuhui Yuan.*\u003cbr\u003e\nECCV 2024. [[PDF](https://arxiv.org/abs/2403.09622)]\n\n**DivCon: Divide and Conquer for Progressive Text-to-Image Generation.**\u003cbr\u003e\n*Yuhao Jia, Wenhan Tan.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2403.06400)]\n\n**Enhancing Semantic Fidelity in Text-to-Image Synthesis: Attention Regulation in Diffusion Models.**\u003cbr\u003e\n*Yang Zhang, Teoh Tze Tzun, Lim Wei Hern, Tiviatis Sim, Kenji Kawaguchi.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2403.06381)]\n\n**ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignment.**\u003cbr\u003e\n*Xiwei Hu, Rui Wang, Yixiao Fang, Bin Fu, Pei Cheng, Gang Yu.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2403.05135)]\n\n**MuLan: Multimodal-LLM Agent for Progressive Multi-Object Diffusion.**\u003cbr\u003e\n*Sen Li, Ruochen Wang, Cho-Jui Hsieh, Minhao Cheng, Tianyi Zhou.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2402.12741)]\n\n**Learning Continuous 3D Words for Text-to-Image Generation.**\u003cbr\u003e\n*Ta-Ying Cheng, Matheus Gadelha, Thibault Groueix, Matthew Fisher, Radomir Mech, Andrew Markham, Niki Trigoni.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2402.08654)]\n\n**Seek for Incantations: Towards Accurate Text-to-Image Diffusion Synthesis through Prompt Engineering.**\u003cbr\u003e\n*Chang Yu, Junran Peng, Xiangyu Zhu, Zhaoxiang Zhang, Qi Tian, Zhen Lei.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2401.06345)]\n\n**EmoGen: Emotional Image Content Generation with Text-to-Image Diffusion Models.**\u003cbr\u003e\n*Jingyuan Yang, Jiawei Feng, Hui Huang.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2401.04608)]\n\n**TokenCompose: Grounding Diffusion with Token-level Supervision.**\u003cbr\u003e\n*Zirui Wang, Zhizhou Sha, Zheng Ding, Yilin Wang, Zhuowen Tu.*\u003cbr\u003e\nCVPR 2024. [[PDF](https://arxiv.org/abs/2312.03626)]\n\n**PEA-Diffusion: Parameter-Efficient Adapter with Knowledge Distillation  in non-English Text-to-Image Generation.**\u003cbr\u003e\n*Jian Ma, Chen Chen, Qingsong Xie, Haonan Lu.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2311.17086)]\n\n**Paragraph-to-Image Generation with Information-Enriched Diffusion Model.**\u003cbr\u003e\n*Weijia Wu, Zhuang Li, Yefei He, Mike Zheng Shou, Chunhua Shen, Lele Cheng, Yan Li, Tingting Gao, Di Zhang, Zhongyuan Wang.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2311.14284)]\n\n**Tailored Visions: Enhancing Text-to-Image Generation with Personalized Prompt Rewriting.**\u003cbr\u003e\n*Zijie Chen, Lichao Zhang, Fangsheng Weng, Lili Pan, Zhenzhong Lan.*\u003cbr\u003e\nCVPR 2024. [[PDF](https://arxiv.org/abs/2310.08129)]\n\n**Divide \u0026 Bind Your Attention for Improved Generative Semantic Nursing.**\u003cbr\u003e\n*Yumeng Li, Margret Keuper, Dan Zhang, Anna Khoreva.*\u003cbr\u003e\nBMVC 2023. [[PDF](https://arxiv.org/abs/2307.10864)]\n\n**Linguistic Binding in Diffusion Models-  Enhancing Attribute Correspondence  through Attention Map Alignment.**\u003cbr\u003e\n*Royi Rassin, Eran Hirsch, Daniel Glickman, Shauli Ravfogel, Yoav Goldberg, Gal Chechik.*\u003cbr\u003e\nNeurIPS 2023. [[PDF](https://arxiv.org/abs/2306.08877)]\n\n**Expressive Text-to-Image Generation with Rich Text.**\u003cbr\u003e\n*Songwei Ge, Taesung Park, Jun-Yan Zhu, Jia-Bin Huang.*\u003cbr\u003e\nICCV 2023. [[PDF](https://arxiv.org/abs/2304.06720)]\n\n**GlueGen: Plug and Play Multi-modal Encoders for X-to-image Generation.**\u003cbr\u003e\n*Can Qin, Ning Yu, Chen Xing, Shu Zhang, Zeyuan Chen, Stefano Ermon, Yun Fu, Caiming Xiong, Ran Xu.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2303.10056)]\n\n**Attend-and-Excite- Attention-Based Semantic Guidance for Text-to-Image  Diffusion Models.**\u003cbr\u003e\n*Hila Chefer, Yuval Alaluf, Yael Vinker, Lior Wolf, Daniel Cohen-Or.*\u003cbr\u003e\nTOG 2023. [[PDF](https://arxiv.org/abs/2301.13826)]\n\n**TRAINING-FREE STRUCTURED DIFFUSION GUIDANCE FOR COMPOSITIONAL TEXT-TO-I MAGE SYNTHESIS.**\u003cbr\u003e\n*Weixi Feng, Xuehai He, Tsu-Jui Fu, Varun Jampani, Arjun Akula, Pradyumna Narayana, Sugato Basu, Xin Eric Wang, William Yang Wang.*\u003cbr\u003e\narXiv 2022. [[PDF](https://arxiv.org/abs/2212.05032)]\n\n\n### 🧁In-Context Generation\n**Analogist: Out-of-the-box Visual In-Context Learning with Image Diffusion Model.**\u003cbr\u003e\n*Zheng Gu, Shiyuan Yang, Jing Liao, Jing Huo, Yang Gao.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2405.10316)]\n\n**Improving In-Context Learning in Diffusion Models with Visual Context-Modulated Prompts.**\u003cbr\u003e\n*Tianqi Chen, Yongfei Liu, Zhendong Wang, Jianbo Yuan, Quanzeng You, Hongxia Yang, Mingyuan Zhou.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2312.01408)]\n\n**In-Context Learning Unlocked for Diffusion Models.**\u003cbr\u003e\n*Zhendong Wang, Yifan Jiang, Yadong Lu, Yelong Shen, Pengcheng He, Weizhu Chen, Zhangyang Wang, Mingyuan Zhou.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2305.01115)]\n\n\n### 🍹Brain-Guided Generation\n**BrainVis: Exploring the Bridge between Brain and Visual Signals via Image Reconstruction.**\u003cbr\u003e\n*Honghao Fu, Zhiqi Shen, Jing Jih Chin, Hao Wang.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2312.14871)]\n\n**DreamDiffusion: Generating High-Quality Images from Brain EEG Signals.**\u003cbr\u003e\n*Yunpeng Bai, Xintao Wang, Yan-pei Cao, Yixiao Ge, Chun Yuan, Ying Shan.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2306.16934)]\n\n**Natural Image Reconstruction from fMRI Based on Self-supervised Representation Learning and Latent Diffusion Model.**\u003cbr\u003e\n*Pengyu Ni, Yifeng Zhang.*\u003cbr\u003e\nACM MM 2023. \n\n**MindDiffuser: Controlled Image Reconstruction from Human Brain Activity with Semantic and Structural Diffusion.**\u003cbr\u003e\n*Yizhuo Lu, Changde Du, Dianpeng Wang, Huiguang He.*\u003cbr\u003e\nACM MM 2023. [[PDF](https://arxiv.org/abs/2303.14139)]\n\n**Natural scene reconstruction from fMRI signals using generative latent diffusion.**\u003cbr\u003e\n*Furkan Ozcelik, Rufin VanRullen.*\u003cbr\u003e\nScientific Reports 2023. [[PDF](https://arxiv.org/abs/2303.05334)]\n\n**High-Resolution Image Reconstruction With Latent Diffusion Models From Human Brain Activity.**\u003cbr\u003e\n*Yu Takagi, Shinji Nishimoto.*\u003cbr\u003e\nCVPR 2023. \n\n**Seeing Beyond the Brain: Conditional Diffusion Model with Sparse Masked Modeling for Vision Decoding.**\u003cbr\u003e\n*Zijiao Chen, Jiaxin Qing, Tiange Xiang, Wan Lin Yue, Juan Helen Zhou.*\u003cbr\u003e\nCVPR 2023. [[PDF](https://arxiv.org/abs/2211.06956)]\n\n\n### 🍭Sound-Guided Generation\n**SonicDiffusion: Audio-Driven Image Generation and Editing with  Pretrained Diffusion Models.**\u003cbr\u003e\n*Burak Can Biner, Farrin Marouf Sofian, Umur Berkay Karakaş, Duygu Ceylan, Erkut Erdem, Aykut Erdem.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2405.00878)]\n\n**Align, Adapt and Inject: Sound-guided\nUnified Image Generation.**\u003cbr\u003e\n*Yue Yang, Kaipeng Zhang, Yuying Ge, Wenqi Shao, Zeyue Xue, Yu Qiao, Ping Luo.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2306.11504)]\n\n**GlueGen: Plug and Play Multi-modal Encoders for X-to-image Generation.**\u003cbr\u003e\n*Can Qin, Ning Yu, Chen Xing, Shu Zhang, Zeyuan Chen, Stefano Ermon, Yun Fu, Caiming Xiong, Ran Xu.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2303.10056)]\n\n\n### 🧉Text Rendering\n**\nGlyphDraw2: Automatic Generation of Complex Glyph Posters with Diffusion Models and Large Language Models.**\u003cbr\u003e\n*Jian Ma, Yonglin Deng, Chen Chen, Haonan Lu, Zhenyu Yang.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2407.02252)]\n\n**High Fidelity Scene Text Synthesis.**\u003cbr\u003e\n*Yibin Wang, Weizhong Zhang, Changhai Zhou, Cheng Jin.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2405.14701)]\n\n**CustomText: Customized Textual Image Generation using Diffusion Models.**\u003cbr\u003e\n*Shubham Paliwal, Arushi Jain, Monika Sharma, Vikram Jamwal, Lovekesh Vig.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2405.12531)]\n\n**Refining Text-to-Image Generation: Towards Accurate Training-Free Glyph-Enhanced Image Generation.**\u003cbr\u003e\n*Sanyam Lakhanpal, Shivang Chopra, Vinija Jain, Aman Chadha, Man Luo.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2403.16422)]\n\n**Typographic Text Generation with Off-the-Shelf Diffusion Model.**\u003cbr\u003e\n*KhayTze Peong, Seiichi Uchida, Daichi Haraguchi.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2402.14314)]\n\n**Brush Your Text: Synthesize Any Scene Text on Images via Diffusion Model.**\u003cbr\u003e\n*Lingjun Zhang, Xinyuan Chen, Yaohui Wang, Yue Lu, Yu Qiao.*\u003cbr\u003e\nAAAI 2024. [[PDF](https://arxiv.org/abs/2312.12232)]\n\n**UDiffText: A Unified Framework for High-quality Text Synthesis in Arbitrary Images via Character-aware Diffusion Models.**\u003cbr\u003e\n*Yiming Zhao, Zhouhui Lian.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2312.04884)]\n\n**TextDiffuser-2: Unleashing the Power of Language Models for Text Rendering.**\u003cbr\u003e\n*Jingye Chen, Yupan Huang, Tengchao Lv, Lei Cui, Qifeng Chen, Furu Wei.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2311.16465)]\n\n**AnyText: Multilingual Visual Text Generation And Editing.**\u003cbr\u003e\n*Yuxiang Tuo, Wangmeng Xiang, Jun-Yan He, Yifeng Geng, Xuansong Xie.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2311.03054)]\n\n**GlyphControl: Glyph Conditional Control for Visual Text Generation.**\u003cbr\u003e\n*Yukang Yang, Dongnan Gui, Yuhui Yuan, Weicong Liang, Haisong Ding, Han Hu, Kai Chen.*\u003cbr\u003e\nNeurIPS 2023. [[PDF](https://arxiv.org/abs/2305.18259)]\n\n**TextDiffuser: Diffusion Models as Text Painters.**\u003cbr\u003e\n*Jingye Chen, Yupan Huang, Tengchao Lv, Lei Cui, Qifeng Chen, Furu Wei.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2305.10855)]\n\n**GlyphDraw: Seamlessly Rendering Text with Intricate Spatial Structures in Text-to-Image.**\u003cbr\u003e\n*Jian Ma, Mingjun Zhao, Chen Chen, Ruichen Wang, Di Niu, Haonan Lu, Xiaodong Lin.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2303.17870)]\n\n**Character-Aware Models Improve Visual Text Rendering.**\u003cbr\u003e\n*Rosanne Liu, Dan Garrette, Chitwan Saharia, William Chan, Adam Roberts, Sharan Narang, Irina Blok, RJ Mical, Mohammad Norouzi, Noah Constant.*\u003cbr\u003e\nACL 2022. [[PDF](https://arxiv.org/abs/2212.10562)]\n\n\n## 🌕Generation with Multiple Conditions\n### 🌒Joint Training\n**FreeCustom: Tuning-Free Customized Image Generation for Multi-Concept Composition.**\u003cbr\u003e\n*Ganggui Ding, Canyu Zhao, Wen Wang, Zhen Yang, Zide Liu, Hao Chen, Chunhua Shen.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2405.13870)]\n\n**Non-confusing Generation of Customized Concepts in Diffusion Models.**\u003cbr\u003e\n*Wang Lin, Jingyuan Chen, Jiaxin Shi, Yichen Zhu, Chen Liang, Junzhong Miao, Tao Jin, Zhou Zhao, Fei Wu, Shuicheng Yan, Hanwang Zhang.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2405.06914)]\n\n**MaxFusion: Plug\u0026Play Multi-Modal Generation in Text-to-Image Diffusion Models.**\u003cbr\u003e\n*Nithin Gopalakrishnan Nair, Jeya Maria Jose Valanarasu, Vishal M Patel.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2404.09977)]\n\n**Identity Decoupling for Multi-Subject Personalization of Text-to-Image Models.**\u003cbr\u003e\n*Sangwon Jang, Jaehyeong Jo, Kimin Lee, Sung Ju Hwang.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2404.04243)]\n\n**Concept Weaver: Enabling Multi-Concept Fusion in Text-to-Image Models.**\u003cbr\u003e\n*Gihyun Kwon, Simon Jenni, Dingzeyu Li, Joon-Young Lee, Jong Chul Ye, Fabian Caba Heilbron.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2404.03913)]\n\n**Cocktail: Mixing Multi-Modality Controls for Text-Conditional Image Generation.**\u003cbr\u003e\n*Minghui Hu, Jianbin Zheng, Daqing Liu, Chuanxia Zheng, Chaoyue Wang, Dacheng Tao, Tat-Jen Cham.*\u003cbr\u003e\nNeurIPS 2023. [[PDF](https://arxiv.org/abs/2306.00964)]\n\n**FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attention.**\u003cbr\u003e\n*Guangxuan Xiao, Tianwei Yin, William T. Freeman, Frédo Durand, Song Han.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2305.10431)]\n\n**SVDiff- Compact Parameter Space for Diffusion Fine-Tuning.**\u003cbr\u003e\n*Ligong Han, Yinxiao Li, Han Zhang, Peyman Milanfar, Dimitris Metaxas, Feng Yang.*\u003cbr\u003e\nICCV 2023. [[PDF](https://arxiv.org/abs/2303.11305)]\n\n**Composer: Creative and controllable image synthesis with composable conditions.**\u003cbr\u003e\n*Lianghua Huang, Di Chen, Yu Liu, Yujun Shen, Deli Zhao, Jingren Zhou.*\u003cbr\u003e\nICML 2023. [[PDF](https://arxiv.org/abs/2302.09778)]\n\n\n### 🌓Continual Learning\n**MuseumMaker: Continual Style Customization without Catastrophic Forgetting.**\u003cbr\u003e\n*Chenxi Liu, Gan Sun, Wenqi Liang, Jiahua Dong, Can Qin, Yang Cong.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2404.16612)]\n\n**Diffscaler: Enhancing the Generative Prowess of Diffusion Transformers.**\u003cbr\u003e\n*Nithin Gopalakrishnan Nair, Jeya Maria Jose Valanarasu, Vishal M. Patel.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2404.09976)]\n\n**Continual Diffusion with STAMINA: STack-And-Mask INcremental Adapters.**\u003cbr\u003e\n*James Seale Smith, Yen-Chang Hsu, Zsolt Kira, Yilin Shen, Hongxia Jin.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2311.18763)]\n\n**Create Your World: Lifelong Text-to-Image\nDiffusion.**\u003cbr\u003e\n*Gan Sun, Wenqi Liang, Jiahua Dong, Jun Li, Zhengming Ding, Yang Cong.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2309.04430)]\n\n**Continual Diffusion- Continual Customization of  Text-to-Image Diffusion with C-LoRA.**\u003cbr\u003e\n*James Seale Smith, Yen-Chang Hsu, Lingyu Zhang, Ting Hua, Zsolt Kira, Yilin Shen, Hongxia Jin.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2304.06027)]\n\n\n### 🌔Weight Fusion\n**CLoRA: A Contrastive Approach to Compose Multiple LoRA Models.**\u003cbr\u003e\n*Tuna Han Salih Meral, Enis Simsar, Federico Tombari, Pinar Yanardag.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2403.19776)]\n\n**Break-for-Make: Modular Low-Rank Adaptations for Composable Content-Style Customization.**\u003cbr\u003e\n*Yu Xu, Fan Tang, Juan Cao, Yuxin Zhang, Oliver Deussen, Weiming Dong, Jintao Li, Tong-Yee Lee.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2403.19456)]\n\n**LoRA-Composer: Leveraging Low-Rank Adaptation for Multi-Concept Customization in Training-Free Diffusion Models.**\u003cbr\u003e\n*Yang Yang, Wen Wang, Liang Peng, Chaotian Song, Yao Chen, Hengjia Li, Xiaolong Yang, Qinglin Lu, Deng Cai, Boxi Wu, Wei Liu.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2403.11627)]\n\n**Orthogonal Adaptation for Modular Customization of Diffusion Models.**\u003cbr\u003e\n*Ryan Po, Guandao Yang, Kfir Aberman, Gordon Wetzstein.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2312.02432)]\n\n**ZipLoRA- Any Subject in Any Style by Effectively Merging LoRAs.**\u003cbr\u003e\n*Viraj Shah, Nataniel Ruiz, Forrester Cole, Erika Lu, Svetlana Lazebnik, Yuanzhen Li, Varun Jampani.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2311.13600)]\n\n**Mix-of-Show- Decentralized Low-Rank Adaptation for  Multi-Concept Customization of Diffusion Models.**\u003cbr\u003e\n*Yuchao Gu, Xintao Wang, Jay Zhangjie Wu, Yujun Shi, Yunpeng Chen, Zihan Fan, Wuyou Xiao, Rui Zhao, Shuning Chang, Weijia Wu, Yixiao Ge, Ying Shan, Mike Zheng Shou.*\u003cbr\u003e\nNeurIPS 2023. [[PDF](https://arxiv.org/abs/2305.18292)]\n\n**Cones- Concept Neurons in Diffusion Models for Customized Generation.**\u003cbr\u003e\n*Zhiheng Liu, Ruili Feng, Kai Zhu, Yifei Zhang, Kecheng Zheng, Yu Liu, Deli Zhao, Jingren Zhou, Yang Cao.*\u003cbr\u003e\nICML 2023. [[PDF](https://arxiv.org/abs/2303.05125)]\n\n**Multi-Concept Customization of Text-to-Image Diffusion.**\u003cbr\u003e\n*Nupur Kumari, Bingliang Zhang, Richard Zhang, Eli Shechtman, Jun-Yan Zhu.*\u003cbr\u003e\nCVPR 2023. [[PDF](https://arxiv.org/abs/2212.04488)]\n\n\n### 🌖Attention-based Integration\n**Training-free Subject-Enhanced Attention Guidance for Compositional Text-to-image Generation.**\u003cbr\u003e\n*Shengyuan Liu, Bo Wang, Ye Ma, Te Yang, Xipeng Cao, Quan Chen, Han Li, Di Dong, Peng Jiang.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2405.06948)]\n\n**MultiBooth: Towards Generating All Your Concepts in an Image from Text.**\u003cbr\u003e\n*Chenyang Zhu, Kai Li, Yue Ma, Chunming He, Li Xiu.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2404.14239)]\n\n**Cones 2- Customizable Image Synthesis  with Multiple Subjects .**\u003cbr\u003e\n*Zhiheng Liu, Yifei Zhang, Yujun Shen, Kecheng Zheng, Kai Zhu, Ruili Feng, Yu Liu, Deli Zhao, Jingren Zhou, Yang Cao.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2305.19327)]\n\n**Mix-of-Show- Decentralized Low-Rank Adaptation for  Multi-Concept Customization of Diffusion Models.**\u003cbr\u003e\n*Yuchao Gu, Xintao Wang, Jay Zhangjie Wu, Yujun Shi, Yunpeng Chen, Zihan Fan, Wuyou Xiao, Rui Zhao, Shuning Chang, Weijia Wu, Yixiao Ge, Ying Shan, Mike Zheng Shou.*\u003cbr\u003e\nNeurIPS 2023. [[PDF](https://arxiv.org/abs/2305.18292)]\n\n\n### 🌗Guidance Composition\n**DreamWalk: Style Space Exploration using Diffusion Guidance.**\u003cbr\u003e\n*Michelle Shu, Charles Herrmann, Richard Strong Bowen, Forrester Cole, Ramin Zabih.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2404.03145)]\n\n**Isolated Diffusion: Optimizing Multi-Concept Text-to-Image Generation Training-Freely with Isolated Diffusion Guidance.**\u003cbr\u003e\n*Jingyuan Zhu, Huimin Ma, Jiansheng Chen, Jian Yuan.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2403.16954)]\n\n**Concept-centric Personalization with Large-scale Diffusion Priors.**\u003cbr\u003e\n*Pu Cao, Lu Yang, Feng Zhou, Tianrui Huang, Qing Song.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2312.08195)]\n\n**High-fidelity Person-centric Subject-to-Image Synthesis.**\u003cbr\u003e\n*Yibin Wang, Weizhong Zhang, Jianwei Zheng, Cheng Jin.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2311.10329)]\n\n**Decompose and Realign: Tackling Condition Misalignment in Text-to-Image Diffusion Models.**\u003cbr\u003e\n*Luozhou Wang, Guibao Shen, Wenhang Ge, Guangyong Chen, Yijun Li, Ying-cong Chen.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2306.14408)]\n\n**Collaborative Diffusion for Multi-Modal Face Generation and Editing.**\u003cbr\u003e\n*Ziqi Huang, Kelvin C. K. Chan, Yuming Jiang, Ziwei Liu.*\u003cbr\u003e\nCVPR 2023. [[PDF](https://arxiv.org/abs/2304.10530)]\n\n\n## 🔥Universal Controllable Generation\n### 🌤Universal Conditional Score Prediction\n**Image Anything: Towards Reasoning-coherent and Training-free Multi-modal Image Generation.**\u003cbr\u003e\n*Yuanhuiyi Lyu, Xu Zheng, Lin Wang.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2401.17664)]\n\n**UNIMO-G: Unified Image Generation through Multimodal Conditional Diffusion.**\u003cbr\u003e\n*Wei Li, Xue Xu, Jiachen Liu, Xinyan Xiao.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2401.13388)]\n\n**Instruct-Imagen: Image Generation with Multi-modal Instruction.**\u003cbr\u003e\n*Hexiang Hu, Kelvin C. K. Chan, Yu-Chuan Su, Wenhu Chen, Yandong Li, Kihyuk Sohn, Yang Zhao, Xue Ben, Boqing Gong, William Cohen, Ming-Wei Chang, Xuhui Jia.*\u003cbr\u003e\narXiv 2024. [[PDF](https://arxiv.org/abs/2401.01952)]\n\n**Generative Multimodal Models are In-Context Learners.**\u003cbr\u003e\n*Quan Sun, Yufeng Cui, Xiaosong Zhang, Fan Zhang, Qiying Yu, Zhengxiong Luo, Yueze Wang, Yongming Rao, Jingjing Liu, Tiejun Huang, Xinlong Wang.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2312.13286)]\n\n**DiffBlender: Scalable and Composable Multimodal Text-to-Image Diffusion Models.**\u003cbr\u003e\n*Sungnyun Kim, Junsoo Lee, Kibeom Hong, Daesik Kim, Namhyuk Ahn.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2305.15194)]\n\n\n### 🌧Universal Condition-Guided Score Estimation\n**Towards Accurate Guided Diffusion Sampling through Symplectic Adjoint Method.**\u003cbr\u003e\n*Jiachun Pan, Hanshu Yan, Jun Hao Liew, Jiashi Feng, Vincent Y. F. Tan.*\u003cbr\u003e\narXiv 2023. [[PDF](https://arxiv.org/abs/2312.12030)]\n\n**FreeDoM: Training-Free Energy-Guided Conditional Diffusion Model.**\u003cbr\u003e\n*Jiwen Yu, Yinhuai Wang, Chen Zhao, Bernard Ghanem, Jian Zhang.*\u003cbr\u003e\nICCV 2023. [[PDF](https://arxiv.org/abs/2303.09833)]\n\n**Universal Guidance for Diffusion Models.**\u003cbr\u003e\n*Arpit Bansal, Hong-Min Chu, Avi Schwarzschild, Soumyadip Sengupta, Micah Goldblum, Jonas Geiping, Tom Goldstein.*\u003cbr\u003e\nCVPRW 2023. [[PDF](https://arxiv.org/abs/2302.07121)]\n\n\n\n\u003c!-- end --\u003e\n\n## Star History\n\n[![Star History Chart](https://api.star-history.com/svg?repos=PRIV-Creation/Awesome-Controllable-T2I-Diffusion-Models\u0026type=Date)](https://star-history.com/#PRIV-Creation/Awesome-Controllable-T2I-Diffusion-Models\u0026Date)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FPRIV-Creation%2FAwesome-Controllable-T2I-Diffusion-Models","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FPRIV-Creation%2FAwesome-Controllable-T2I-Diffusion-Models","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FPRIV-Creation%2FAwesome-Controllable-T2I-Diffusion-Models/lists"}