Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/Monalissaa/DisenDiff
[CVPR`2024, Oral] Attention Calibration for Disentangled Text-to-Image Personalization
https://github.com/Monalissaa/DisenDiff
Last synced: 4 months ago
JSON representation
[CVPR`2024, Oral] Attention Calibration for Disentangled Text-to-Image Personalization
- Host: GitHub
- URL: https://github.com/Monalissaa/DisenDiff
- Owner: Monalissaa
- License: mit
- Created: 2023-11-24T06:22:26.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2024-04-10T08:13:26.000Z (11 months ago)
- Last Synced: 2024-07-06T17:11:23.381Z (8 months ago)
- Language: Python
- Homepage:
- Size: 30.8 MB
- Stars: 70
- Watchers: 3
- Forks: 1
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# DisenDiff
This repository is the official implementation of [DisenDiff](https://arxiv.org/abs/2403.18551) [CVPR-2024 Oral Presentation].
> **Attention Calibration for Disentangled Text-to-Image Personalization**
> Yanbing Zhang, Mengping Yang, Qin Zhou, Zhe Wang
> [pdf](https://arxiv.org/abs/2403.18551)
## Datasets
The training images are located in `datasets/images`, the test prompts are located in `datasets/prompts`, and the processed images for evaluating image-alignment can be found in `datasets/data_eval`.
## Key modules
The crucial constraints for optimization are implemented in the function `p_losses` within `src/model.py`.
## Results
## Getting Started
```
conda env create -f environment.yml
conda activate ldm
git clone https://github.com/CompVis/stable-diffusion.git
```
## Fine-tuning
```
## run training
bash run.sh
## sample and evaluate
bash eval.sh
```
The `run.sh` and `eval.sh` scripts include several hyperparameters such as `classes` in the input image,`data_path`, `save_path`, training `caption`, random `seed`, and more. Please modify these executable files to suit your specific requirements.
## Contact Us
**Yanbing Zhang**: [[email protected]](mailto:[email protected])
**Mengping Yang**: [[email protected]](mailto:[email protected])
## BibTeX
```
@article{zhang2024attention,
title={Attention Calibration for Disentangled Text-to-Image Personalization},
author={Zhang, Yanbing and Yang, Mengping and Zhou, Qin and Wang, Zhe},
journal={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
year={2024}
}
```
## Acknowledgement
Our code is built upon the excellent codebase of [Custom-Diffusion](https://github.com/adobe-research/custom-diffusion), we thank a lot for their work.
We also kindly refer interesting researchers to these wonderful relted works:
* [DreamBooth](https://dreambooth.github.io/)
* [Break-A-Scene](https://omriavrahami.com/break-a-scene/)
* [Textual Inversion](https://textual-inversion.github.io/)
We also thank the anonymous reviewers for their valuable suggestions during the rebuttal, which greatly help us improve the paper.
## Disclaimer
This project is released for academic use. We disclaim responsibility for user-generated content. Users are solely liable for their actions. The project contributors are not legally affiliated with, nor accountable for, users' behaviors. Use the generative model responsibly, adhering to ethical and legal standards.