https://github.com/Shentao-YANG/Dense_Reward_T2I
Source code for "A Dense Reward View on Aligning Text-to-Image Diffusion with Preference" (ICML'24).
https://github.com/Shentao-YANG/Dense_Reward_T2I
dense-reward-for-direct-preference-optimization preference-alignment text-to-image-generation
Last synced: 24 days ago
JSON representation
Source code for "A Dense Reward View on Aligning Text-to-Image Diffusion with Preference" (ICML'24).
- Host: GitHub
- URL: https://github.com/Shentao-YANG/Dense_Reward_T2I
- Owner: Shentao-YANG
- Created: 2024-02-07T23:44:26.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2024-05-09T02:39:49.000Z (about 1 year ago)
- Last Synced: 2024-08-03T01:12:00.598Z (9 months ago)
- Topics: dense-reward-for-direct-preference-optimization, preference-alignment, text-to-image-generation
- Language: Python
- Homepage:
- Size: 11.3 MB
- Stars: 22
- Watchers: 3
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
- awesome-RLHF - official
README
# A Dense Reward View on Aligning Text-to-Image Diffusion with Preference (ICML'24)
Source code for the single and multiple experiments in *A Dense Reward View on Aligning Text-to-Image Diffusion with Preference*.
[[Paper]](https://arxiv.org/abs/2402.08265).
Bibtex:
```angular2html
@inproceedings{
yang2024adensereward,
title={A Dense Reward View on Aligning Text-to-Image Diffusion with Preference},
author={Shentao Yang and Tianqi Chen and Mingyuan Zhou},
booktitle={Forty-first International Conference on Machine Learning},
year={2024},
url={https://openreview.net/forum?id=xVXnXk9I3I}
}
```
## Dependency
To install the required packages, please run the following command:
```angular2html
bash install_packages.sh
```
## Experiments
### Single Prompt Experiments
As a minimal example, our single prompt experiments can be run by the following command
```angular2html
accelerate launch train_t2i.py --expid="single"
```
We provide our checkpoints in `./ckpts`.
To evaluate our checkpoints, please use the following command
```angular2html
cd eval_ckpts
torchrun --nproc_per_node $NUM_GPUS_TO_USE --standalone main.py --type="both_seen_unseen" --eval_generated_imgs=1 --metrics="image_reward,aesthetic" --outdir="./outputs"
```
Note:
- `$NUM_GPUS_TO_USE` is the number of gpus you want to use.
- Set `--eval_generated_imgs=0` if evaluating the generated images is not needed, *i.e.* only want to generate images.
- Add the flag `--return_traj` if you want to store the generation trajectory corresponding to each image as well.
### Multiple Prompt Experiments
As a minimal example, our multiple prompt experiments can be run by the following command
```angular2html
accelerate launch train_t2i.py --single_flag=0 --expid="multiple"
```
The above command is a minimal example, please check `parse_args.py` for available flags.
*Declaimer: The HPSv2 train set has not been officially released at this moment. We are currently in the process of consulting with the HPSv2's authors on including those prompts in our repository. For now, we temporarily use the drawbench prompts as a substitution*
## Acknowledgement
This codebase builds on the following codebases:
* [**DPOK**](https://github.com/google-research/google-research/tree/master/dpok)
* [**ImageReward**](https://github.com/THUDM/ImageReward)
* [**HPSv2**](https://github.com/tgxs002/HPSv2)
* [**Hugging Face**](https://github.com/huggingface/diffusers/blob/v0.23.1/examples/text_to_image/train_text_to_image_lora.py)