https://github.com/kodenii/ores

ORES: Open-vocabulary Responsible Visual Synthesis
https://github.com/kodenii/ores

Last synced: 5 days ago
JSON representation

ORES: Open-vocabulary Responsible Visual Synthesis

• Host: GitHub
• URL: https://github.com/kodenii/ores
• Owner: kodenii
• License: mit
• Created: 2023-08-15T08:57:49.000Z (about 1 year ago)
• Default Branch: main
• Last Pushed: 2023-12-12T11:10:48.000Z (11 months ago)
• Last Synced: 2024-08-01T18:37:27.550Z (3 months ago)
• Language: Python
• Homepage:
• Size: 5.63 MB
• Stars: 12
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

• awesome-diffusion-categorized - [Code

README

        
# ORES

[[Read Paper](https://arxiv.org/abs/2308.13785)]

Thanks for your interest in our work, ORES: Open-vocabulary Responsible Visual Synthesis. We release our code, checkpoint, and dataset here.

[News🤩] ORES has been accepted by AAAI 2024!

## Cite ORES!

```

@article{ni2023ores,

  title={ORES: Open-vocabulary Responsible Visual Synthesis},

  author={Ni, Minheng and Wu, Chenfei and Wang, Xiaodong and Yin, Shengming and Wang, Lijuan and Liu, Zicheng and Duan, Nan},

  journal={arXiv preprint arXiv:2308.13785},

  year={2023}

}

```

## ORES Samples

![image](img/vis-main.png)

![image](img/samples.png)

## Our FIN framework

### Environment

We follow Stable Diffusion to set up environment.yaml.

```

conda env create -f environment.yaml

conda activate ldm

```

### Quick Start

We design a WebUI for an out-of-the-box start. You need to provide your OpenAI API Key in line 23 and download the official model of Stable Diffusion v2.1 from this [url](https://huggingface.co/stabilityai/stable-diffusion-2-1/blob/main/v2-1_768-ema-pruned.ckpt) to ```TIN/checkpoints/v2-1_768-ema-pruned.ckpt```.

To start up WebUI, use

```

cd TIN

python ui.py

```

### Re-train Learnable Instruction

We also provide the training code for Learnable Instruction. You need to provide your OpenAI API Key in line 9.

To start training, use

```

cd Instruction

python instruction.py

```

## Dataset

We release this dataset under an MIT license. Refer to ```Dataset/ORES_data.txt``` for details.

## Reproducibility

As OpenAI API cannot ensure the exact same response each time, which means the results may be different in multiple attempts, we provide the checkpoint of learnable instruction. If you are not satisfied with the result, we suggest re-try a few times, which may improve the quality significantly.