https://github.com/0606zt/twindiffusion
[ECAI 2024] Official code for "TwinDiffusion: Enhancing Coherence and Efficiency in Panoramic Image Generation with Diffusion Models".
https://github.com/0606zt/twindiffusion
diffusion-models image-generation multidiffusion panorama-image twindiffusion
Last synced: about 2 months ago
JSON representation
[ECAI 2024] Official code for "TwinDiffusion: Enhancing Coherence and Efficiency in Panoramic Image Generation with Diffusion Models".
- Host: GitHub
- URL: https://github.com/0606zt/twindiffusion
- Owner: 0606zt
- Created: 2024-06-29T18:27:29.000Z (11 months ago)
- Default Branch: main
- Last Pushed: 2025-02-15T14:09:47.000Z (4 months ago)
- Last Synced: 2025-04-14T01:54:10.194Z (about 2 months ago)
- Topics: diffusion-models, image-generation, multidiffusion, panorama-image, twindiffusion
- Language: Python
- Homepage: https://ebooks.iospress.nl/doi/10.3233/FAIA240512
- Size: 3.23 MB
- Stars: 28
- Watchers: 1
- Forks: 0
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# ♊TwinDiffusion: Enhancing Coherence and Efficiency in Panoramic Image Generation with Diffusion Models
## Brief Look
**TwinDiffusion** is a MultiDiffusion-based framework that integrates two straightforward but effective methods to generate panoramic images with
improved quality and efficiency.
> **Abstract** Diffusion models have emerged as effective tools for generating diverse and high-quality content. However, their capability in high-resolution image generation, particularly for panoramic images, still faces challenges such as visible seams and incoherent transitions. In this paper, we propose TwinDiffusion, an optimized framework designed to address these challenges through two key innovations: the Crop Fusion for quality enhancement and the Cross Sampling for efficiency optimization. We introduce a training-free optimizing stage to refine the similarity of adjacent image areas, as well as an interleaving sampling strategy to yield dynamic patches during the cropping process. A comprehensive evaluation is conducted to compare TwinDiffusion with the prior works, considering factors including coherence, fidelity, compatibility, and efficiency. The results demonstrate the superior performance of our approach in generating seamless and coherent panoramas, setting a new standard in quality and efficiency for panoramic image generation.
For more details, please visit our [paper page](https://ebooks.iospress.nl/doi/10.3233/FAIA240512).
## Quick Start
**Installation** Set up and configure the environment by installing the required packages:
```bash
pip install -r requirements.txt
```
**Generation** We support SDXL and batch generation, you can generate twin images and panoramic images with the following command:
```bash
python twin.py --prompt "A photo of dolomites" --seed 5 --n 3
```
```bash
python panorama.py \
--prompt "Landscape ink painting" \
--sd_version 2.0 \
--H 512 \
--W 4096 \
--seed -1 \
--lam 1 \
--view_stride 16 \
--cross_time 2 \
--n 1
```
**App** We also provide a gradio app for interactive testing:
```bash
python app_gradio.py
```
## Citation
If you find our work helpful, please consider citing:
```bibtex
@incollection{zhou2024twindiffusion,
title={TwinDiffusion: Enhancing Coherence and Efficiency in Panoramic Image Generation with Diffusion Models},
author={Zhou, Teng and Tang, Yongchuan},
booktitle={ECAI 2024},
pages={386--393},
year={2024},
publisher={IOS Press}
}
```