https://github.com/ztMotaLee/Sagiri
Official implementation of our paper "Sagiri: LDR image enhancement by Generation".
https://github.com/ztMotaLee/Sagiri
Last synced: 6 months ago
JSON representation
Official implementation of our paper "Sagiri: LDR image enhancement by Generation".
- Host: GitHub
- URL: https://github.com/ztMotaLee/Sagiri
- Owner: ztMotaLee
- License: apache-2.0
- Created: 2024-03-06T05:31:06.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2025-01-18T01:43:46.000Z (9 months ago)
- Last Synced: 2025-01-18T02:35:36.998Z (9 months ago)
- Language: Python
- Size: 7.52 MB
- Stars: 20
- Watchers: 2
- Forks: 4
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-diffusion-categorized - [Code
README
## Sagiri v2 is under construction!
## Sagiri: Low Dynamic Range Image Enhancement with Generative Diffusion Prior
[Paper](https://arxiv.org/pdf/2406.09389) | [Project Page](https://sagiri0208.github.io/)
[Baiang Li](ztmotalee.github.io)1, 5, [Sizhuo Ma](https://sizhuoma.netlify.app/)3, [Yanhong Zeng](https://zengyh1900.github.io/)1, [Xiaogang Xu](https://xuxiaogang.com/)2, 4, [Youqing Fang]()1, [Zhao Zhang](https://sites.google.com/site/cszzhang)5, [Jian Wang](https://jianwang-cmu.github.io/)3✝, [Kai Chen](https://chenkai.site/)1✝
✝Corresponding Authors.
1Shanghai AI Laboratory
2The Chinese University of Hong Kong
3Snap Inc.
4Zhejiang University
5Hefei University of Technology
## Our task:
## :book:Table Of Contents
- [Visual Results](#visual_results)
- [Update](#update)
- [Installation](#installation)
- [Pretrained Models](#pretrained_models)
- [Inference](#inference)
- [Train](#train)
## Visual results on entire real-world image
Left: Input image;
Medium: After previous methods;
Right: LS-Sagiri(Ours)
## Visual results on selected region
Left: Input image;
Medium: Previous methods;
Right: LS-Sagiri(Ours)
## Sagiri plugged after other methods
Left: Input image;
Medium: SingleHDR;
Right: SingleHDR+Sagiri(Ours)
Left: Input image;
Medium: LCDPNet;
Right: LCDPNet+Sagiri(Ours)
## Controlling where and what to generate
First: Input image;
Second: SingleHDR;
Third: SingleHDR+Sagiri (with prompt a);
Fourth: SingleHDR+Sagiri (with prompt b);
Prompt a: ``A white waterfall is flowing down from the cliff, surrounded by rocks and trees.'';
Prompt b: ``The sun is setting, and the sky is filled with clouds.''
## Update
- **2024.12**: A new version is under reconstruction! We will release the improved mask strategy, and support more customized generation options for selected regions.
- **2024.06**: This repo is released.
```shell
# clone this repo
git clone https://github.com/ztMotaLee/Sagiri.git
cd Sagiri
# create an environment with python >= 3.9
conda create -n sagiri python=3.9
conda activate sagiri
pip install -r requirements.txt
```
| Model Name | Description | BaiduNetdisk |
| :--------- | :---------- | :---------- |
| stage1.ckpt | Stage1 for brightness and color adjustment. | [download](https://pan.baidu.com/s/1StNZdmnLx5uPsXIz-zXZSw?pwd=sgri)|
| stage2.ckpt | Sagiri for conditional image generation. | [download](https://pan.baidu.com/s/14bPVDza-gRbpF3qYeJuYHQ?pwd=sgri) |
## Inference
### Stage 1 inference
Note that we can use other restoration models to finish stage 1's process.
```shell
python scripts/inference_stage1.py \
--config configs/model/swinir.yaml \
--ckpt /path/to/stage1/model \
--input /path/to/input/images \
--output /path/to/output/images
```
### Sagiri inference
```shell
python infer_Sagiri.py \
--config configs/model/cldm.yaml \
--ckpt /path/to/stage2/model\
--steps 30 \
--input /path/to/input/images \
--output /path/to/output/images \
--disable_preprocess_model \
--device cuda
```
### LS-Sagiri inference
```shell
python infer_LSSagiri.py \
--config configs/model/cldm.yaml \
--ckpt /path/to/stage2/model\
--steps 30 \
--input /path/to/input/images \
--output /path/to/output/images \
--device cuda
```
```shell
wget https://huggingface.co/stabilityai/stable-diffusion-2-1-base/resolve/main/v2-1_512-ema-pruned.ckpt --no-check-certificate
```
and
```shell
python train.py --config [training_config_path]
```
## Citation
Please cite us if our work is useful for your research.
```
@article{li2024sagiri,
author = {Baiang Li and Sizhuo Ma and Yanhong Zeng and Xiaogang Xu and Youqing Fang and Zhao Zhang and Jian Wang and Kai Chen},
title = {Sagiri: Low Dynamic Range Image Enhancement with Generative Diffusion Prior},
journal = {arxiv},
year = {2024},
}
```
## License
This project is released under the [Apache 2.0 license](LICENSE).
## Acknowledgement
This project is based on [ControlNet](https://github.com/lllyasviel/ControlNet), [BasicSR](https://github.com/XPixelGroup/BasicSR) and [DiffBIR](https://github.com/XPixelGroup/DiffBIR). Thanks for their awesome work.
## Contact
Should you have any questions, please feel free to contact with me at ztmotalee@gmail.com.