https://github.com/compphoto/intrinsichdr
Repo for the paper "Intrinsic Single-Image HDR Reconstruction" (ECCV 2024)
https://github.com/compphoto/intrinsichdr
computational-photography computer-graphics computer-vision hdr machine-learning
Last synced: over 1 year ago
JSON representation
Repo for the paper "Intrinsic Single-Image HDR Reconstruction" (ECCV 2024)
- Host: GitHub
- URL: https://github.com/compphoto/intrinsichdr
- Owner: compphoto
- License: other
- Created: 2024-11-08T01:57:44.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2024-11-10T23:39:36.000Z (over 1 year ago)
- Last Synced: 2025-01-11T00:28:01.561Z (over 1 year ago)
- Topics: computational-photography, computer-graphics, computer-vision, hdr, machine-learning
- Language: Python
- Homepage: https://yaksoy.github.io/intrinsicHDR/
- Size: 3.62 MB
- Stars: 8
- Watchers: 4
- Forks: 1
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
---
# Intrinsic Single-Image HDR Reconstruction
[](https://yaksoy.github.io/intrinsicHDR/)
[](https://www.youtube.com/watch?v=EiyH52BcKkw)
[](https://arxiv.org/abs/2409.13803)
[](https://yaksoy.github.io/papers/ECCV24-IntrinsicHDR-supp.pdf)
[](https://colab.research.google.com/github/compphoto/IntrinsicHDR/blob/main/notebooks/intrinsicHDR.ipynb)
## Description
The low dynamic range (LDR) of common cameras fails to capture the rich contrast in natural scenes, resulting in loss of color and details in saturated pixels. Reconstructing the high dynamic range (HDR) of luminance present in the scene from single LDR photographs is an important task with many applications in computational photography and realistic display of images. The HDR reconstruction task aims to infer the lost details using the context present in the scene, requiring neural networks to understand high-level geometric and illumination cues. This makes it challenging for data-driven algorithms to generate accurate and high-resolution results.
In this work, we introduce a physically-inspired remodeling of the HDR reconstruction problem in the intrinsic domain. The intrinsic model allows us to train separate networks to extend the dynamic range in the shading domain and to recover lost color details in the albedo domain. We show that dividing the problem into two simpler sub-tasks improves performance in a wide variety of photographs.
Try out our pipeline on your own images in Colab! [](https://colab.research.google.com/github/compphoto/IntrinsicHDR/blob/main/notebooks/intrinsicHDR.ipynb)
## How to run
First, install dependencies. The code was tested with Python 3.9. It is recommended to start with a fresh environment:
```bash
# create empty env
python3 -m venv intrHDR_env
# activate env
source intrHDR_env/bin/activate
```
Next, clone this repository and install the requirements. Make sure that pip is up-to-date (python3 -m pip install --upgrade pip):
```bash
# clone project
git clone https://github.com/compphoto/IntrinsicHDR
# install project
cd IntrinsicHDR
pip install .
```
The pipeline expects input images to be linear.
To dequantize and linearize images, run:
```bash
# download pretrained weights 'model.ckpt.*' and put them into "./baselines/SingleHDR/checkpoints"
wget https://github.com/compphoto/IntrinsicHDR/releases/download/v1.0/model.ckpt.data-00000-of-00001
wget https://github.com/compphoto/IntrinsicHDR/releases/download/v1.0/model.ckpt.index
wget https://github.com/compphoto/IntrinsicHDR/releases/download/v1.0/model.ckpt.meta
# create checkpoint directory
mkdir ./baselines/SingleHDR/checkpoints
# move weights to checkpoint directory.
mv model.ckpt* ./baselines/SingleHDR/checkpoints/.
# run linearization, e.g.
python3 dequantize_and_linearize.py --test_imgs /path/to/input/imgs --output_path /path/to/results --root .
```
Now, run our HDR reconstruction pipeline. The results will be saved as EXR files in --output_path:
```bash
# run module, e.g.
python3 inference.py --test_imgs /path/to/input/imgs --output_path /path/to/results --use_exr
```
### Citation
This implementation is provided for academic use only. Please cite our paper if you use this code or any of the models.
```
@INPROCEEDINGS{dilleIntrinsicHDR,
author={Sebastian Dille and Chris Careaga and Ya\u{g}{\i}z Aksoy},
title={Intrinsic Single-Image HDR Reconstruction},
booktitle={Proc. ECCV},
year={2024},
}
```
### Credits
"./baselines/SingleHDR/" is adapted from [SingleHDR](https://github.com/alex04072000/SingleHDR) for their dequantization and linearization network.
".intrinsic_decomposition" is adapted from [IntrinsicImageDecomposition](https://github.com/compphoto/Intrinsic) for the decomposition network.
".src/midas/" is adapted from [MiDaS](https://github.com/intel-isl/MiDaS/tree/v2) for their EfficientNet implementation.