https://github.com/luanfujun/deep-painterly-harmonization

Code and data for paper "Deep Painterly Harmonization": https://arxiv.org/abs/1804.03189
https://github.com/luanfujun/deep-painterly-harmonization

Last synced: 2 days ago
JSON representation

Code and data for paper "Deep Painterly Harmonization": https://arxiv.org/abs/1804.03189

• Host: GitHub
• URL: https://github.com/luanfujun/deep-painterly-harmonization
• Owner: luanfujun
• Created: 2018-04-09T19:13:29.000Z (almost 7 years ago)
• Default Branch: master
• Last Pushed: 2021-08-02T02:46:03.000Z (over 3 years ago)
• Last Synced: 2025-01-16T18:19:22.291Z (9 days ago)
• Language: Cuda
• Homepage:
• Size: 87.3 MB
• Stars: 6,075
• Watchers: 172
• Forks: 624
• Open Issues: 34
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

• awesome - deep-painterly-harmonization - Code and data for paper "Deep Painterly Harmonization": https://arxiv.org/abs/1804.03189 (Cuda)
• awesome-neural-art - Deep Painterly Harmonization - Photoshopping an object into a painting, and then neural network changes the style (pallete, strokes, luminosity, etc) of the object to match the painting style. [See paper](https://arxiv.org/abs/1804.03189). (Style Transfer)

README

        
# deep-painterly-harmonization

Code and data for paper "[Deep Painterly Harmonization](https://arxiv.org/abs/1804.03189)"  

## Disclaimer 

**This software is published for academic and non-commercial use only.**

## Setup

This code is based on torch. It has been tested on Ubuntu 16.04 LTS.

Dependencies:

* [Torch](https://github.com/torch/torch7) (with [loadcaffe](https://github.com/szagoruyko/loadcaffe))

* [Matlab](https://www.mathworks.com/) or [Octave](https://www.gnu.org/software/octave/)

CUDA backend:

* [CUDA](https://developer.nvidia.com/cuda-downloads)

* [cudnn](https://developer.nvidia.com/cudnn)

Download VGG-19:

```

sh models/download_models.sh

```

Compile ``cuda_utils.cu`` (Adjust ``PREFIX`` and ``NVCC_PREFIX`` in ``makefile`` for your machine):

```

make clean && make

```

## Usage

To generate all results (in  ``data/``) using the provided scripts, simply run

```

python gen_all.py

```

in Python and then 

```

run('filt_cnn_artifact.m')

```

in Matlab or Octave. The final output will be in ``results/``.

Note that in the paper we trained a CNN on a dataset of 80,000 paintings collected from [wikiart.org](https://www.wikiart.org), which estimates the stylization level of a given painting and adjust weights accordingly. We will release the pre-trained model in the next update. Users will need to set those weights manually if running on their new paintings for now. 

**Removed a few images due to copyright issue. Full set [here](https://github.com/luanfujun/deep-painterly-harmonization/blob/master/README2.md) for testing use only.**

## Examples

Here are some results from our algorithm (from left to right are original painting, naive composite and our output):



  

  

  





  

  

  





  

  

  





  

  

  





  

  

  





  

  

  





  

  

  





  

  

  





  

  

  





  

  

  





  

  

  





  

  

  





  

  

  





  

  

  





  

  

  





  

  

  





  

  

  





  

  

  





  

  

  





  

  

  





  

  

  





  

  

  





  

  

  





  

  

  





  

  

  





  

  

  





  

  

  





  

  

  





  

  

  





  

  

  





  

  

  





  

  

  





  

  

  





  

  

  





  

  

  





  

  

  



## Acknowledgement

* Our torch implementation is based on Justin Johnson's [code](https://github.com/jcjohnson/neural-style);

* Histogram loss is inspired by [Risser et al.](https://arxiv.org/abs/1701.08893)

## Citation

If you find this work useful for your research, please cite:

```

@article{luan2018deep,

  title={Deep Painterly Harmonization},

  author={Luan, Fujun and Paris, Sylvain and Shechtman, Eli and Bala, Kavita},

  journal={arXiv preprint arXiv:1804.03189},

  year={2018}

}

```

## Contact

Feel free to contact me if there is any question (Fujun Luan [email protected]).