https://github.com/ReshotAI/gaussian-painters

Gaussian Painters using 3D Gaussian Splatting
https://github.com/ReshotAI/gaussian-painters

3d ai art gaussian-splatting machine-learning nerf painting python pytorch

Last synced: about 2 months ago
JSON representation

Gaussian Painters using 3D Gaussian Splatting

• Host: GitHub
• URL: https://github.com/ReshotAI/gaussian-painters
• Owner: ReshotAI
• License: other
• Created: 2023-09-02T16:19:15.000Z (about 1 year ago)
• Default Branch: main
• Last Pushed: 2024-04-27T12:31:36.000Z (5 months ago)
• Last Synced: 2024-06-08T10:32:56.453Z (4 months ago)
• Topics: 3d, ai, art, gaussian-splatting, machine-learning, nerf, painting, python, pytorch
• Language: Python
• Homepage: https://www.lingosub.com
• Size: 2.83 MB
• Stars: 360
• Watchers: 10
• Forks: 19
• Open Issues: 1
• Metadata Files:
  • Readme: README.md
  • License: LICENSE.md

Awesome Lists containing this project

README

        
# Gaussian Painters

**Sponsored by  [LingoSub](https://www.lingosub.com): Learn languages by watching videos with AI-powered translations.**

This is a fork of 3D Gaussian Splatting. Refer to the original repo for instructions on how to run the code.


![](assets/gaussian_painter.jpg)

## How to create a Gaussian Painter dataset

After having installed the 3D Gaussian Splatting code, run the following command:

```bash

python create_dataset.py --img_path /path/to/image --output_dir /path/to/output_dir

```

You can disable the `opacity_reset_interval` argument by setting it to 30_000.

You can also set `sh_degree` to 0 to disable viewdependent effects.

This will create a dataset ready to be trained with the Gaussian Splatting code.

## Experiments

- Orthogonal images (using `create_dataset2.py`)

https://github.com/ReshotAI/gaussian-painters/assets/16474636/4799f0b6-ed29-412e-9875-4a790ecbbaaf

- Steganography (using `create_dataset3.py`)

https://github.com/ReshotAI/gaussian-painters/assets/16474636/9a391361-7d5b-40cc-ab67-97e15e53a913

- Lenticular effect (using `create_dataset5.py`)

This code requires to install [kornia](https://github.com/kornia/kornia) using `pip install kornia`

https://github.com/ReshotAI/gaussian-painters/assets/16474636/356ad0f6-3bcb-46fe-a6f8-421138e54222

## Visualize the "painting" process

Using the SIBR visualizer, you can visualize the "painting" process during the Gaussian Splatting optimization.

https://github.com/ReshotAI/gaussian-painters/assets/16474636/b29731b6-5fcc-43f5-a169-bfed2b109ce0

## How it works?

The `create_dataset` script simply creates a COLMAP output directory with a single camera pointing at a plane. 100 points are sampled from the image and used as initial point cloud for the Gaussian Splatting optimization. A second perpendicular image is also created with a black image as target.

![](assets/colmap.jpg)