Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/tmcclintock/panoramai
Panoramic images using AI.
https://github.com/tmcclintock/panoramai
Last synced: 14 days ago
JSON representation
Panoramic images using AI.
- Host: GitHub
- URL: https://github.com/tmcclintock/panoramai
- Owner: tmcclintock
- License: mit
- Created: 2020-01-21T18:44:26.000Z (about 5 years ago)
- Default Branch: master
- Last Pushed: 2020-02-06T20:19:45.000Z (almost 5 years ago)
- Last Synced: 2024-12-14T17:09:35.105Z (about 1 month ago)
- Language: Python
- Homepage:
- Size: 265 KB
- Stars: 1
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# PanoramAI [](https://travis-ci.com/tmcclintock/PanoramAI) [](https://opensource.org/licenses/MIT) [](https://coveralls.io/github/tmcclintock/PanoramAI?branch=master&service=github)
Panoramic images using generative networks, and (in the future) panoramic images conditional on smaller field-of-view images.
Much of the training data for this project was generated using the synthetic panoramic image tool called the [LandscapeGenerator](https://github.com/tmcclintock/LandscapeGenerator); a stand-alone tool for making synthetic landscape images.
## Installation
Install the requirements and then this repository
```bash
pip install -r requirements.txt
python setup.py install
```
The requirements are:
* [`numpy` and `scipy`](https://scipy.org/install.html)
* [`scikit-learn`](https://scikit-learn.org/stable/install.html)
* [`tensorflow`](https://www.tensorflow.org/install) >= version 2.0.0
* [`notebook`](https://jupyter.readthedocs.io/en/latest/install.html) (for running the example notebooks)
* [`matplotlib`](https://matplotlib.org/users/installing.html) (for notebooks)
* [`pytest`](https://docs.pytest.org/en/latest/getting-started.html) (for testing)
These can all be installed together using the `requirements.txt` file as shown above (assuming you have `pip`).
## Usage
PanoramAI contains models to generate panoramic images. Models provided in this package are untrained. This section documents how to instantiate and train a model, and then how to generate new panoramic images.
First, import and create a model. As an example here, we consider a convolutional variational autoencoder (`VAEorama`).
```python
import PanoramAI
model = PanoramAI.VAEorama(dataset)
```
In this example, `dataset` is a `numpy.ndarray` containing the input panoramic images for training. It should have `N` RGB images in total, meaning its shape must be (`N`,`Height`,`Width`,`3`) where the height and width are in pixels. Note that pixels are assumed to be normalized to the range `[0,1]`.
Once created, we train the model for some number of epochs.
```python
epochs = 100
model.train(epochs)
```
Predictions can be made in batches.
```python
#Create 10 sample panoramas
samples = model.generate_samples(10)
```
### Models
At present, PanoramAI includes two fully generative models: a DCGAN and a convolutional variational autoencoder (VAE). A conditional VAE is in development.
Training images should be landscapes. Development of this project used synthetic landscape images made with the [LandscapeGenerator](https://github.com/tmcclintock/LandscapeGenerator) tool. Example training images are shown below:
![alt text][example1]
![alt text][example2]
![alt text][example3]
[example1]: https://github.com/tmcclintock/PanoramAI/blob/master/images/ex1.png "Example sunset with trees"
[example2]: https://github.com/tmcclintock/PanoramAI/blob/master/images/ex2.png "Example autumn day"
[example3]: https://github.com/tmcclintock/PanoramAI/blob/master/images/ex3.png "Another example sunset with trees"
Once trained, the model produces novel landscape images, like this one:
![alt text][example4]
[example4]: https://github.com/tmcclintock/PanoramAI/blob/master/images/ex4.png "Generated sunset with trees"
This landscape was created with a VAE (hence the blurriness) with not much training data, with a narrow field of view. Smaller fields of view have been able to successfully generate high resolution landscapes, but they are more difficult to render in VR.
To see generated images in a VR environment check out [this link](https://tmcclintock.github.io/) (works best in mobile). This was created using [React 360](https://facebook.github.io/react-360/).