Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/jooohyunpark/Uncanny-Forest
A virtual world synthesized with the latent representation of forest sounds.
https://github.com/jooohyunpark/Uncanny-Forest
Last synced: about 2 months ago
JSON representation
A virtual world synthesized with the latent representation of forest sounds.
- Host: GitHub
- URL: https://github.com/jooohyunpark/Uncanny-Forest
- Owner: jooohyunpark
- Created: 2018-12-18T05:39:18.000Z (almost 6 years ago)
- Default Branch: master
- Last Pushed: 2023-03-04T05:56:49.000Z (over 1 year ago)
- Last Synced: 2024-07-18T05:40:21.459Z (2 months ago)
- Language: JavaScript
- Homepage:
- Size: 20.2 MB
- Stars: 4
- Watchers: 1
- Forks: 0
- Open Issues: 5
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Uncanny-Forest
Uncanny forest is an alpha version of my thesis project. This project is designed as an immersive web VR experience combined with machine learning. All the sounds here are LATENT REPRESENTATION of real forests encoded and decoded through latent space with deep neural networks. You can see how neural networks interpret and extract meaningful features of forest sounds and reconstruct them.
* [View Project](https://www.uncannyforest.parkjoohyun.com/)
* [Demo(old)](https://vimeo.com/307056674)
## Generation
#### WaveNet
The dataset for training WaveNet model consists of 922 30-sec wav files(5.27 GB). It is 8 hour length that has even amount for 4 different categories: birds, creek, wind, bugs&frogs. Training process took 3 days with NVIDIA GPU showing 3.6 sec/step and stopped at 38244 steps with 1.531 loss. Command I used was:
```
python train.py --data_dir=/ --num_steps=200000 --silence_threshold=0 --batch_size=4 --checkpoint_every=1000
-- max_checkpoints=100
```
#### NSynth
Generation process took 61 hours for reconstructing 35-40 10-sec wav files using pre-trained model with following command.
```
nsynth_generate --checkpoint_path=//wavenet-ckpt/model.ckpt-200000 --source_path=/ --save_path=/ --batch_size=1
```
## Tech Stack
* [WaveNet](https://github.com/ibab/tensorflow-wavenet)
* [NSynth(Magenta)](https://github.com/tensorflow/magenta/tree/master/magenta/models/nsynth)
* [A-frame](https://aframe.io/)
## Sound Reference
* [BBC Sound Library](http://bbcsfx.acropolis.org.uk/)
* [NPS Sound Library](https://www.nps.gov/romo/learn/photosmultimedia/soundlibrary.htm)
* [Wildlife Sound Recording Society](https://www.wildlife-sound.org/sounds-of-nature/radio-wsrs)
* [Free To Use Sounds](https://freetousesounds.com/complete-library/)
* [Personal Recordings](https://recordingsofnature.wordpress.com/)