https://github.com/xasopheno/whoyouare

A Deep Learning Toolkit for Composition and Improvisation
https://github.com/xasopheno/whoyouare

audio keras pyaudio python real-time

Last synced: 10 days ago
JSON representation

A Deep Learning Toolkit for Composition and Improvisation

• Host: GitHub
• URL: https://github.com/xasopheno/whoyouare
• Owner: xasopheno
• Created: 2017-12-28T17:48:17.000Z (about 7 years ago)
• Default Branch: master
• Last Pushed: 2022-12-08T18:34:22.000Z (about 2 years ago)
• Last Synced: 2023-02-28T11:47:02.657Z (almost 2 years ago)
• Topics: audio, keras, pyaudio, python, real-time
• Language: Jupyter Notebook
• Size: 324 MB
• Stars: 4
• Watchers: 2
• Forks: 2
• Open Issues: 30
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# WhoYouAre

A Recurrent Neural Network trained on a corpus of melodic time series data is used to train a network to produce new melodies with structural similarities to that of the training corpus. Midi data is generated from microphone input. This data is then used to train an RNN using techniques commonly employed in the context of Natural Language Processing. Trained model makes predictions in under a 1/10 second and can be used in real-time situations. 

The time needed to train a RNN increases with the amount of data presented to the network, but predict time remains relatively the same. Therefore, it is possible to supplement initial training data with additional melodies which will influence the types of melodic structures generated by the network while negligibly reducing network prediction speed. As the dataset in this case is generated by a musician, therefore uniquely personal, this process opens up an interesting window into the general study of RNNs. 

## Setup 

`pip3 install requirements.txt`

`cd UI/front-end/ && yarn`

## Use

#### Generate a .csv file for training:

`python3 Record.py -val -csv`

Copy .csv values from `Audio/data/output_new.csv` to `Audio/data/input.csv`.

#### Train on the .csv file:

Open `WhoYouAre.ipynb` with `jupyter notebook`

Copy `model.h5` and `model.json` to `NN/trained_networks`

#### You can then improvise with the trained model with 

From the directory `WhoYouAre` run `./client.sh` and `./server.sh`

`python3 Record.py -nn -model=model -cli -val`

#### `Record.py`

```

  -h, --help    show this help message and exit

  -vol          Specify if input volume should be displayed.

  -val          Specify if prediction values should be displayed.

  -csv          Specify if a csv should be written.

  -midi         Specify if midi should be sent with python-rt-midi.

  -socket       Specify if frequencies should be sent to a local websocket.

  -filtered     filter out redundant notes).

  -py_osc       Specify if frequencies should be sent to the python

                oscillator.

  -nn           Improvise with neural network.

  -model MODEL  specify model to be used with network.

  -cli          set flag if using webclient

  

```