https://github.com/halfrost/bachgpt
https://github.com/halfrost/bachgpt
Last synced: about 1 year ago
JSON representation
- Host: GitHub
- URL: https://github.com/halfrost/bachgpt
- Owner: halfrost
- License: mit
- Created: 2023-11-16T08:49:40.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2024-06-25T13:36:45.000Z (about 2 years ago)
- Last Synced: 2025-04-08T04:05:30.441Z (over 1 year ago)
- Language: Jupyter Notebook
- Size: 74.1 MB
- Stars: 9
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# BachGPT
# Background
The BachGPT aims to explore the intersection of artificial intelligence and classical music composition. This project utilizes LSTM (Long Short-Term Memory) + attention neural networks, which are particularly suited for sequential data like music. The focus is on generating compositions in the style of Johann Sebastian Bach, a renowned composer known for his complex harmonies and counterpoint techniques.
This endeavor follows in the footsteps of previous works in AI-driven music generation, aiming to capture the essence of Bach's style while exploring the creative capacities of machine learning models in arts.
# Dataset
For this project, the dataset consists of MIDI files of various compositions by Bach. These files were processed to extract patterns and sequences characteristic of Bach's musical style. The pipeline involves converting MIDI files into a format suitable for training the LSTM model, ensuring the network learns to mimic the intricacies and nuances of Bach's compositions.
I have 4 sets of dataset
1. Raw MIDI music
2. Contrapunctus XIV
3. [JSB Chorales Dataset](https://github.com/czhuang/JSB-Chorales-dataset) - A dataset with 4 voice midi files from bach
4. [JS Fake Chorales](https://github.com/omarperacha/js-fakes) - A similar dataset to JSB Chorales, but fake, generated by AI and annotated by humans
I have used all 4 of these data sources, but when I finally trained the model with dataset 2. Finally, 3 pieces of music were output. The first and third pieces of music came from dataset 2, and the second piece of music came from dataset 3.
# Train methods
LSTM with Attention
Our approach employs a Long Short-Term Memory (LSTM) network enhanced with an attention mechanism. LSTM networks are adept at handling sequences with long-range dependencies, making them ideal for music generation tasks.
- LSTM Overview: LSTMs are capable of learning order dependence in sequence prediction problems.
- Attention Mechanism: This feature allows the network to focus on specific parts of the input sequence, enhancing the quality and coherence of the generated music.
# Evaluations
The model's performance was evaluated based on its ability to authentically replicate Bach's musical style. We conducted several experiments to optimize the model's architecture and parameters.
- Experiments and Results: The results of these experiments were quantified and are presented in tables and figures, demonstrating the model's accuracy in generating music that aligns with Bach's style.
- Comparison with Baselines: The model's performance was also compared against established baselines to ascertain its efficacy and uniqueness.
# How to run
I highly recommend running the notebook in [Colaboratory](https://colab.research.google.com/) so you can view the results and change parameters in real-time. The jupyter notebook code is [here](https://github.com/halfrost/BachGPT/blob/main/bachGPT.ipynb). However, you can run Python code on your local machine.
## 1. Train model
```python
python3 bachGPT.py
```
## 2. Use the model to generate predicted output
```python
python3 bach_predict.py ./models/{your_model_name}.pth
```
## 3. Create midi music file
```python
python3 create_midi.py {your_model_name}.pth/output.txt
```
## 4. Generate mp3 file
```python
timidity output/output.midi -Ow -o - | ffmpeg -i - -acodec libmp3lame -ab 320k output/output.mp3
timidity output/complete.midi -Ow -o - | ffmpeg -i - -acodec libmp3lame -ab 320k output/complete.mp3
```
The final generated Bach-style music is in the output folder.
# Generated music
https://github.com/halfrost/BachGPT/assets/10825609/ba1f8a7b-5317-4cce-898a-98ca69de66de
https://github.com/halfrost/BachGPT/assets/10825609/cb740315-36c5-4722-bac3-a619888c18b9
https://github.com/halfrost/BachGPT/assets/10825609/069e8bf7-a2c9-4624-820a-deccf4ef2aac
# Reference
1. [Generating Original Classical Music with an LSTM Neural Network and Attention](https://medium.com/@alexissa122/generating-original-classical-music-with-an-lstm-neural-network-and-attention-abf03f9ddcb4)
2. [Generating Long-Term Structure in Songs and Stories](https://magenta.tensorflow.org/2016/07/15/lookback-rnn-attention-rnn)
3. [Neural Machine Translation by Jointly Learning to Align and Translate](https://arxiv.org/abs/1409.0473)
4. [Baroque Music Generator](https://github.com/Pudkip/Bach-Bot)
5. [Generating Music using LSTM](https://www.researchgate.net/publication/351502178_Generating_Music_using_LSTM)
6. [Music Generation using Deep Learning](https://medium.com/@sabadejuyee21/music-generation-using-deep-learning-7d3dbb2254af)
7. [How to Generate Music using a LSTM Neural Network in Keras](https://towardsdatascience.com/how-to-generate-music-using-a-lstm-neural-network-in-keras-68786834d4c5)
8. [Classical-Piano-Composer](https://github.com/Skuldur/Classical-Piano-Composer)
9. [Classic_Music_Generation](https://github.com/thebeyonder001/Classic_Music_Generation)
10. [conversion-tool](https://www.conversion-tool.com/audiotomidi/)