Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/salu133445/arranger
Official Implementation of "Towards Automatic Instrumentation by Learning to Separate Parts in Symbolic Multitrack Music" (ISMIR 2021)
https://github.com/salu133445/arranger
machine-learning music music-information-retrieval tensorflow
Last synced: about 2 months ago
JSON representation
Official Implementation of "Towards Automatic Instrumentation by Learning to Separate Parts in Symbolic Multitrack Music" (ISMIR 2021)
- Host: GitHub
- URL: https://github.com/salu133445/arranger
- Owner: salu133445
- License: mit
- Created: 2021-01-14T03:28:25.000Z (over 3 years ago)
- Default Branch: main
- Last Pushed: 2023-06-26T07:40:36.000Z (about 1 year ago)
- Last Synced: 2024-05-02T03:16:36.925Z (5 months ago)
- Topics: machine-learning, music, music-information-retrieval, tensorflow
- Language: Python
- Homepage: https://salu133445.github.io/arranger/
- Size: 193 MB
- Stars: 53
- Watchers: 4
- Forks: 7
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- Funding: .github/FUNDING.yml
- License: LICENSE
- Citation: CITATION.cff
Awesome Lists containing this project
README
# Arranger
This repository contains the official implementation of "Towards Automatic Instrumentation by Learning to Separate Parts in Symbolic Multitrack Music" (ISMIR 2021).
__Towards Automatic Instrumentation by Learning to Separate Parts in Symbolic Multitrack Music__
Hao-Wen Dong, Chris Donahue, Taylor Berg-Kirkpatrick and Julian McAuley
_Proceedings of the International Society for Music Information Retrieval Conference (ISMIR)_, 2021
[[homepage](https://salu133445.github.io/arranger/)]
[[paper](https://arxiv.org/pdf/2107.05916.pdf)]
[[video](https://youtu.be/-KncOGouAh8)]
[[slides](https://salu133445.github.io/arranger/pdf/arranger-ismir2021-slides.pdf)]
[[video (long)](https://youtu.be/RMhzOuHJ5UI)]
[[slides (long)](https://salu133445.github.io/arranger/pdf/arranger-research-exam-slides.pdf)]
[[code](https://github.com/salu133445/arranger)]
## Content
- [Content](#content)
- [Prerequisites](#prerequisites)
- [Directory structure](#directory-structure)
- [Data Collection](#data-collection)
- [Data Preprocessing](#data-preprocessing)
- [Models](#models)
- [Baseline algorithms](#baseline-algorithms)
- [Configuration](#configuration)
- [Citation](#citation)
## Prerequisites
You can install the dependencies by running `pipenv install` (recommended) or `python3 setup.py install -e .`. Python>3.6 is required.
## Directory structure
```text
├─ analysis Notebooks for analysis
├─ scripts Scripts for running experiments
├─ models Pretrained models
└─ arranger Main Python module
├─ config.yaml Configuration file
├─ data Code for collecting and processing data
├─ common Most-common algorithm
├─ zone Zone-based algorithm
├─ closest Closest-pitch algorithm
├─ lstm LSTM model
└─ transformer Transformer model
```
## Data Collection
### Bach Chorales
```python
# Collect Bach chorales from the music21 corpus
import shutil
import music21.corpus
for path in music21.corpus.getComposer("bach"):
if path.suffix in (".mxl", ".xml"):
shutil.copyfile(path, "data/bach/raw/" + path.name)
```
### MusicNet
```sh
# Download the metadata
wget -O data/musicnet https://homes.cs.washington.edu/~thickstn/media/musicnet_metadata.csv
```
### NES Music Database
```sh
# Download the dataset
wget -O data/nes http://deepyeti.ucsd.edu/cdonahue/nesmdb/nesmdb_midi.tar.gz
# Extract the archive
tar zxf data/nes/nesmdb_midi.tar.gz
# Rename the folder for consistency
mv nesmdb_midi/ raw/
```
### Lakh MIDI Dataset (LMD)
```sh
# Download the dataset
wget -O data/lmd http://hog.ee.columbia.edu/craffel/lmd/lmd_matched.tar.gz
# Extract the archive
tar zxf data/lmd/lmd_matched.tar.gz
# Rename the folder for consistency
mv lmd_matched/ raw/
# Download the filenames
wget -O data/lmd http://hog.ee.columbia.edu/craffel/lmd/md5_to_paths.json
```
## Data Preprocessing
> The following commands assume Bach chorales. You might want to replace the dataset identifier `bach` with identifiers of other datasets (`musicnet` for MusicNet, `nes` for NES Music Database and `lmd` for Lakh MIDI Dataset).
```sh
# Preprocess the data
python3 arranger/data/collect_bach.py -i data/bach/raw/ -o data/bach/json/ -j 1
# Collect training data
python3 arranger/data/collect.py -i data/bach/json/ -o data/bach/s_500_m_10/ -d bach -s 500 -m 10 -j 1
```
## Models
- LSTM model
- `arranger/lstm/train.py`: Train the LSTM model
- `arranger/lstm/infer.py`: Infer with the LSTM model
- Transformer model
- `arranger/transformer/train.py`: Train the Transformer model
- `arranger/transformer/infer.py`: Infer with the Transformer model
## Pretrained Models
Pretrained models can be found in the `models/` directory.
To run a pretrained model, please pass the corresponding command line options to the `infer.py` scripts. You may want to follow the commands used in the experiment scripts provided in `scripts/infer_*.sh`.
For example, use the following command to run the pretrained BiLSTM model with embeddings.
```sh
# Assuming we are at the root of the repository
cp models/bach/lstm/bidirectional_embedding/best_models.hdf5 OUTPUT_DIRECTORY
python3 arranger/lstm/infer.py \
-i {INPUT_DIRECTORY} -o {OUTPUT_DIRECTORY} \
-d bach -g 0 -bi -pe -bp -be -fi
```
The input directory (`INPUT_DIRECTORY`) contains the input JSON files, which can be generated by `muspy.save()`. The output directory (`OUTPUT_DIRECTORY`) should contain the pretrained model and will contain the output files. The `-d bach` option indicates that we are using the Bach chorale dataset. The `-g 0` option will run the model on the first GPU. The `-bi -pe -bp -be -fi` specifies the model options (run `python3 arranger/lstm/infer.py -h` for more information).
## Baseline algorithms
- Most-common algorithm
- `arranger/common/learn.py`: Learn the most common label
- `arranger/common/infer.py`: Infer with the most-common algorithm
- Zone-based algorithm
- `arranger/zone/learn.py`: Learn the optimal zone setting
- `arranger/zone/infer.py`: Infer with the zone-based algorithm
- Closest-pitch algorithm
- `arranger/closest/infer.py`: Infer with the closest-pitch algorithm
- MLP model
- `arranger/mlp/train.py`: Train the MLP model
- `arranger/mlp/infer.py`: Infer with the MLP model
## Configuration
In `arranger/config.yaml`, you can configure the MIDI program numbers used for each track in the sample files generated. You can also configure the color of the generated sample piano roll visualization.
## Citation
Please cite the following paper if you use the code provided in this repository.
> Hao-Wen Dong, Chris Donahue, Taylor Berg-Kirkpatrick and Julian McAuley, "Towards Automatic Instrumentation by Learning to Separate Parts in Symbolic Multitrack Music," _Proceedings of the International Society for Music Information Retrieval Conference (ISMIR)_, 2021.
```bibtex
@inproceedings{dong2021arranger,
author = {Hao-Wen Dong and Chris Donahue and Taylor Berg-Kirkpatrick and Julian McAuley},
title = {Towards Automatic Instrumentation by Learning to Separate Parts in Symbolic Multitrack Music},
booktitle = {Proceedings of the International Society for Music Information Retrieval Conference (ISMIR)},
year = 2021,
}
```