{"id":13861765,"url":"https://github.com/salu133445/mmt","last_synced_at":"2025-03-25T20:32:06.584Z","repository":{"id":45693720,"uuid":"482164212","full_name":"salu133445/mmt","owner":"salu133445","description":"Official Implementation of \"Multitrack Music Transformer\" (ICASSP 2023)","archived":false,"fork":false,"pushed_at":"2024-03-14T23:51:59.000Z","size":430003,"stargazers_count":135,"open_issues_count":2,"forks_count":23,"subscribers_count":5,"default_branch":"main","last_synced_at":"2024-10-03T19:25:29.020Z","etag":null,"topics":["machine-learning","music","music-generation","music-information-retrieval","python"],"latest_commit_sha":null,"homepage":"https://salu133445.github.io/mmt/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/salu133445.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":"salu133445"}},"created_at":"2022-04-16T05:24:58.000Z","updated_at":"2024-09-27T14:41:49.000Z","dependencies_parsed_at":"2024-08-05T06:14:11.340Z","dependency_job_id":null,"html_url":"https://github.com/salu133445/mmt","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/salu133445%2Fmmt","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/salu133445%2Fmmt/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/salu133445%2Fmmt/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/salu133445%2Fmmt/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/salu133445","download_url":"https://codeload.github.com/salu133445/mmt/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":222090822,"owners_count":16929472,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["machine-learning","music","music-generation","music-information-retrieval","python"],"created_at":"2024-08-05T06:01:29.735Z","updated_at":"2024-10-29T18:21:06.539Z","avatar_url":"https://github.com/salu133445.png","language":"Python","funding_links":["https://github.com/sponsors/salu133445"],"categories":["Python"],"sub_categories":[],"readme":"# Multitrack Music Transformer\n\nThis repository contains the official implementation of \"Multitrack Music Transformer\" (ICASSP 2023).\n\n__Multitrack Music Transformer__\u003cbr\u003e\nHao-Wen Dong, Ke Chen, Shlomo Dubnov, Julian McAuley and Taylor Berg-Kirkpatrick\u003cbr\u003e\n_IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_, 2023\u003cbr\u003e\n[[homepage](https://salu133445.github.io/mmt/)]\n[[paper](https://arxiv.org/pdf/2207.06983.pdf)]\n[[code](https://github.com/salu133445/mmt)]\n[[reviews](https://salu133445.github.io/pdf/mmt-icassp2023-reviews.pdf)]\n\n## Content\n\n- [Prerequisites](#prerequisites)\n- [Preprocessing](#preprocessing)\n  - [Preprocessed Datasets](#preprocessed-datasets)\n  - [Preprocessing Scripts](#preprocessing-scripts)\n- [Training](#training)\n  - [Pretrained Models](#pretrained-models)\n  - [Training Scripts](#training-scripts)\n- [Evaluation](#evaluation)\n- [Generation (Inference)](#generation-inference)\n- [Citation](#citation)\n\n## Prerequisites\n\nWe recommend using Conda. You can create the environment with the following command.\n\n```sh\nconda env create -f environment.yml\n```\n\n## Preprocessing\n\n### Preprocessed Datasets\n\nThe preprocessed datasets can be found [here](https://ucsdcloud-my.sharepoint.com/:f:/g/personal/h3dong_ucsd_edu/Er7nrsVc7NhNtYVSdWpHMQwBS5U1dXo0q0eQEi2LW-DVGw).\n\nExtract the files to `data/{DATASET_KEY}/processed/json` and `data/{DATASET_KEY}/processed/notes`, where `DATASET_KEY` is `sod`, `lmd`, `lmd_full` or `snd`.\n\n### Preprocessing Scripts\n\n__You can skip this section if you download the preprocessed datasets.__\n\n#### Step 1 -- Download the datasets\n\nPlease download the [Symbolic orchestral database (SOD)](https://qsdfo.github.io/LOP/database.html). You may download it via command line as follows.\n\n```sh\nwget https://qsdfo.github.io/LOP/database/SOD.zip\n```\n\nWe also support the following two datasets:\n\n- [Lakh MIDI Dataset (LMD)](https://qsdfo.github.io/LOP/database.html):\n\n  ```sh\n  wget http://hog.ee.columbia.edu/craffel/lmd/lmd_full.tar.gz\n  ```\n\n- [SymphonyNet Dataset](https://symphonynet.github.io/):\n\n  ```sh\n  gdown https://drive.google.com/u/0/uc?id=1j9Pvtzaq8k_QIPs8e2ikvCR-BusPluTb\u0026export=download\n  ```\n\n#### Step 2 -- Prepare the name list\n\nGet a list of filenames for each dataset.\n\n```sh\nfind data/sod/SOD -type f -name *.mid -o -name *.xml | cut -c 14- \u003e data/sod/original-names.txt\n```\n\n\u003e Note: Change the number in the cut command for different datasets.\n\n#### Step 3 -- Convert the data\n\nConvert the MIDI and MusicXML files into MusPy files for processing.\n\n```sh\npython convert_sod.py\n```\n\n\u003e Note: You may enable multiprocessing with the `-j` option, for example, `python convert_sod.py -j 10` for 10 parallel jobs.\n\n#### Step 4 -- Extract the note list\n\nExtract a list of notes from the MusPy JSON files.\n\n```sh\npython extract.py -d sod\n```\n\n#### Step 5 -- Split training/validation/test sets\n\nSplit the processed data into training, validation and test sets.\n\n```sh\npython split.py -d sod\n```\n\n## Training\n\n### Pretrained Models\n\nThe pretrained models can be found [here](https://ucsdcloud-my.sharepoint.com/:f:/g/personal/h3dong_ucsd_edu/EqYq6KHrcltHvgJTmw7Nl6MBtv4szg4RUZUPXc4i_RgEkw).\n\n### Training Scripts\n\nTrain a Multitrack Music Transformer model.\n\n- Absolute positional embedding (APE):\n\n  `python mmt/train.py -d sod -o exp/sod/ape -g 0`\n\n- Relative positional embedding (RPE):\n\n  `python mmt/train.py -d sod -o exp/sod/rpe --no-abs_pos_emb --rel_pos_emb -g 0`\n\n- No positional embedding (NPE):\n\n  `python mmt/train.py -d sod -o exp/sod/npe --no-abs_pos_emb --no-rel_pos_emb -g 0`\n\n## Generation (Inference)\n\nGenerate new samples using a trained model.\n\n```sh\npython mmt/generate.py -d sod -o exp/sod/ape -g 0\n```\n\n## Evaluation\n\nEvaluate the trained model using objective evaluation metrics.\n\n```sh\npython mmt/evaluate.py -d sod -o exp/sod/ape -ns 100 -g 0\n```\n\n## Acknowledgment\n\nThe code is based largely on the [x-transformers](https://github.com/lucidrains/x-transformers) library developed by [lucidrains](https://github.com/lucidrains).\n\n## Citation\n\nPlease cite the following paper if you use the code provided in this repository.\n\n \u003e Hao-Wen Dong, Ke Chen, Shlomo Dubnov, Julian McAuley, and Taylor Berg-Kirkpatrick, \"Multitrack Music Transformer,\" _IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_, 2023.\n\n```bibtex\n@inproceedings{dong2023mmt,\n    author = {Hao-Wen Dong and Ke Chen and Shlomo Dubnov and Julian McAuley and Taylor Berg-Kirkpatrick},\n    title = {Multitrack Music Transformer},\n    booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},\n    year = 2023,\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsalu133445%2Fmmt","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsalu133445%2Fmmt","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsalu133445%2Fmmt/lists"}