{"id":13862254,"url":"https://github.com/YatingMusic/remi","last_synced_at":"2025-07-14T11:33:04.949Z","repository":{"id":41191410,"uuid":"237456772","full_name":"YatingMusic/remi","owner":"YatingMusic","description":"\"Pop Music Transformer: Beat-based Modeling and Generation of Expressive Pop Piano Compositions\", ACM Multimedia 2020","archived":false,"fork":false,"pushed_at":"2022-11-15T14:34:43.000Z","size":47,"stargazers_count":540,"open_issues_count":24,"forks_count":84,"subscribers_count":14,"default_branch":"master","last_synced_at":"2024-08-05T06:05:29.711Z","etag":null,"topics":["music-generation","tensorflow","transformer"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/YatingMusic.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-01-31T15:21:46.000Z","updated_at":"2024-08-03T11:00:22.000Z","dependencies_parsed_at":"2023-01-22T10:15:18.646Z","dependency_job_id":null,"html_url":"https://github.com/YatingMusic/remi","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/YatingMusic%2Fremi","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/YatingMusic%2Fremi/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/YatingMusic%2Fremi/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/YatingMusic%2Fremi/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/YatingMusic","download_url":"https://codeload.github.com/YatingMusic/remi/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225974375,"owners_count":17553941,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["music-generation","tensorflow","transformer"],"created_at":"2024-08-05T06:01:40.526Z","updated_at":"2024-11-22T22:30:48.712Z","avatar_url":"https://github.com/YatingMusic.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# REMI\nAuthors: [Yu-Siang Huang](https://remyhuang.github.io/), [Yi-Hsuan Yang](http://mac.citi.sinica.edu.tw/~yang/)\n\n[**Paper (arXiv)**](https://arxiv.org/abs/2002.00212) | [**Blog**](https://ailabs.tw/human-interaction/pop-music-transformer/) | [**Audio demo (Google Drive)**](https://drive.google.com/open?id=1LzPBjHPip4S0CBOLquk5CNapvXSfys54) | [**Online interactive demo**](https://vibertthio.com/transformer/)\n\nREMI, which stands for `REvamped MIDI-derived events`, is a new event representation we propose for converting MIDI scores into text-like discrete tokens.  Compared to the MIDI-like event representation adopted in exising Transformer-based music composition models, REMI provides sequence models a metrical context for modeling the rhythmic patterns of music. Using REMI as the event representation, we train a Transformer-XL model to generate minute-long Pop piano music with expressive, coherent and clear structure of rhythm and harmony, without needing any post-processing to refine the result. The model also provides controllability of local tempo changes and chord progression.\n\n## Citation\n```\n@inproceedings{10.1145/3394171.3413671,\n  author = {Huang, Yu-Siang and Yang, Yi-Hsuan},\n  title = {Pop Music Transformer: Beat-Based Modeling and Generation of Expressive Pop Piano Compositions},\n  year = {2020},\n  isbn = {9781450379885},\n  publisher = {Association for Computing Machinery},\n  address = {New York, NY, USA},\n  url = {https://doi.org/10.1145/3394171.3413671},\n  doi = {10.1145/3394171.3413671},\n  pages = {1180–1188},\n  numpages = {9},\n  location = {Seattle, WA, USA},\n  series = {MM '20}\n}\n```\n\n## Getting Started\n### Install Dependencies\n* python 3.6 (recommend using [Anaconda](https://www.anaconda.com/distribution/))\n* tensorflow-gpu 1.14.0 (`pip install tensorflow-gpu==1.14.0`)\n* [miditoolkit](https://github.com/YatingMusic/miditoolkit) (`pip install miditoolkit`)\n\n### Download Pre-trained Checkpoints\nWe provide two pre-trained checkpoints for generating samples.\n* `REMI-tempo-checkpoint` [(428 MB)](https://drive.google.com/open?id=1gxuTSkF51NP04JZgTE46Pg4KQsbHQKGo)\n* `REMI-tempo-chord-checkpoint` [(429 MB)](https://drive.google.com/open?id=1nAKjaeahlzpVAX0F9wjQEG_hL4UosSbo)\n\n### Obtain the MIDI Data\nWe provide the MIDI files including local tempo changes and estimated chord. [(5 MB)](https://drive.google.com/open?id=1JUDHGrVYGyHtjkfI2vgR1xb2oU8unlI3)\n* `data/train`: 775 files used for training models\n* `data/evaluation`: 100 files (prompts) used for the continuation experiments\n\n## Generate Samples\nSee `main.py` as an example:\n```python\nfrom model import PopMusicTransformer\nimport os\nos.environ['CUDA_VISIBLE_DEVICES'] = '0'\n\ndef main():\n    # declare model\n    model = PopMusicTransformer(\n        checkpoint='REMI-tempo-checkpoint',\n        is_training=False)\n        \n    # generate from scratch\n    model.generate(\n        n_target_bar=16,\n        temperature=1.2,\n        topk=5,\n        output_path='./result/from_scratch.midi',\n        prompt=None)\n        \n    # generate continuation\n    model.generate(\n        n_target_bar=16,\n        temperature=1.2,\n        topk=5,\n        output_path='./result/continuation.midi',\n        prompt='./data/evaluation/000.midi')\n        \n    # close model\n    model.close()\n\nif __name__ == '__main__':\n    main()\n```\n\n## Convert MIDI to REMI\nYou can find out how to convert the MIDI messages into REMI events in the `midi2remi.ipynb`.\n\n## FAQ\n#### 1. How to synthesize the audio files (e.g., mp3)?\nWe strongly recommend using DAW (e.g., Logic Pro) to open/play the generated MIDI files. Or, you can use [FluidSynth](https://github.com/FluidSynth/fluidsynth) with a [SoundFont](https://sites.google.com/site/soundfonts4u/). However, it may not be able to correctly handle the tempo changes (see [fluidsynth/issues/141](https://github.com/FluidSynth/fluidsynth/issues/141)).\n\n#### 2. What is the function of the inputs \"temperature\" and \"topk\"?\nIt is the temperature-controlled stochastic sampling methods are used for generating text from a trained language model. You can find out more details in the reference paper [CTRL: 4.1 Sampling](https://einstein.ai/presentations/ctrl.pdf).\n\u003e It is worth noting that the sampling method used for generation is very critical to the quality of the output, which is a research topic worthy of further exploration. \n\n#### 3. How to finetune with my personal MIDI data?\nPlease see [issue/Training on custom MIDI corpus](https://github.com/YatingMusic/remi/issues/2)\n\n## Acknowledgement\n- The content of `modules.py` comes from the [kimiyoung/transformer-xl](https://github.com/kimiyoung/transformer-xl) repository.\n- Thanks [@vibertthio](https://github.com/vibertthio) for the awesome online interactive demo.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FYatingMusic%2Fremi","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FYatingMusic%2Fremi","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FYatingMusic%2Fremi/lists"}