{"id":13862154,"url":"https://github.com/yunlong10/LaunchpadGPT","last_synced_at":"2025-07-14T11:32:33.132Z","repository":{"id":179504757,"uuid":"641971295","full_name":"yunlong10/LaunchpadGPT","owner":"yunlong10","description":"[ICMC 2023] LaunchpadGPT: Language Model as Music Visualization Designer on Launchpad","archived":false,"fork":false,"pushed_at":"2024-01-01T05:33:58.000Z","size":2427,"stargazers_count":17,"open_issues_count":0,"forks_count":3,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-05-12T23:07:20.047Z","etag":null,"topics":["computer-music","icmc","language-model","launchpad"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/yunlong10.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-05-17T14:40:46.000Z","updated_at":"2025-03-08T20:49:06.000Z","dependencies_parsed_at":"2024-01-01T06:46:22.312Z","dependency_job_id":null,"html_url":"https://github.com/yunlong10/LaunchpadGPT","commit_stats":null,"previous_names":["yunlong10/launchpadgpt"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/yunlong10/LaunchpadGPT","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yunlong10%2FLaunchpadGPT","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yunlong10%2FLaunchpadGPT/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yunlong10%2FLaunchpadGPT/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yunlong10%2FLaunchpadGPT/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/yunlong10","download_url":"https://codeload.github.com/yunlong10/LaunchpadGPT/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yunlong10%2FLaunchpadGPT/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265285697,"owners_count":23740582,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["computer-music","icmc","language-model","launchpad"],"created_at":"2024-08-05T06:01:38.349Z","updated_at":"2025-07-14T11:32:32.166Z","avatar_url":"https://github.com/yunlong10.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# LaunchpadGPT\nThis repo is for paper accepted by ICMC 2023: LaunchpadGPT: Language Model as Music Visualization Designer on Launchpad \n\n\u003cimg src=\"/framework.png\"\u003e \n\n### Quick Start\n\n1. download the prompt-completion pairs' data `prompt_completion.txt` from [[here]](https://drive.google.com/file/d/1ZBQtDop9jGhcSjpZwWKHC-k6QK4pVNe-/view?usp=sharing), and put it in the `./data` folder. \n\n2. turn `prompt_completion.txt` from raw text into one large stream of integers:\n\n```shell\npython data/prompt_completion/prepare.py\n```\nThis creates a `train.bin` and `val.bin` in the prompt_completion directory.\n\n3. train a LaunchpadGPT:\n\n```shell\npython train.py config/train_launchpad_gpt.py\n```\n\n### Sampling\n\nThis is an example of generate RGB-X tuples (completion) given the MFCC features with 128 dims of input music (prompt):\n\n```shell\npython sample.py --out_dir=out-launchpad-gpt --start='{\"prompt\": [-29.44, 108.58, -15.65, 36.5, 2.3, 14.21, 4.92, 20.2, -2.59, 9.43, 10.56, 20.83, -0.24, 1.78, -12.75, 2.06, -4.75, 0.09, -4.64, -7.97, -0.51, -4.5, -3.58, -9.82, -1.73, 8.06, 1.05, -1.21, -1.25, -5.44, -9.97, -16.69, -5.6, 2.49, 0.04, 5.14, -0.37, -8.98, -5.22, -8.35, -14.0, 5.34, 3.24, -0.72, -4.3, -1.48, -3.27, 1.1, -2.93, -5.9, -3.68, 2.54, 5.99, 2.21, -6.68, 1.52, 0.23, 1.74, 1.14, -1.17, 1.01, -0.78, -5.34, -0.31, 1.09, 4.35, -0.25, -0.52, -0.14, -1.47, 9.78, 1.56, -1.56, 5.22, -1.96, -0.0, 1.63, 0.18, -0.63, 3.86, -1.81, 3.28, -0.89, 1.4, -0.75, -2.01, -0.78, 1.12, -0.02, 1.75, 0.24, -0.99, -1.75, 3.75, 1.06, 1.01, 2.99, 1.59, 3.54, -1.33, -3.71, 1.18, 1.11, -0.47, 0.76, -0.96, -1.03, 1.0, -0.48, -0.51, 0.9, 0.36, -0.4, 1.28, -0.78, 1.92, 0.57, 2.5, 1.79, 0.8, -0.5, -0.19, -2.1, -1.51, -0.57, -1.17, 0.08, 0.45], \"completion\":'\n```\n\nThe generated Launchpad can be found in `./outputs/sample_outs`\n\n### Inference\n\nThe `infer.py` can generate the Launchpads with validation data `val_prompts.json`.\n\n\n```shell\npython infer.py\n```\nThe results will be saved in the `./outputs/val_outs`\n\n### Evaluation\n\nTo evaluate the results, you can download the ground-truth data `gt_frame` from [[here]](https://drive.google.com/file/d/13UNtQgTKaUJomo3vxwBuUC7W25sRYqgz/view?usp=sharing) to `./outputs`.\n\nThen run the script to calculate the scores:\n\n```shell\npython -m pytorch_fid outputs/gt_frames outputs/val_outs\n```\n\n\nThe original Launchpad-playing video can be downloaded from [[here]](https://drive.google.com/file/d/1ikugWFBwkRm0V6AlDoRswdC3knrvLZCt/view?usp=sharing).\n\n### Acknowledgement\nThe project is based on [NanoGPT](https://github.com/karpathy/nanoGPT), [midifox](http://midifox.com/). Thanks for the authors for their efforts.\n\n\nIf you find our survey useful for your research, please cite the following paper:\n```bibtex\n@article{xu2023launchpadgpt,\n  title={Launchpadgpt: Language model as music visualization designer on launchpad},\n  author={Xu, Siting and Tang, Yunlong and Zheng, Feng},\n  journal={arXiv preprint arXiv:2307.04827},\n  year={2023}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyunlong10%2FLaunchpadGPT","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyunlong10%2FLaunchpadGPT","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyunlong10%2FLaunchpadGPT/lists"}