{"id":19547333,"url":"https://github.com/emilamaj/syntomid","last_synced_at":"2025-08-21T16:23:17.673Z","repository":{"id":166016700,"uuid":"620777457","full_name":"emilamaj/SynToMid","owner":"emilamaj","description":"Extract notes from Synthesia piano videos on youtube, and export songs to MIDI files.","archived":false,"fork":false,"pushed_at":"2023-05-16T13:13:27.000Z","size":579,"stargazers_count":4,"open_issues_count":0,"forks_count":2,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-19T18:51:06.173Z","etag":null,"topics":["converter","midi","opencv","piano","synthesia","youtube"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/emilamaj.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-03-29T10:59:55.000Z","updated_at":"2024-03-27T13:58:26.000Z","dependencies_parsed_at":null,"dependency_job_id":"cf22e4cd-10e2-41dd-961e-df1b66bc24e0","html_url":"https://github.com/emilamaj/SynToMid","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/emilamaj/SynToMid","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/emilamaj%2FSynToMid","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/emilamaj%2FSynToMid/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/emilamaj%2FSynToMid/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/emilamaj%2FSynToMid/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/emilamaj","download_url":"https://codeload.github.com/emilamaj/SynToMid/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/emilamaj%2FSynToMid/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":271506911,"owners_count":24771821,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-21T02:00:08.990Z","response_time":74,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["converter","midi","opencv","piano","synthesia","youtube"],"created_at":"2024-11-11T03:49:19.737Z","updated_at":"2025-08-21T16:23:17.622Z","avatar_url":"https://github.com/emilamaj.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# SynToMid\r\nThis tool helps you convert synthesia-like piano videos of Youtube to a MIDI (.mid) file that can be read by most music tools. It makes extensive use of OpenCV to process the video frames, and to extract the notes played.\r\n\r\nCurrently implemented:\r\n- YoutubeStitch.py: Stitch together the frames of the Youtube video into a tall png image of the keys pressed.\r\n- WaterfallProcess.py: Convert the stitched image to a list of list of rectangles representing the keys pressed. **Very buggy, not yet usable for note extraction.**\r\n\r\nTodo:\r\n- [ ] Better processing of the notes, especially close ones. (Try again pseudo-gradient descent but with cost function in absolute pixels, like L=[10*black_pixels - 1*white_pixels], where the pixels proposed are the ones enclosed by a rectangle. Beware of the fact that close rectangles will mask each otehr, L also needs an overlap term.)\r\n- [ ] Implement ReadNotes.py, which converts the processed rectangles to a list of notes.\r\n- [ ] Fix YoutubeStitch.py so that the note timings are respected. (Find average scroll rate, and blindly stitch ? Scroll rate might not be constant...)\r\n\r\n## YoutubeStitch.py\r\nThis is the tool that stitches together the frames of the Youtube video into a tall png image of the keys pressed.\r\n\r\n![Example output of YoutubeStitch.py](/output_stitch.png)\r\n\r\nThis is the result of the script for the first 30 seconds of Chopin - Ballade No. 1 played by Rousseau (https://www.youtube.com/watch?v=Zj_psrTUW_w)\r\nThis project will consist in 3 parts:\r\n- YoutubeStitch.py\r\n- WaterfallProcess.py\r\n- ReadNotes.py\r\n\r\n# YoutubeStitch.py\r\nHere is the usage for YoutubeStitch.py:\r\n```bash\r\npython YoutubeStitch.py \u003curl\u003e \u003cheight\u003e \u003cinterval\u003e \u003cstart\u003e \u003cstop\u003e\r\n```\r\n\r\n- url (string): Url of the Youtube video\r\n- height (string): Percent of the height of the video to process (starting from the top, to ignore the hands of the player)\r\n- interval (float): Interval in seconds between the frames, to allow the script to run faster\r\n- start (float): Start position of the video, in seconds (to ignore intros)\r\n- stop (float): Stop position of the video, in seconds\r\n\r\nThe process can run for several minutes.\r\nYou can speedup the script by lowering the height processed. It also removes a lot of the visual artefacts that might remain at the end.\r\nReducing the height however completely messes up the duration of the long notes and the long silences.\r\n\r\nIt produces a file called `output.png`, which is the stitched image of the video.\r\n\r\n# WaterfallProcess.py\r\nThis is the tool that converts the stitched image to a list of list of rectangles representing the keys pressed.\r\nUsage:\r\n```bash\r\npython WaterfallProcess.py \u003cinput_png\u003e \u003coutput_mid\u003e\r\n```\r\nHere is the current state of the image processing done with OpenCV:\r\n\r\n![Example output of WaterfallProcess.py](/output_process.png)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Femilamaj%2Fsyntomid","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Femilamaj%2Fsyntomid","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Femilamaj%2Fsyntomid/lists"}