{"id":20681527,"url":"https://github.com/googlecreativelab/semi-conductor","last_synced_at":"2025-04-05T14:04:27.380Z","repository":{"id":36302252,"uuid":"223071774","full_name":"googlecreativelab/semi-conductor","owner":"googlecreativelab","description":"Semi-Conductor allows you to conduct a virtual orchestra using only your web browser \u0026 webcam.","archived":false,"fork":false,"pushed_at":"2025-02-13T06:46:57.000Z","size":8045,"stargazers_count":104,"open_issues_count":17,"forks_count":40,"subscribers_count":9,"default_branch":"master","last_synced_at":"2025-03-29T13:05:27.620Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://semiconductor.withgoogle.com","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/googlecreativelab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-11-21T02:35:35.000Z","updated_at":"2025-03-19T03:30:20.000Z","dependencies_parsed_at":"2024-11-16T22:11:16.832Z","dependency_job_id":"a1d86031-25f0-4137-b543-f52a845b9d2f","html_url":"https://github.com/googlecreativelab/semi-conductor","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/googlecreativelab%2Fsemi-conductor","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/googlecreativelab%2Fsemi-conductor/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/googlecreativelab%2Fsemi-conductor/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/googlecreativelab%2Fsemi-conductor/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/googlecreativelab","download_url":"https://codeload.github.com/googlecreativelab/semi-conductor/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247345850,"owners_count":20924102,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-16T22:11:11.341Z","updated_at":"2025-04-05T14:04:27.359Z","avatar_url":"https://github.com/googlecreativelab.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Semi-Conductor 🎻\n\n![An animation of Semi-Conductor in action.](semi-conductor.gif)\n\n[Semi-Conductor](https://semiconductor.withgoogle.com) allows you to conduct a virtual orchestra using only your web browser \u0026 webcam. Try it out [live here](https://semiconductor.withgoogle.com)!\n\nIt uses [Posenet](https://github.com/tensorflow/tfjs-models/tree/master/posenet) to detect your body pose, and from that how fast you are moving your hands. Using this data and the [Tone.js](https://github.com/Tonejs/Tone.js/) web audio library, it plays real samples of orchestral instruments playing individual notes at the speed of your conducting, which play live from a score you that can edit.\n\nIt's been build to be remixed, so read on to learn how to build and edit your own version of Semi-Conductor, with custom music and instruments.\n\n## Quick Start\n\n```sh\nyarn  # Install dependencies\n```\n```sh\nyarn start  # Start a local dev server\n```\n```sh\nyarn build  # Build static site into the /dist folder\n```\n\n## Code Structure\n\nSemi-Conductor is built in vanilla JS without a framework. Each JS file in the `/scripts` folder contains a class module, with each class controlling one portion of the app (also there are some helper files with useful functions). It's designed so that modules can be used or deleted according to how you want to remix the app.\n\n- `main.js` controls the primary app state \u0026 functions, including loading the app and instantiating the other classes\n- `renderer.js` handles all the UI/DOM updating\n- `orchestra.js` controls the graphic of the orchestra, made with PIXI.js\n- `pose-controller.js` uses TFJS to get the pose state from the webcam with Posenet\n- `posenet-renderer.js` renders the pose skeleton in the interface\n- `audio-player.js` handles the MIDI playback \u0026 loading of samples using Tone.js\n\n## How to remix this\n\n#### Adding instrument samples\n\nWe haven't included the original audio samples used in the live [Semi-Conductor experiment](https://semiconductor.withgoogle.com), but you can use an open sound library like [Sonatina Symphonic Orchestra](https://github.com/peastman/sso) (just make sure you have the rights to use samples you choose in accordance to their licence agreement), or even record your own.\n\nAdd your samples to `/static/samples`, and then add to `/src/assets/samples.json` the relative paths from the `/static/samples` directory to each sample. Samples are organised first by instrument name, then by note. Note descriptions are of the format `C4` for middle C, or `D#5` for the D sharp the octave above middle C. You don't need to have all the notes for a given instrument, Tone.js will interpolate between the samples you have provided.\n\n#### Changing instrumentation\n\nSemi-Conductor is designed to work with a string quartet, however you can change the instrumentation as you desire. Create a rock band, percussion ensemble or synth orchestra if you want! If you're low on disk space, you could even replicate John Cage's [4'33\"](https://youtu.be/Oh-o3udImy8?t=57).\n\nFollow the steps above with whichever instruments \u0026 samples you want to add. Then, edit `config.js` to include the instrument names you want to have play for each zone (instruments change as you move your hands from the left to the right of the screen while conducting). \n\nYou will also need to have a `song.json` file (see below) that only contains instrument names that correspond the ones in `samples.json`. You may need to remove or edit the `orchestra.js` module, as it will break if it doesn't recognise the instrument names.\n\n#### Changing the MIDI song\n\nThe current MIDI song is a custom arrangement of Mozart's *Eine Kleine Nachtmusik* made by us. But you can swap this out with any MIDI file, you just need to convert it to JSON and make sure you have samples listed in `samples.json` with the same instrument names as the tracks.\n\nTo build a `song.json` file, create or acquire a MIDI file you want to use. Then, head to [this Tone.js website](http://tonejs.github.io/Midi/) to convert the MIDI into a JSON file that Tone.js can read. Go through the JSON file and edit the instrument names to correspond to the instrument names in `samples.json` and then move this file to `/src/assets/song.json`.\n\n## Contributors\n\nBuilt by [Rupert Parry](https://www.rparry.me/), [Melissa Lu](https://melissaludesigns.com/), Samantha Cordingley, [Haylie Craig](https://www.hayliecraig.com/), and the team at Google Creative Lab, Sydney.\n\n## License \u0026 Notes\n\nSemi-Conductor has been made available under the [Apache 2.0 license](https://www.apache.org/licenses/LICENSE-2.0) (see LICENSE file in this repository for details).\n\nWe encourage open sourcing projects as a way of learning from each other. Please respect our and other creators’ rights, including copyright and trademark rights when present when sharing these works and creating derivative work. If you want more info on Google's policy, you can find it [here](https://www.google.com/permissions/). To contribute to the project, please refer to the CONTRIBUTING file in this repository.\n\nThis is an experiment, not an official Google product. We do our best to support and maintain this experiment, but your mileage may vary. \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgooglecreativelab%2Fsemi-conductor","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgooglecreativelab%2Fsemi-conductor","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgooglecreativelab%2Fsemi-conductor/lists"}