{"id":24689854,"url":"https://github.com/agarnung/cv-theremin","last_synced_at":"2026-05-13T05:36:20.846Z","repository":{"id":270570454,"uuid":"910783358","full_name":"agarnung/cv-theremin","owner":"agarnung","description":"A handtracking and 2D camera based digital theremin","archived":false,"fork":false,"pushed_at":"2025-02-25T11:48:41.000Z","size":44,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-25T12:35:32.627Z","etag":null,"topics":["computer-vision","handtracking","image-processing","music","python","theremin"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/agarnung.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-01-01T12:14:51.000Z","updated_at":"2025-02-25T11:48:44.000Z","dependencies_parsed_at":"2025-02-25T12:27:09.535Z","dependency_job_id":"3c728a2d-6e90-45ae-b594-16db47850c26","html_url":"https://github.com/agarnung/cv-theremin","commit_stats":null,"previous_names":["agarnung/cv-theremin"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/agarnung%2Fcv-theremin","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/agarnung%2Fcv-theremin/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/agarnung%2Fcv-theremin/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/agarnung%2Fcv-theremin/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/agarnung","download_url":"https://codeload.github.com/agarnung/cv-theremin/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244885700,"owners_count":20526296,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["computer-vision","handtracking","image-processing","music","python","theremin"],"created_at":"2025-01-26T18:17:58.211Z","updated_at":"2026-05-13T05:36:20.841Z","avatar_url":"https://github.com/agarnung.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# cv-theremin\nA handtracking and 2D camera based digital theremin 🎶.\n\nThe Theremin operates in one (and only one) of the following modes:\n\n🖐️ **Naïve Method** – Maps 2D structural and morphological features of the hand to tones.\n\n🧠 **Fuzzy Logic** – Uses hand pose and geometry to intelligently determine tones.\n\n📏 **Pinhole Depth Estimation** – Calculates hand distance from the camera and translates it into sound (see [theory](./cameraCalibration/README.md)).\n\n\u003ch2\u003eTools used\u003c/h2\u003e\n\n- [Python v3.10.12](https://docs.python-guide.org/starting/install3/linux/) for programming\n- [`opencv_contrib_python`](https://github.com/opencv/opencv-python?tab=readme-ov-file#installation-and-usage) for camera usage and frames manipulation\n- [`mediapipe`](https://github.com/google-ai-edge/mediapipe) for hand tracking\n- [`pyo`](https://github.com/belangeo/pyo) for audio managment\n- [`pipreqs`](https://github.com/bndr/pipreqs) for listing all requirements\n- [`skfuzzy`](https://pythonhosted.org/scikit-fuzzy/) for fuzzy sets implementation of tone control\n\n\u003ch2\u003eUsage of the theremin in a virtual environment\u003c/h2\u003e\n\nInstall venv, e.g.\n`sudo apt install python3.10-venv `\n\nCreate a virtual environment:\n`python3 -m venv test`\n\nActivate it:\n`source ./bin/activate`\n\nOpen VS Code:\n`code .`\n\nBe sure to use the correct Python interpreter, e.g.:\n`../bin/python3`\n\nInstall required dependencies:\n`python3 -m pip install -r requirements.txt`\n\nRun main program:\n`python3 main.py`\n\n\u003ch2\u003eTo run tests\u003c/h2\u003e\n\n`pytest tests/camera_test.py -v --tb=short --camera 0`\n\n`python3 tests/handtracking_test.py`\n\n\u003ch2\u003eOther libraries considered but not used\u003c/h2\u003e\n\n- [pygame](https://www.pygame.org/news) \n- [sounddevice](https://python-sounddevice.readthedocs.io/en/0.5.1/)\n- [pyaudio](https://people.csail.mit.edu/hubert/pyaudio/) \n\n\u003ch2\u003eInteresting links\u003c/h2\u003e\n\n- https://mediapipe.readthedocs.io/en/latest/solutions/hands.html\n- https://medium.com/@stevehiehn/how-to-generate-music-with-python-the-basics-62e8ea9b99a5\n- https://www.youtube.com/watch?v=m_rmwcUREeY\n- https://gist.github.com/sahithyen/b20922c902620e5bd6fd926263a93836\n- https://splice.com/blog/how-theremin-works/\n\n\u003ch2\u003eReferences\u003c/h2\u003e\n- Bartelt, T. L. M. (2001). Industrial Control Electronics: Devices, Systems \u0026 Applications (2ª ed.). Thomson Delmar Learning (p. 73).\n\n\u003ch2\u003eTODO\u003c/h2\u003e\n\n- Versión para móvil o web.\n\n- Hacer interfaz estática tipo HUD como [aqui](https://www.linkedin.com/posts/francastano_visiaejnartificial-inteligenciaartificial-activity-7307811734059151362-2_2m?utm_source=share\u0026utm_medium=member_desktop\u0026rcm=ACoAAFXLzVABTf15btKvx3DmtCu91bAxFIwl-gs).\n\n- Otro modo de funcionamiento: monocular dephtmap estimation con DL, en vez de la aproximación pinhole que hice, ver [esto](https://stackoverflow.com/questions/64685185/is-there-a-way-to-generate-real-time-depthmap-from-single-camera-video-in-python) y [esto](https://openaccess.thecvf.com/content_ICCV_2019/html/Godard_Digging_Into_Self-Supervised_Monocular_Depth_Estimation_ICCV_2019_paper.html).\n\n- Usar barras de estado de [este estilo](https://www.linkedin.com/posts/siddhartha-reddy-054321341_python-opencv-mediapipe-activity-7360493210772598784-o8ko?utm_source=share\u0026utm_medium=member_android\u0026rcm=ACoAAFXLzVABTf15btKvx3DmtCu91bAxFIwl-gs)\n\n- Hacer otro plugin que use kalman filter pal seguimiento mano theremin (tamaño posición,..), ver pag 183 simon prince\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fagarnung%2Fcv-theremin","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fagarnung%2Fcv-theremin","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fagarnung%2Fcv-theremin/lists"}