https://github.com/nikhilr612/handlenz
Mediapipe Application to generate sound from real-time hand landmarks.
https://github.com/nikhilr612/handlenz
c-bindings hand-landmark hand-landmark-detection libmp mediapipe music music-generation sound
Last synced: about 2 months ago
JSON representation
Mediapipe Application to generate sound from real-time hand landmarks.
- Host: GitHub
- URL: https://github.com/nikhilr612/handlenz
- Owner: nikhilr612
- License: apache-2.0
- Created: 2024-05-18T05:26:16.000Z (about 1 year ago)
- Default Branch: master
- Last Pushed: 2024-05-20T15:27:52.000Z (12 months ago)
- Last Synced: 2025-02-09T09:14:47.025Z (3 months ago)
- Topics: c-bindings, hand-landmark, hand-landmark-detection, libmp, mediapipe, music, music-generation, sound
- Language: CMake
- Homepage:
- Size: 20.5 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# HandLenz
A MediaPipe Application that performs real-time hand landmark detection, and uses the resulting landmarks to generate MIDI messages, by mapping landmark space to pitch space using a provided scale.
# HandLenz Application Usage
- See [LibMP - MediaPipe as a Shared Library](https://github.com/rajkundu/mediapipe) for more information on LibMP
- This application does the following in realtime:
1. Streams video frames from an attached camera/webcam (OpenCV Device #0)
2. Runs MediaPipe Hand Landmark on each frame
3. Uses LibMP to get the XYZ coordinates of detected landmarks
4. Draws a circle at each landmark's location on the input frame
5. Displays the result using OpenCV/HighGUI
6. Produces sound
- Compilation is managed using CMake - see [`CMakeLists.txt`](/CMakeLists.txt) and [`libprotobuf.cmake`](/libprotobuf.cmake)
- Change `${MEDIAPIPE_DIR}` in `CMakeLists.txt` to match your installation of LibMP
- This application uses a custom [fork](https://github.com/nikhilr612/libmp) of the original LibMP project.
- Tested on Windows (MSVC 17.9.8)
## Prerequisites
1. Clone and build [LibMP](https://github.com/nikhilr612/libmp)
## Building
1. Compile using CMake
2. `cd` into output directory containing `main` binary
3. Run `main` binary