{"id":23628463,"url":"https://github.com/prodev717/cv-controller","last_synced_at":"2025-11-08T00:30:37.352Z","repository":{"id":269815066,"uuid":"908540767","full_name":"prodev717/cv-controller","owner":"prodev717","description":"Open-source project to control games using physical movements and computer vision","archived":false,"fork":false,"pushed_at":"2025-01-24T16:14:25.000Z","size":9,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-01-24T17:23:21.842Z","etag":null,"topics":["computer-vision","game","gamecontroller","keyboard","mediapipe","opencv","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/prodev717.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-12-26T10:19:45.000Z","updated_at":"2025-01-24T16:14:28.000Z","dependencies_parsed_at":null,"dependency_job_id":"4b83e639-d334-49fd-a81d-cd2e90e7d2d0","html_url":"https://github.com/prodev717/cv-controller","commit_stats":null,"previous_names":["prodev717/cv-controller"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prodev717%2Fcv-controller","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prodev717%2Fcv-controller/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prodev717%2Fcv-controller/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prodev717%2Fcv-controller/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/prodev717","download_url":"https://codeload.github.com/prodev717/cv-controller/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":239543405,"owners_count":19656353,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["computer-vision","game","gamecontroller","keyboard","mediapipe","opencv","python"],"created_at":"2024-12-28T00:45:35.571Z","updated_at":"2025-02-18T20:25:54.458Z","avatar_url":"https://github.com/prodev717.png","language":"Python","readme":"# CV Controller\n\n**CV Controller** is an innovative computer vision project that transforms physical movements detected via a webcam into interactive controls for gaming and desktop navigation. By leveraging **OpenCV**, **Mediapipe**, and **pydirectinput**, this project offers two main functionalities:\n- Hands-free directional controls using body movements.\n- Mouse control through hand gestures, complete with drag-and-drop support.\n\n---\n\n## Features\n### 1. **Real-Time Movement Detection**\n- Tracks body movements using Mediapipe's Pose module.\n- Detects directional motion (up, down, left, right) based on keypoint displacement.\n\n### 2. **Dynamic Game Controls**\n- Simulates keyboard presses (`W`, `A`, `S`, `D`) corresponding to detected movement direction.\n- Supports smooth, real-time control for games or applications requiring directional input.\n\n### 3. **Mouse Control with Hand Gestures**\n- Tracks hand movements using Mediapipe's Hand module.\n- Converts wrist position into mouse movements with adjustable acceleration.\n- Detects hand closure to enable drag-and-drop functionality:\n  - **Hand Closed**: Activates drag mode.\n  - **Hand Open**: Releases drag mode.\n\n---\n\n## How It Works\n### Game Controls:\n1. **Input via Webcam**:\n   - Captures real-time video feed for body landmark tracking.\n2. **Landmark Tracking**:\n   - Uses the nose as the primary movement point.\n3. **Direction Detection**:\n   - Calculates movement direction based on the nose's position changes:\n     - **Up (`W`)**: Nose moves upward.\n     - **Down (`S`)**: Nose moves downward.\n     - **Left (`A`)**: Nose moves left.\n     - **Right (`D`)**: Nose moves right.\n4. **Simulated Keyboard Input**:\n   - Sends the detected direction as a keyboard press using the `pydirectinput` library.\n\n### Hand Mouse Control:\n1. **Hand Detection**:\n   - Tracks hand landmarks using Mediapipe's Hands solution.\n2. **Mouse Movement**:\n   - Maps wrist movements to mouse cursor movements, with configurable acceleration:\n     - **Acceleration Factor**: Adjusts cursor speed based on hand velocity.\n3. **Drag-and-Drop**:\n   - Detects hand closure to enable drag-and-drop:\n     - **Mouse Down**: Hand closes.\n     - **Mouse Up**: Hand opens.\n\n---\n\n## Key Components\n- **OpenCV**:\n  - Handles video feed capture and real-time visualization.\n  \n- **Mediapipe**:\n  - Provides landmark detection for hands and pose estimation.\n\n- **PyDirectInput**:\n  - Simulates mouse movements, clicks, and drag-and-drop functionality.\n  - Simulates keyboard inputs for directional control.\n\n---\n\n## Use Cases\n- **Gaming**:\n  - Hands-free gaming using directional controls.\n- **Accessibility**:\n  - Use hand gestures to replace traditional mouse functions.\n- **Interactive Applications**:\n  - Intuitive interaction with desktop environments.\n\n---\n\n## How to Quit\n- Press the `q` key to stop the application.\n\n---\n\n## Output Display\n1. **Game Controls**:\n   - Displays movement details, such as position differences (`dif`), distance (`dis`), and movement direction (`mov`) on the video feed.\n2. **Hand Mouse Control**:\n   - Visualizes hand landmarks and tracks wrist position on the video feed.\n\n---\n\n## Notes\n- The hand mouse control feature is designed for single-hand tracking.\n- Adjustable parameters:\n  - **Acceleration (`acc`)**: Controls cursor speed for normal movements.\n  - **Drag Acceleration (`accd`)**: Adjusts speed during drag-and-drop.\n  - **Hand Closure Sensitivity (`close`)**: Defines the threshold for detecting a closed hand.\n\n---\n\n## Contributions\nContributions to improve or extend the functionality of this project are welcome! Feel free to open issues or submit pull requests.\n\n---\n\n## License\nThis project is open-source and available under the MIT License.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprodev717%2Fcv-controller","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fprodev717%2Fcv-controller","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprodev717%2Fcv-controller/lists"}