{"id":16467228,"url":"https://github.com/norgul/handy","last_synced_at":"2026-06-09T04:31:46.182Z","repository":{"id":117332843,"uuid":"595469216","full_name":"Norgul/handy","owner":"Norgul","description":null,"archived":false,"fork":false,"pushed_at":"2024-04-08T18:04:28.000Z","size":140,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-11-23T04:03:33.285Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Norgul.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-01-31T06:17:08.000Z","updated_at":"2024-04-16T11:11:08.000Z","dependencies_parsed_at":"2025-02-27T20:15:11.929Z","dependency_job_id":"b596bc83-973c-4699-aa0c-939b28e66b81","html_url":"https://github.com/Norgul/handy","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Norgul/handy","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Norgul%2Fhandy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Norgul%2Fhandy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Norgul%2Fhandy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Norgul%2Fhandy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Norgul","download_url":"https://codeload.github.com/Norgul/handy/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Norgul%2Fhandy/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34092253,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-09T02:00:06.510Z","response_time":63,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-11T11:46:38.765Z","updated_at":"2026-06-09T04:31:46.157Z","avatar_url":"https://github.com/Norgul.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Computer vision hand-controller\n\nThis is a side-project I created as a result of me wanting to control Blender by waving my hands in front of the \nscreen instead of using keyboard and mouse. Turned out to be loads of fun along the way!\n\n# Quick run\n\nTo try it out, just run the main function and have fun. When camera starts, put both hands in front of the screen.\n\n## Gestures\n### Left hand\n\n- `Cursor \"normal\" mode` (or rather, hand operated cursor) - make a fist, and then open your hand fully (for a greater \ncool effect do it like you're throwing sand in someone's face, whoosh!). Your monitor will be represented in smaller \nscale on the screen, and you can move hand within those bounds to control the mouse.\n- ``Cursor \"fine\" mode`` - while in normal mode, flick your thumb inwards to switch to a fine mode which will move the \nmouse gradually depending on how far away you drifted from original location where you turned the fine mode on. To\nbetter understand, just try it, and you'll see a line from origin point to your hand. Longer the line, faster the movement.\n\n- `Thumb + Index finger` touch - left mouse click (as long as you hold them together, click is active)\n- `Thumb + Middle finger` touch - middle mouse click \n- `Thumb + Ring finger` touch - right mouse click \n\n### Right hand\n\n- ``Grab`` (make a fist) - keyboard Shift\n\n- ``Thumb + Index finger`` - keyboard G (Blender move)\n- ``Thumb + Middle finger`` - keyboard R (Blender rotate)\n- ``Thumb + Ring finger`` - keyboard S (Blender scale)\n- ``Thumb + Pinky finger`` - keyboard E (Blender extrude)\n\n- ``Pointing one`` (index finger up) - keyboard X (isolate X axis)\n- ``Pointing two`` (index and middle finger up) - keyboard Y (isolate Y axis)\n- ``Pointing three`` (index and middle finger up, thumb pointing out) - keyboard Z (isolate Z axis)\n\n# Concepts and project structure\n\nWe start in the `main` function where events and listeners are registered in a following way:\n\n```python\nEvent: [ \n    Listener,\n    Listener2\n]\n```\n\nEvents are representing gestures (indicating something _has happened_), and listeners represent what is to be executed \nafter event has happened (indicating to _do something_). I.e. thumb and index finger touched (event), now execute a \nmouse click (listener).\n\nCode has a concept of inverse events which are simply event opposites to be able to trigger an action when opposite happens.\nI.e. thumb and index touch is an event, opposite event is thumb and index release. We would, for example, trigger mouse \nclick on touch, keep it clicked until release when we'd execute a mouse click release. \n\n```python\nEvent: InverseEvent\n```\n\n```\nroot\n├── Events\n│   └── ...\n└── Listeners\n    └── ...\n```\n\nNext we extract hands (using [MediaPipe](https://developers.google.com/mediapipe/solutions/vision/hand_landmarker)) and \ncreate ``Hand`` objects from them, making sure we separate left and right hand. Each `Hand` has many `Fingers`.\n\n```\nroot\n└── Models\n    ├── Fingers\n    ├── Hand\n    └── MpHands\n```\n\nLastly, we load the gestures. Loading of gestures happens in a way that events are dispatched under certain condition.\nSo when you see ``Event.when(something happens).dispatch()``, it means (given that this is triggering each frame) that\nas long as condition is satisfied, this event will keep on triggering. \n\nIf you registered a reverse event, you don't need to specifically call ``InverseEvent.when(opposite happens).dispatch()``,\nit will happen behind the scenes.\n\nOne thing to mention is that although events will keep on firing each and every frame the condition is satisfied, \nlisteners will fire either:\n- each and every frame \n- once\n\nThe determining factor is what I call a ``flip_flop`` mode on `Listener` object. If turned on, it will make sure to flip\nthe listener only once per satisfied event. Only when inverse event is triggered will that state return to false enabling\nit to be fired next time the condition is satisfied. ``flip_flop`` is false by default, meaning it is operating in \"stream\"\nmode. \n\nSome examples when this makes sense:\n- ``flip_flop`` - mouse click. You want to execute a click once per gesture satisfied, otherwise you'd get X amount of \nclicks.\n- ``stream`` - mouse move. You want each frame to read new coordinates and move mouse as long as condition is satisfied,\notherwise you'd get a single mouse nudge and that would be it.\n\n# Contributing\n\nI welcome all to join the project and contribute! \n\nAlso, feel free to reach out with any suggestions, bug reports or objections, those are valuable contributions as well,\nand are inspiring me to continue.\n\nUsing [Black formatter](https://marketplace.visualstudio.com/items?itemName=ms-python.black-formatter) for code formatting,\nbut set with 119 line length within `pyproject.toml` file.\n\nSome of TODOs and ideas:\n- Stabilize and smoothen the mouse movement (having a hand in front of the camera perfectly still is rather hard)\n- Fill out the rest of the gestures still unassigned (probably also test out with Blender what makes the most sense)\n- Introduce blocking events (i.e. when cursor mode is active, and you enter camera move, \ndisable cursor mode until you release the camera)\n- Introduce new point left, right, up, down gestures to enable Blender camera view front, back, up, down\n- Introduce complex modes (i.e. have one set of gestures when cursor mode is active, and other set when inactive)\n- Introduce dynamic sensitivity for cursor mode (i.e. when cursor mode is active, do some gesture with right hand to make\ndrawn canvas bigger or smaller for finer or faster mouse movements)\n- Flip hands through config\n- Improve UX by dynamically changing points on hands which are currently active, and other which can be activated \nwithin that mode\n- Prevent detecting random movements as gestures (i.e. drinking coffee in front of the screen would trigger events)\n- (continuous) Measure the time of the functions and make sure that all actions are smooth and non-blocking\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnorgul%2Fhandy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnorgul%2Fhandy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnorgul%2Fhandy/lists"}