{"id":51542784,"url":"https://github.com/damiansire/web-ar-hand-tracking","last_synced_at":"2026-07-09T15:30:20.668Z","repository":{"id":266814578,"uuid":"899401971","full_name":"damiansire/web-ar-hand-tracking","owner":"damiansire","description":"Web-based Augmented Reality app featuring hand-tracking with MediaPipe (in a Web Worker) and 3D figures rendered with Three.js.","archived":false,"fork":false,"pushed_at":"2026-07-09T04:13:36.000Z","size":426,"stargazers_count":15,"open_issues_count":7,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2026-07-09T06:10:05.325Z","etag":null,"topics":["angular","augmented-reality","computer-vision","hand-tracking","mediapipe","test-management","threejs","typescript","web-ar","web-worker"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/damiansire.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2024-12-06T07:34:06.000Z","updated_at":"2026-07-09T04:10:40.000Z","dependencies_parsed_at":null,"dependency_job_id":"a18f8f9c-39e9-494f-aa78-e281ee1edb04","html_url":"https://github.com/damiansire/web-ar-hand-tracking","commit_stats":null,"previous_names":["damiansire/test-case-manager","damiansire/ai-examples","damiansire/web-ar-hand-tracking"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/damiansire/web-ar-hand-tracking","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/damiansire%2Fweb-ar-hand-tracking","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/damiansire%2Fweb-ar-hand-tracking/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/damiansire%2Fweb-ar-hand-tracking/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/damiansire%2Fweb-ar-hand-tracking/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/damiansire","download_url":"https://codeload.github.com/damiansire/web-ar-hand-tracking/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/damiansire%2Fweb-ar-hand-tracking/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35304874,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-09T02:00:07.329Z","response_time":57,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["angular","augmented-reality","computer-vision","hand-tracking","mediapipe","test-management","threejs","typescript","web-ar","web-worker"],"created_at":"2026-07-09T15:30:18.633Z","updated_at":"2026-07-09T15:30:20.660Z","avatar_url":"https://github.com/damiansire.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AR Hand Figures\n\n[![CI](https://github.com/damiansire/artificial-intelligence-augmented-reality-figures/actions/workflows/ci.yml/badge.svg)](https://github.com/damiansire/artificial-intelligence-augmented-reality-figures/actions/workflows/ci.yml)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)\n\nAugmented reality in the browser: it detects your hand with the camera and draws\na 3D figure that follows it in real time. Detection runs in a **Web Worker**\n(MediaPipe Hand Landmarker) so the main thread stays unblocked, and the 3D\nrendering is done with **Three.js**.\n\n**▶ Live demo:** https://damiansire.github.io/artificial-intelligence-augmented-reality-figures/\n_(requires a camera; the video never leaves your device)_\n\n\u003e Complete rewrite of the original version (p5.js + ml5.js on the main thread).\n\u003e The stack was modernized, the pure domain was separated from the imperative\n\u003e _shells_, and inference was moved to a worker.\n\n## What you can do\n\nThe app ships **5 creative experiences** selectable from the bottom picker (the\ndifferentiator from the original p5.js + ml5.js version, which only tracked the\nhand with a single figure). Each mode decides internally how it uses the hands:\n\n| Mode           | What it does                                                                        |\n| -------------- | ----------------------------------------------------------------------------------- |\n| **3D Figures** | Move your hand: the figure follows it.                                              |\n| **Draw**       | Draw with your index finger · pinch your fingers to move · open your hand to erase. |\n| **Catch**      | Catch the circles with your hand and score points.                                  |\n| **Cosmos**     | Move your hand: the nebula orbits · pinch to form a planet · release for the burst. |\n| **Lasers**     | Your hand lights up in neon · show both hands for beams between them.               |\n\n\u003e This table mirrors `EXPERIENCES` in\n\u003e [`src/domain/experiences.ts`](src/domain/experiences.ts) (labels + hints); if\n\u003e you add or change a mode, update both so they don't drift out of sync.\n\n### 3D Figures mode — detail\n\nChoose between 6 3D figures that follow the hand (with **perspective**: closer =\nbigger), adjust size/speed/opacity/material/color, show edges or wireframe,\nshadow, **two hands** at once, colored background, **occlusion** (the figure goes\nbehind when you flip your hand — _calibrated for the right hand_: with the left\nhand, occlusion triggers on the palm instead of the back), and **take a photo**\n(downloads a PNG). When there's no hand, the figure stays as a preview in the\ncorner.\n\n## How it works\n\n```\n┌───────────────── main thread ────────────────────┐      ┌──── Web Worker ────┐\n│  camera (getUserMedia) ──► \u003cvideo\u003e                │      │  MediaPipe         │\n│        │ ImageBitmap (transferable)               │ ───► │  HandLandmarker    │\n│        ▼                                          │      │  (WASM + GPU)      │\n│  Three.js  ◄── landmarks ──────────────────────── │ ◄─── │  detectForVideo()  │\n│  (3D figure over the hand)                        │      └────────────────────┘\n└───────────────────────────────────────────────────┘\n```\n\n- **`src/domain/`** — pure, tested logic (state machine, landmark-to-screen\n  mapping, figure catalog). No DOM, no dependencies.\n- **`src/camera/`** — camera access with typed errors.\n- **`src/inference/`** — the MediaPipe worker and its client with back-pressure\n  (a single frame in flight; if another arrives before the previous finishes, it\n  is dropped).\n- **`src/render/`** — Three.js scene with an orthographic camera mapped to pixels.\n- **`src/ui/`** — screens (permission / loading / error) and the `\u003cfigure-selector\u003e`.\n\n## Requirements\n\n- Node.js ≥ 20\n- A browser with WebGL and `getUserMedia` (HTTPS or `localhost`).\n\n## Development\n\n```bash\nnpm install\nnpm run dev        # development server (Vite)\nnpm test           # domain tests (Vitest)\nnpm run typecheck  # TypeScript in strict mode\nnpm run format     # format with Prettier\nnpm run build      # production build to dist/\n```\n\n\u003e The camera only works on `localhost` or over HTTPS (a browser requirement).\n\n## Deployment\n\nA workflow ([`.github/workflows/deploy.yml`](.github/workflows/deploy.yml))\npublishes `dist/` to **GitHub Pages** on every push to `main`/`master`. To\nenable it: Settings → Pages → Source: **GitHub Actions**. The `base` is relative\n(`./`), so it works both at the root and under a project sub-path.\n\n## Model configuration\n\nThe MediaPipe assets (JS bundle + WASM + the `.task` model) are loaded from the\nofficial CDN, pinned by version in [`src/config.ts`](src/config.ts). To\nself-host them, copy those files to `public/` and change the URLs.\n\nThe worker is **classic** (not a module worker) and loads MediaPipe with\n`importScripts`: MediaPipe requires it, and this way the same code runs the same\nin the dev server and in the build. Details in\n[`hand-landmarker.worker.ts`](src/inference/hand-landmarker.worker.ts).\n\n## Stack\n\nVite · TypeScript · Three.js · @mediapipe/tasks-vision · Vitest\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdamiansire%2Fweb-ar-hand-tracking","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdamiansire%2Fweb-ar-hand-tracking","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdamiansire%2Fweb-ar-hand-tracking/lists"}