{"id":44284027,"url":"https://github.com/wengmister/hand-tracking-sdk","last_synced_at":"2026-03-11T10:01:25.047Z","repository":{"id":337563295,"uuid":"1152470174","full_name":"wengmister/hand-tracking-sdk","owner":"wengmister","description":"Python SDK for Hand Tracking Streamer VR App","archived":false,"fork":false,"pushed_at":"2026-03-08T23:36:18.000Z","size":33601,"stargazers_count":6,"open_issues_count":1,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-03-09T00:53:06.698Z","etag":null,"topics":["handtracking","robotics"],"latest_commit_sha":null,"homepage":"https://hand-tracking-sdk.readthedocs.io/en/latest/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/wengmister.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-02-07T23:16:29.000Z","updated_at":"2026-02-25T11:24:39.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/wengmister/hand-tracking-sdk","commit_stats":null,"previous_names":["wengmister/hand-tracking-sdk"],"tags_count":5,"template":false,"template_full_name":null,"purl":"pkg:github/wengmister/hand-tracking-sdk","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wengmister%2Fhand-tracking-sdk","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wengmister%2Fhand-tracking-sdk/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wengmister%2Fhand-tracking-sdk/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wengmister%2Fhand-tracking-sdk/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/wengmister","download_url":"https://codeload.github.com/wengmister/hand-tracking-sdk/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wengmister%2Fhand-tracking-sdk/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30377837,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-11T06:09:32.197Z","status":"ssl_error","status_checked_at":"2026-03-11T06:09:17.086Z","response_time":84,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["handtracking","robotics"],"created_at":"2026-02-10T22:10:33.033Z","updated_at":"2026-03-11T10:01:25.034Z","avatar_url":"https://github.com/wengmister.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n  \u003cimg width=\"512\" height=\"362\" alt=\"sdk_logo\" src=\"https://github.com/user-attachments/assets/33cbe2c6-2da9-4868-b5f0-c66e7abc6e3e\" /\u003e\n  \u003ch3 align=\"center\"\u003e\n    Python SDK for consuming telemetry from\n    \u003ca href=\"https://github.com/wengmister/hand-tracking-streamer\"\u003eHand Tracking Streamer (HTS)\u003c/a\u003e\n  \u003c/h3\u003e\n\u003c/div\u003e\n\u003cp align=\"center\"\u003e\n\n  \u003ca href=\"https://www.meta.com/experiences/hand-tracking-streamer/26303946202523164/\"\u003e\n   \u003cimg src=\"https://img.shields.io/badge/VR_app-Meta_Quest_Store-FF5757?labelColor=grey\" alt=\"Horizon Store Release\"\u003e\n  \u003c/a\u003e\n\n  \u003ca href=\"https://github.com/wengmister/hand-tracking-streamer\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/VR_app-GitHub-orange?labelColor=grey\" alt=\"Python SDK\"\u003e\n  \u003c/a\u003e\n\n  \u003ca href=\"https://github.com/wengmister/hand-tracking-streamer/blob/main/LICENSE\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/license-Apache%20License%202.0-yellow.svg\" alt=\"Apache 2.0\"\u003e\n  \u003c/a\u003e\n\n  \u003ca href=\"https://hand-tracking-sdk.readthedocs.io/\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/API-ReadTheDocs-green.svg\" alt=\"API Documentation\"\u003e\n  \u003c/a\u003e\n\n\n\u003c/p\u003e\n\n**Hand Tracking SDK** is a Python package for consuming HTS hand-tracking telemetry (UDP/TCP), parsing wrist/landmark data into typed frames, and providing conversion, visualization, and integration-ready APIs.\n\nThis SDK is hosted on [PyPI](https://pypi.org/project/hand-tracking-sdk/), with API documentation [Here](https://hand-tracking-sdk.readthedocs.io/)\n\n## Installation\n\n```bash\npip install hand-tracking-sdk\n```\n\nOptional visualization support with Rerun:\n\n```bash\npip install \"hand-tracking-sdk[visualization]\"\n```\n\n## Quickstart\n\n```python\nfrom hand_tracking_sdk import HTSClient, HTSClientConfig, StreamOutput\n\nclient = HTSClient(\n    HTSClientConfig(\n        output=StreamOutput.BOTH,  # packets + assembled frames\n    )\n)\n\nfor event in client.iter_events():\n    print(event)\n```\n\n## What HTS Sends\n\nHTS emits UTF-8 CSV lines:\n- wrist packet: 7 floats (`x, y, z, qx, qy, qz, qw`)\n- landmarks packet: 63 floats (`21 x [x, y, z]`)\n\nThe SDK validates packet labels, hand side, and exact value counts.\n\n## Streaming Client\n\n`HTSClient` provides a high-level sync stream with filtering and error policy controls.\n\n- **Transport**: UDP, TCP server, TCP client\n- **Output**: raw packets, assembled frames, or both\n- **Hand filter**: left, right, or both\n- **Error policy**: strict (raise) or tolerant (skip malformed)\n- **Observability**: `get_stats()` counters, `log_hook` for structured events\n\n### Frame Assembly\n\n`HandFrameAssembler` correlates wrist + landmark packets into per-hand `HandFrame`\nobjects. Head pose packets produce `HeadFrame` events. Stale out-of-order updates\nare discarded.\n\nA `HandFrame` includes:\n- `side`: Left or Right\n- `wrist`: `WristPose(x, y, z, qx, qy, qz, qw)`\n- `landmarks.points`: 21 MediaPipe-style `(x, y, z)` joints\n- Per-joint access: `frame.get_joint(JointName.INDEX_TIP)`\n- Per-finger access: `frame.get_finger(\"index\")`\n- Timing metadata: `recv_ts_ns`, `source_ts_ns`, `sequence_id`\n\n### Coordinate Conversion\n\nExplicit Unity left-handed to right-handed converters:\n\n```python\nfrom hand_tracking_sdk.convert import (\n    convert_hand_frame_unity_left_to_right,\n    unity_left_to_rfu_position,          # right-forward-up\n    unity_left_to_rfu_rotation_matrix,\n)\n```\n\n### Joint \u0026 Finger Access\n\nTo get telemetry for a specific joint from a frame, use `get_joint(...)`.\nJoint names and order follow the HTS streamed contract (wrist is `JointName.WRIST`).\n\n```python\nfrom hand_tracking_sdk import HTSClient, HTSClientConfig, JointName, StreamOutput\n\nclient = HTSClient(HTSClientConfig(output=StreamOutput.FRAMES))\n\nfor frame in client.iter_events():\n    x, y, z = frame.get_joint(JointName.INDEX_TIP)\n    print(\n        f\"side={frame.side.value} joint={JointName.INDEX_TIP.value} \"\n        f\"xyz=({x:.5f}, {y:.5f}, {z:.5f}) recv_ts_ns={frame.recv_ts_ns}\"\n    )\n```\n\nYou can also query by finger group:\n\n```python\nindex_points = frame.get_finger(\"index\")\n# returns dict[JointName, tuple[float, float, float]]\n# keys include JointName.INDEX_PROXIMAL, JointName.INDEX_TIP, ...\n```\n\n## Examples\n\n### Telemetry\n\n| Script | Description |\n|--------|-------------|\n| `examples/visualize_rerun.py` | Rerun 3D visualization with coordinate frames and jitter metrics |\n| `examples/stream_frames.py` | Print assembled frames to console |\n| `examples/log_to_jsonl.py` | JSONL capture for replay and analysis |\n| `examples/jitter_report.py` | Timing jitter report |\n\n```bash\nuv run python examples/visualize_rerun.py --transport tcp_server --host 0.0.0.0 --port 8000\n```\n\n### Video Host (WebRTC)\n\nHost-side scripts that stream video back to the Quest headset over WebRTC.\nSee [`examples/video/README.md`](examples/video/README.md) for details.\n\n| Script | Source | Description |\n|--------|--------|-------------|\n| `test_pattern_video_host.py` | Test pattern | Colour bars — no hardware needed |\n| `webcam_video_host.py` | USB webcam | Streams a local camera feed |\n| `inspire_hand_video_host.py` | MuJoCo | Bimanual Inspire Hand with vector retargeting |\n| `shadow_hand_video_host.py` | MuJoCo | Bimanual Shadow Hand E3M5 with vector retargeting |\n| `aloha_video_host.py` | MuJoCo | ALOHA 2 bimanual arms with IK |\n\n```bash\n# Test pattern — no extra dependencies:\nuv run examples/video/test_pattern_video_host.py\n\n# Shadow Hand bimanual retargeting:\nuv run examples/video/shadow_hand_video_host.py --mocap-tcp-port 5555\n```\n\n### Simulation Teleop\n\nThe MuJoCo video hosts close a full teleoperation loop: Quest sends hand + head\nmocap over TCP, Python drives a MuJoCo simulation, and the rendered camera view\nstreams back to the headset over WebRTC.\n\n```\nQuest 3/3S ──TCP──► HTSClient ──► MuJoCo pre_step + mj_step + render\n                                                     │\nQuest 3/3S ◄────────────── WebRTC H.264 ◄────────────┘\n```\n\n## Protocol and Docs\n\n- HTS protocol: [hand-tracking-streamer README](https://github.com/wengmister/hand-tracking-streamer/blob/main/README.md)\n- SDK API docs: [hand-tracking-sdk.readthedocs.io](https://hand-tracking-sdk.readthedocs.io/)\n\n## License\n\nApache-2.0\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwengmister%2Fhand-tracking-sdk","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwengmister%2Fhand-tracking-sdk","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwengmister%2Fhand-tracking-sdk/lists"}