{"id":20339047,"url":"https://github.com/szczyglis-dev/js-ai-body-tracker","last_synced_at":"2025-04-24T03:45:29.818Z","repository":{"id":47366968,"uuid":"516079032","full_name":"szczyglis-dev/js-ai-body-tracker","owner":"szczyglis-dev","description":"[JS/TensorFlow] JavaScript library that implements machine learning-based models for human pose estimation and human movement analysis. It allows you to easily implement three neural network models for human pose estimation: MoveNet, PoseNet, and BlazePose. No Node.js is required. TensorFlowJS was used, and an example app is included.","archived":false,"fork":false,"pushed_at":"2024-08-26T04:47:09.000Z","size":185,"stargazers_count":64,"open_issues_count":1,"forks_count":11,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-04-24T03:45:23.166Z","etag":null,"topics":["artificial-intelligence","blazepose","body-tracking","iptv","javascript","library","m3u8","machine-learning","movenet","mp4-video","neural-network","pose-estimation","posenet","realtime","tensorflow","tfjs","tfjs-models","video-streaming","webbrowser","webcam-capture"],"latest_commit_sha":null,"homepage":"https://szczyglis.dev/js-ai-body-tracker","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/szczyglis-dev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-07-20T17:41:19.000Z","updated_at":"2025-04-06T13:31:10.000Z","dependencies_parsed_at":"2025-02-10T02:00:29.894Z","dependency_job_id":"93b9f64d-eba3-4a7c-9f00-15e618db13c3","html_url":"https://github.com/szczyglis-dev/js-ai-body-tracker","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/szczyglis-dev%2Fjs-ai-body-tracker","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/szczyglis-dev%2Fjs-ai-body-tracker/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/szczyglis-dev%2Fjs-ai-body-tracker/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/szczyglis-dev%2Fjs-ai-body-tracker/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/szczyglis-dev","download_url":"https://codeload.github.com/szczyglis-dev/js-ai-body-tracker/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250559993,"owners_count":21450168,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["artificial-intelligence","blazepose","body-tracking","iptv","javascript","library","m3u8","machine-learning","movenet","mp4-video","neural-network","pose-estimation","posenet","realtime","tensorflow","tfjs","tfjs-models","video-streaming","webbrowser","webcam-capture"],"created_at":"2024-11-14T21:15:14.232Z","updated_at":"2025-04-24T03:45:29.800Z","avatar_url":"https://github.com/szczyglis-dev.png","language":"JavaScript","funding_links":["https://www.buymeacoffee.com/szczyglis"],"categories":[],"sub_categories":[],"readme":"Release: **1.0.1** | build: **2024.08.26** | Javascript / TensorFlow JS\n\n# JS AI BODY TRACKER - tracker.js\n\n**JS AI Body Tracker `(tracker.js)` is a JavaScript library that implements machine learning-based models for human pose estimation and movement analysis. The library is written in JavaScript and does not require Node.js. It supports three different models for detecting human poses in videos: `MoveNet`, `PoseNet`, and `BlazePose`. The library offers real-time video analysis from three different sources: web/smartphone cameras, video files (mp4, mkv, avi, webm), and online streaming (IPTV/m3u8).**\n## Features\n\n- Easy to implement in your own JavaScript application\n- Works in real-time directly in the browser\n- Supports multiple video sources: web cameras, video files (mp4, mkv, avi, webm), online streams (IPTV/m3u8)\n- Offers three different neural network models: `MoveNet`, `PoseNet`, and `BlazePose`\n- Provides real-time 3D mapping (BlazePose)\n- Can be easily extended with events and hooks\n- Consists of only one file for direct import into your application; no Node.js required\n- Uses `TensorFlow JS`, `ScatterGL`, and `videoJS` libraries\n\nReal-time Human Pose Estimation on MP4 Video:\n\n![gif1](https://user-images.githubusercontent.com/61396542/180048048-ebd4fdbe-9b0e-43a8-b34a-255bd092e366.gif)\n![gif2](https://user-images.githubusercontent.com/61396542/180047990-523f6706-1241-4cd6-9114-8e1e887cce7f.gif)\n\nReal-time human pose estimation on IPTV/m3u8 online stream:\n\n![gif3](https://user-images.githubusercontent.com/61396542/180048146-80cf69b3-26ac-4841-bbe5-7c16cc757491.gif)\n\n\n## Live demo\n\nA demo of a real-time working sample application using the library can be found here:\n\n## https://szczyglis.dev/js-ai-body-tracker\n\n\nThere are three input video sources available in the demo application: web camera, video file, and IPTV/m3u8 stream. The entire process of image analysis takes place in real-time.\n\n## Basic usage\n\n```js\n// javascript\n\ntracker.setModel('MoveNetSinglePoseLightning'); // define model to use\n\ntracker.elCanvas = '#canvas'; // define HTML canvas container\ntracker.elVideo = '#video'; // define HTML video container\ntracker.el3D = '#view_3d'; // define HTML container for 3D view \n\ntracker.run('camera'); // run\n```\nThe repository includes a sample application that illustrates the operation and use of the library.\n\n## Usage step by step\n\nFirst, you need to define two HTML elements: a video and a canvas:\n\n```html\n\u003ccanvas id=\"canvas\" width=\"500\" height=\"500\"\u003e\u003c/canvas\u003e\n\u003cvideo id=\"video\" width=\"500\" height=\"500\"\u003e\n\t\u003csource src=\"\"\u003e\n\u003c/video\u003e\n```\nNext, you need to import the library and configure it:\n\n```html\n\u003cscript src=\"./js/tracker.js\"\u003e\u003c/script\u003e\n\u003cscript\u003e\n\ttracker.setModel('MoveNetSinglePoseLightning');\t\n\ttracker.elCanvas = '#canvas';\n\ttracker.elVideo = '#video';\n\ttracker.run('camera');\n\u003c/script\u003e\n```\n\n### Examples of use\n\n**1) VIDEO INPUT: Webcam / Smartphone Camera**\n\n```html\n\u003c!DOCTYPE html\u003e\n\u003chtml\u003e\n\u003chead\u003e\n\u003c/head\u003e\n\u003cbody\u003e\n\u003cdiv class=\"container\"\u003e\n\t\u003ccanvas id=\"canvas\" width=\"500\" height=\"500\"\u003e\u003c/canvas\u003e\n\t\u003cvideo id=\"video\" width=\"500\" height=\"500\" style=\"display:none\"\u003e\n\t\t\u003csource src=\"\"\u003e\n\t\u003c/video\u003e\t\t\t\t\n\u003c/div\u003e\n\n\u003c!-- Load Tensor Flow --\u003e\n\u003cscript src=\"https://cdn.jsdelivr.net/npm/@tensorflow/tfjs-core\"\u003e\u003c/script\u003e\n\u003cscript src=\"https://cdn.jsdelivr.net/npm/@tensorflow/tfjs-converter\"\u003e\u003c/script\u003e\n\u003cscript src=\"https://cdn.jsdelivr.net/npm/@tensorflow/tfjs-backend-webgl\"\u003e\u003c/script\u003e\n\u003cscript src=\"https://cdn.jsdelivr.net/npm/@tensorflow-models/pose-detection\"\u003e\u003c/script\u003e\n\n\u003c!-- Load three.js --\u003e\n\u003cscript src=\"https://cdnjs.cloudflare.com/ajax/libs/three.js/r134/three.min.js\"\u003e\u003c/script\u003e\n\u003c!-- Load scatter-gl.js --\u003e\n\u003cscript src=\"https://cdn.jsdelivr.net/npm/scatter-gl@0.0.13/lib/scatter-gl.min.js\"\u003e\u003c/script\u003e\n\n\u003c!-- Load tracker.js --\u003e\n\u003cscript src=\"./js/tracker.js\"\u003e\u003c/script\u003e\n\u003cscript\u003e\n\ttracker.setModel('MoveNetSinglePoseLightning');\n\ttracker.elCanvas = '#canvas';\n\ttracker.elVideo = '#video';\n\ttracker.run('camera');\n\u003c/script\u003e\n\u003c/body\u003e\n\u003c/html\u003e\n```\nThe above code activates the camera, initializes the model for detecting human poses, and then displays the detected points on the video from the camera on the canvas element in real time, superimposing them on the input from the webcam.\n\n**2) VIDEO INPUT: Movie File (mp4, mkv, avi, webm)**\n\n```html\n\u003c!DOCTYPE html\u003e\n\u003chtml\u003e\n\u003chead\u003e\n\u003c/head\u003e\n\u003cbody\u003e\n\u003cdiv class=\"container\"\u003e\n\t\u003ccanvas id=\"canvas\" width=\"500\" height=\"500\"\u003e\u003c/canvas\u003e\n\t\u003cvideo id=\"video\" width=\"500\" height=\"500\" style=\"display:none\"\u003e\n\t\t\u003csource src=\"movie.mp4\"\u003e\n\t\u003c/video\u003e\t\t\t\t\n\u003c/div\u003e\n\n\u003c!-- Load Tensor Flow --\u003e\n\u003cscript src=\"https://cdn.jsdelivr.net/npm/@tensorflow/tfjs-core\"\u003e\u003c/script\u003e\n\u003cscript src=\"https://cdn.jsdelivr.net/npm/@tensorflow/tfjs-converter\"\u003e\u003c/script\u003e\n\u003cscript src=\"https://cdn.jsdelivr.net/npm/@tensorflow/tfjs-backend-webgl\"\u003e\u003c/script\u003e\n\u003cscript src=\"https://cdn.jsdelivr.net/npm/@tensorflow-models/pose-detection\"\u003e\u003c/script\u003e\n\n\u003c!-- Load three.js --\u003e\n\u003cscript src=\"https://cdnjs.cloudflare.com/ajax/libs/three.js/r134/three.min.js\"\u003e\u003c/script\u003e\n\u003c!-- Load scatter-gl.js --\u003e\n\u003cscript src=\"https://cdn.jsdelivr.net/npm/scatter-gl@0.0.13/lib/scatter-gl.min.js\"\u003e\u003c/script\u003e\n\n\u003c!-- Load tracker.js --\u003e\n\u003cscript src=\"./js/tracker.js\"\u003e\u003c/script\u003e\n\u003cscript\u003e\n\ttracker.setModel('MoveNetSinglePoseLightning');\n\ttracker.elCanvas = '#canvas';\n\ttracker.elVideo = '#video';\n\ttracker.run('video');\n\u003c/script\u003e\n\u003c/body\u003e\n\u003c/html\u003e\n```\nThe above code loads the movie `movie.mp4`, initializes the human pose analysis model, and then displays the detected points on the video in real time on the canvas, superimposing them on the input video.\n\n**3) VIDEO INPUT: Online Stream (IPTV/m3u8)**\n\n```html\n\u003c!DOCTYPE html\u003e\n\u003chtml\u003e\n\u003chead\u003e\n\t\u003clink href=\"https://cdnjs.cloudflare.com/ajax/libs/video.js/7.0.0/video-js.css\" rel=\"stylesheet\"\u003e\n    \u003cscript src=\"https://cdnjs.cloudflare.com/ajax/libs/video.js/7.0.0/video.min.js\"\u003e\u003c/script\u003e\n\u003c/head\u003e\n\u003cbody\u003e\n\u003cdiv class=\"container\"\u003e\n\t\u003ccanvas id=\"canvas\" width=\"500\" height=\"500\"\u003e\u003c/canvas\u003e\n\t\u003cvideo id=\"video\" class=\"video-js vjs-fluid vjs-default-skin\" preload=\"metadata\" width=\"500\" height=\"500\" style=\"display:none\"\u003e\n\t\t\u003csource src=\"https://multiplatform-f.akamaihd.net/i/multi/will/bunny/big_buck_bunny_,640x360_400,640x360_700,640x360_1000,950x540_1500,.f4v.csmil/master.m3u8\"\u003e\n\t\u003c/video\u003e\t\t\t\t\n\u003c/div\u003e\n\n\u003c!-- Load Tensor Flow --\u003e\n\u003cscript src=\"https://cdn.jsdelivr.net/npm/@tensorflow/tfjs-core\"\u003e\u003c/script\u003e\n\u003cscript src=\"https://cdn.jsdelivr.net/npm/@tensorflow/tfjs-converter\"\u003e\u003c/script\u003e\n\u003cscript src=\"https://cdn.jsdelivr.net/npm/@tensorflow/tfjs-backend-webgl\"\u003e\u003c/script\u003e\n\u003cscript src=\"https://cdn.jsdelivr.net/npm/@tensorflow-models/pose-detection\"\u003e\u003c/script\u003e\n\n\u003c!-- Load three.js --\u003e\n\u003cscript src=\"https://cdnjs.cloudflare.com/ajax/libs/three.js/r134/three.min.js\"\u003e\u003c/script\u003e\n\u003c!-- Load scatter-gl.js --\u003e\n\u003cscript src=\"https://cdn.jsdelivr.net/npm/scatter-gl@0.0.13/lib/scatter-gl.min.js\"\u003e\u003c/script\u003e\n\n\u003c!-- Load tracker.js --\u003e\n\u003cscript src=\"./js/tracker.js\"\u003e\u003c/script\u003e\n\u003cscript\u003e\n\ttracker.setModel('MoveNetMultiPoseLightning');\n\ttracker.elCanvas = '#canvas';\n\ttracker.elVideo = '#video';\n\ttracker.run('stream');\n\u003c/script\u003e\n\u003c/body\u003e\n\u003c/html\u003e\n```\nThe above code opens an `.m3u8` online video stream, initializes the human pose detection model, and then displays the detected points on the video stream superimposed on the stream image in the canvas element in real-time. **Note that the `videoJS` library is used to handle the stream.**\n\n### Configuration\n\n**Model**\n\nThe library has preconfigured settings for three different neural network models. You can use these settings or define all options manually. To use the predefined model, use the `setModel()` method:\n```js\ntracker.setModel('MoveNetSinglePoseLightning');\n```\n\n**Available models**\n\n- `MoveNetSinglePoseLightning`\t\t\t\n- `MoveNetSinglePoseThunder`\n- `MoveNetMultiPoseLightning`\n- `PoseNetMobileNetV1`\n- `PoseNetResNet50`\n- `BlazePoseLite`\n- `BlazePoseHeavy`\n- `BlazePoseFull`\n\nYou can also define the settings manually, for example, using the MoveNet network with the MULTIPOSE settings:\n```js\ntracker.detectorModel = poseDetection.SupportedModels.MoveNet;\ntracker.detectorConfig = {\n    modelType: poseDetection.movenet.modelType.MULTIPOSE_LIGHTNING,\n    enableSmoothing: true,\n    multiPoseMaxDimension: 256,\n    enableTracking: true,\n    trackerType: poseDetection.TrackerType.BoundingBox\n}\ntracker.minScore = 0.35;\n```\nThe models differ in efficiency and accuracy. You can read more about these neural network models on the Google website:\nhttps://github.com/tensorflow/tfjs-models/tree/master/pose-detection\n\n\n**Input video source**\n\nYou can define three different video sources:\n\n- `camera`\n- `video`\n- `stream`\n\nTo select a source, call the method `run()` with the source name passed as an argument, e.g.:\n```js\ntracker.run('camera') // takes video from the webcam\n```\n```js\ntracker.run('video') // takes video from a movie file (e.g., mp4)\n```\n```js\ntracker.run('stream') // takes video from an m3u8 online stream\n```\n\n**DOM Elements Configuration**\n\nYou need to define elements for video, canvas, and for 3D output (only available for the BlazePose model). To do this, set the appropriate values:\n```js\ntracker.elCanvas = '#canvas';\ntracker.elVideo = '#video';\ntracker.el3D = '#view_3d';\n```\n\n**Display Settings**\n\nYou can define the size of points drawn on the canvas:\n\n```js\ntracker.pointWidth = 6; // point connection width\ntracker.pointRadius = 8; // point circle radius\n```\n\n**Events / Hooks**\n\nYou can define your own functions to handle specific tasks, such as processing points in the image detected by the neural network. The event/hook system is used for this purpose. To define a hook, use the `on()` method. For example, to define a hook that displays all detected points in real-time, write the following code:\n```js\ntracker.on('beforeupdate', function(poses) {\n\tconsole.log(poses);\n});\n````\n\nThe above code will make the console display a set of points detected in the image when rendering each frame.\n\nYou can define hooks for five types of events:\n\n- `beforeupdate` - Executes before the canvas is rendered in every frame\n- `afterupdate` - Executes after the canvas is rendered in every frame\n- `statuschange` - Executes when the status changes\n- `detectorerror` - Executes when a detector error occurs\n- `videoerror` - Executes when a video/stream error occurs\n\nDefining a hook is done as follows:\n\n```js\ntracker.on('HOOK_TYPE', function(value) {\n\t// do something with value\n});\n```\n**3D Pose Estimation**\n\nTo display the detected points in 3D space, enable the `enable3D` option, as shown below:\n```js\ntracker.enable3D = true;\n```\nAn HTML element (e.g., a div) must also be created to act as a container for the 3D output. Define the element as shown below:\n```html\n\u003cbody\u003e\n\t...\n\t\u003cdiv id=\"view_3d\"\u003e\u003c/div\u003e\n\n\t\u003cscript\u003e\n\t\ttracker.el3D = '#view_3d';\n\t\ttracker.enable3D = true;\n```\n\n3D keypoints are displayed using the `ScatterGL` library.\n\n![3dok](https://user-images.githubusercontent.com/61396542/180082333-e7793d65-5214-4354-bf61-97c3494a08ec.gif)\n\n**Options reference**\n\n`tracker.detectorModel` - detector instance, default: `poseDetection.SupportedModels.MoveNet`\n\n`tracker.detectorConfig` - detector config\n\n`tracker.autofit` bool, enable autofit/rescale points on canvas CSS auto-scaling, default: `false`\n\n`tracker.enableAI` bool, enable or disable tracking, default: `true`\n\n`tracker.enableVideo` bool, enable or disable display original video on canvas, default: `true`\n\n`tracker.enable3D` bool, enable or disable 3D keypoints, default: `false`\n\n`tracker.pointWidth` int, width of line between points, default: `6`\n\n`tracker.pointRadius` int, point circle radius, default: `8`\n\n`tracker.minScore` float, minimum threshold (score) for estimated points, default: `0.35`\n\n`tracker.log` bool, enable logging to console, default: `true`\n\n`tracker.el3D` string, HTML element for 3D keypoint, default: `#view_3d`\n\n`tracker.elCanvas` string, HTML element for canvas, default: `#canvas`\n\n`tracker.elVideo` string, HTML element for video, default: `#video`\n\n\n\nThe following options should be defined before calling the `run()` method: `detectorModel`, `detectorConfig`, `elCanvas`, `elVideo`, `el3D`.\n\n\n___\n\n## Live demo: https://szczyglis.dev/js-ai-body-tracker\n\n![demo_www](https://user-images.githubusercontent.com/61396542/180053192-4342567b-1cab-49bb-813d-b96f3a337f5e.jpg)\n\n\n## Changelog\n\n**1.0.0** -- First release published (2022-07-20)\n\n**1.0.1** -- Improved documentation (2024-08-26)\n\n\n--- \n**JS AI Body Tracker is free to use, but if you like it, you can support my work by buying me a coffee ;)**\n\nhttps://www.buymeacoffee.com/szczyglis\n\n**Enjoy!**\n\nMIT License | 2022 Marcin 'szczyglis' Szczygliński\n\nhttps://github.com/szczyglis-dev/js-ai-body-tracker\n\nhttps://szczyglis.dev/js-ai-body-tracker\n\nContact: szczyglis@protonmail.com\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fszczyglis-dev%2Fjs-ai-body-tracker","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fszczyglis-dev%2Fjs-ai-body-tracker","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fszczyglis-dev%2Fjs-ai-body-tracker/lists"}