{"id":26584302,"url":"https://github.com/webar-rocks/webar.rocks.facedepth","last_synced_at":"2025-07-29T03:38:02.478Z","repository":{"id":65591536,"uuid":"528487591","full_name":"WebAR-rocks/WebAR.rocks.faceDepth","owner":"WebAR-rocks","description":"Insert your face, detected in your camera feed, in a web 3D scene in real-time.","archived":false,"fork":false,"pushed_at":"2023-07-06T15:57:10.000Z","size":61175,"stargazers_count":40,"open_issues_count":0,"forks_count":5,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-09T07:01:39.592Z","etag":null,"topics":["3d","camera","depth","depth-map","face","face-detection","face-tracking","javascript","javascript-library","metaverse","real-time","threejs","tracking","webar","webgl"],"latest_commit_sha":null,"homepage":"https://webar.rocks","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/WebAR-rocks.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2022-08-24T15:39:24.000Z","updated_at":"2025-03-26T11:17:34.000Z","dependencies_parsed_at":"2024-01-03T04:16:29.552Z","dependency_job_id":null,"html_url":"https://github.com/WebAR-rocks/WebAR.rocks.faceDepth","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/WebAR-rocks/WebAR.rocks.faceDepth","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WebAR-rocks%2FWebAR.rocks.faceDepth","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WebAR-rocks%2FWebAR.rocks.faceDepth/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WebAR-rocks%2FWebAR.rocks.faceDepth/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WebAR-rocks%2FWebAR.rocks.faceDepth/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/WebAR-rocks","download_url":"https://codeload.github.com/WebAR-rocks/WebAR.rocks.faceDepth/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WebAR-rocks%2FWebAR.rocks.faceDepth/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":267624231,"owners_count":24117377,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-29T02:00:12.549Z","response_time":2574,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d","camera","depth","depth-map","face","face-detection","face-tracking","javascript","javascript-library","metaverse","real-time","threejs","tracking","webar","webgl"],"created_at":"2025-03-23T09:33:37.400Z","updated_at":"2025-07-29T03:38:02.430Z","avatar_url":"https://github.com/WebAR-rocks.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# WebAR.rocks.faceDepth\n\n**Import your face in a 3D scene, in live!**\n\nThis JavaScript library:\n\n1. Get the camera video stream,\n2. Detects and track the user's face\n3. Crop the face and evaluate the depth\n\nAll is done in real-time, in a standard web browser. We provide a THREE.js demo where the 3D face of the user is inserted into a 3D scene. However, this library is framework agnostic and can be user with any web 3D engine.\n\n\n## Table of contents\n\n* [Features](#features)\n* [Architecture](#architecture)\n* [Demonstrations](#demonstrations)\n* [Specifications](#specifications)\n  * [Get started](#get-started)\n  * [Init arguments](#init-arguments)\n  * [Error codes](#error-codes)\n  * [Miscellaneous methods](#miscellaneous-methods)\n  * [Optimization](#optimization)\n* [Hosting](#hosting)\n* [About the tech](#about-the-tech)\n  * [Under the hood](#under-the-hood)\n  * [Compatibility](#compatibility)\n* [License](#license)\n* [References](#references)\n\n\n## Features\n\nHere are the main features of the library:\n\n* video acquisition,\n* face detection,\n* face tracking,\n* face cropping,\n* inference of face depth,\n* global 3D face pose estimation,\n* robust to difficult lighting conditions,\n* mobile friendly.\n\n\n## Architecture\n\n* `/demos/`: demonstration source code,\n* `/dist/`: core of the library: \n  * `WebARRocksFaceDepth.js`: main minified script,\n  * `WebARRocksFaceDepth.module.js`: main minified script for module use (with `import` or `require`),\n* `/helpers/`: scripts which can help you to use this library in some specific use cases,\n  * `WebARRocksFaceDepthThreeHelper.js`: makes the bridge between *THREE.js* and this lib,\n* `/neuralNets/`: neural networks models,\n  * `NN_FACEDEPTH_DEPTH_\u003cversion\u003e.json`: neural networks computing the depth from a face cropped image,\n  * `NN_FACEDEPTH_TRACK_\u003cversion\u003e.json`: neural networks detecting and tracking the face,\n* `/libs/`: 3rd party libraries and 3D engines used in the demos,\n\n\n## Demonstrations\n\nHere are the demonstrations included in this repository:\n\n* Three.js avatar: [live demo](https://webar.rocks/demos/faceDepth/demos/threeAvatar/), [source code](/demos/threeAvatar/)\n* Ready Player Me avatar: [live demo](https://webar.rocks/demos/faceDepth/demos/threeReadyPlayerMeAvatar/), [source code](/demos/threeReadyPlayerMeAvatar/)\n\n\n## Specifications\n\n### Get started\n\nThe best way to get started is to take at the demo.\n\n\n### Init arguments\n\n* `\u003cfunction\u003e callbackReady`: This function is called when the lib is initialized, with 2 argument: `\u003cstring|false\u003e errorCode, \u003cobject\u003e spec`. This function is called again if an error happens. The object `spec` has the following properties:\n\n  * `GL`: the WebGL context. The rendering 3D engine should use this WebGL context,\n  * `canvasElement`: the `\u003ccanvas\u003e` element,\n  * `videoTexture`: a WebGL texture displaying the camera video. It has the same resolution as the camera video,\n  * `[\u003cfloat\u003e, \u003cfloat\u003e, \u003cfloat\u003e, \u003cfloat\u003e]` videoTransformMat2: flatten 2x2 matrix encoding a scaling and a rotation. We should apply this matrix to viewport coordinates to render `videoTexture` in the viewport,\n  * `\u003cHTMLVideoElement\u003e video`: the video used as source for the webgl texture `videoTexture`,\n\n* `\u003cfunction\u003e callbackTrack`: This function is called at each iteration loop, with an object as argument with these properties:\n  * `\u003cfloat\u003e detected`: the face detection probability, between `0` and `1`,\n  * `\u003cboolean\u003e isDetected`: whether the face is detected or not\n  * `\u003cfloat\u003e x`, `\u003cfloat\u003e y`: The 2D coordinates of the center of the detection frame in the viewport (each between -1 and 1, `x` from left to right and `y` from bottom to top),\n  * `\u003cfloat\u003e s`: the scale along the horizontal axis of the detection frame, between 0 and 1 (1 for the full width). The detection frame is always square,\n  * `\u003cfloat\u003e rx`, `\u003cfloat\u003e ry`, `\u003cfloat\u003e rz`: the Euler angles of the head rotation in radians,\n  * `\u003cUint8Array\u003e RGBDBuf`: RGBD buffer of the face,\n  * `\u003cint\u003e RGBDRes`: resolution of the RGBD buffer.  \n\n* `\u003cstring\u003e canvasId` or `\u003cHTMLCanvasElement\u003e canvas`: the canvas where the computation will be performed,\n* `\u003cstring\u003eNNTrackPath` or `\u003cobject\u003e NNTrack`: the neural network model used for face detection and tracking,\n* `\u003cstring\u003eNNDepthPath` or `\u003cobject\u003e NNDepth`: the neural network model used for depth inference,\n* `\u003cinteger\u003e animateDelay`: With this statement you can set accurately the number of milliseconds during which the browser wait at the end of the rendering loop before starting another detection. If you use the canvas of this library as a secondary element (for example in *PACMAN* or *EARTH NAVIGATION* demos) you should set a small `animateDelay` value (for example 2 milliseconds) in order to avoid rendering lags.\n* `\u003cfunction\u003e onWebcamAsk`: Function launched just before asking for the user to allow its camera access,\n* `\u003cfunction\u003e onWebcamGet`: Function launched just after the user has accepted to share its video. It is called with the video element as argument,\n* `\u003cdict\u003e videoSettings`: override MediaStream API specified video settings, which are by default:\n\n```javascript\n{\n  'videoElement' // not set by default. \u003cvideo\u003e element used\n   // If you specify this parameter,\n   // all other settings will be useless\n   // it means that you fully handle the video aspect\n\n  'deviceId'            // not set by default\n  'facingMode': 'user', // to use the rear camera, set to 'environment'\n\n  'idealWidth': 800,  // ideal video width in pixels\n  'idealHeight': 600, // ideal video height in pixels\n  'minWidth': 480,    // min video width in pixels\n  'maxWidth': 1280,   // max video width in pixels\n  'minHeight': 480,   // min video height in pixels\n  'maxHeight': 1280,  // max video height in pixels,\n  'rotate': 0         // rotation in degrees possible values: 0,90,-90,180\n},\n```\n\n* `\u003cdict\u003e scanSettings`: overrides face scan settings - see `set_scanSettings(...)` method for more information.\n* `\u003cdict\u003e stabilizationSettings`: overrides tracking stabilization settings - see `set_stabilizationSettings(...)` method for more information.\n* `\u003cboolean\u003e isKeepRunningOnWinFocusLost`: Whether we should keep the detection loop running even if the user switches the browser tab or minimizes the browser window. Default value is `false`. This option is useful for a videoconferencing app, where a face mask should be still computed if the *FaceFilter* window is not the active window. Even with this option toggled on, the face tracking is still slowed down when the FaceFilter window is not active.\n\n\n### Error codes\n\nThe initialization function ( `callbackReady` in the code snippet ) will be called with an error code ( `errCode` ). It can have these values:\n* `false`: no error occurs,\n* `\"GL_INCOMPATIBLE\"`: WebGL is not available, or this WebGL configuration is not enough (there is no WebGL2, or there is WebGL1 without OES_TEXTURE_FLOAT or OES_TEXTURE_HALF_FLOAT extension),\n* `\"ALREADY_INITIALIZED\"`: the library has been already initialized,\n* `\"NO_CANVASID\"`: no canvas ID was specified,\n* `\"INVALID_CANVASID\"`: cannot find the `\u003ccanvas\u003e` element in the DOM,\n* `\"INVALID_CANVASDIMENSIONS\"`: the dimensions `width` and `height` of the canvas are not specified,\n* `\"WEBCAM_UNAVAILABLE\"`: cannot get access to the camera (the user has no camera, or it has not accepted to share the device, or the camera is already busy),\n* `\"GLCONTEXT_LOST\"`: The WebGL context was lost. If the context is lost after the initialization, the `callbackReady` function will be launched a second time with this value as error code,\n\n\n### Miscellaneous methods\n\n* `resize()`: should be called after resizing the `\u003ccanvas\u003e` element to adapt the cut of the video,\n\n* `toggle_pause(\u003cboolean\u003e isPause)`: pauses/resumes,\n\n* `set_animateDelay(\u003cinteger\u003e delay)`: Changes the `animateDelay` (see `init()` arguments),\n\n* `set_inputTexture(\u003cWebGLTexture\u003e tex, \u003cinteger\u003e width, \u003cinteger\u003e height)`: Changes the video input by a WebGL Texture instance. The dimensions of the texture, in pixels, should be provided,\n\n* `reset_inputTexture()`: Comes back to the user's video as input texture,\n\n* `get_videoDevices(\u003cfunction\u003e callback)`: Should be called before the `init` method. 2 arguments are provided to the callback function:\n  * `\u003carray\u003e mediaDevices`: an array with all the devices founds. Each device is a javascript object having a `deviceId` string attribute. This value can be provided to the `init` method to use a specific camera. If an error happens, this value is set to `false`,\n  * `\u003cstring\u003e errorLabel`: if an error happens, the label of the error. It can be: `NOTSUPPORTED`, `NODEVICESFOUND` or `PROMISEREJECTED`.\n\n* `set_scanSettings(\u003cobject\u003e scanSettings)`: Overrides scan settings. `scanSettings` is a dictionnary with the following properties:\n  * `\u003cfloat\u003e threshold`: detection threshold, between `0` and `1`. Default value is `0.75`. You can decrease it if you want to make the detection more sensitive (but it will increase the false positive detections),\n  * `\u003cint\u003e nDetectsPerLoop`: specifies the number of detections per drawing loop. `0` for adaptative value. Default: `0`\n  * `\u003cint\u003e nScaleLevels`: number of detection steps for the scale. Default: `3`,\n  * `[\u003cfloat\u003e, \u003cfloat\u003e, \u003cfloat\u003e] overlapFactors`: overlaps between 2 scan positions for `X`, `Y` and `scale`. Default: `[2, 2, 3]`,\n  * `\u003cfloat\u003e scale0Factor`: scale factor for the largest scan level. Default is `0.8`.\n\n* `set_stabilizationSettings(\u003cobject\u003e stabilizationSettings)`: Overrides detection stabilization settings. The output of the neural network is always noisy, so we need to stabilize it using a floating average to avoid shaking artifacts. The internal algorithm computes first a stabilization factor `k` between `0` and `1`. If `k==0.0`, the detection is bad and we favor responsivity against stabilization. It happens when the user is moving quickly, rotating the head or when the detection is bad. On the contrary, if `k` is close to `1`, the detection is nice and the user does not move a lot so we can stabilize a lot. `stabilizationSettings` is a dictionnary with the following properties:\n  * `[\u003cfloat\u003e minValue, \u003cfloat\u003e maxValue] translationFactorRange`: multiply `k` by a factor `kTranslation` depending on the translation speed of the head (relative to the viewport). `kTranslation=0` if `translationSpeed\u003cminValue` and `kTranslation=1` if `translationSpeed\u003emaxValue`. The regression is linear. Default value: `[0.0015, 0.005]`,\n  * `[\u003cfloat\u003e minValue, \u003cfloat\u003e maxValue] rotationFactorRange`: analogous to `translationFactorRange` but for rotation speed. Default value: `[0.12, 0.25]`,\n  * `[\u003cfloat\u003e minValue, \u003cfloat\u003e maxValue] qualityFactorRange`: analogous to `translationFactorRange` but for the head detection coefficient. Default value: `[0.85, 0.95]`,\n  * `[\u003cfloat\u003e minValue, \u003cfloat\u003e maxValue] alphaRange`: it specifies how to apply `k`. Between 2 successive detections, we blend the previous `detectState` values with the current detection values using a mixing factor `alpha`. `alpha=\u003cminValue\u003e` if `k\u003c0.0` and `alpha=\u003cmaxValue\u003e` if `k\u003e1.0`. Between the 2 values, the variation is quadratic. Default value is `[0.05, 0.9]`,\nIt only applies to global pose stabilization. Landmarks are stabilized using helpers (`/helpers/WebARRocksLMStabilizer\u003cX\u003e.js`).\n\n* `update_videoElement(\u003cvideo\u003e vid, \u003cfunction|False\u003e callback)`: changes the video element used for the face detection (which can be provided via `VIDEOSETTINGS.videoElement`) by another video element. A callback function can be called when it is done.\n\n* `update_videoSettings(\u003cobject\u003e videoSettings)`: dynamically change the video settings (see [Optional init arguments](optional-init-arguments) for the properties of `videoSettings`). It is useful to change the camera from the selfie camera (user) to the back (environment) camera. A `Promise` is returned. If `videoSettings = null`, the video is stopped and the camera is toggled off.\n\n* `destroy()`: Cleans both graphic memory and JavaScript memory, uninit the library. After that you need to init the library again. A `Promise` is returned.\n\n* `is_winFocus()`: Return if the current window has focus or not (For example if the user has changed the browser tab if will return `false`). This function works only if init option `isKeepRunningOnWinFocusLost` is set to `true`.\n\n\n## Hosting\n\nYou should host the content of this repository using a HTTPS static server.\n\nBe careful to enable gzip HTTP/HTTPS compression for JSON and JS files. Indeed, the neuron network JSON file, `neuralNets/NN_\u003cxxx\u003e.json` is quite heavy, but very well compressed with GZIP. You can check the gzip compression of your server [here](https://checkgzipcompression.com/).\n\n\n## About the tech\n\n### Under the hood\n\nThis library relies on [WebAR.rocks](https://webar.rocks) WebGL Deep Learning technology to detect and track the user's face using a neural network. The accuracy is adaptative: the best is the hardware, the more detections are processed per second. All is done on the client-side.\n\n### Compatibility\n\n* If `WebGL2` is available, it uses `WebGL2` and no specific extension is required,\n* If `WebGL2` is not available but `WebGL1`, we require either `OES_TEXTURE_FLOAT` extension or `OES_TEXTURE_HALF_FLOAT` extension,\n* If `WebGL2` is not available, and if `WebGL1` is not available or neither `OES_TEXTURE_FLOAT` or `OES_HALF_TEXTURE_FLOAT` are implemented, the user is not compatible.\n\nIf a compatibility error is triggered, please post an issue on this repository. If this is a problem with the camera access, please first retry after closing all applications which could use your device (Skype, Messenger, other browser tabs and windows, ...). Please include:\n\n* a screenshot of [webglreport.com - WebGL1](http://webglreport.com/?v=1) (about your `WebGL1` implementation),\n* a screenshot of [webglreport.com - WebGL2](http://webglreport.com/?v=2) (about your `WebGL2` implementation),\n* the log from the web console,\n* the steps to reproduce the bug, and screenshots.\n\n\n## License\n\nThis code repository is dual licensed. You have to choose between these 2 licenses:\n\n1. [GPLv3](GPLv3.txt) (free default option)\n2. Nominative commercial license: please contact-us for more information\n\nFor more information, please read the [LICENSE](/LICENSE) file.\n\n\n## References\n\n* [WebAR.rocks website](https://webar.rocks)\n* [Webgl Academy: tutorials on WebGL and THREE.JS](http://www.webglacademy.com)\n* [Three.js official website](https://threejs.org/)\n* [WebAR.rocks on Linkedin](https://www.linkedin.com/company/webar-rocks)\n* [WebAR.rocks on Twitter](https://twitter.com/WebARRocks)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwebar-rocks%2Fwebar.rocks.facedepth","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwebar-rocks%2Fwebar.rocks.facedepth","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwebar-rocks%2Fwebar.rocks.facedepth/lists"}