https://github.com/needle-engine/facefilter
Face Filter support for Needle Engine: https://needle-face-filter-examples-z23hmxb18gxoo-latest.needle.run/
https://github.com/needle-engine/facefilter
Last synced: 8 months ago
JSON representation
Face Filter support for Needle Engine: https://needle-face-filter-examples-z23hmxb18gxoo-latest.needle.run/
- Host: GitHub
- URL: https://github.com/needle-engine/facefilter
- Owner: needle-engine
- Created: 2025-03-19T15:56:54.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2025-07-24T16:15:00.000Z (11 months ago)
- Last Synced: 2025-07-24T21:51:44.936Z (11 months ago)
- Language: TypeScript
- Homepage: https://needle-face-filter-examples-z23hmxb18gxoo-latest.needle.run/
- Size: 51.4 MB
- Stars: 20
- Watchers: 3
- Forks: 3
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Needle Engine FaceFilter
Add face tracking to your [Needle Engine](https://needle.tools) projects with ease.
This project contains the sourcecode for the facefilter package as well as an example Unity project (see [quickstart](#quickstart) below for how to get started without an editor)
---
# Features
- Blendshape mesh face tracking
- Texture face tracking with google/mediapipe canonical or procreate texture layouts (Use the `FaceMeshTexture` class)
- Video face tracking: Play a video as a face texture (Use the `FaceMeshVideo` class)
- Custom shader face meshes: Use custom materials on your face mesh (Use the `FaceMeshCustomShader` class)
- Tracking for multiple faces at once (with smoothing)
- *Can be used with Unity to create filters, animations, materials...*
# Examples
- [Example with Blendshapes on Stackblitz](https://stackblitz.com/edit/needle-engine-facefilter-blendshapes?file=src%2Fmain.ts)
- [Example with Sunglasses on Stackblitz](https://stackblitz.com/edit/needle-engine-facefilter-glasses?file=src%2Fmain.ts)
- [Example with Texture Mesh on Stackblitz](https://stackblitz.com/edit/needle-engine-facefilter)
- [Example with ShaderToy on Stackblitz](https://stackblitz.com/edit/needle-engine-shadertoy-facefilter)
- [Example without a bundler on Stackblitz](https://stackblitz.com/edit/needle-engine-facefilter-html?file=index.html)
- [Example with HTML only 2D filter on Stackblitz](https://stackblitz.com/edit/needle-engine-facefilter-html-only?file=index.html) ([or github](https://github.com/needle-engine/facefilter/blob/main/package/examples/html/index.html))
- [Example with HTML only 3D filter on Stackblitz](https://stackblitz.com/edit/needle-engine-facefilter-html-only-3d?file=index.html) ([or github](https://github.com/needle-engine/facefilter/blob/main/package/examples/html/model.html))
- [Demo Video](https://github.com/user-attachments/assets/51300430-6290-4672-b2aa-f1e870b9e99c)
## Quickstart
Run `npm i @needle-tools/facefilter` in your web project
Then see the code below or [full examples](#examples):
### Face Filter with Unity
1) Clone this repository
2) Open the Unity project at `Unity FaceFilter Example`
3) Open the Example scene in Unity and click play
**Note**: The Unity project uses Needle Engine 4.4 alpha.
### Face Filter with HTML only
See full examples in [/examples/html/](/package/examples/)
#### Supported attributes
| | |
| -- | -- |
| `face-filter` | URL to either a image or model file. Supported formats: `glTF`, `GLB`, `FBX`, `OBJ` **or 2D** `jpeg`, `jpg`, `png`, `webp` |
| `face-filter-mask` | (optional, 2D only) URL to image file that will be used to mask out the face filter texture
| `face-filter-layout` | (optional, 2D only) Either `procreate`, `mediapipe` or `canonical`. Default `mediapipe`
| `face-filter-scale` | (optional, 3D only) Apply scale to the 3D face filter model (e.g. `face-filter-scale=".5"`)
| `face-filter-offset` | (optional, 3D only) Offset the 3D face filter model (e.g. `face-filter-offset="0.0, 0.1, 0.1"`)
| `face-filter-max-faces` | (optional) How many faces should be tracked automatically. Default: `1`
| `face-filter-show-video` | (optional) Should the camera videofeed be rendered in the background? Default: `true`. Can be set to `0` to hide the videofeed in the background.
| `face-filter-video-selector` | (optional) HTML selector for a HTMLVideoElement. Useful if you want to render provide your own video element on the website elsewhere. If none is provided a hidden video element will be created automatically.
#### HTML Example
```html
{
"imports": {
"three": "https://cdn.jsdelivr.net/npm/@needle-tools/engine@4.4.0-alpha.5/dist/three.min.js",
"@needle-tools/engine": "https://cdn.jsdelivr.net/npm/@needle-tools/engine@4.4.0-alpha.5/dist/needle-engine.min.js",
"@needle-tools/facefilter": "https://cdn.jsdelivr.net/npm/@needle-tools/facefilter/dist/facefilter.min.js"
}
}
```
[Open 2D Example](https://stackblitz.com/edit/needle-engine-facefilter-html-only?file=index.html) –
[Open 3D Example](https://stackblitz.com/edit/needle-engine-facefilter-html-only-3d?file=index.html)
### Face Mesh Texture Filter
```ts
import { onStart } from '@needle-tools/engine';
import { FaceMeshTexture, NeedleFaceFilterTrackingManager } from '@needle-tools/facefilter';
onStart(context => {
const scene = context.scene;
// Create a face filter tracking manager and add it to the scene
const filtermanager = new NeedleFaceFilterTrackingManager();
filtermanager.createMenuButton = true;
scene.addComponent(filtermanager);
// Creating a filter
const filter = new FaceMeshTexture({
layout: 'procreate', // we support both the google/mediapipe canonical layout and procreate/arkit layouts
texture: {
url: './assets/crocodile.webp', // provide a URL to the texture
// texture: // alternatively you can assign an existing texture directly
},
});
// Activate one of your filters
filtermanager.activateFilter(filter);
});
```
[Open Example](https://stackblitz.com/edit/needle-engine-facefilter)
### Face Mesh Blendshapes Filter
```ts
import { onStart } from '@needle-tools/engine';
import { FaceMeshTexture, NeedleFaceFilterTrackingManager } from '@needle-tools/facefilter';
onStart(context => {
const scene = context.scene;
// Create a face filter tracking manager and add it to the scene
const filtermanager = new NeedleFaceFilterTrackingManager();
filtermanager.createMenuButton = false;
scene.addComponent(filtermanager);
// Creating a filter using a GLB/glTF URL model that has blendshapes
const filter = await FaceFilterRoot.create('https://cloud.needle.tools/-/assets/Z23hmXBZWllze-ZWllze/file', {
scale: 0.5,
offset: { x: 0, y: 0.01, z: 0 },
});
if (filter) filtermanager.activateFilter(filter);
});
```
[Open Example](https://stackblitz.com/edit/needle-engine-facefilter-blendshapes?file=src%2Fmain.ts)
## Video
https://github.com/user-attachments/assets/51300430-6290-4672-b2aa-f1e870b9e99c
# Contributing
⚠️ TODO
# Contact
[needle](https://needle.tools) •
[Twitter](https://twitter.com/NeedleTools) •
[Forum](https://forum.needle.tools) •
[Youtube](https://www.youtube.com/@needle-tools)