https://github.com/noboomu/arfacekit

Full featured AR face magic for iOS.
https://github.com/noboomu/arfacekit

arkit augmented-reality face-detection face-tracking ios

Last synced: about 1 year ago
JSON representation

Full featured AR face magic for iOS.

• Host: GitHub
• URL: https://github.com/noboomu/arfacekit
• Owner: noboomu
• Created: 2019-07-17T05:00:59.000Z (almost 7 years ago)
• Default Branch: master
• Last Pushed: 2019-07-17T05:30:43.000Z (almost 7 years ago)
• Last Synced: 2025-04-22T14:54:44.527Z (about 1 year ago)
• Topics: arkit, augmented-reality, face-detection, face-tracking, ios
• Language: C++
• Size: 9.69 MB
• Stars: 10
• Watchers: 1
• Forks: 4
• Open Issues: 0
• Metadata Files:
  • Readme: ReadMe.md

Awesome Lists containing this project

README

          
### ARFaceGeometry

- we add an additional texureCoordinate element to the geometry

- this enables us to support geometry warping without having to override SceneKit’s rendering system

 - the second textureCoordinate element is designed to work with Daz3D face textures

### SCNScene Info

#### face

- special node names

    - lut

        - if this node's material has a diffuse texture, that texture will be used to color the face node 

    - eyeLeft

        - eye tracking will be enabled if this node is present

        - the first material of this node will be used as the left eye's material

    - eyeRight

        - eye tracking will be enabled if this node is present

        - the first material of this node will be used as the right eye's material

    - geometry

        - this node's materials are applied to the ARFaceGeometry

        - custom modifiers are applied to the ARFaceGeometry 

#### world

- child nodes are placed into the scene and are fixed relative to the camera

- special node names

    - lut

        - if this node's material has a diffuse texture, that texture will be used to color all nodes in the scene 

    - overlay

        - if this node has a material name with `*.sks` or `*.plist` than an SpriteKit scene will be created and the scene's `overlaySKScene` property will be set to this SpriteKit scene

- lights

    - constraints are preserved (i.e. lookAt)

#### overlay 

> *this name should be changed to mask* to avoid confusion

- all child nodes will be attached to the face geometry

- special node names

    - lut

        - if this node's material has a diffuse texture, that texture will be used to color the mask node 

#### camera

- if a camera is present, use the color grading values for saturation and contrast, these are used by the colorProcessing pipeline

> *All lut textures are assumed to be 256x16 RGB images

### Special Material Names

- *.mp4

    - if a material has this name, an `AVPlayer` will be created and the media file with the material's name will be loaded into it

    - playback will be looped

    - the node will use the `AVPlayer` as it's diffuse material property

- *.sks

    - if a material has this name, a `SKScene` will be loaded using the material's name as the file name

    - the node will use the `SKScene` as it's diffuse material property

- *.plist

    - if a material has this name, an `AnimatedSpriteNode` will be created using the material's name as the file name

    - it is assumed the format of this file matches that of that exported by the `TexturePacker` application when using `plist` mode

    - the node will use the `AnimatedSpriteNode` as it's diffuse material property

    - `AnimatedSpriteNode` inherits from `SKSpriteNode` but is extended to support `TexturePacker`'s capabiltities

 

### Custom Material Properties

- cameraTexture

    - If this custom property is present, the renderer will set this texture with the output of the camera every render pass

- displacementMap

    - When present:

        - the contents of the materials's `SCNShaderModifierEntryPoint` is set to the contents of `displacementGeometryModifier.txt` file

        - the `SCNGeometryModifierEntryPoint` modifier is set from the contents of the  `displacementShaderModifier.txt` file

    - The texture must be an [OpenEXR](http://www.openexr.com/) image

    - Values are interpreted as follows:

        - R: x displacement

        - G: y displacement

        - B: z displacement

        - values = 0.5 have 0 displacement

        - values < 0.5 have negative displacement 

        - values > 0.5 have positive displacement

- backgroundInfluence

    - determines the amount if influence the camera texture has on the diffuse texture

    - default is 1.2

- backgroundAverage

    - determines the average background luminance used when blending with the camera texture

    - default is 0.5