Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.
Awesome Lists | Featured Topics | Projects
https://github.com/react-native-ar/react-native-arkit

React Native binding for iOS ARKit
https://github.com/react-native-ar/react-native-arkit
arkit augmented-reality ios objective-c react-native
Last synced: 3 days ago
JSON representation
React Native binding for iOS ARKit
Host: GitHub
URL: https://github.com/react-native-ar/react-native-arkit
Owner: react-native-ar
License: mit
Created: 2017-07-10T06:17:34.000Z (over 7 years ago)
Default Branch: master
Last Pushed: 2023-01-11T18:46:48.000Z (almost 2 years ago)
Last Synced: 2024-12-21T05:03:50.353Z (10 days ago)
Topics: arkit, augmented-reality, ios, objective-c, react-native
Language: Objective-C
Size: 4.38 MB
Stars: 1,725
Watchers: 49
Forks: 141
Open Issues: 43
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
Awesome Lists containing this project

awesome-react-native - react-native-arkit ★1290 - React Native binding for iOS ARKit (Components / System)
README

        ⚠️  **LOOKING FOR MAINTAINERS - This Repo is currently not maintained. Give https://github.com/ViroCommunity/viro/ a try which has been open sourced and also supports android (ARCORE)** ⚠️ 

# react-native-arkit 

[![npm version](https://img.shields.io/npm/v/react-native-arkit.svg?style=flat)](https://www.npmjs.com/package/react-native-arkit)

[![npm downloads](https://img.shields.io/npm/dm/react-native-arkit.svg?style=flat)](https://www.npmjs.com/package/react-native-arkit)

React Native binding for iOS ARKit.

**Made with React Native Arkit**:

- Homestory: An AI powered interior design assistant ([App store](https://itunes.apple.com/us/app/homestory-augmented-reality/id1292552232?ls=1&mt=8))

**Tutorial**: [How to make an ARKit app in 5 minutes using React Native](https://medium.com/@HippoAR/how-to-make-your-own-arkit-app-in-5-minutes-using-react-native-9d7ce109a4c2)

**Sample Project**: https://github.com/HippoAR/ReactNativeARKit

**Note**: ARKit is only supported by devices with A9 or later processors (iPhone 6s/7/SE/8/X, iPad 2017/Pro) on iOS 11. You also need Xcode 9 to build the project.

There is a Slack group that anyone can join for help / support / general questions.

[**Join Slack**](https://join.slack.com/t/react-native-ar/shared_invite/enQtMjUzMzg3MjM0MTQ5LWU3Nzg2YjI4MGRjMTM1ZDBlNmIwYTE4YmM0M2U0NmY2YjBiYzQ4YzlkODExMTA0NDkwMzFhYWY4ZDE2M2Q4NGY)

## Getting started

`$ yarn add react-native-arkit`

make sure to use the latest version of yarn (>=1.x.x)

(npm does not work properly at the moment. See https://github.com/HippoAR/react-native-arkit/issues/103)

### Mostly automatic installation

⚠️ **Currently automatic installation does not work as PocketSVG is missing. Follow the manual installation.**

`$ react-native link react-native-arkit`

### Manual installation

#### iOS

1. In XCode, in the project navigator, right click `Libraries` ➜ `Add Files to [your project's name]`

2. Go to `node_modules` ➜ add `react-native-arkit/ios/RCTARKit.xcodeproj` and `react-native-arkit/ios/PocketSVG/PocketSVG.xcodeproj`

3. In XCode, in the project navigator, select your project. Add `libRCTARKit.a` `and PocketSVG.framework` to your project's `Build Phases` ➜ `Link Binary With Libraries`

4. In Tab `General` ➜ `Embedded Binaries` ➜ `+` ➜ Add `PocketSVG.framework ios`

5. Run your project (`Cmd+R`)<

##### iOS Project configuration

These steps are mandatory regardless of doing a manual or automatic installation:

1. Give permissions for camera usage. In `Info.plist` add the following:

```

NSCameraUsageDescription

Your message to user when the camera is accessed for the first time

```

2. ARKit only runs on arm64-ready devices so the default build architecture should be set to arm64: go to `Build settings` ➜ `Build Active Architecture Only` and change the value to `Yes`.

## Usage

A simple sample React Native ARKit App

```javascript

// index.ios.js

import React, { Component } from 'react';

import { AppRegistry, View } from 'react-native';

import { ARKit } from 'react-native-arkit';

export default class ReactNativeARKit extends Component {

  render() {

    return (

      

         console.log(e.nativeEvent)}

          // event listener for (horizontal) plane detection

          onPlaneDetected={anchor => console.log(anchor)}

          // event listener for plane update

          onPlaneUpdated={anchor => console.log(anchor)}

          // arkit sometimes removes detected planes

          onPlaneRemoved={anchor => console.log(anchor)}

          // event listeners for all anchors, see [Planes and Anchors](#planes-and-anchors)

          onAnchorDetected={anchor => console.log(anchor)}

          onAnchorUpdated={anchor => console.log(anchor)}

          onAnchorRemoved={anchor => console.log(anchor)}

          // you can detect images and will get an anchor for these images

          detectionImages={[{ resourceGroupName: 'DetectionImages' }]}

          onARKitError={console.log} // if arkit could not be initialized (e.g. missing permissions), you will get notified here

        >

          

          

          

          

          

          

          

          

          

          

          

          

          

          

                

              `,

              pathFlatness: 0.1,

              // it's also possible to specify a chamfer profile:

              chamferRadius: 5,

              chamferProfilePathSvg: `

                

              `,

              extrusion: 10,

            }}

          />  

        

      

    );

  }

}

AppRegistry.registerComponent('ReactNativeARKit', () => ReactNativeARKit);

```



### ``-Component

#### Props

| Prop | Type | Note |

|---|---|---|

| `debug` | `Boolean` | Debug mode will show the 3D axis and feature points detected.

| `planeDetection` | `ARKit.ARPlaneDetection.{ Horizontal \| Vertical \| HorizontalVertical \| None }` | ARKit plane detection. Defaults to `Horizontal`. `Vertical` is available with IOS 11.3

| `lightEstimationEnabled` | `Boolean` | ARKit light estimation (defaults to false).

| `worldAlignment` | `ARKit.ARWorldAlignment.{ Gravity \| GravityAndHeading \| Camera }` | **ARWorldAlignmentGravity** 
 The coordinate system's y-axis is parallel to gravity, and its origin is the initial position of the device. **ARWorldAlignmentGravityAndHeading** 
 The coordinate system's y-axis is parallel to gravity, its x- and z-axes are oriented to compass heading, and its origin is the initial position of the device. **ARWorldAlignmentCamera** 
 The scene coordinate system is locked to match the orientation of the camera. Defaults to `ARKit.ARWorldAlignment.Gravity`.  [See](https://developer.apple.com/documentation/arkit/arworldalignment)|

| `origin` | `{position, transition}` | Usually `{0,0,0}` is where you launched the app. If you want to have a different origin, you can set it here. E.g. if you set `origin={{position: {0,-1, 0}, transition: {duration: 1}}}` the new origin will be one meter below. If you have any objects already placed, they will get moved down using the given transition. All hit-test functions or similar will report coordinates relative to that new origin as `position`. You can get the original coordinates with `positionAbsolute` in these functions |  

| `detectionImages` | `Array` | An Array of `DetectionImage` (see below), only available on IOS 11.3 |  

##### `DetectionImage`

An `DetectionImage` is an image or image resource group that should be detected by ARKit.

See https://developer.apple.com/documentation/arkit/arreferenceimage?language=objc how to add these images.

You will then receive theses images in `onAnchorDetected/onAnchorUpdated`. See also [Planes and Anchors](#planes-and-anchors) for more details.

`DetectionImage` has these properties

| Prop | Type | Notes

|---|---|---|

| `resourceGroupName` | `String` | The name of the resource group |

We probably will add the option to load images from other sources as well (PRs encouraged).

#### Events

| Event Name | Returns | Notes

|---|---|---|

| `onARKitError` | `ARKiterror` | will report whether an error occured while initializing ARKit. A common error is when the user has not allowed camera access. Another error is, if you use `worldAlignment=GravityAndHeading` and location service is turned off |

| `onLightEstimation` | `{ ambientColorTemperature, ambientIntensity }` | Light estimation on every frame. Called rapidly, better use polling. See `ARKit.getCurrentLightEstimation()`

| `onFeaturesDetected` | `{ featurePoints}` | Detected Features on every frame (currently also not throttled). Usefull to display custom dots for detected features. You can also poll this information with `ARKit.getCurrentDetectedFeaturePoints()`

| `onAnchorDetected` | `Anchor` | When an anchor (plane or image) is first detected.

| `onAnchorUpdated` | `Anchor` | When an anchor is updated

| `onAnchorRemoved` | `Anchor` | When an anchor is removed

| `onPlaneDetected` | `Anchor` | When a plane anchor is first detected.

| `onPlaneUpdated` | `Anchor` | When a detected plane is updated

| `onPlaneRemoved` | `Anchor` | When a detected plane is removed

See [Planes and Anchors](#planes-and-anchors) for Details about anchors

#### Planes and Anchors

ARKit can detect different anchors in the real world:

- `plane` horizontal and vertical planes

- `image`, image-anchors [See DetectionImage](#DetectionImage)

- face with iphone X or similar (not implemented yet)

You then will receive anchor objects in the `onAnchorDetected`, `onAnchorUpdated`, `onAnchorRemoved` callbacks on your ``-component.

You can use `onPlaneDetected`, `onPlaneUpdated`, `onPlaneRemoved` to only receive plane-anchors (may be deprecated later).

The `Anchor` object has the following properties:

| Property | Type | Description

|---|---|---|

| `id` | `String` | a unique id identifying the anchor |

| `type` | `String` | The type of the anchor (plane, image) |

| `position` | `{ x, y, z }` | the position of the anchor (relative to the origin) |

| `positionAbsolute` | `{ x, y, z }` | the absolute position of the anchor |

| `eulerAngles` | `{ x, y, z }` | the rotation of the plane |

If its a `plane`-anchor, it will have these additional properties:

| Property | Description

|---|---|

| `alignment` | `ARKit.ARPlaneAnchorAlignment.Horizontal` or `ARKit.ARPlaneAnchorAlignment.Vertical` 
 so you can check whether it was a horizontal or vertical plane |

| `extent` | see https://developer.apple.com/documentation/arkit/arplaneanchor?language=objc |

| `center` | see https://developer.apple.com/documentation/arkit/arplaneanchor?language=objc |

`image`-Anchor:

| Property | type | Description

|---|---|---|

| `image` | `{name}` | an object with the name of the image.

### Static methods

Static Methods can directly be used on the `ARKit`-export:

```

import { ARKit } from 'react-native-arkit'

//...

const result = await ARKit.hitTestSceneObjects(point);

```

All methods return a *promise* with the result.

| Method Name | Arguments |  Notes

|---|---|---|

| `snapshot` |  |  | Take a screenshot (will save to Photo Library) |

| `snapshotCamera` |  | Take a screenshot without 3d models (will save to Photo Library) |

| `getCameraPosition` |  | Get the current position of the `ARCamera` |

| `getCamera` |  | Get all properties of the `ARCamera` |

| `getCurrentLightEstimation` |  | Get current light estimation  `{ ambientColorTemperature, ambientIntensity}` |

| `getCurrentDetectedFeaturePoints` |  | Get current detected feature points (in last current frame)  (array) |

| `focusScene` |  | Sets the scene's position/rotation to where it was when first rendered (but now relative to your device's current position/rotation) |

| `hitTestPlanes` | point, type  |  check if a plane has ben hit by point (`{x,y}`) with detection type (any of `ARKit.ARHitTestResultType`). See https://developer.apple.com/documentation/arkit/arhittestresulttype?language=objc for further information |

| `hitTestSceneObjects` | point |  check if a scene object has ben hit by point (`{x,y}`) |

| `isInitialized` | boolean | check whether arkit has been initialized (e.g. by mounting). See https://github.com/HippoAR/react-native-arkit/pull/152 for details |

| `isMounted` | boolean | check whether arkit has been mounted. See https://github.com/HippoAR/react-native-arkit/pull/152 for details |

### 3D-Components

This project allows you to work with 3d elements like with usual react-components.

We provide some primitive shapes like cubes, spheres, etc. as well as

a component to load model-files.

You can also nest components to create new Components. Child-elements will

be relative to the parent:

```

const BigExclamationMark = ({ position, eulerAngles, color = '#ff0000' }) => (

  

    

    

  

)

// somewhere else

```

#### General props

Most 3d object have these common properties

| Prop | Type | Description |

|---|---|---|

| `position` | `{ x, y, z }` | The object's position (y is up) |

| `scale` | Number | The scale of the object. Defaults to 1 |

| `eulerAngles` | `{ x, y, z }` | The rotation in eulerAngles |

| `id` | String | a unique identifier. Only provide one, if you need to find the node later in hit-testing. |

| `shape` | depends on object | the shape of the object (will probably renamed to geometry in future versions)

| `material` | `{ diffuse, metalness, roughness, lightingModel, shaders }` | the material of the object |

| `transition` | `{duration: 1}` | Some property changes can be animated like in css transitions. Currently you can specify the duration (in seconds). |

Advanced properties:

| Prop | Type | Description |

|---|---|---|

| `rotation` | TODO | see scenkit documentation |

| `orientation` | TODO | see scenkit documentation |

| `renderingOrder` | Number | Order in which object is rendered. Usefull to place elements "behind" others, although they are nearer. |

| `categoryBitMask` | Number / bitmask | control which lights affect this object |

| `castsShadow` | `boolean` | whether this object casts shadows |

| `constraint` | `ARKit.Constraint.{ BillboardAxisAll \| BillboardAxisX \| BillboardAxisY \| BillboardAxisZ \| None }` | Constrains the node to always point to the camera |

*New experimental feature:*

You can switch properties on mount or onmount by specifying `propsOnMount` and `propsOnUnmount`.

E.g. you can scale an object on unmount:

```

```

#### Material

Most objects take a material property with these sub-props:

| Prop | Type | Description |

|---|---|---|

| `diffuse` | `{ ...mapProperties }` (see below) | [diffuse](https://developer.apple.com/documentation/scenekit/scnmaterial/1462589-diffuse?language=objc)

| `specular` | `{ ...mapProperties }` (see below) | [specular](https://developer.apple.com/documentation/scenekit/scnmaterial/1462516-specular?language=objc)

| `displacement` | `{ ...mapProperties }` (see below) | [displacement](https://developer.apple.com/documentation/scenekit/scnmaterial/2867516-displacement?language=objc)

| `normal` | `{ ...mapProperties }` (see below) |  [normal](https://developer.apple.com/documentation/scenekit/scnmaterial/1462542-normal)

| `metalness` | number | metalness of the object |

| `roughness` | number | roughness of the object |

| `doubleSided` | boolean | render both sides, default is `true` |

| `litPerPixel` | boolean | calculate lighting per-pixel or vertex [litPerPixel](https://developer.apple.com/documentation/scenekit/scnmaterial/1462580-litperpixel) |

| `lightingModel` | `ARKit.LightingModel.*` | [LightingModel](https://developer.apple.com/documentation/scenekit/scnmaterial.lightingmodel) |

| `blendMode` | `ARKit.BlendMode.*` | [BlendMode](https://developer.apple.com/documentation/scenekit/scnmaterial/1462585-blendmode) |

| `transparencyMode` | `ARKit.TransparencyMode.*` | [TransparencyMode](https://developer.apple.com/documentation/scenekit/scnmaterial/1462549-transparencymode?language=objc) |

| `fillMode` | `ARKit.FillMode.*` | [FillMode](https://developer.apple.com/documentation/scenekit/scnmaterial/2867442-fillmode)

| `shaders` | Object with keys from `ARKit.ShaderModifierEntryPoint.*` and shader strings as values | [Shader modifiers](https://developer.apple.com/documentation/scenekit/scnshadable) |

| `colorBufferWriteMask` | `ARKit.ColorMask.*` | [color mask](https://developer.apple.com/documentation/scenekit/scncolormask). Set to ARKit.ColorMask.None so that an object is transparent, but receives deferred shadows. |

Map Properties:

| Prop | Type | Description |

|---|---|---|

| `path` | string | Currently `require` is not supported, so this is an absolute link to a local resource placed for example in .xcassets |

| `color` | string | Color string, only used if path is not provided |

| `wrapS` | `ARKit.WrapMode.{ Clamp \| Repeat \| Mirror }` | [wrapS](https://developer.apple.com/documentation/scenekit/scnmaterialproperty/1395384-wraps?language=objc) |

| `wrapT` | `ARKit.WrapMode.{ Clamp \| Repeat \| Mirror }` |  [wrapT](https://developer.apple.com/documentation/scenekit/scnmaterialproperty/1395382-wrapt?language=objc) |

| `wrap` | `ARKit.WrapMode.{ Clamp \| Repeat \| Mirror }` |  Shorthand for setting both wrapS & wrapT |

| `translation` | `{ x, y, z }` | Translate the UVs, equivalent to applying a translation matrix to SceneKit's `transformContents`  |

| `rotation` | `{ angle, x, y, z }` | Rotate the UVs, equivalent to applying a rotation matrix to SceneKit's `transformContents` |

| `scale` | `{ x, y, z }` | Scale the UVs, equivalent to applying a scale matrix to SceneKit's `transformContents`  |

#### ``

This Object has no geometry, but is simply a wrapper for other components.

It receives all common properties like position, eulerAngles, scale, opacity, etc.

but no shape or material.

#### [``](https://developer.apple.com/documentation/scenekit/scnbox)

| Prop | Type |

|---|---|

| `shape` | `{ width, height, length, chamfer }` |

And any common object property (position, material, etc.)

#### [``](https://developer.apple.com/documentation/scenekit/scnsphere)

| Prop | Type |

|---|---|

| `shape` | `{ radius }` |

#### [``](https://developer.apple.com/documentation/scenekit/scncylinder)

| Prop | Type |

|---|---|

| `shape` | `{ radius, height }` |

#### [``](https://developer.apple.com/documentation/scenekit/scncone)

| Prop | Type |

|---|---|

| `shape` | `{ topR, bottomR, height }` |

#### [``](https://developer.apple.com/documentation/scenekit/scnpyramid)

| Prop | Type |

|---|---|

| `shape` | `{ width, height, length }` |

#### [``](https://developer.apple.com/documentation/scenekit/scntube)

| Prop | Type |

|---|---|

| `shape` | `{ innerR, outerR, height }` |

#### [``](https://developer.apple.com/documentation/scenekit/scntorus)

| Prop | Type |

|---|---|

| `shape` | `{ ringR, pipeR }` |

#### [``](https://developer.apple.com/documentation/scenekit/scncapsule)

| Prop | Type |

|---|---|

| `shape` | `{ capR, height }` |

#### [``](https://developer.apple.com/documentation/scenekit/scnplane)

| Prop | Type |

|---|---|

| `shape` | `{ width, height }` |

Notice: planes are veritcally aligned. If you want a horizontal plane, rotate it around the x-axis.

*Example*:

This is a horizontal plane that only receives shadows, but is invisible otherwise:

```

```

#### [``](https://developer.apple.com/documentation/scenekit/scntext)

| Prop | Type |

|---|---|

| `text` | `String` |

| `font` | `{ name, size, depth, chamfer }` |

#### ``

SceneKit only supports `.scn` and `.dae` formats.

| Prop | Type |

|---|---|

| `model` | `{ file, node, scale, alpha }` |

Objects currently don't take material property.

#### ``

Creates a extruded shape by an svg path.

See https://github.com/HippoAR/react-native-arkit/pull/89 for details

| Prop | Type |

|---|---|

| `shape` | `{ pathSvg, extrusion, pathFlatness, chamferRadius, chamferProfilePathSvg, chamferProfilePathFlatness }` |

#### [``](https://developer.apple.com/documentation/scenekit/scnlight)

Place lights on the scene!

You might set `autoenablesDefaultLighting={false}` on The `` component to disable default lighting. You can use `lightEstimationEnabled` and `ARKit.getCurrentLightEstimation()` to find values for intensity and temperature. This produces much nicer results then `autoenablesDefaultLighting`.

| Prop | Type | Description |

|---|---|---|

| `position` | `{ x, y, z }` |  |

| `eulerAngles` | `{ x, y, z }` |  |

| `type` | any of `ARKit.LightType` | see [here for details](https://developer.apple.com/documentation/scenekit/scnlight.lighttype) |

| `color` | `string` | the color of the light |

| `temperature` | `Number` | The color temperature of the light |

| `intensity` | `Number` | The light intensity |

| `lightCategoryBitMask` | `Number`/`bitmask` | control which objects are lit by this light |

| `castsShadow` | `boolean` | whether to cast shadows on object |

| `shadowMode`| `ARKit.ShadowMode.* | Define the shadowmode. Set to `ARKit.ShadowMode.Deferred` to cast shadows on invisible objects (like an invisible floor plane) |

Most properties described here are also supported: https://developer.apple.com/documentation/scenekit/scnlight

This feature is new. If you experience any problem, please report an issue!

### HOCs (higher order components)

#### withProjectedPosition()

this hoc allows you to create 3D components where the position is always relative to the same point on the screen/camera, but sticks to a plane or object.

Think about a 3D cursor that can be moved across your table or a 3D cursor on a wall.

You can use the hoc like this:

```

const Cursor3D = withProjectedPosition()(({positionProjected, projectionResult}) => {

  if(!projectionResult) {

    // nothing has been hit, don't render it

    return null;

  }

  return (

    

  )

})

```

It's recommended that you specify a transition duration (0.1s works nice), as the position gets updated rapidly, but slightly throttled.

Now you can use your 3D cursor like this:

##### Attach to a given detected horizontal plane

Given you have detected a plane with onPlaneDetected, you can make the cursor stick to that plane:

```

```

If you don't have the id, but want to place the cursor on a certain plane (e.g. the first or last one), pass a function for plane. This function will get all hit-results and you can return the one you need:

```

 results.length > 0 ? results[0] : null

  }}

/>

```

You can also add a property `onProjectedPosition` to your cursor which will be called with the hit result on every frame

It uses https://developer.apple.com/documentation/arkit/arframe/2875718-hittest with some default options. Please file an issue or send a PR if you need more control over the options here!

##### Attach to a given 3D object

You can attach the cursor on a 3D object, e.g. a non-horizontal-plane or similar:

Given there is some 3D object on your scene with `id="my-nodeId"`

```

```

Like with planes, you can select the node with a function.

E.gl you have several "walls" with ids "wall_1", "wall_2", etc.

```

 results.find(r => r.id.startsWith('wall_')),

  }}

/>

```

It uses https://developer.apple.com/documentation/scenekit/scnscenerenderer/1522929-hittest with some default options. Please file an issue or send a PR if you need more control over the options here!

## FAQ:

#### Which permissions does this use?

- **camera access** (see section iOS Project configuration above). The user is asked for permission, as soon as you mount an `` component or use any of its API. If user denies access, you will get an error in `onARKitError`

- **location service**: only needed if you use `ARKit.ARWorldAlignment.GravityAndHeading`.

#### Is there an Android / ARCore version?

Not yet, but there has been a proof-of-concept: https://github.com/HippoAR/react-native-arkit/issues/14. We are looking for contributors to help backporting this proof-of-conept to react-native-arkit.

#### I have another question...

[**Join Slack!**](https://join.slack.com/t/react-native-ar/shared_invite/enQtMjUzMzg3MjM0MTQ5LWU3Nzg2YjI4MGRjMTM1ZDBlNmIwYTE4YmM0M2U0NmY2YjBiYzQ4YzlkODExMTA0NDkwMzFhYWY4ZDE2M2Q4NGY)

## Contributing

If you find a bug or would like to request a new feature, just [open an issue](https://github.com/HippoAR/react-native-arkit/issues/new). Your contributions are always welcome! Submit a pull request and see [`CONTRIBUTING.md`](CONTRIBUTING.md) for guidelines.