{"id":25265819,"url":"https://github.com/holokit/holokit-unity-sdk","last_synced_at":"2025-10-27T04:31:17.070Z","repository":{"id":167121877,"uuid":"642256658","full_name":"holokit/holokit-unity-sdk","owner":"holokit","description":"Unity SDK for the optical see-through mixed reality headset HoloKit X","archived":false,"fork":false,"pushed_at":"2025-01-07T02:28:48.000Z","size":2624,"stargazers_count":29,"open_issues_count":3,"forks_count":2,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-07T03:35:57.456Z","etag":null,"topics":["holokit","spatial-computing","unity-sdk","unity3d","unity3d-plugin"],"latest_commit_sha":null,"homepage":"https://docs.holokit.io","language":"C#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/holokit.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":"Supported_Android_Devices.md","governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2023-05-18T06:56:33.000Z","updated_at":"2025-01-07T02:28:08.000Z","dependencies_parsed_at":"2023-12-22T03:55:30.666Z","dependency_job_id":"f81c7478-3116-4f05-b1ad-218e81c54552","html_url":"https://github.com/holokit/holokit-unity-sdk","commit_stats":null,"previous_names":["holoi/holokit-unity-sdk","holokit/holokit-unity-sdk"],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/holokit%2Fholokit-unity-sdk","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/holokit%2Fholokit-unity-sdk/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/holokit%2Fholokit-unity-sdk/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/holokit%2Fholokit-unity-sdk/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/holokit","download_url":"https://codeload.github.com/holokit/holokit-unity-sdk/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":238437413,"owners_count":19472401,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["holokit","spatial-computing","unity-sdk","unity3d","unity3d-plugin"],"created_at":"2025-02-12T08:30:52.719Z","updated_at":"2025-10-27T04:31:17.065Z","avatar_url":"https://github.com/holokit.png","language":"C#","funding_links":[],"categories":[],"sub_categories":[],"readme":"# HoloKit Unity SDK\n\n[HoloKit](https://holokit.io/) is an optical see-through mixed reality headset that transforms your iPhone into a powerful stereoscopic AR device. With the HoloKit Unity SDK, developers can create immersive AR experiences in Unity, which can be viewed with HoloKit headset.\n\nThe SDK currently provides three core features:\n\n- Stereoscopic Rendering\n- Hand Tracking\n- Hand Gesture Recognition\n\nStereoscopic rendering is the central feature of the SDK, allowing for the display of stereo images on the iPhone screen. By attaching your iPhone to the HoloKit, you can enjoy a captivating AR experience. Utilizing [Apple Vision framework](https://developer.apple.com/documentation/vision?language=objc), the SDK detects user hand poses. Together with LiDAR depth sensor, it allows your iPhone to track the 3D positions of the user's hands. Furthermore, the SDK can recognize hand gestures such as pinching, serving as a trigger for specific operations in your project.\n\nThe SDK offers a built-in video recording feature which allows the user to record videos under both screen AR mode and stereoscopic mode.\n\nHoloKit Unity SDK, built on the foundation of ARFoundation, is compatible with most ARFoundation features such as image tracking and plane detection. Upgrading your ARFoundation project to a stereoscopic AR project is straightforward with the HoloKit Unity SDK.\n\n## How to Install\n\nYou can install HoloKit Unity SDK from the following git URL in Package Manager:\n```\nhttps://github.com/holokit/holokit-unity-sdk.git\n```\n\u003cimg width=\"279\" alt=\"image\" src=\"https://github.com/holokit/holokit-unity-sdk/assets/44870300/a82656c9-aa73-4158-83b5-20d5178c8a81\"\u003e\n\nor by directly adding the following line to the `Packages/manifest.json` file.\n```\n\"io.holokit.unity-sdk\": \"https://github.com/holokit/holokit-unity-sdk.git\"\n```\n\n## Supported Software Versions\n\nHoloKit Unity SDK has been tested and found compatible with the following software versions:\n\n### Unity\n- Unity 6 LTS\n\n### ARFoundation\n- ARFoundation 6.0.5\n\nWe aim to continually test and verify compatibility with newer versions of these softwares.\n\n## Phone Compatibility\n\n### iOS\n- iOS 18.0 or higher\n\nHoloKit was primarily built for iOS devices. Please see the [Supported iOS Devices](Supported_iOS_Devices.md) to check if your iPhone is supported.\n\n### Android\n\nHoloKit currently only supports a limited number of Android phones. Please see the [Supported Android Devices](Supported_Android_Devices.md) for more details.\n\nIf your Android phone is not supported, we also provide a calibration method to let you figure out the specs of your Android phone yourself. You can then enter your Android phone specs to the SDK to have your device rendering stereo images. Please see [Phone Calibration Guide](Phone_Calibration_Guide.md), which is a step-by-step guide on how to figure out the specs of your phone model.\n\n## Project Environment Setup\n\nBefore using the HoloKit Unity SDK, ensure to adjust the project settings to meet ARFoundation's requirements. If you are already comfortable with ARFoundation, feel free to skip this section.\n\nUpon successfully installation of the SDK package, the ARFoundation package will be installed automatically. Please note that the HoloKit Unity SDK is only compatible with ARFoundation 6.+. \n\nFor iOS-targeted projects, install the `Apple ARKit XR Plugin` package. For Android-targeted projects, the `Google ARCore XR Plugin` is required.\n\n### iOS Project Settings\n\n1. **Set Camera and Microphone Usage Descriptions**: Navigate to `Project Settings \u003e Player \u003e Other Settings \u003e Configuration` and provide a user-friendly text for `Camera Usage Description` and `Microphone usage description`. The `Camera Usage Description` will pop up when the iPhone requests access to the camera. The `Microphone usage description` is required because the video recording feature of the SDK will access the microphone to record audio.\n\n\u003cimg width=\"722\" alt=\"image\" src=\"https://github.com/holokit/holokit-unity-sdk/assets/44870300/848a1bf3-b877-48a0-8430-92d2d448c7a5\"\u003e\n\n2. **Enable Apple ARKit**: Under `Project Settings \u003e XR Plug-in Management \u003e Plug-in Providers`, enable the `Apple ARKit` option.\n\n\u003cimg width=\"600\" alt=\"Enable Apple ARKit\" src=\"https://github.com/holokit/holokit-unity-sdk/assets/44870300/ab5f3554-9530-464d-a336-12957a857192\"\u003e\n\n### Android Project Settings\n\n1. **Disable Auto Graphics API**: Navigate to `Project Settings \u003e Player \u003e Other Settings \u003e Rendering` and disable `Auto Graphics API`. In the `Graphics APIs` list, remove `Vulkan` and ensure only `OpenGLES3` is listed.\n\n\u003cimg width=\"500\" alt=\"Graphics APIs\" src=\"https://github.com/holokit/holokit-unity-sdk/assets/44870300/4f66611d-d912-4fe8-b4d9-ca1a08e15a5d\"\u003e\n\n2. **Set Minimum API Level**: Proceed to `Project Settings \u003e Player \u003e Other Settings \u003e Identification` and set `Minimum API Level` to `Android 7.0 'Nougat' (API level 24)` or higher.\n\n\u003cimg width=\"500\" alt=\"Minimum API Level\" src=\"https://github.com/holokit/holokit-unity-sdk/assets/44870300/fb16a187-651c-4ace-b266-77f2e18f67dc\"\u003e\n\n3. **Configure Scripting Backend**: In `Project Settings \u003e Player \u003e Other Settings \u003e Configuration`, set `Scripting Backend` to `IL2CPP`. Under `Target Architectures`, uncheck `ARMv7` and check `ARM64`.\n\n\u003cimg width=\"500\" alt=\"Configuration\" src=\"https://github.com/holokit/holokit-unity-sdk/assets/44870300/049fc7a9-e8a2-4b21-8e1c-cf241df6aa3c\"\u003e\n\n4. **Enable Google ARCore**: Under `Project Settings \u003e XR Plug-in Management \u003e Plug-in Providers`, enable the `Google ARCore` option.\n\n\u003cimg width=\"600\" alt=\"Enable Google ARCore\" src=\"https://github.com/holokit/holokit-unity-sdk/assets/44870300/daccfe62-c90e-4ed3-8fcc-218bcae59a31\"\u003e\n\n## Core Samples\n\nThe SDK package offers a samples for each of the three key features: stereoscopic rendering, hand tracking, and hand gesture recognition. Import these samples into your project and deploy them to mobile devices for a quick overview of each feature. These samples can also be used as templates for your projects.\n\n### Stereoscopic Rendering\n\nStereoscopic rendering is the core feature of the SDK, your app needs to display stereo images on the phone screen so that it can be viewed with the HoloKit headset.\n\nThe SDK provides two rendering modes: mono and stereo. Mono mode renders the background camera image with virtual content layered on top, akin to a standard ARFoundation project. In constrast, stereo mode renders two stereo images on a black background one the phone's screen, so that you can insert your phone onto the HoloKit headset to have stereoscopic AR experience.\n\n| \u003cimg src=\"https://github.com/holokit/holokit-unity-sdk/assets/44870300/a9d048a0-b2b5-4f0e-a7f8-03ca782b9725\" alt=\"ScreenRenderMode.Mono\" width=\"450\"/\u003e | \u003cimg src=\"https://github.com/holokit/holokit-unity-sdk/assets/44870300/01bf3231-41ec-4a29-9d94-a48dcebfeefb\" alt=\"ScreenRenderMode.Stereo\" width=\"450\"/\u003e |\n|:---:|:---:|\n| Mono rendering mode | Stereo rendering mode |\n\nTo integrate stereoscopic rendering into your project, the `HoloKitCameraManager` component is required. In fact, `HoloKitCameraManager` is the only mandatory component in the SDK that you must add into the scene. Instead of adding a default ARFoundation `XR Origin` object, we add a `HoloKit XR Origin` object into the scene by right-clicking in Unity Editor's hierarchy window and selecting `XR/HoloKit/HoloKit XR Origin`. The `HoloKit XR Origin` contains a `HoloKit Camera` object with the `HoloKitCameraManager` script, which enables both mono and stereo rendering modes.\n\n\u003cimg width=\"903\" alt=\"image\" src=\"https://github.com/holokit/holokit-unity-sdk/assets/44870300/a5d5a3d8-c600-405f-8743-a5266d024368\"\u003e\n\nWhen you start the game, `HoloKitCameraManager` initially enters mono mode. A UI button is required to transition between rendering modes. The current rendering mode can be retrieved and set with `HoloKitCameraManager`. The following code snippet illustrates how to switch the rendering mode.\n\n```\n// A reference to the HoloKitCameraManager component is required\n[SerializedField] HoloKitCameraManager m_HoloKitCameraManager;\n\npublic void SwitchRenderMode()\n{\n    // Get the current ScreenRenderMode value and assign a new value\n    holokitCamera.ScreenRenderMode = holokitCamera.ScreenRenderMode == ScreenRenderMode.Mono ? ScreenRenderMode.Stereo : ScreenRenderMode.Mono;\n}\n```\n\nWhen the stereo mode is activated, the SDK spawns an alignment marker UI on the screen's upper right corner, enabling you to align your phone accurately once attached to the HoloKit headset.\n\n### Hand Tracking\n\nThe SDK tracks the user's hand, providing the 3D positions of the [21 hand joints](https://developer.apple.com/documentation/vision/vnhumanhandposeobservationjointname?language=objc) of each hand.\n\nTo incorporate the hand tracking into your project, create an empty GameObject and add `HandTrackingManager` component to it. The script will automatically setup all necessary objects as child GameObjects.\n\n\u003cimg src=\"https://github.com/holokit/holokit-unity-sdk/assets/44870300/54c5dc5e-ca2b-4cac-b132-d1aab5f47fa0\" alt=\"HandTrackingManager\" width=\"800\"/\u003e\n\n`HandTrackingManager` requires depth information to calculate 3D hand positions, so add `AROcclusionManager` component to the `HoloKit Camera` GameObject and adjust its settings accordingly.\n\n\u003cimg width=\"903\" alt=\"image\" src=\"https://github.com/holokit/holokit-unity-sdk/assets/44870300/e38f3c36-a673-46ec-a41c-339572fc11a5\"\u003e\n\nYou can now build the project onto an iPhone to view the results.\n\n\u003cimg src=\"https://github.com/holokit/holokit-unity-sdk/assets/44870300/aba052ed-6433-41ba-9329-a6219364f7ea\" alt=\"Hand Tracking Result\" width=\"600\"/\u003e\n\n`HandTrackingManager` can be configured to track either one hand or both hands. As hand tracking is energy-intensive, if dual hand tracking isn't necessary, we advise tracking only one hand to conserve energy. Set `HandTrackingManager.MaxHandCount` in its inspector within the Unity editor.\n\nTo obtain specific hand joint positions in code, use `HandTrackingManager.GetHandJointPosition(int handIndex, JointName jointName)`. As the hand tracking algorithm can't distinguish handedness, the parameter `handIndex` simply represents the hand detection sequence. When only one hand is detected, `handIndex` should always be 0. If two hands are detected, `handIndex` can be either 0 or 1. The number of detected hands can be accessed via `HandTrackingManager.HandCount`.\n\nPlease note, the hand tracking feature is only compatible with iOS 18.0 or higher devices equipped with a LiDAR depth sensor.\n\n### Hand Gesture Recognition\n\nThe SDK can also recognizes user's hand gesture, which can serve as input triggers in your project. Currently there are only two available hand gestures: `HandGesture.None` and `HandGesture.Pinched`.\n\n\u003cimg src=\"https://github.com/holokit/holokit-unity-sdk/assets/44870300/0fe0010f-7854-41ad-b9b8-95ee8039fb4c\" alt=\"Hand Gesture Recognition\" width=\"600\"/\u003e\n\nTo utilize hand gesture recognition, add `HandGestureRecognitionManager` to the scene. The callback function `HandGestureRecognitionManager.OnHandGestureChanged` is invoked when a hand gesture transition is detected. Register this callback to receive notifications. You can also access the user's current hand gesture via `HandGestureRecognitionManager.HandGesture`. The following code snippet illustrates how to capture user hand gesture transitions.\n\n```\n[SerializeField] HandGestureRecognitionManager m_HandGestureRecognitionManager;\n\nprivate void Start()\n{\n    // Register the callback\n    m_HandGestureRecognitionManager.OnHandGestureChanged += OnHandGestureChanged;\n}\n\nprivate void OnHandGestureChanged(HandGesture handGesture)\n{\n    // Execute desired action\n}\n```\n\nPlease note, the hand gesture recognition feature is only available on iOS 18.0 or higher devices.\n\nYou can use hand tracking and hand gesture recognition at the same time, simply add both `HandTrackingManager` and `HandGestureRecognitionManager` components to your scene.\n\n## Quick Scene Setup\n\nHoloKit Unity SDK is based on ARFoundation. For a minimal ARFoundation scene requires two primary objects: an `AR Session` object and an `XR Origin` object with the Main Camera Object as its child.\n\n\u003cimg width=\"377\" alt=\"image\" src=\"https://github.com/holokit/holokit-unity-sdk/assets/44870300/feeb658f-ec65-4461-bf5a-e42fc423cafa\"\u003e\n\nTo enable stereoscopic rendering, we need to upgrade the default Main Camera to a HoloKit Camera. To create an `XR Origin` with a HoloKit Camera as its child, right-click in the Unity Editor and select `XR/HoloKit/HoloKit XR Origin` to create a `HoloKit XR Origin`.\n\n\u003cimg width=\"660\" alt=\"image\" src=\"https://github.com/holokit/holokit-unity-sdk/assets/44870300/ad56a592-147f-4969-9942-d3445bf038f6\"\u003e\n\nThe `HoloKit XR Orign` object includes a HoloKit Camera capable of both screen AR and stereoscopic rendering.\n\n\u003cimg width=\"355\" alt=\"image\" src=\"https://github.com/holokit/holokit-unity-sdk/assets/44870300/cdd3b0fa-6d69-45eb-9a8b-f129ce14df2a\"\u003e\n\nTo switch between the rendering modes, a UI button is necessary. To create a default UI panel which includes this mode-switching button, right-click and navigate to `UI/HoloKit/HoloKit Default UI Canvas`.\n\n\u003cimg width=\"690\" alt=\"image\" src=\"https://github.com/holokit/holokit-unity-sdk/assets/44870300/d3c5a561-43c2-48b4-9f24-cbb770b3e773\"\u003e\n\nThe default canvas also has a button for controlling video recording.\n\n\u003cimg width=\"374\" alt=\"image\" src=\"https://github.com/holokit/holokit-unity-sdk/assets/44870300/6daa7737-49e6-4403-9f95-a1eedd88506d\"\u003e\n\nAfter setting up the `AR Session`, `HoloKit XROrigin`, and UI components, you can add any 3D object to the scene, such as a cube. When you build your project onto an iPhone, you'll be able to view your object in both screen AR mode and stereoscopic rendering mode with a HoloKit headset.\n\n## Advanced Features\n\n### Reset World Origin\n\nThe SDK enables resetting the ARKit `ARSession`'s world origin to a specified position and rotation by using the `WorldOriginResetter` script. Please notice that there must be a `HoloKitARKitManager` component in the scene, which is used by the `WorldOriginResetter` script.\n\nImport the `Reset World Origin` package sample for how to use this feature.\n\n### AR Background Video Enhancement\n\nThe 4K HDR (High Dynamic Range) AR background video is a feature of ARKit 6, available on iOS 16 and later. This enhancement improves the quality of ARBackground video, leading to better video recording quality. To enable the 4K HDR option, use the `ARBackgroundVideoEnhancementManager` component. Please notice that there must be a `HoloKitARKitManager` component in the scene, which is used by the `ARBackgroundVideoEnhancementManager` script.\n\n\u003cimg width=\"675\" alt=\"image\" src=\"https://github.com/holokit/holokit-unity-sdk/assets/44870300/24c9274f-d30c-47e3-acb3-10b7d0bce816\"\u003e\n\nImport the `AR Background Video Enhancement` package sample for how to use this feature. Please note that not all iOS devices support 4K or HDR. For more information, please visit the [ARKit's website](https://developer.apple.com/augmented-reality/arkit/) and check out this [WWDC session](https://developer.apple.com/videos/play/wwdc2022/10126/#:~:text=In%204K%20mode%2C%20an%20image,at%2030%20frames%20per%20second.) on ARKit 6.\n\n## Community and Feedback\n\nIf you encounter any issues, have queries, suggestions, or discover any bugs, we welcome you to join our [Discord](https://discord.gg/9Stseyje) community or submit an issue. Your feedback is invaluable as we continually strive to update and improve the HoloKit Unity SDK.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fholokit%2Fholokit-unity-sdk","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fholokit%2Fholokit-unity-sdk","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fholokit%2Fholokit-unity-sdk/lists"}