{"id":19235561,"url":"https://github.com/stephenguanqi/unity-arkit-plugin","last_synced_at":"2025-04-21T05:32:13.727Z","repository":{"id":230167170,"uuid":"96098434","full_name":"StephenGuanqi/Unity-ARKit-Plugin","owner":"StephenGuanqi","description":"Modified plugin source and Add custom ARKit projects implement on Unity","archived":false,"fork":false,"pushed_at":"2017-11-03T22:15:27.000Z","size":71383,"stargazers_count":83,"open_issues_count":6,"forks_count":21,"subscribers_count":13,"default_branch":"master","last_synced_at":"2025-04-01T10:04:05.812Z","etag":null,"topics":["arframe","arkit","augmented-reality","cvpixelbuffer","ios","objective-c","scenekit","unity"],"latest_commit_sha":null,"homepage":null,"language":"C#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/StephenGuanqi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"License.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2017-07-03T10:15:44.000Z","updated_at":"2025-03-31T03:18:52.000Z","dependencies_parsed_at":"2024-03-28T06:55:36.026Z","dependency_job_id":null,"html_url":"https://github.com/StephenGuanqi/Unity-ARKit-Plugin","commit_stats":null,"previous_names":["stephenguanqi/unity-arkit-plugin"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/StephenGuanqi%2FUnity-ARKit-Plugin","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/StephenGuanqi%2FUnity-ARKit-Plugin/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/StephenGuanqi%2FUnity-ARKit-Plugin/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/StephenGuanqi%2FUnity-ARKit-Plugin/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/StephenGuanqi","download_url":"https://codeload.github.com/StephenGuanqi/Unity-ARKit-Plugin/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250002274,"owners_count":21359084,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["arframe","arkit","augmented-reality","cvpixelbuffer","ios","objective-c","scenekit","unity"],"created_at":"2024-11-09T16:17:19.650Z","updated_at":"2025-04-21T05:32:08.705Z","avatar_url":"https://github.com/StephenGuanqi.png","language":"C#","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Unity-ARKit-Plugin\n\n[![License: GPL v3](https://img.shields.io/badge/License-GPL%20v3-blue.svg)](#license)\n\nModified plugin source and add custom ARKit projects implemented by Unity\n\n## Introduction\nThe unity-arkit-plugin enables using all the functionality of the ARKit SDK simply within your Unity projects for iOS. The plugin exposes ARKit SDK's world tracking capabilities, rendering the camera video input, plane detection and update, point cloud extraction, light estimation, and hit testing API to Unity developers for their AR projects. For more information, you can go to the official [repo](https://bitbucket.org/Unity-Technologies/unity-arkit-plugin) for downloading.\n\nThis repository has two extensions for the official plugin. First it modifies the existing `UnityARKitScene` project to change it to a zombie walk AR application. The app support hitTest function, changing the zombie's euler angle and pos, as well as scaling the zombie up and down with walking animation. Second the repository has make changes to the plugin's source code in the `NativeInterface` folder, in order to get the captured image `CVPixelBuffer` from the `ARFrame` object, as well as the camera pos, timestamp, lightestimation, etc. and saved those info into the native document directory of the app container asyncronously. And the `UnityARRecorder` app has realize this function.\n\n\u003e Environment Requirements:\n\u003e - Xcode 9 that contains ARKit framework\n\u003e - iOS 11\n\u003e - iOS devices that support A9 or better processor for ARWorldTrackingSessionConfiguration (iPhone6s or higher)\n\u003e - Unity v5.6.1p1+ with iOS plugin\n\n## Usage\n\n1. ` $ git clone https://github.com/StephenGuanqi/Unity-ARKit-Plugin.git `\n2. Create a new empty project in your Unity, and drag the `unity-arkit-plugin.unitypackage` into your project.\n3. You could see multiple unity projects whose names start with `UnithyAR`\n4. Try any projects and made modifications you like, and build with the iOS plugin.\n5. Open the XCode projects, add the `CoreImage` framework and finish building.\n\n\n## Extended Projects\n\n### ZombieWalk\n\n![gif](https://raw.githubusercontent.com/StephenGuanqi/AVResources/unity-arkit-plugin/zombie.gif)\n\nThe project enables cloudpoint and plane detection, and perform hitTest against existing plane to change the zombie's transform. It also changes the zombie's euler angle to face towards the position of the camera in world coordinates.\n\n### UnityARRecorder\n\n![gif](https://raw.githubusercontent.com/StephenGuanqi/AVResources/unity-arkit-plugin/unityrecording.gif)\n\nThe `ARSessionNative.mm` in the plugin source code export various functions to C# layer to let users configure the `ARSession` using C#, and in turn the objective-C++ class take the passed in paramters and conduct processing like parsing the UnityARKitWorldTrackingConfiguration and set it to the ARSession, or transforming left-hand coordinates(Unity) to right-hand coordinaets(ARKit).\n\nThe core in `ARSessionNative.mm` is that it provides callback method for `ARFrame` update and `ARAnchor` update. This callback is intended to be implemented by user in C#, and when the `ARFrame` and `ARAnchor` id updated, the corresponding delegate method in objective-c like `(void)session:(ARSession *)session didUpdateFrame:(ARFrame *)frame` and `(void)session:(ARSession *)session didAddAnchors:(NSArray\u003cARAnchor*\u003e*)anchors` will call the callback function and passed in the `ARFrame` and `ARAnchor` parameter.\n\nHowever, the official plugin provides many interfaces for the native ARKit API, except for method to get `CVPixelBuffer` in `ARFrame`. The `ARSessionNative.mm` only gives user the opportunity to **manipulate the `ARCamera`, not the `ARFrame`**, therefore it's hard for us to get the captured image and conduct our own research.\n\nThis repository has modified the `ARSessionNative.mm` and `UnityARSessionNativeInterface`, meanwhile added `ARFrameHandler` class to get and convert the `CVPixelBuffer` to UIImage, save it to jpg file, and save the camera pos and other info into json file every time the frame is updated.\n\nThe sample json file is as follows:\n\n```json\n{\n  \"27255.603090583.jpg\" : {\n    \"lightEstimate\" : 944.16241455078125,\n    \"imageResolution\" : {\n      \"width\" : 1280,\n      \"height\" : 720\n    },\n    \"timeStamp\" : \"27255.603090583\",\n    \"cameraPos\" : {\n      \"x\" : -0.0040707914158701897,\n      \"y\" : 0.0039433315396308899,\n      \"z\" : 0.0042630583047866821\n    },\n    \"cameraTransform\" : [\n      [\n        0.83647698163986206,\n        -0.21551112830638885,\n        0.50384646654129028,\n        -0.0040707914158701897\n      ],\n      [\n        0.084991201758384705,\n        0.95931828022003174,\n        0.26923003792762756,\n        0.0039433315396308899\n      ],\n      [\n        -0.54137128591537476,\n        -0.18238219618797302,\n        0.82076418399810791,\n        0.0042630583047866821\n      ],\n      [\n        0,\n        0,\n        0,\n        1\n      ]\n    ],\n    \"cameraIntrinsics\" : [\n      [\n        1088.72119140625,\n        0,\n        639.5\n      ],\n      [\n        0,\n        1088.72119140625,\n        359.5\n      ],\n      [\n        0,\n        0,\n        1\n      ]\n    ],\n    \"imageName\" : \"27255.603090583.jpg\",\n    \"cameraEulerAngle\" : {\n      \"x\" : -0.27259346842765808,\n      \"y\" : 0.55055916309356689,\n      \"z\" : 0.088364705443382263\n    }\n  }\n}\n\n```\nYou can find more sample json and image data in the sampleAppContainer.xcappdata.\n\n## Original Official Projects\n\n### ParticlePainter\n![gif](https://raw.githubusercontent.com/StephenGuanqi/AVResources/unity-arkit-plugin/particlepainter.gif)\nThe app paint the gameobject only when the distance between the current camera transform and the previous camera transform is larger than the threshold.\n\n### ARBallz\n![gif](https://raw.githubusercontent.com/StephenGuanqi/AVResources/unity-arkit-plugin/ballz.gif)\nThe app put balls on the coordinates derived by the hitTest result, when a plane has been detected.\n\n## Communication\n- If you **found a bug**, open an issue.\n- If you **have a feature request**, open an issue.\n- If you **want to contribute**, submit a pull request.\n\n## Thanks\n[Augmented Reality Tutorial: APPLE ARkit is AMAZING!!!](https://www.youtube.com/watch?v=S7kKQZuOdlk)\n\n## License\n\nThis custom Unity-ARKit-Plugin is under GNU GPL v3.0 License. [SEE LICENSE](https://github.com/StephenGuanqi/Unity-ARKit-Plugin/blob/master/License.md)for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstephenguanqi%2Funity-arkit-plugin","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fstephenguanqi%2Funity-arkit-plugin","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstephenguanqi%2Funity-arkit-plugin/lists"}