{"id":16580201,"url":"https://github.com/fabio914/realitymixer","last_synced_at":"2025-05-16T18:08:19.273Z","repository":{"id":38390426,"uuid":"305156553","full_name":"fabio914/RealityMixer","owner":"fabio914","description":"Mixed Reality app for iOS","archived":false,"fork":false,"pushed_at":"2024-06-13T18:29:59.000Z","size":142301,"stargazers_count":798,"open_issues_count":53,"forks_count":64,"subscribers_count":43,"default_branch":"main","last_synced_at":"2025-04-12T16:57:52.626Z","etag":null,"topics":["arkit","ios","mixed-reality","oculus-quest","virtual-reality"],"latest_commit_sha":null,"homepage":"","language":"Swift","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fabio914.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-10-18T17:21:12.000Z","updated_at":"2025-04-04T19:42:11.000Z","dependencies_parsed_at":"2024-10-26T00:28:30.707Z","dependency_job_id":null,"html_url":"https://github.com/fabio914/RealityMixer","commit_stats":null,"previous_names":[],"tags_count":14,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fabio914%2FRealityMixer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fabio914%2FRealityMixer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fabio914%2FRealityMixer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fabio914%2FRealityMixer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fabio914","download_url":"https://codeload.github.com/fabio914/RealityMixer/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254582907,"owners_count":22095518,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["arkit","ios","mixed-reality","oculus-quest","virtual-reality"],"created_at":"2024-10-11T22:24:13.709Z","updated_at":"2025-05-16T18:08:19.248Z","avatar_url":"https://github.com/fabio914.png","language":"Swift","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cimg src=\"Images/rounded.png\" width=\"100\" /\u003e\n\n# Reality Mixer \u003cbr/\u003e *Mixed Reality app for iOS*\n\nReality Mixer lets you use your iPhone or iPad to capture your favorite VR games and applications in Mixed Reality without a PC and without a green screen.\n\nClick on [this link](https://apps.apple.com/us/app/reality-mixer/id1539307552) to get it from the App Store, or follow the [instructions](#installation) below to install it using alternative methods.\n\nFollow us on [Twitter](https://twitter.com/reality_mixer) for more updates!\n\n\u003ca href=\"https://www.youtube.com/watch?v=KhoyKABFvzA\"\u003e\u003cimg src=\"Images/video.jpg\" width=\"400\"/\u003e\u003c/a\u003e\n\n## ATTENTION!\n\nMixed Reality Capture is **broken** on the Quest 2/3/Pro after version 51 was released (that uses Android 12). The regular Mixed Reality Capture calibration **app is no longer able to save the calibration**!\n\nIf you still want to try to use the app, make sure to use install [SoapyMan's calibration app](https://github.com/SoapyMan/MRCCameraCalibrationApp/releases/latest) with [SideQuest](https://sidequestvr.com/setup-howto) and use this app instead of the regular Mixed Reality Capture calibration app. This app will ask for the right permissions to save the calibration. Notice that most Quest apps no longer support Mixed Reality Capture.\n\nAccess [this link](Additional-instructions.md) to read more about it.\n\nIf you're a developer, and Mixed Reality Capture is no longer working with your Unity app, then make sure to add this to your Android manifest file:\n\n```xml\n\u003cuses-native-library android:name=\"libOVRMrcLib.oculus.so\" android:required=\"true\" /\u003e\n```\n\n## Features\n\n - Mixed Reality capture without green screen (using ARKit's person segmentation).\n - Avatars (using ARKit's body tracking) and Ready Player Me avatars. [Example](https://www.youtube.com/watch?v=fOPpnZKSK4I).\n - \"AR\" mode. [Example](https://www.youtube.com/watch?v=jWCI-Ha1IJU).\n - Moving camera. [Example](https://www.youtube.com/watch?v=_FSoRw4pILk).\n - Spectator camera. [Example](https://www.youtube.com/watch?v=inp_TRWU5QM).\n - Mixed Reality with green screen. [Example](https://twitter.com/fabio914/status/1409331072832266242).\n\n## Examples\n\n| Game | YouTube video | Photos |\n|------|---------------|--------|\n| Beat Saber | [Link](https://www.youtube.com/watch?v=JL5e_moZ7XM) | \u003cimg src=\"Images/beatsaber/1.jpg\" width=\"300\" /\u003e \u003cimg src=\"Images/beatsaber/2.jpg\" width=\"300\" /\u003e |\n| SUPERHOT | [Link](https://youtu.be/ZnOY8juMw4k) | \u003cimg src=\"Images/superhot/1.jpg\" width=\"300\" /\u003e \u003cimg src=\"Images/superhot/2.jpg\" width=\"300\" /\u003e |\n| The Thrill of the Fight | [Link](https://youtu.be/aPSBmej4ppc) | \u003cimg src=\"Images/the_thrill_of_the_fight/1.jpg\" width=\"300\" /\u003e \u003cimg src=\"Images/the_thrill_of_the_fight/2.jpg\" width=\"300\" /\u003e |\n| Space Pirate Trainer | [Link](https://youtu.be/44Nmv7Es5yI) | \u003cimg src=\"Images/space_pirate_trainer/1.jpg\" width=\"300\" /\u003e \u003cimg src=\"Images/space_pirate_trainer/2.jpg\" width=\"300\" /\u003e |\n\n## How to use it?\n\nYou can find instructions and tutorials [on this page](Instructions.md).\n\n**ATTENTION!** Mixed Reality Capture is broken on the Quest 2/Pro after version 51 was released (that uses Android 12). You'll need to follow additional steps with [SideQuest](https://sidequestvr.com/). You can read these steps [here](Additional-instructions.md).\n\n## Requirements\n\n - Oculus Quest 1 or 2, or the Meta Quest Pro, with the [Oculus MRC Calibration app](https://www.oculus.com/experiences/quest/2532132800176262/) version 1.7 installed.\n - **Virtual Green Screen and Avatars**: iPhone or iPad with an A12 chip or newer, running iOS 14. The LiDAR sensor is optional but recommended for better results.\n - **Green Screen and Spectator**: iPhone or iPad runnning iOS 14.\n - 5 Ghz WiFi network (802.11ac).\n - A [compatible Quest VR application/game](Compatibility.md).\n\n## Compatible Applications/Games\n\nThe app is currently only compatible with the Oculus Quest 1 and 2, and the Meta Quest Pro, check this [link](Compatibility.md) for a list of the compatible games/applications.\n\n## Installation\n\n### App Store\n\n[Link](https://apps.apple.com/us/app/reality-mixer/id1539307552)\n\n### AltStore\n\n - Follow [these instructions](https://altstore.io/) to install and configure AltServer on your PC or Mac, then install the AltStore app on your iPhone/iPad.\n \n - Open the camera app on your iPhone/iPad and scan this QR code:\n\n![download](https://user-images.githubusercontent.com/2430631/129988363-ea0ab68a-6211-4ade-bedc-cc6dbcc354d5.png)\n\n**OR**\n\n- Open this URL on your iPhone/iPad: \n\n`altstore://install?url=https://github.com/fabio914/RealityMixer/releases/download/0.6.0/RealityMixer.ipa`\n\n### Xcode\n\nTo build the app using your own developer account, create a `DeveloperSettings.xcconfig` config file inside `RealityMixer/` with your development team identifier and your organization identifier:\n\n```\nDEVELOPMENT_TEAM = YOUR_TEAM_ID\nORGANIZATION_IDENTIFIER = com.yourcompany\n```\n\n## TO-DOs\n\n - [ ] Investigate the audio delay;\n\n - [ ] Improve the calibration screen and fix the positioning of the controller models;\n \n## Wishlist\n\nThese are features that could be implemented in the future:\n\n - Create a custom calibration app for the Quest to replace the Oculus MRC app, allowing the user to calibrate from VR itself;\n \n - Allow the user to adjust the FOV of the mixed reality capture;\n \n - Allow the app to cast the video output (although this should already be possible if you long press the \"Screen recording\" button on \"Control Center\" and then select a streaming app);\n \n - Android version;\n\n## Credits\n\nThis project uses [SwiftSocket](https://github.com/swiftsocket/SwiftSocket) to handle the TCP connection with the Oculus Quest, Apple's VideoToolbox to decode the stream, and ARKit for the \"virtual green screen\". Its video decoder is based on [zerdzhong's SwiftH264Demo](https://github.com/zerdzhong/SwfitH264Demo).\n\nOld versions of Reality Mixer also used FFMPEG, and code from the [Oculus MRC plugin for OBS](https://github.com/facebookincubator/obs-plugins/tree/master/oculus-mrc).\n\nThe right and left controller models are modified versions of the `oculus-touch-v3` models from the [WebXR Input Profiles](https://github.com/immersive-web/webxr-input-profiles).\n\nThis project also uses sounds from [Notification Sounds](https://notificationsounds.com) and avatars from [Ready Player Me](https://readyplayer.me/).\n\n\u003cimg src=\"Images/rpm-badge-light-dark-bg.png\" width=\"200\" /\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffabio914%2Frealitymixer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffabio914%2Frealitymixer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffabio914%2Frealitymixer/lists"}