{"id":50414190,"url":"https://github.com/ibrews/godot-avp-cascade","last_synced_at":"2026-06-03T08:00:53.637Z","repository":{"id":360830021,"uuid":"1251844165","full_name":"ibrews/godot-avp-cascade","owner":"ibrews","description":"Cascade Countdown — a hand-tracked physics arcade game for Apple Vision Pro, built on the Godot engine. RigidBody3D + procedural audio at 90 FPS locked in immersive mode. Live on TestFlight.","archived":false,"fork":false,"pushed_at":"2026-06-02T03:18:29.000Z","size":17640,"stargazers_count":5,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-02T07:22:45.762Z","etag":null,"topics":["apple-vision-pro","arcade-game","cascade-countdown","godot","hand-tracking","spatial-computing","testflight","visionos"],"latest_commit_sha":null,"homepage":null,"language":"GDScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ibrews.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-05-28T00:53:59.000Z","updated_at":"2026-06-02T03:18:33.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/ibrews/godot-avp-cascade","commit_stats":null,"previous_names":["ibrews/godot-avp-cascade"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/ibrews/godot-avp-cascade","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ibrews%2Fgodot-avp-cascade","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ibrews%2Fgodot-avp-cascade/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ibrews%2Fgodot-avp-cascade/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ibrews%2Fgodot-avp-cascade/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ibrews","download_url":"https://codeload.github.com/ibrews/godot-avp-cascade/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ibrews%2Fgodot-avp-cascade/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33854119,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-03T02:00:06.370Z","response_time":59,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apple-vision-pro","arcade-game","cascade-countdown","godot","hand-tracking","spatial-computing","testflight","visionos"],"created_at":"2026-05-31T05:02:50.981Z","updated_at":"2026-06-03T08:00:53.592Z","avatar_url":"https://github.com/ibrews.png","language":"GDScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# godot-avp-cascade\n\n**Falling Cascade** — a self-running physics + procedural-audio demo for Apple Vision Pro, built on the Godot game engine via Apple's official upstream visionOS XR contribution (PR [#109975](https://github.com/godotengine/godot/pull/109975)).\n\nTwenty-five emissive cubes — 8-colour fire-plasma palette, randomised sizes — drop onto a tilted catch plate in your immersive space. Each collision triggers a procedurally-synthesized chime (plate chimes have an octave harmonic; cube-on-cube tinks are brighter and shorter). Reach in with your hand and **pinch to grab and throw** any cube. Walk around it.\n\nThe first publicly-documented Godot `RigidBody3D` physics scene rendering in immersive mode on real AVP at locked 90 FPS, with working hand-tracking pickup.\n\n![Cascade demo](captures/cascade.gif)\n\n## ⚠️ Which Godot engine you need (read this first)\n\nHand tracking is an **engine capability, not a project setting** — no amount of GDScript or Info.plist tweaking enables it. There are two visionOS Godot engine paths and **they are not interchangeable**:\n\n| Engine | Hand tracking | Use it if… |\n|--------|:---:|--------|\n| [rsanchezsaez/godot](https://github.com/rsanchezsaez/godot) `apple/visionos-xr` — **Apple's official PR branch** | ❌ render-only | you only want the falling cascade (no grab/throw) |\n| [Clancey/godot](https://github.com/Clancey/godot/tree/visionos_master_pr) `visionos_master_pr` — HEAD `2b2f749` | ✅ pinch grab + throw | you want the **full demo as shown above** |\n\n**If the cubes render but you can't grab anything and you never saw a hand-tracking permission prompt — you're on the rsanchezsaez engine. Switch to Clancey's fork.** The missing permission prompt is the tell: rsanchezsaez never asks visionOS for hand data because the hand-tracking code isn't compiled in. See [Troubleshooting](#troubleshooting).\n\n\u003e Apple has stated visionOS hand tracking is \"ready but unsubmitted.\" When it lands in rsanchezsaez/upstream, this repo will migrate to the official path and Clancey's fork won't be needed. Until then, **hand tracking = Clancey's fork.**\n\n## Status\n\nExperimental WIP. Rendering rides on Apple's official visionOS contribution — [rsanchezsaez/godot](https://github.com/rsanchezsaez/godot)'s `apple/visionos-xr` branch (PR open, not yet merged upstream; Ricardo Sanchez-Saez, Apple visionOS team, is lead author — see the [PR thread](https://github.com/godotengine/godot/pull/109975)). **Hand tracking** rides on [Clancey's fork](https://github.com/Clancey/godot/tree/visionos_master_pr), which rebases hand-interaction support on top — see [Which Godot engine you need](#️-which-godot-engine-you-need-read-this-first) above.\n\nVerified working **2026-05-28** on Apple Vision Pro M2 (visionOS 26.5, RealityDevice14,1) using Xcode 26.\n\n- 90 FPS locked, **zero variance across a 95-second sample** (19 × 5-second windows)\n- ~475 physics collisions / 95s\n- Mobile renderer + Metal driver (Forward+ silently doesn't render on this path)\n- Mixed immersion (passthrough) — cubes composite into your real room\n- **Pinch-to-grab and throw** any cube with either hand\n\n## What this demo proves\n\nMost public Godot-on-AVP material to date shows either: (a) flat-plane Godot apps in a shared-space window, (b) the rsanchezsaez reference scene (a third-person platformer ported from mobile VR), or (c) the GodotVision/SwiftGodotKit RealityKit-bridged approach.\n\nThis is the first published example of:\n\n- `RigidBody3D` simulation running in immersive mode on AVP via the Apple-official native path\n- `AudioStreamGenerator` + `push_frame` real-time procedural audio synchronized to physics collisions\n- A fully procedural scene (no `.glb`, no `.scn`, no textures) that walks the full XR pipeline\n- **Hand-tracking pickup via `XRHandTracker` joint data and `XRController3D`** — pinch distance drives a smooth analog threshold; cubes snap to the pinch midpoint and inherit hand velocity on throw\n\n## How it's built\n\n```\ntest-project/\n  main_v2.tscn        # XR boilerplate scene; gameplay built procedurally in script\n  main_v2.gd          # Cascade script — spawn, physics, audio, hand setup\n  pickup/\n    pickup_handler.gd # PickupHandler3D — pinch detection, fingertip anchoring (Marshall Nowak)\n    pickup_able_body.gd # PickupAbleBody3D — grab/snap/throw logic with velocity tracking\n  shaders/\n    highlight_shader.tres    # Yellow outline on nearest grabbable cube\n    highlight_material.tres\n  project.godot       # mobile renderer, OpenXR + hand_interaction_profile, alpha-0 clear\n  export_presets.cfg  # visionOS preset, app_role=Immersive\n\nout/xcode-visionos/   # generated by Godot --export-pack, packaged by xcodebuild\ncaptures/             # screen recordings + stills from AVP runs\n```\n\n`main_v2.tscn` has only `XROrigin3D`, `XRCamera3D`, `WorldEnvironment`, `DirectionalLight3D`, and a script. Everything else — the cubes, the catch plate, the deflector wall, the kill-plane Area3D, the audio player, the physics material, the two `XRController3D` + `PickupHandler3D` hand nodes — is created in code in `_ready()`. Saves editor round-trips during iteration.\n\n## Quickstart (build + deploy)\n\nYou need:\n- macOS Apple Silicon (M1+) with Xcode 26 installed\n- Apple Vision Pro paired and trusted to your Xcode\n- Your Apple Developer Team ID\n- A built `libgodot.a` + matching `Godot.app` editor from **[Clancey/godot](https://github.com/Clancey/godot/tree/visionos_master_pr) `visionos_master_pr` (HEAD `2b2f749`)** for hand tracking — or rsanchezsaez `apple/visionos-xr` if you only want the render-only cascade. See [building-the-engine](#building-the-engine) below. **Build both from the same commit** so the editor that exports the PCK matches the runtime lib.\n\n```bash\n# Re-export the PCK from the test-project\n~/godot-visionos-pilot/Godot.app/Contents/MacOS/Godot --headless \\\n  --path test-project \\\n  --export-pack \"visionOS\" \\\n  out/xcode-visionos/GodotVisionPilot.pck\n\n# Build and sign the visionOS app\nxcodebuild \\\n  -project out/xcode-visionos/GodotVisionPilot.xcodeproj \\\n  -scheme GodotVisionPilot \\\n  -configuration Debug \\\n  -destination \"platform=visionOS,id=\u003cYOUR_DEVICE_UDID\u003e\" \\\n  CODE_SIGN_IDENTITY=\"Apple Development\" \\\n  DEVELOPMENT_TEAM=\"\u003cYOUR_TEAM_ID\u003e\" \\\n  build\n\n# Install\nxcrun devicectl device install app \\\n  --device \u003cYOUR_DEVICE_UDID\u003e \\\n  $(ls -d ~/Library/Developer/Xcode/DerivedData/GodotVisionPilot-*/Build/Products/Debug-xros/GodotVisionPilot.app)\n```\n\n**You then put on the headset and tap the app icon.** Remote-launch via `devicectl device process launch` does not work for immersive apps (Apple's CoreDevice returns `connection invalidated` for immersive-space launches).\n\n## Things to Try\n\n1. **Confirm the scene renders at 90 FPS.** Pull diagnostic frame counts after a run: `xcrun devicectl device copy from --device \u003cUDID\u003e --source Documents/xr_diag.txt --destination /tmp/xr_diag.txt --domain-type appDataContainer --domain-identifier com.agilelens.godotvisionpilot`. Per-5s frame deltas should be exactly 450.\n2. **Grab and throw a cube.** Bring your index finger and thumb together (pinch) near any glowing cube — it will highlight yellow and snap to your pinch point. Flick your wrist and release to throw. Try deflecting a cascade of falling cubes.\n3. **Reposition the rig.** The catch plates and deflector wall are grabbable too — pinch one and move it, and it stays where you release it (it won't fall). Redirect the whole cascade mid-stream.\n4. **Change the spawn rate.** Edit `SPAWN_INTERVAL` in `test-project/main_v2.gd`. Try `0.15` for a downpour, `0.8` for a trickle. Re-export the PCK (`--export-pack`) and re-deploy.\n5. **Switch to full immersion.** Change `UISceneInitialImmersionStyle` from `UIImmersionStyleMixed` to `UIImmersionStyleFull` in `out/xcode-visionos/GodotVisionPilot/GodotVisionPilot-Info.plist`, then rebuild. The Digital Crown already works to blend in progressive immersion if you change the style key to `UIImmersionStyleProgressive`.\n6. **Add a second cascade tier.** Drop another tilted plate beneath the first at `y=−0.5` rotated `+15°` on X so it catches cubes that fall off the first plate. Coin-pusher feel.\n\n## Building the engine\n\nFor **hand tracking**, build from [Clancey/godot](https://github.com/Clancey/godot/tree/visionos_master_pr) branch `visionos_master_pr` (HEAD `2b2f749`, 2026-03-03 — \"Full hand tracking with pinch-based interaction\"). For **render-only**, build from rsanchezsaez `apple/visionos-xr`. Either way you produce a `Godot.app` editor and a `libgodot.a` (xros-arm64) static lib. Building takes 30–90 minutes on an M1 Max.\n\n\u003e The binary this repo was verified against was built from a *pre-rebase* commit of the same branch (`85f0afd`, now diverged from the tip). Both have hand tracking; `2b2f749` is the current branch HEAD and the right target for a fresh build.\n\n**Build the editor and the lib from the same commit.** The editor exports the PCK; the lib runs it. If their Godot versions differ (e.g. exporting with a 4.6.3 editor against a 4.6.2 lib), tokenized GDScript can fail to load at runtime — the app renders nothing but passthrough, with no crash. Two ways to avoid it:\n1. Use a matched editor+lib pair (preferred), **or**\n2. Set `script_export_mode=0` (Text) in `export_presets.cfg` so the runtime compiles scripts from source and the token format no longer has to match. This repo currently uses Text mode because its editor (4.6.3) and lib (Clancey 4.6.2) differ.\n\nRecipe + gotchas documented in detail at:\n- [agile-lens-kb/intelligence/techniques/godot-visionos-xr.md](https://github.com/AgileLens/agile-lens-kb/blob/master/intelligence/techniques/godot-visionos-xr.md) — full engine-build gotchas, scene-recipe details, `XROrigin3D.current=true` requirement\n- [agile-lens-kb/intelligence/techniques/godot-avp-hand-tracking-engine-swap.md](https://github.com/AgileLens/agile-lens-kb/blob/master/intelligence/techniques/godot-avp-hand-tracking-engine-swap.md) — engine-fork requirement, the version-gap fix, swap recipe\n- [agile-lens-kb/intelligence/techniques/godot-avp-falling-cascade.md](https://github.com/AgileLens/agile-lens-kb/blob/master/intelligence/techniques/godot-avp-falling-cascade.md) — this demo's gotchas, perf numbers, code patterns\n\nThe engine source tree is `.gitignore`'d here (regenerable from the fork above).\n\n## Troubleshooting\n\n| Symptom | Cause | Fix |\n|---------|-------|-----|\n| **Cubes fall and render, but I can't grab anything, and I never got a hand-tracking permission prompt** | You're on the **rsanchezsaez** engine (render-only) — it never requests hand data | Rebuild/link against **Clancey's fork** (HEAD `2b2f749`). See [Which Godot engine you need](#️-which-godot-engine-you-need-read-this-first) |\n| **App opens but I only see passthrough — no cubes at all** | The main scene's GDScript failed to load (compile error, or a tokenized-script version mismatch between editor and lib) | Run the project headless first: `Godot --headless --path test-project --quit-after 120` and fix any `SCRIPT ERROR`. If editor/lib versions differ, set `script_export_mode=0` (Text). |\n| **`--export-pack` \"succeeded\" but the app runs old code / nothing changed** | Output path was relative — it resolves from the **project dir**, not your shell's cwd | Use an **absolute** path for the PCK output |\n| **`devicectl install` fails: \"unable to locate device\"** | AVP is asleep (it sleeps when not worn) | Put the headset on, retry 2–3× |\n| **Cubes render but no permission prompt AND grab still dead after switching engines** | Missing Info.plist privacy strings | Ensure `NSHandsTrackingUsageDescription` + `NSWorldSensingUsageDescription` are in the built bundle (`plutil -extract … \u003capp\u003e/Info.plist`) |\n\n## Known limitations\n\n- **Mobile renderer only.** Forward+ doesn't render on this path. No Lumen/Nanite-equivalent quality.\n- **Hand mesh not rendered.** Visual hand skeleton (`XRHandModifier3D`) is excluded from this build to keep the scene minimal. Only the pickup sphere and fingertip anchor are active.\n- **Manual app launch on the headset.** `xcrun devicectl device process launch` returns `connection invalidated` for immersive-space apps. The user has to tap the icon.\n- **MSAA does not work on this branch yet.** Blocked on Godot PR [#78598](https://github.com/godotengine/godot/pull/78598). Don't enable it.\n- **Hand-as-collision-mesh not yet implemented.** The hand can grab and throw but does not act as a physics collider for passive deflection. The plan is five `AnimatableBody3D` capsules driven by `XRHandTracker` joint positions.\n\n## Credits\n\n- Engine: [Godot](https://godotengine.org/) — open source\n- visionOS XR port: [Ricardo Sanchez-Saez @ Apple](https://github.com/rsanchezsaez) + community contributors (huisedenanhai, stuartcarnie, BastiaanOlij)\n- **Hand tracking pickup system:** [Marshall Nowak (Nocxr)](https://github.com/Nocxr) @ Agile Lens — `PickupHandler3D` / `PickupAbleBody3D` from [visionosxr_hand_tracking](https://github.com/Clancey/godot/tree/visionos_master_pr), ported from [Clancey's hand-tracking fork](https://github.com/Clancey/godot/tree/visionos_master_pr)\n- Demo + writeup: [Agile Lens](https://agilelens.com/)\n\n## License\n\nMIT. See [LICENSE](LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fibrews%2Fgodot-avp-cascade","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fibrews%2Fgodot-avp-cascade","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fibrews%2Fgodot-avp-cascade/lists"}