{"id":51342510,"url":"https://github.com/sun-lab-nbb/sollertia-virtual-reality","last_synced_at":"2026-07-02T09:02:24.523Z","repository":{"id":316802459,"uuid":"991426369","full_name":"Sun-Lab-NBB/sollertia-virtual-reality","owner":"Sun-Lab-NBB","description":"Provides assets for creating and executing Virtual Reality (VR) tasks for Sollertia platform data acquisition systems.","archived":false,"fork":false,"pushed_at":"2026-06-21T14:47:23.000Z","size":13155,"stargazers_count":0,"open_issues_count":0,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-21T15:24:57.630Z","etag":null,"topics":["data-acquisition","experiment","game-engine","sollertia","unity","virtual-reality"],"latest_commit_sha":null,"homepage":"","language":"C#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Sun-Lab-NBB.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-05-27T15:52:26.000Z","updated_at":"2026-06-21T14:47:46.000Z","dependencies_parsed_at":"2025-09-26T20:52:02.558Z","dependency_job_id":"5ff79cad-2dee-4f9e-a1ed-ab609e16507b","html_url":"https://github.com/Sun-Lab-NBB/sollertia-virtual-reality","commit_stats":null,"previous_names":["sun-lab-nbb/sl-unity-tasks","sun-lab-nbb/sollertia-unity-tasks"],"tags_count":5,"template":false,"template_full_name":null,"purl":"pkg:github/Sun-Lab-NBB/sollertia-virtual-reality","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Sun-Lab-NBB%2Fsollertia-virtual-reality","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Sun-Lab-NBB%2Fsollertia-virtual-reality/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Sun-Lab-NBB%2Fsollertia-virtual-reality/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Sun-Lab-NBB%2Fsollertia-virtual-reality/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Sun-Lab-NBB","download_url":"https://codeload.github.com/Sun-Lab-NBB/sollertia-virtual-reality/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Sun-Lab-NBB%2Fsollertia-virtual-reality/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35040024,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-02T02:00:06.368Z","response_time":173,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-acquisition","experiment","game-engine","sollertia","unity","virtual-reality"],"created_at":"2026-07-02T09:02:21.407Z","updated_at":"2026-07-02T09:02:24.515Z","avatar_url":"https://github.com/Sun-Lab-NBB.png","language":"C#","funding_links":[],"categories":[],"sub_categories":[],"readme":"# sollertia-virtual-reality\n\nProvides assets for creating and executing Virtual Reality (VR) tasks for Sollertia platform data acquisition systems.\n\n[![C#](https://tinyurl.com/bdd689s9)](https://docs.microsoft.com/en-us/dotnet/csharp/)\n[![Unity](https://img.shields.io/badge/Unity-6000.3.15f1_LTS-000000?logo=unity\u0026logoColor=white)](https://unity.com/)\n[![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](LICENSE)\n\n___\n\n## Detailed Description\n\nThis project is part of the [Sollertia](https://github.com/Sun-Lab-NBB/sollertia) AI-assisted scientific data\nacquisition and processing platform, built on the [Ataraxis](https://github.com/Sun-Lab-NBB/ataraxis) framework and\ndeveloped in the Sun (NeuroAI) lab at Cornell University. It provides the Unity-side assets and runtime bindings for\nbuilding VR tasks consumed by Sollertia platform data acquisition systems. The current task surface targets an\n**infinite linear corridor** environment displayed to the animal during runtime across a three-monitor VR rig.\n\nThis project is the Unity counterpart of [sollertia-experiment](https://github.com/Sun-Lab-NBB/sollertia-experiment),\nthe Python acquisition runtime. The two libraries communicate over an [MQTT 5.0](https://mqtt.org/) broker:\nsollertia-experiment publishes treadmill motion, lick events, and runtime toggles; this project publishes cue sequences,\nscene metadata, stimulus events, and brake-activation requests. Task templates, experiment configurations, and the data\nschema are owned by [sollertia-shared-assets](https://github.com/Sun-Lab-NBB/sollertia-shared-assets), whose `slsa mcp`\nserver doubles as the agentic Unity Editor relay for AI-driven task authoring.\n\nThe Unity-side runtime is a ground-up refactor of the [GIMBL](https://github.com/winnubstj/Gimbl) VR framework,\ninlined under `Assets/Gimbl/`. The refactor preserves GIMBL's core actor, controller, and display abstractions with\nextensive editor, MQTT, and runtime enhancements focused on agentic task authoring and Sollertia platform integration.\n\n___\n\n## Features\n\n- Generates infinite-corridor VR tasks from YAML templates via the Editor menu or the `slsa mcp` Unity relay.\n- Supports five stimulus trigger modes (interaction, collision, occupancy-disarm, occupancy-arm, occupancy-trigger)\n  with optional guidance modes.\n- Supports probabilistic transitions between trial structures within a single task template.\n- Exposes HTTP-based McpBridge that exposes 14 Editor operations to AI agents (task lifecycle, scene management,\n  asset inspection, Play Mode control, parameter read/write).\n- Maintains bidirectional MQTT 5.0 contract with \n  [sollertia-experiment](https://github.com/Sun-Lab-NBB/sollertia-experiment), centralized in a single `MQTTTopics` \n  constant set.\n- Apache 2.0 License.\n\n___\n\n## Table of Contents\n\n- [Dependencies](#dependencies)\n- [Installation](#installation)\n- [Usage](#usage)\n  - [Project Structure](#project-structure)\n  - [Task Runtime Structure](#task-runtime-structure)\n  - [Task Asset Hierarchy](#task-asset-hierarchy)\n  - [Authoring Task Templates](#authoring-task-templates)\n  - [Creating Tasks](#creating-tasks)\n  - [Loading and Running Tasks](#loading-and-running-tasks)\n  - [Task Parameters Window](#task-parameters-window)\n  - [MQTT Contract](#mqtt-contract)\n  - [Editor MCP Bridge](#editor-mcp-bridge)\n- [Developer Notes](#developer-notes)\n  - [Project Layout Conventions](#project-layout-conventions)\n  - [Formatting and Style](#formatting-and-style)\n  - [Extending the Library](#extending-the-library)\n  - [AI-Assisted Development](#ai-assisted-development)\n- [Versioning](#versioning)\n- [Authors](#authors)\n- [License](#license)\n- [Acknowledgments](#acknowledgments)\n\n___\n\n## Dependencies\n\nExternal requirements that must be installed before working with this Unity project:\n\n- [Unity Game Engine](https://unity.com/products/unity-engine) **6000.3.15f1 LTS** (Unity 6). Installed via\n  [Unity Hub](https://unity.com/download).\n- An [MQTT broker](https://mosquitto.org/) supporting **MQTT 5.0**, such as Mosquitto 2.0 or later. The project\n  defaults to `127.0.0.1:1883` for the broker; both the IP and port are configurable from the Task Parameters window.\n- [Blender](https://www.blender.org/download/) **4.5.0 LTS** is required only for authoring or modifying 3D assets\n  (corridor models). It is not required to build or run existing tasks.\n- [.NET SDK](https://dotnet.microsoft.com/download) **8.0 or later** and\n  [CSharpier](https://csharpier.com/) only when contributing source changes (see\n  [Formatting and Style](#formatting-and-style)).\n\nTwo managed dependencies ship as committed DLLs under `Assets/Plugins/` and require no separate installation:\n\n- [MQTTnet](https://github.com/dotnet/MQTTnet) — the MQTT 5.0 client used by `MQTTClient.cs`.\n- [YamlDotNet](https://github.com/aaubry/YamlDotNet) — the YAML deserializer used by `ConfigLoader.cs`.\n\n___\n\n## Installation\n\nThis project is a Unity 6 application that is not distributed via package managers. To install:\n\n1. Install [Unity Hub](https://unity.com/download) and use it to install the required Unity Editor version\n   (**6000.3.15f1 LTS**).\n2. Download this repository to the local machine using the preferred method, such as git-cloning. Use one of the\n   [stable releases](https://github.com/Sun-Lab-NBB/sollertia-virtual-reality/tags) when available.\n3. From Unity Hub, select **Add project from disk** and navigate to the local folder containing the downloaded\n   repository:\n   \u003cbr\u003e\n   ![Adding the project to Unity Hub](imgs/AddProjectFromDisk.png)\n\n***Note,*** if the correct Unity version is not installed when the project is imported, Unity Hub displays a warning\nnext to the project name. Selecting the warning offers to install the recommended Unity version:\n\u003cbr\u003e\n![Unity Hub installing the recommended version](imgs/InstallRecommendedVersion.png)\n\n___\n\n## Usage\n\n### Project Structure\n\nThe runtime and editor code lives under `Assets/`. Hand-authored assets are protected from agentic deletion via the\nMcpBridge's protected-paths list; generated assets live under separate folders and are rebuilt on demand.\n\n| Directory                                     | Purpose                                                                    |\n|-----------------------------------------------|----------------------------------------------------------------------------|\n| `Assets/InfiniteCorridorTask/Scripts/`        | Runtime C# (`Task`, zone scripts, `ConfigLoader`, schema mirror classes)   |\n| `Assets/InfiniteCorridorTask/Scripts/Editor/` | `CreateTask` pipeline, `McpBridge` HTTP listener, `TaskEditor`, `MiniJson` |\n| `Assets/InfiniteCorridorTask/Configurations/` | YAML task templates                                                        |\n| `Assets/InfiniteCorridorTask/Cues/`           | Generated cue prefabs (length-suffixed, shared across templates)           |\n| `Assets/InfiniteCorridorTask/Prefabs/`        | Hand-authored zone prefabs and generated segment prefabs                   |\n| `Assets/InfiniteCorridorTask/Tasks/`          | Generated task prefabs (one per template)                                  |\n| `Assets/InfiniteCorridorTask/Materials/`      | Generated cue materials and the canonical `_CueShaderReference.mat`        |\n| `Assets/InfiniteCorridorTask/Textures/`       | Cue textures referenced by YAML templates                                  |\n| `Assets/UI-lick-reward/`                      | On-screen lick and stimulus feedback canvas                                |\n| `Assets/Gimbl/`                               | Inlined GIMBL runtime and the consolidated Task Parameters window          |\n| `Assets/Scenes/`                              | `ExperimentTemplate.unity` plus per-task generated scenes                  |\n| `Assets/Plugins/`                             | Inlined `MQTTnet.dll` and `YamlDotNet.dll`                                 |\n\n### Task Runtime Structure\n\nA task represents an **infinite linear corridor sequence** built from a fixed catalog of reusable (prefab) parts. \nUnder this hierarchy, a **prefab** is Unity's serializable template for a hierarchy of GameObjects — a piece of a scene \nsaved as a file so it can be instantiated repeatedly and updated in one place.\n\nAny task hierarchy can be described in terms of four distinc levels, finest to coarsest:\n\n```text\nTask\n  │\n  ├── Corridor          ─┐  A corridor is a fixed sequence of segments stacked along the animal's\n  ├── Corridor           │  motion axis. A task pre-instantiates one corridor for every possible\n  ├── Corridor           │  combination of segments, so corridors enumerate the configuration space\n  └── … (every          ─┘  the task can take.\n       combination)\n\n      one Corridor\n        ├── Segment          ← active: the trial currently driving behavior\n        ├── Segment          ← lookahead: visible only, no behavior\n        └── Segment          ← lookahead: visible only, no behavior\n\n      one Segment (= one trial)\n        ├── Cue\n        ├── Cue              ← the cue sequence declared by this trial, laid out along the corridor\n        └── …\n```\n\n- **Cues** are individual visual panels displayed along the walls of the corridor. They are the smallest unit of the\n  corridor and are shared across every trial — and every task — that declares the same cue identity.\n- **Segments are trials.** Each trial declared by the task produces exactly one segment. A segment owns its cue\n  sequence and the behavioral element associated with that trial (the stimulus trigger zone, the reward or aversive\n  contingency, etc.).\n- **Corridors** are fixed-length windows of segments. The first segment in a corridor is the **active** trial — it\n  drives behavior. The remaining segments are pure visual lookahead, so the animal can see what is coming without yet\n  experiencing it. The number of segments per corridor is a per-task parameter; setting it to one collapses corridor\n  and segment into the same thing.\n- **Tasks** are the full set of corridors plus a transition graph that describes how trials chain during a session.\n  Each trial may declare a probability distribution over the trials that can follow it; trials without an explicit\n  distribution are followed by a uniformly random trial.\n\n**Iterative corridor traversal.** At session start, the task walks its trial-transition graph to build a flat\nsequence of trials that overshoots the configured track length. The runtime then slides a window the size of the\ncorridor (in segments) over that sequence. The current window names one corridor in the pre-built catalog; the\nanimal is teleported to that corridor's lane (its x-position). Whenever the animal finishes the first segment of\nthe current corridor, the window slides one trial forward and the animal jumps to the corridor that matches the\nnew window. Adjacent corridors share all but one segment, so the visible cue sequence stays continuous across the \nteleport and the animal experiences a single infinite track.\n\nThe corridor count grows exponentially with the number of segments per corridor, so raising the lookahead depth is a\ndeliberate choice: it adds visual context at the cost of an exponentially larger task. Most paradigms use one or two\nsegments per corridor.\n\n### Task Asset Hierarchy\n\nOn disk, a task lives as three artifacts that share the same basename. The template is the authoritative\ndescription; the task prefab and the task scene are derived from it, and the three files together represent one\ntask end to end:\n\n```text\n\u003cname\u003e.yaml      ─┐  the template — an abstract description of the task's cues, trials, and transitions\n                  ▼\n\u003cname\u003e.prefab    ─┐  the task prefab — the runtime hierarchy of corridors, segments, and cues\n                  ▼  introduced in the previous section, built from the template\n\u003cname\u003e.unity     ─┐  the task scene — a runnable scene that instantiates the task prefab and wraps\n                  ▼  it in the auxiliary infrastructure a session needs\n```\n\n- The **template** is the only artifact authored by hand. It is a plain text description of the task — its cues,\n  the trials those cues compose into, and the transition probabilities between trials.\n- The **task prefab** is the runtime hierarchy described in the previous section: every corridor the task can take,\n  with the segments and cues that fill them. It is fully regenerable — the same template always produces the same\n  task prefab.\n- The **task scene** wraps one instance of the task prefab in the auxiliary GameObjects a session needs (the animal\n  avatar, the display rig, the broker client, the controllers that drive avatar motion). Play mode runs against the\n  task scene, not the bare prefab. One hand-authored base scene serves as the template that every task scene is\n  copied from; that base is the only scene that is not a task scene.\n\nThe basename convention is enforced end to end: regenerating from a template named `\u003cname\u003e.yaml` always produces a\n`\u003cname\u003e.prefab` and a `\u003cname\u003e.unity`. One name identifies the task across all three layers.\n\nTwo more file types complete the picture:\n\n- **Segment prefabs** live alongside the task prefab and are owned by their parent template — their filenames embed\n  both the template name and the trial name, so each task's segments are addressable on their own without colliding\n  with any other task's segments.\n- **Cue prefabs** are the one shared layer. They are keyed by cue identity and cue length, so two tasks that declare\n  the same cue identity reuse the same cue prefab file. The generation pipeline aborts up front if two tasks declare\n  the same cue identity with different visuals, so the shared file can never silently corrupt a sibling task.\n\n### Authoring Task Templates\n\nTask templates are YAML files under `Assets/InfiniteCorridorTask/Configurations/`. The schema mirrors the Python\n`TaskTemplate` dataclasses in [sollertia-shared-assets](https://github.com/Sun-Lab-NBB/sollertia-shared-assets), and\nthe deserializer in `ConfigLoader.cs` uses `UnderscoredNamingConvention` to map snake_case YAML keys to camelCase C#\nfields.\n\nA task template defines:\n\n- **cues**: A list of visual cue panels. Each cue has a unique `name`, a `code` (0–255 byte) used for MQTT and\n  downstream data analysis, a `length_cm`, and a `texture` filename resolved against \n  `Assets/InfiniteCorridorTask/Textures/`.\n- **vr_environment**: The Unity corridor configuration — `corridor_spacing_cm`, `segments_per_corridor`,\n  `padding_prefab_name`, `cm_per_unity_unit`, and `cue_offset_cm`.\n- **trial_structures**: A dictionary mapping trial names (e.g., `ABCD`) to their spatial configuration: the cue\n  sequence, the stimulus trigger zone start and end positions, the stimulus location, an optional collision-boundary\n  visibility flag, a trigger type (one of `\"interaction\"`, `\"collision\"`, `\"occupancy_disarm\"`, `\"occupancy_arm\"`, or\n  `\"occupancy_trigger\"`), and an optional probability distribution over successor trials.\n\nThe five trigger modes share the same `Stimulus` event but differ in how that event is fired:\n\n- **interaction**: an animal interaction (e.g., a lick) detected while inside the trigger zone fires the stimulus.\n- **collision**: crossing an invisible boundary wall — a thin collider at `stimulus_location` — fires the stimulus\n  unconditionally, with no sensor and no occupancy requirement. The `showStimulusCollisionBoundary` flag toggles the\n  boundary's visibility.\n- **occupancy_disarm**: colliding with the boundary while the occupancy requirement is **not** met fires the stimulus.\n- **occupancy_arm**: the inverse of `occupancy_disarm` — occupying the zone for the required duration arms the\n  boundary, and colliding with the now-armed boundary fires the stimulus.\n- **occupancy_trigger**: occupying the zone for the required duration fires the stimulus immediately, with no boundary\n  collision.\n\nAll three occupancy modes keep the occupancy-guidance brake: the `OccupancyGuidanceZone` publishes `Delay` to guide\nthe animal toward completing the occupancy requirement when running in guidance mode.\n\nTemplate filenames follow the `ProjectAbbreviation_TaskDescription.yaml` convention (for example, `SSO_Merging.yaml`).\nThe template name is derived from the filename and is reused verbatim as the Unity scene name, the task prefab name,\nand the prefix for every generated segment prefab. Each template must include a YAML comment header with `Project`,\n`Purpose`, `Layout`, and `Related` fields:\n\n```yaml\n# Project: StateSpaceOdyssey\n# Purpose: Merges ABC and AGFE trial structures by sharing the A cue.\n# Layout:  Segment ABC with the rewarding stimulus trigger zone in cue C.\n#          Segment AGFE with the rewarding stimulus trigger zone in cue E.\n# Related: SSO_Shared_Base (ABC base training), SSO_Merging_Base (AGFE base training)\n```\n\n***Note,*** detailed schema authoring guidance is owned by the `/task-templates` skill in the sollertia marketplace's\n**assets** plugin. See [sollertia-shared-assets](https://github.com/Sun-Lab-NBB/sollertia-shared-assets) for the\nauthoritative dataclass definitions.\n\n### Creating Tasks\n\nTasks can be generated from the Editor menu or programmatically via the McpBridge.\n\n**Editor menu flow.** Select `CreateTask → New Task` from the Unity menu bar. A file dialog seeded at\n`Assets/InfiniteCorridorTask/Configurations/` opens; only templates inside that directory are accepted. After selecting\na template, the pipeline:\n\n1. Runs a cross-template cue-texture preflight (`ValidateCueDefinitionsAcrossTemplates`) — if two templates declare the\n   same `(cue name, length_cm)` pair with different textures, the generation aborts before any asset is written.\n2. Wipes every segment prefab the template owns (`CleanGeneratedSegments`) so trial-parameter edits take effect.\n3. Builds or reuses cue prefabs keyed by `(name, length_cm)` under `Assets/InfiniteCorridorTask/Cues/`.\n4. Builds every segment prefab from scratch under \n   `Assets/InfiniteCorridorTask/Prefabs/\u003cTemplateName\u003e_\u003cTrialName\u003e.prefab`.\n5. Assembles the task prefab at `Assets/InfiniteCorridorTask/Tasks/\u003cTemplateName\u003e.prefab`.\n6. Copies `Assets/Scenes/ExperimentTemplate.unity` to `Assets/Scenes/\u003cTemplateName\u003e.unity`, instantiates the task\n   prefab into it, and runs `MainWindow.EnsureControllers` so both the `LinearTreadmill` (hardware) and\n   `SimulatedLinearTreadmill` (keyboard testing) controllers are present.\n\n**MCP-driven flow.** The same pipeline is reachable via AI agents over the `slsa mcp` server's Unity relay. A single\n`create_task_tool` call builds both the task prefab and the matching scene from the same `CreateTask.CreateFromTemplate`\nand `CreateTask.CreateSceneFromTemplate` methods used by the Editor menu, so the resulting assets are byte-equivalent.\n\n### Loading and Running Tasks\n\nWhen a task is generated via the Editor menu, the scene is created and opened automatically. To load an existing task\ninto a new scene manually:\n\n1. Open the scene at `Assets/Scenes/\u003cTemplateName\u003e.unity` via `File → Open Scene` (the file already contains the task\n   prefab instance, the Display rig, the Actor, both controllers, and the MQTT client).\n2. Configure displays and per-scene parameters via the [Task Parameters window](#task-parameters-window).\n3. Enter Play Mode (`Window → Task Parameters` is always available, and the Play button starts execution).\n\n***Note,*** scene-specific settings such as the camera-to-monitor mapping are scene-bound. Always verify the mapping\nafter every reboot, since the operating system may reorder display ports.\n\n### Task Parameters Window\n\n`Window → Task Parameters` opens the consolidated editor surface for every per-scene configuration. The window is\ndocked next to the Inspector tab and auto-opens on Editor start, scene open, and Play Mode entry. The surface exposes\nfive sections:\n\n| Section          | Controls                                                                                                                                    |\n|------------------|---------------------------------------------------------------------------------------------------------------------------------------------|\n| Actor            | Animal model selection and active controller (Linear or Simulated Linear)                                                                   |\n| MQTT             | Broker IP and port; the Test Connection button performs a one-shot connect/disconnect probe                                                 |\n| Display          | Brightness, height in VR, and a Blank/Show toggle for the active display                                                                    |\n| Camera Mapping   | Refresh Monitor Positions plus a per-monitor row (one per OS-detected monitor) with a camera dropdown (`Left View`, `Center View`, `Right View` for the default display rig) and a Show Full-Screen Views action |\n| Task             | Require Interaction, Require Wait, Track Length, and Track Seed for the active scene's `Task` component                                     |\n\nThe `Task` component's public fields are marked `[HideInInspector]`; `TaskEditor` replaces the default Inspector with\na HelpBox pointing at this window. Configure every task field through Task Parameters, not the Inspector. The\n`Require Interaction` and `Require Wait` controls are hidden when the active scene lacks the corresponding\n`GuidanceZone` or `OccupancyZone`.\n\n***Warning!*** Verify monitor assignments after every system reboot. The operating system can reassign display ports,\nand the camera-to-monitor mapping is scene-bound — a mismatch causes the wrong camera to render to each physical\nmonitor.\n\nFor manual testing without hardware, select **Simulated Linear** as the Actor's controller. The\n`SimulatedLinearTreadmill` reads keyboard input via the Unity Input System and publishes a synthetic `Interaction`\nmessage on every press of the Jump action (spacebar).\n\n### MQTT Contract\n\nThis project communicates with [sollertia-experiment](https://github.com/Sun-Lab-NBB/sollertia-experiment) over MQTT\n5.0. All topics are flat PascalCase identifiers (no hierarchical separators), declared as `public const string` values\nin `Assets/Gimbl/Scripts/MQTT/MQTTTopics.cs`.\n\n| Topic                | Direction (Unity)   | Payload                                               |\n|----------------------|---------------------|-------------------------------------------------------|\n| `SessionStart`       | Publish             | Empty trigger                                         |\n| `SessionStop`        | Publish             | Empty trigger                                         |\n| `Motion`             | Subscribe           | `{movement: float}`                                   |\n| `Interaction`        | Publish + Subscribe | Empty trigger                                         |\n| `Stimulus`           | Publish + Subscribe | `{trialName: string, delivered: bool, cause: string}` |\n| `Delay`              | Publish             | `{delayMilliseconds: uint}`                           |\n| `CueSequenceTrigger` | Subscribe           | Empty trigger                                         |\n| `CueSequence`        | Publish             | `{cueSequence: byte[]}`                               |\n| `SceneNameTrigger`   | Subscribe           | Empty trigger                                         |\n| `SceneName`          | Publish             | `{name: string}`                                      |\n| `RequireInteraction` | Subscribe           | `{value: bool}`                                       |\n| `RequireWait`        | Subscribe           | `{value: bool}`                                       |\n\nWhen the broker is unreachable, `MQTTClient.Publish` routes messages in-process so keyboard-only test runs still reach\nlocal subscribers (for example, the on-screen `LickStimulusSpawner` indicator). Only `Interaction` and `Stimulus`\nhave a local Unity-side subscriber, so they alone can be exercised without a broker; every other topic requires a\nreal MQTT 5.0 broker because sollertia-experiment is the counterparty.\n\n***Note,*** the `/mqtt-contract` skill in the sollertia marketplace's **unity** plugin is the canonical reference for\ntopic ownership and payload shape. Any topic addition or rename must be coordinated with sollertia-experiment in the\nsame release.\n\n### Editor MCP Bridge\n\nThe `McpBridge` editor plugin (`Assets/InfiniteCorridorTask/Scripts/Editor/McpBridge.cs`) starts an `HttpListener` on\n`127.0.0.1:8090`, `[::1]:8090`, and `localhost:8090` when the Unity Editor loads. The listener exposes Editor\noperations to AI agents via JSON request/response. The backing MCP server (`slsa mcp` from\n[sollertia-shared-assets](https://github.com/Sun-Lab-NBB/sollertia-shared-assets)) relays each agent tool call to this\nbridge over HTTP.\n\nThe bridge dispatches **14 tools**:\n\n| Tool                    | Description                                                                       |\n|-------------------------|-----------------------------------------------------------------------------------|\n| `create_task`           | Builds the task prefab and the matching scene from a YAML template in one call    |\n| `delete_task`           | Removes the scene + companion + task prefab + every segment prefab for a template |\n| `inspect_prefab`        | Returns hierarchy, components, transforms, and collider details                   |\n| `clone_zone_prefab`     | Clones a base zone prefab into a new trigger-zone prefab (script + field swaps)    |\n| `delete_asset`          | Deletes a regenerable non-scene asset (refuses hand-authored protected paths)     |\n| `list_assets`           | Lists Unity assets by type filter within a search path                            |\n| `list_scenes`           | Enumerates every `.unity` asset and reports the active scene                      |\n| `open_scene`            | Opens a scene with explicit `unsaved_changes` policy                              |\n| `inspect_scene`         | Returns the active scene's root hierarchy and dirty flag                          |\n| `enter_play_mode`       | Triggers `EditorApplication.EnterPlaymode`                                        |\n| `exit_play_mode`        | Triggers `EditorApplication.ExitPlaymode`                                         |\n| `get_play_state`        | Returns `playing`, `compiling`, or `edit` plus the active scene name              |\n| `read_task_parameters`  | Snapshots Actor, MQTT, Display, Camera Mapping, and Task fields                   |\n| `write_task_parameters` | Applies a subset of Task Parameters fields and returns a new snapshot             |\n\nAll responses are JSON objects carrying a `success` boolean plus a payload or error string. `delete_asset` is bounded\nby an allow-prefix list (`Assets/InfiniteCorridorTask/Tasks/`, `Prefabs/`, `Cues/`, `Materials/`) and rejects scene\npaths under `Assets/Scenes/` — scene cleanup goes through `delete_task` exclusively so the cascade-delete of the\nmatching `Assets/VRSettings/Displays/\u003cscene\u003e-savedFullScreenViews.asset` companion can never be bypassed. A\nprotected-paths set covers the four hand-authored prefabs (`StimulusTriggerZone.prefab`, `OccupancyTriggerZone.prefab`,\n`ResetZone.prefab`, `Padding.prefab`), the four hand-authored materials (`_CueShaderReference.mat`, `Floor.mat`,\n`Wall.mat`, `TargetMat.mat`), and the scene base template (`ExperimentTemplate.unity`). Path traversal sequences and\nabsolute paths are rejected.\n\n***Note,*** AI agents do not call this bridge directly. They use the `slsa mcp` server's Unity relay tools, which are\nlisted in the [sollertia-shared-assets](https://github.com/Sun-Lab-NBB/sollertia-shared-assets) README. The bridge\nexists so that any MCP-driven action against this project shares the same code paths as the Editor menu.\n\n___\n\n## Developer Notes\n\nThese notes apply to project developers and task authors who modify source code, segment prefabs, or templates.\n\n### Project Layout Conventions\n\nThree categories of assets coexist in this project:\n\n- **Hand-authored** (protected): `StimulusTriggerZone.prefab`, `OccupancyTriggerZone.prefab`, `ResetZone.prefab`,\n  `Padding.prefab`, `Materials/_CueShaderReference.mat`, `Materials/Floor.mat`, `Materials/Wall.mat`,\n  `Materials/TargetMat.mat`, and `Scenes/ExperimentTemplate.unity`. These are the source templates and shared assets\n  that `CreateTask` references at generation time (and that the trigger zone prefabs reference in turn via\n  `TargetMat.mat`). They must remain untouched; `McpBridge.DeleteProtectedPaths` refuses to delete them.\n- **Generated** (regenerable): every cue prefab under `Cues/`, every segment prefab under `Prefabs/` matching\n  `\u003cTemplateName\u003e_\u003cTrialName\u003e.prefab`, every cue material under `Materials/Cue_*_*cm.mat`, every prefab under `Tasks/`,\n  and every scene other than `ExperimentTemplate.unity`. These are produced by `CreateTask` and are safe to delete via\n  `delete_task` (whole-task cleanup) or `delete_asset` (individual cue prefab / material) so the next generation pass\n  rebuilds them.\n- **Shared inputs**: the YAML templates under `Configurations/` and the cue textures under `Textures/`. Templates are\n  the source of truth for trial structure; textures are imported from external sources (for example,\n  [vr-visual-cues](https://github.com/sprustonlab/vr-visual-cues)) and referenced by the `cues[].texture` field.\n\n***Note,*** cue prefabs are shared across templates by `(name, length_cm)`. Editing a cue's texture without renaming\nthe cue requires deleting the affected `Cue_\u003cname\u003e_\u003clength\u003ecm.prefab` and `Cue_\u003cname\u003e_\u003clength\u003ecm.mat` before\nregenerating; the cross-template cue-texture preflight catches conflicts before they corrupt downstream assets.\n\n### Formatting and Style\n\nThis project uses [CSharpier](https://csharpier.com/) for code formatting and an `.editorconfig` for naming, brace,\nand spacing conventions. Run `csharpier .` before committing, or `csharpier --check .` to verify without modifying.\nSee the `/csharp-style` skill in the ataraxis marketplace's **automation** plugin for the complete C# convention\nreference.\n\n### Extending the Library\n\nThe project exposes six concentrated extension points. Each has a matching skill in the sollertia marketplace's\n**unity** or **assets** plugin, listed alongside the touch points below.\n\n| Extension                | Touch points                                                                                    | Owner skill        |\n|--------------------------|-------------------------------------------------------------------------------------------------|--------------------|\n| New task template        | YAML in `Configurations/`; generate via `/task-prefabs`                                         | `/task-templates`  |\n| New cue texture          | PNG in `Textures/`; reference it from a YAML `texture` field                                    | `/task-templates`  |\n| New trigger zone type    | New zone script + prefab (via `/zone-prefabs`) + `ConfigLoader` literal + `CreateTask` branch   | `/zone-prefabs`    |\n| New MQTT topic           | `MQTTTopics` constant + matching publisher / subscriber on Unity and sollertia-experiment sides | `/mqtt-contract`   |\n| New `McpBridge` tool     | `Dispatch` switch case + handler method + `@mcp.tool()` wrapper in `unity_tools.py`             | n/a (manual)       |\n| New treadmill controller | `ControllerObject` subclass + `ControllerTypes` enum entry                                      | `/gimbl-framework` |\n\n**Adding a new trigger zone type** is the most cross-cutting extension. The `/zone-prefabs` skill documents the\ncopy-and-edit workflow for the prefab itself: start from `StimulusTriggerZone.prefab` (interaction and collision modes)\nor `OccupancyTriggerZone.prefab` (the three occupancy modes) under `Prefabs/`, swap the modifier script GUIDs, rename\nregions, and override field defaults. The new prefab path must then be added to `McpBridge.DeleteProtectedPaths`, a new\nbranch must be added in `CreateTask.BuildSegmentPrefabs` with a matching `Place...Zone` helper, and \n`ConfigLoader.ValidateTemplate` must accept the new `trigger_type` literal. `CreateTask` sets the `TriggerMode` enum\nfield on `StimulusTriggerZone` from the `trigger_type`, and the zone dispatches on that enum. The Python side requires\na matching `TriggerType` registry update via the `/library-extension` skill in the **assets** plugin. Adding a\n`TriggerType` member does **not** require a `from_task_template` branch in every acquisition system: the platform\n`TriggerType` enum carries all members, but each system maps only the subset it supports and may leave a mode\nunmapped. A config that uses an unmapped mode raises a clear \"not mapped to a runtime trial class\" error. The\nMesoscope-VR system, for example, maps `interaction` (`MesoscopeWaterRewardTrial`) and `occupancy_disarm`\n(`MesoscopeGasPuffTrial`), and does not map `collision`, `occupancy_arm`, or `occupancy_trigger`.\n\n**Adding a new MQTT topic** requires the constant in `MQTTTopics.cs` (with `Direction`, `Payload`, and `Callers`\nremarks), a runtime script that publishes or subscribes, an in-lockstep update in sollertia-experiment, and a refresh\nof the `/mqtt-contract` skill catalog.\n\n**Adding a new McpBridge tool** requires a new switch case in `McpBridge.Dispatch`, a handler method that returns\n`Ok(...)` or `Error(...)`, optional integration with `AcquireSceneComponents` and `BuildSnapshot` when the tool reads\nor writes scene state, and a matching `@mcp.tool()` wrapper in\n`sollertia-shared-assets/src/sollertia_shared_assets/interfaces/unity_tools.py`.\n\n### AI-Assisted Development\n\nClaude Code skills and AI development assets for this project are distributed through two marketplaces:\n\n- [sollertia](https://github.com/Sun-Lab-NBB/sollertia) marketplace:\n  - **unity** plugin — Unity Editor skills that drive McpBridge tools, document the MQTT contract, document the\n    `CreateTask` pipeline, and guide manufacturing of new trigger zone prefabs.\n  - **assets** plugin — registers the `slsa mcp` server (which fronts the Unity relay), and provides configuration and\n    experiment-authoring skills (task templates, experiment configurations, library extension).\n- [ataraxis](https://github.com/Sun-Lab-NBB/ataraxis) marketplace:\n  - **automation** plugin — shared development skills that enforce coding conventions (C# style, README style, commit\n    messages, project layout) and general-purpose codebase exploration tools.\n\nInstall all three plugins to make the full skill set available to compatible AI coding agents. The **unity** plugin\ndepends on the **assets** plugin for the backing MCP server that drives the Unity Editor relay.\n\n___\n\n## Versioning\n\nThis project uses [semantic versioning](https://semver.org/). See the\n[tags on this repository](https://github.com/Sun-Lab-NBB/sollertia-virtual-reality/tags) for the available project\nreleases.\n\n___\n\n## Authors\n\n- Ivan Kondratyev ([Inkaros](https://github.com/Inkaros))\n- Jacob Groner ([Jgroner11](https://github.com/Jgroner11))\n\n___\n\n## License\n\nThis project is licensed under the Apache 2.0 License: see the [LICENSE](LICENSE) file for details.\n\n___\n\n## Acknowledgments\n\n- All Sun lab [members](https://neuroai.github.io/sunlab/people) for providing the inspiration and comments during the\n  development of this library.\n- The creators of the original [GIMBL](https://github.com/winnubstj/Gimbl) package, whose framework is inlined under\n  `Assets/Gimbl/`.\n- The creators of [MQTTnet](https://github.com/dotnet/MQTTnet) and [YamlDotNet](https://github.com/aaubry/YamlDotNet),\n  whose libraries ship as committed DLLs under `Assets/Plugins/`.\n- The [vr-visual-cues](https://github.com/sprustonlab/vr-visual-cues) project for the cue texture set.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsun-lab-nbb%2Fsollertia-virtual-reality","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsun-lab-nbb%2Fsollertia-virtual-reality","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsun-lab-nbb%2Fsollertia-virtual-reality/lists"}