{"id":14961096,"url":"https://github.com/timcsy/gymize","last_synced_at":"2025-10-24T20:31:11.670Z","repository":{"id":187409130,"uuid":"575785076","full_name":"timcsy/gymize","owner":"timcsy","description":"Unity and Python Reinforcement and Imitation Learning with Gymnasium and PettingZoo API.","archived":false,"fork":false,"pushed_at":"2024-01-25T06:44:34.000Z","size":13642,"stargazers_count":12,"open_issues_count":1,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-10-30T00:05:03.246Z","etag":null,"topics":["3d","gym","gymnasium","imitation-learning","pettingzoo","reinforcement-learning","rl","unity"],"latest_commit_sha":null,"homepage":"","language":"C#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/timcsy.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-12-08T09:42:08.000Z","updated_at":"2024-07-23T11:53:56.000Z","dependencies_parsed_at":"2024-01-25T07:43:26.325Z","dependency_job_id":null,"html_url":"https://github.com/timcsy/gymize","commit_stats":{"total_commits":67,"total_committers":1,"mean_commits":67.0,"dds":0.0,"last_synced_commit":"889f28917dfb7ce7124d84d91a631de9ccbfed9e"},"previous_names":["timcsy/gymize"],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timcsy%2Fgymize","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timcsy%2Fgymize/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timcsy%2Fgymize/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timcsy%2Fgymize/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/timcsy","download_url":"https://codeload.github.com/timcsy/gymize/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":238030288,"owners_count":19404859,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d","gym","gymnasium","imitation-learning","pettingzoo","reinforcement-learning","rl","unity"],"created_at":"2024-09-24T13:23:49.100Z","updated_at":"2025-10-24T20:31:01.768Z","avatar_url":"https://github.com/timcsy.png","language":"C#","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Gymize - Gym Reinforcement Learning with Unity 3D\n\nUnity and Python Reinforcement and Imitation Learning with Gymnasium and PettingZoo API.\n\n![Gymize_Framework](docs/images/gymize_framework.png)\n\n## Installations\n\n### Installation for Python 3\n\n```\npip install gymize\n```\n\nIf you want to render videos, please install [ffmpeg](https://ffmpeg.org/download.html) additionally.\n\nYou can also install ffmpeg with Anaconda:\n```\nconda install -c conda-forge ffmpeg\n```\n\n### Installation for Unity\n\nGo to Window -\u003e Package Manager -\u003e Add package from git URL...\n\nThen paste the following git URL:\n```\nhttps://github.com/timcsy/gymize.git?path=/unity\n```\n\nIf git hasn't been installed, then you can download gymize first and install it from disk, specify the path to `/unity/package.json`, or you can install from tarball, downloaded from Releases.\n\n\n## Usages\n\n### Usage for Python 3\n\n1. Assign the `env_name`, which should be same as in the Unity. For example, `kart`.\n2. `file_name` is the path to the built Unity game, or leave `None` if using Unity Editor.\n3. For multi-agent, `agent_names` is the list of agent names.\n4. Define the `observation_space` in the format of [Gym Spaces](https://gymnasium.farama.org/api/spaces/). For multi-agent, `observation_spaces` is a dictionary whose keys are agent names and values are observation spaces.\n5. Define the `action_space` in the format of [Gym Spaces](https://gymnasium.farama.org/api/spaces/). For multi-agent, `action_spaces` is a dictionary whose keys are agent names and values are action spaces.\n6. Assign `render_mode='video'` if you want to record video from Unity, otherwise omit `render_mode`.\n7. Assign `views=['', ...]` if you want to record video from Unity, which given a list of view names, the empty string will be the default view, otherwise omit `views`.\n8. Make the Gymnasium or PettingZoo environment by the following commands:\n\nSingle-Agent with Gymnasium API:\n```\nimport gymnasium as gym\nimport gymize\n\nenv = gym.make(\n    'gymize/Unity-v0',\n    env_name='\u003cyour env name\u003e',\n    file_name=file_name,\n    observation_space=observation_space,\n    action_space=action_space,\n    render_mode='\u003crender_mode\u003e',\n    views=['', ...]\n)\n```\n\nMulti-Agents with PettingZoo AEC API:\n```\nfrom gymize.envs import UnityAECEnv\n\nenv = UnityAECEnv(\n    env_name='\u003cyour env name\u003e',\n    file_name=file_name,\n    agent_names=agent_names,\n    observation_spaces=observation_spaces,\n    action_spaces=action_spaces,\n    render_mode='\u003crender_mode\u003e',\n    views=['', ...]\n)\n```\n\nMulti-Agents with PettingZoo Parallel API:\n```\nfrom gymize.envs import UnityParallelEnv\n\nenv = UnityParallelEnv(\n    env_name='\u003cyour env name\u003e',\n    file_name=file_name,\n    agent_names=agent_names,\n    observation_spaces=observation_spaces,\n    action_spaces=action_spaces,\n    render_mode='\u003crender_mode\u003e',\n    views=['', ...]\n)\n```\n\nWell done! Now you can use the environment as the gym environment!\n\nThe environment `env` will have some additional methods other than Gymnasium or PettingZoo:\n- `env.unwrapped.send_info(info, agent=None)`\n    - At anytime, you can send information through `info` parameter in the form of Gymize Instance (see below) to Unity side.\n    - The `agent` parameter is the agent name that will receive info (by `Gymize.Agent.OnInfo()` method in the Unity side), or `None` for the environment to receive info (by `Gymize.GymEnv.OnInfo += (info) =\u003e {}` listener in the Unity side).\n- `env.unwrapped.begin_render(screen_width=-1, screen_height=-1, fullscreen=False)`\n    - Begin to record video in the Unity.\n    - `screen_width` and `screen_height` is to set the width and height of the window, leave `-1` if you want the default window size.\n    - `fullscreen` is to set whether Unity is fullscreen.\n- `env.unwrapped.end_render()`\n    - Stop to record video in the Unity.\n- `env.unwrapped.render_all(video_paths={})`\n    - Render the videos which name and path are given by key value pairs in `video_path`.\n    - Using `None` as path for binary video object.\n- `env.unwrapped.render()` or `env.render()`\n    - Only render the default video, and return a binary video object.\n\nIf you want to open the signaling service only, you can run `gymize-signaling` at the command line.\n\n### Usage for Unity\n\n1. Create Gym Manager Component\n    - Add a game object in the scene.\n    - Add a `Gym Manager` Component to the game object.\n    - Fill in the `Env Name` property with the name of the environment, which should be same as in the Python. For example, `kart`.\n2. Implement a class that inherit the `Agent` class: `class MyAgent : Agent {}`\n    - Note that the `Agent` class is inheritted from `MonoBehaviour` class.\n        - Override these listeners\n        - `public override void OnReset() {}`\n        - `public override void OnAction(object obj) {}`\n        - `public override void OnInfo(object obj) {}`\n      - Add `Terminate()`, `Truncate()` at the proper place.\n      - For example:\n        ```\n        public class MyAgent : Agent\n        {\n            [Obs]\n            private float pi = 3.14159f;\n\n            private int m_Count;\n            private float m_Speed;\n            private string m_NickName;\n\n            public override void OnReset()\n            {\n                // Reload the Unity scene when getting the reset signal from Python\n                SceneManager.LoadScene(SceneManager.GetActiveScene().name, LoadSceneMode.Single);\n            }\n\n            public override void OnAction(object action)\n            {\n                Dictionary\u003cstring, object\u003e actions = action as Dictionary\u003cstring, object\u003e;\n                List\u003cobject\u003e arr = actions[\"arr\"] as List\u003cobject\u003e;\n\n                Debug.Log((long)arr[0]); // arr[0] is Discrete value\n                m_Count = Convert.ToInt32(arr[0]); // arr[0] is Discrete value\n                m_Speed = Convert.ToSingle(actions[\"speed\"]); // actions[\"speed\"] is float value\n            }\n\n            public override void OnInfo(object info)\n            {\n                m_NickName = (string)info;\n            }\n\n            void Update()\n            {\n                // Terminate the game if collision occurred\n                if (m_Collision)\n                {\n                    // This method will tell the Python side to terminate the env\n                    Terminate();\n                }\n            }\n        }\n        ```\n    - Add this Agent Component to the game object, and fill in the `Name` property with the name of the agent, using `agent` for single agent.\n3. Add Observers\n    - Refer [below](#Locator) to learn more about the concept of \"Locator\".\n    - Using Attributes in the Agent class, assign the Locator or use the default Locator, for example:\n        ```\n        [Obs] // Using the default Locator \".UsedTime\", which is same as the field name.\n        private float UsedTime;\n\n        [Obs(\".Progress\")]\n        private float m_Progress;\n\n        [Box]\n        private float Distance;\n\n        [Box(\".Height=$\")]\n        private float m_Height;\n        ```\n    - You can use `Obs` to use the default Gym space type depending on the source variable, or you can use `Box`(for Tensor, including MultiBinrary and MultiDiscrete), `Discrete`, `Text`, `Dict`, `List`, `Graph` attributes, see more details in the [Reflection](https://github.com/timcsy/gymize/blob/main/unity/Runtime/Agent/Observation/Reflection/) folder, check out [TestAgentInstance.cs](https://github.com/timcsy/gymize/blob/main/unity/Tests/Runtime/TestAgentInstance.cs) for more examples.\n    - Add Sensor Component in the scene.\n        - Inheritted from `SensorComponent` class. `CameraSensor` is an example of a sensor component.\n        - Make sure that the `public override IInstance GetObservation() {}` method is implemented in the Sensor Component.\n        - Assign proper Locator in the Unity Editor.\n4. Add Gym Render Recorder Component to the scene if needed\n    - The `Name` property can be empty or the name of the view.\n    - At the Python side, set `render_mode='video'` if you want to render videos.\n5. You can disable the `Gym Manager` component in the Unity Editor to develop the game without Python connection and play the game manually, it is useful for debugging.\n\n!!! Remember to close the channel in MonoBehaviour.OnApplicationQuit !!!\n\n\n## Gymize Instance\n\nThe instance generated from the action, observation space or info is called \"Gymize Instance\".\n\nGymize Instance is defined in the [space.proto](https://github.com/timcsy/gymize/blob/main/proto/definitions/space.proto), which describes how the Gymize exchange data between Unity and Python, using [Protocol Buffers 3](https://protobuf.dev/programming-guides/proto3/). Most of which originates from the [Gym Spaces](https://gymnasium.farama.org/api/spaces/).\n\nIn Unity, check out [GymInstance.cs](https://github.com/timcsy/gymize/blob/main/unity/Runtime/Space/GymInstance.cs) for more information about how to convert the object into a meaningful type. In Python, you can treat the instance as usual object.\n\n### Fundamental Instance\n\n- `Tensor`: numpy array, with dtype and shape\n    - Corresponding to the Gym Spaces as `Box`, `MultiBinrary`, `MultiDiscrete`\n- `Discrete`: int64\n    - Corresponding to the Gym Spaces as `Discrete`\n- `Text`: string\n    - Corresponding to the Gym Spaces as `Text`\n\n### Composite Instance\n\n- `Dict`: `key`, `value` pairs mapping. Type of `key` is string, `value` is Gymize Instance\n    - Corresponding to the Gym Spaces as `Dict`\n- `List`: array of Gymize Instance\n    - Corresponding to the Gym Spaces as `Tuple`, `Sequence`\n- `Graph`: including three tensor objects, which are `nodes`, `edges` and `edge_links`\n    - `nodes`: the numeric information of nodes\n    - `edges`: the numeric information of edges\n    - `edge_links`: the list of edges represent by node pairs (the node index begins with 0)\n    - Corresponding to the Gym Spaces as `Graph`\n\n### Additional Instance (Not defined in Gym Spaces)\n\n- `Raw`: binary data\n- `Image`: image data, include format (PNG, JPG, ...), binary data, dtype, shape, axis permutation\n- `Float`: double precision floating number\n- `Boolean`: boolean value (true/false)\n- `JSON`: JSON after stringifying\n\n\n## Locator\n\n\"Locator\" maps the observations collected from the Unity side to the specified location of Python side observation data, which is transferred by Gymize Instance.\n\nThe followings are valid examples:\n\nexample 1:\n```\n.UsedTime = $\n```\n\nexample 2:\n```\n.Progress\n```\n\nexample 3:\n```\n@.Rays\n```\n\nexample 4:\n```\nagent1@agent2@.key.0[12][\"camera\"]['front'][right][87](2)\n```\n\nexample 5:\n```\n@@agent3@agent4@[\"camera\"](1:10:2) = $(24:29) \u0026 @[11]=$[0] \u0026 @.key = $(3:8)\n```\n\nFor more examples, check out [TestLocator.cs](https://github.com/timcsy/gymize/blob/main/unity/Tests/Runtime/TestLocator.cs) and [TestAgentInstance.cs](https://github.com/timcsy/gymize/blob/main/unity/Tests/Runtime/TestAgentInstance.cs).\n\n### Structure of Locator\n\n- A `Locator` is a sequence of `Mapping`s.\n- A `Mapping` consists of `Agent`, `Destination`, and `Source`.\n    - `Agent` has four kinds: \"all agents\", \"list agents\", \"root agent\", and \"omitted\".\n    - `Destination` and `Source` are in the form of `Selector`.\n- A `Selector` can act on the following types: `Dict`, `Tuple`, `Sequence`, `Tensor`.\n    - \"key\" selector is for `Dict`.\n    - \"index\" or \"slices\" selector is for `Tuple`, `Sequence`, `Tensor`.\n- A `Slice` is Python-like or Numpy-like, which has \"start\", \"stop\", and \"step\".\n    - Usually we write `start:stop:step`, e.g. `1:10:2`.\n    - There are some special cases: \"index\", \"ellipsis\", and \"new axis\".\n\n### Syntax for Locator\n\n- Syntax for `Locator`\n    - Using `\u0026` to connect different `Mapping`s.\n    - e.g. `.field1 \u0026 .field2=$ \u0026 @.field3=$[\"key\"]`\n- Syntax of `Mapping`\n    - {`Agent`} `Selectors(Destination)` {`=$` `Selectors(Source)`}\n        - { } means can be omitted.\n        - If you omit `Agent`, then the Locator will become relative w.r.t Agent.\n        - If you omit `=$` `Selectors(Source)`, it is same as `=$`.\n        - `=` means mapping or assignment. It will map the source (right hand side) to the destination (left hand side).\n        - `$` means the Unity side observation (variable or sensor data).\n- Syntax for `Agent`\n    - Using `agent@` to assign the agent name.\n    - Using `agent1@agent2@` to assign the agent names.\n    - Using `@@` means for all agents.\n    - Using `@@agent3@agent4@` means for all agents, except `agent3` and `agent4`.\n    - Using `@` to represent the root agent itself.\n    - You can omit the `Agent` to use relative location.\n- Syntax for `Selector`\n    - For `Dict`: `.key`, `['key']`, `[\"key\"]`, or `[key]`\n    - For `Tuple`: `[index]` or `[slice]`\n    - For `Sequence`: `[]`, means append\n    - For `Tensor`: `(Slice)` or `(Slice1, Slice2, ..., SliceN)`\n        - The several `Tensor` selectors have to put at the end of the selector sequence.\n- Syntax for `Slice`\n    - Same syntax as Python or numpy.\n    - `start:stop:step`, e.g. `1:10:2`.\n        - omitted \"step\", it becomes `1:10`.\n        - omitted \"stop\", it becomes `1:`.\n        - omitted \"start\", it becomes `:`.\n    - You can just use an integer to represent the index (negative integer means counting from the end).\n    - You can use `...` to represent ellipsis.\n    - You can use `newaxis` or `np.newaxis` to represent new axis.\n\nSee [locator.bnf](https://github.com/timcsy/gymize/blob/main/locator.bnf) for more information about the syntax.\n\n\n## Known issues\n\n- Gymize get the observation space type information by generating a sample instance, so it may not get the type information of `Sequence` if it samples a empty array.\n- On the Python side, it may cause some problems if you edit the observation data directly, because it is not a copy, it is a reference object.\n- Dict source with different keys but with same destination Locator may not merged correctly, check out `space.py` for more details.\n- If you run the built Unity Application and it fails after first OnReset, try:\n    - It may be this issue: `ArgumentNullException: Value cannot be null. Parameter name: shader`.\n    - Go to: Edit -\u003e ProjectSettings -\u003e Graphics\n    - Change the size of Always Included Shaders, and add the `Runtime/Space/Grayscale.shader` into it.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftimcsy%2Fgymize","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftimcsy%2Fgymize","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftimcsy%2Fgymize/lists"}