{"id":22846122,"url":"https://github.com/kajiih/rl_thor","last_synced_at":"2026-04-28T11:32:39.970Z","repository":{"id":244336378,"uuid":"755269821","full_name":"Kajiih/rl_thor","owner":"Kajiih","description":"🤖 Lightweight and customizable reinforcement learning environment based on AI2-THOR","archived":false,"fork":false,"pushed_at":"2024-09-21T16:30:55.000Z","size":49490,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-02-06T10:14:12.689Z","etag":null,"topics":["ai2thor","benchmark","benchmark-environments","embodied-ai","gymnasium","meta-learning","multi-task","reinforcement-learning","reinforcement-learning-environments","stable-baselines3"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Kajiih.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-02-09T19:25:12.000Z","updated_at":"2025-01-21T17:08:00.000Z","dependencies_parsed_at":"2024-06-14T04:52:48.431Z","dependency_job_id":null,"html_url":"https://github.com/Kajiih/rl_thor","commit_stats":null,"previous_names":["kajiih/rl_thor"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Kajiih%2Frl_thor","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Kajiih%2Frl_thor/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Kajiih%2Frl_thor/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Kajiih%2Frl_thor/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Kajiih","download_url":"https://codeload.github.com/Kajiih/rl_thor/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246423494,"owners_count":20774796,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai2thor","benchmark","benchmark-environments","embodied-ai","gymnasium","meta-learning","multi-task","reinforcement-learning","reinforcement-learning-environments","stable-baselines3"],"created_at":"2024-12-13T03:24:52.420Z","updated_at":"2026-04-28T11:32:34.900Z","avatar_url":"https://github.com/Kajiih.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🤖 RL-THOR\n\nRL-THOR is a lightweight and customizable reinforcement learning environment based on [AI2-THOR](https://ai2thor.allenai.org/).\n\nThanks to AI2-THOR simulation environment, AI agents can explore realistic 3D household environments 🏠, interact with a wide variety of objects 🧹, navigate complex scenes 🧭, and complete meaningful tasks 🏆. The environment is designed to support high-level embodied reasoning and complex interactions, enabling agents to learn and generalize from diverse, scalable tasks.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"media/ITHOR_illustration.png\" alt=\"Ithor Illustration\"\u003e\n\u003c/p\u003e\n\n## Contents \u003c!-- omit from toc --\u003e\n\n- [💻 Installation](#-installation)\n  - [Requirements](#requirements)\n- [🏃 Getting Started](#-getting-started)\n- [☁️ Running Headless](#️-running-headless)\n  - [Cloud rendering](#cloud-rendering)\n  - [Starting X-Server](#starting-x-server)\n- [✏️ Environment Configuration](#️-environment-configuration)\n  - [🕹️ Task Configuration](#️-task-configuration)\n- [➕ Creating new tasks - \\[WIP\\]](#-creating-new-tasks---wip)\n- [🧮 The Benchmark](#-the-benchmark)\n  - [Reproducing baselines results](#reproducing-baselines-results)\n- [📔 Citation](#-citation)\n- [🧾 License](#-license)\n- [🤝 Contributing](#-contributing)\n\n## 💻 Installation\n\n1. **Create virtual environment**\\\n    We recommend you use a [conda](https://docs.conda.io/projects/conda/en/latest/user-guide/install/) virtual environment:\n\n    ```bash\n      #  We require python\u003e=3.12\n      conda create -n rl_thor python=3.12\n      conda activate rl_thor\n    ```\n\n2. **Install RL-THOR and its dependencies**\\\n    To install and customize the environment locally:\n\n    ```bash\n    git clone https://github.com/Kajiih/rl_thor.git\n    pip install -r requirements/dev.txt\n    ```\n\n    \u003c!-- **[!! Not supported yet !!]**\n    Alternatively, if you only want to use predefined settings and task, you can install the PyPI package:\n    ```bash\n    pip install rl-thor\n    ``` --\u003e\n\n### Requirements\n\nRL-THOR requires Python 3.12 or later. In addition, it shares the same requirements as AI2-THOR. Notably, Windows systems are not natively supported. For detailed requirements, refer to the [AI2-THOR requirements](https://github.com/allenai/ai2thor/?tab=readme-ov-file#requirements).\n\n## 🏃 Getting Started\n\nRL-THOR uses [Gymnasium](https://github.com/Farama-Foundation/Gymnasium) API, so you can use it as simply as any other Gym/Gymnasium environment.\n\nThis short script runs an ITHOR environment with the basic configuration and random actions:\n\n```python\nfrom rl_thor.envs import ITHOREnv\n\nenv = ITHOREnv()\nobs, info = env.reset()\n\nterminated, truncated = False, False\nwhile not terminated and not truncated:\n    action = env.action_space.sample()\n    obs, reward, terminated, truncated, info = env.step(action)\nenv.close()\n```\n\nNote that the first time that you instantiate the environment, AI2-THOR will download locally the 3D simulator resources to `~/.ai2thor` (~500MB).\n\nMore examples and training scripts can be found in the `examples` folder of this repository.\n\nTo go further, we recommend you to get familiar with the [concepts of the ITHOR simulation environment](https://ai2thor.allenai.org/ithor/documentation/concepts) and [our documentation](https://github.com/Kajiih/rl_thor) **[create actual documentation]** to understand how tasks are defined.\n\n\u003c!-- ## ☁️ Running Headless\n\nBy default AI2-THOR opens up a Unity window each time a Controller is instantiated, but there are several ways to run AI2-THOR in headless mode, which is useful for training agents on a remote server or in a cloud environment. Depending on your needs, you can use one of the following methods:\n\n### Cloud rendering\n\nAI2-THOR supports cloud rendering. You can enable it by setting the `platform` parameter under `controller_parameters` in the configuration file to `CloudRendering`:\n\n```yaml\ncontroller_parameters:\n  platform: CloudRendering\n```\n\nNote that you will need the appropriate Vulkan drivers installed on your machine to use cloud rendering.\n\nYou can refer to the [official AI2-THOR documentation](https://ai2thor.allenai.org/ithor/documentation/#headless-setup) for more direct information.\n\n### Starting X-Server\n\nIf you are using NVIDIA GPUs, an alternative solution is to start an X-Server on your machine. You can use the [script](https://github.com/allenai/ai2thor/blob/main/scripts/ai2thor-xorg) provided by AI2-THOR after installing the package in your environment:\n\n```bash\nai2thor-xorg start\n```\n\nYou should now be able to instantiate the AI2-THOR controllers.\n\nIf you prefer using docker, we have a [Dockerfile](docker/Dockerfile) based on [ai2thor-docker](https://github.com/allenai/ai2thor-docker) repository.\n\nBuild the image:\n\n```bash\n./docker/scripts/build.sh\n```\n\nRun the image:\n\n```bash\n./docker/scripts/run.sh\n```\n\nIn the container, you can instantiate the controller, for example, run the test script:\n\n```bash\npython example_agent.py\n``` --\u003e\n\n## ✏️ Environment Configuration\n\nThe environment can be configured through a `yaml` file or by dictionary when instantiating the environment.\n\nFor a complete enumeration of the configuration options, see [Configuration](https://github.com/Kajiih/rl_thor). \u003c!-- **[create actual documentation]** --\u003e\n\nIf unspecified, the environment configuration is equivalent to the file `configurations/environment_config.yaml` of this repository.\n\n```yaml\n# === General ===\nseed: 1\nmax_episode_steps: 1000\n\n# === Simulator ===\ncontroller_parameters:\n  platform: null # set to \"CloudRendering\" for headless cloud rendering\n  visibility_distance: 1.5\n  # Camera properties\n  frame_width: 300\n  frame_height: 300\n  field_of_view: 90\n\nscene_randomization:\n  random_agent_spawn: False # If True, the agent will spawn at a random location and rotation at the beginning of each episode\n  random_object_spawn: False # If True, pickupable objects will spawn at random locations at the beginning of each episode\n  random_object_materials: False # If True, the materials of the objects will be randomized at the beginning of each episode\n  random_object_colors: False # If True, the colors of the objects will be randomized at the beginning of each episode  # Note: Not realistic\n  random_lighting: False # If True, the lighting conditions will be randomized at the beginning of each episode\n\n# === Actions ===\naction_groups:\n  # === Navigation actions ===\n  movement_actions: True\n  rotation_actions: True\n  head_movement_actions: True\n  crouch_actions: False\n  # === Object manipulation actions ===\n  pickup_put_actions: True\n  drop_actions: False\n  throw_actions: False\n  push_pull_actions: False\n  hand_control_actions: False\n  # === Object interaction actions ===\n  open_close_actions: True\n  toggle_actions: True\n  slice_actions: False\n  use_up_actions: False\n  liquid_manipulation_actions: False\n  break_actions: False\n\naction_modifiers:\n  discrete_actions: True # If True, all actions requiring a parameter will be discretized and use their discrete value\n  target_closest_object: True # If True, the closest operable object to the agent will be used as target for object interaction actions (e.g. pickup, open, etc.)\n  simple_movement_actions: False # Only keep MoveAhead action (no MoveBack, MoveRight and MoveLeft Actions), should at least be used with body_rotation_actions\n  static_pickup: False # Picked up objects don't teleport to hand\n  stationary_placement: True # If False, a placed object will use the physics engine to resolve the final position (no deterministic placement)\n  partial_openness: False # If True, objects can be opened partially with a parameter (only if open_close_actions is already enabled and environment is continuous) -\u003e Adds partial_open_object_action from the \"special\" action category and removes open_object_action and close_object_action\n\naction_discrete_param_values: # If run in discrete mode\n  movement_magnitude: 0.25\n  rotation_degrees: 45\n  head_movement_degrees: 30\n  throw_strength: 50\n  push_pull_strength: 100\n\n# === Tasks ===\ntasks:\n  globally_excluded_scenes: [] # List of scene names to exclude for all tasks(only full names like \"FloorPlan1\", \"FloorPlan201\", ...)\n  task_blueprints: []\n```\n\nWhen instantiating the environment, you can set the relative path to the configuration:\n\n```python\nenv = gym.make(\"rl_thor/ITHOREnv-v0.1\", config_path=\"path/to/config.yaml\")\n```\n\nBy default, this path is set to `config/environment_config.yaml`.\n\nFor convenience, you can also override specific values of the configuration with the `config_override` parameter.\n\n**Example:**\nIf you only want to change the maximum number of steps per episode to 200 and\nthe randomization of the Agent's spawn location and object materials, you can do it like this:\n\n```python\nconfig_override = {\n    \"max_episode_steps\": 200,\n    \"scene_randomization\": {\n        \"random_agent_spawn\": True,\n        \"random_object_materials\": True,\n    },\n}\nenv = gym.make(\n    \"rl_thor/ITHOREnv-v0.1\",\n    config_folder_path=\"config/environment_config.yaml\",  # Default value\n    config_override=config_override,\n)\n```\n\u003c!-- \n### 🕹️ Task Configuration\n\n[WIP]\n[Add more details about the task configuration] --\u003e\n\n## ➕ Creating new tasks \u003c!--- [WIP]--\u003e\n\nIn AI2-THOR RL, we use a specific task description format called [Graph Task](https://github.com/Kajiih/rl_thor) \u003c!--**[link to documentation]**. --\u003e\n\nThanks to graph tasks, you can define a new task by describing its agency list. In practice, it is as simple as creating a python dictionary describing the task items, their properties and their relations:\n\n```python\ntask_description_dict = {\n    \"plate_receptacle\": {\n        \"properties\": {\"objectType\": \"Plate\"},\n    },\n    \"hot_apple\": {\n        \"properties\": {\"objectType\": \"Apple\", \"temperature\": \"Hot\"},\n        \"relations\": {\"plate_receptacle\": [\"contained_in\"]},\n    },\n}\n```\n\nThis code lets you define a new task consisting of putting a hot apple in a plate. `hot_apple` and `plate_receptacle` are identifiers of the items used to defined relations and each property and relation can be found [here](.) \u003c!--**[link to documentation]**.--\u003e\nThis is enough to automatically create the reward function associated to the graph task.\n\n\u003c!-- -\u003e **[Add explanation how to define task blueprints in the configuration]**\n\n-\u003e **[If automatic task creation from dict is not implemented, also add how to create the class corresponding to the task]** --\u003e\n\nFor more details about how to define new tasks, item properties or relations, see the [dedicated part of the documentation](.) \u003c!-- **[create actual documentation]**. --\u003e\n\n## 🧮 The Benchmark\n\n### Reproducing baselines results\n\nThe training script and commands to reproduce the results of the baselines are available in the `examples/benchmark` folder.\n\n## 📔 Citation\n\nNot available yet\n\n## 🧾 License\n\n| Component            | License                                                                  |\n| -------------------- | -------------------------------------------------------------------------|\n| Codebase (this repo) | [MIT License](LICENSE)                                                   |\n| AI2-THOR             | [Apache License Version 2.0](https://www.apache.org/licenses/LICENSE-2.0)|\n\n## 🤝 Contributing\n\nNot available yet\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkajiih%2Frl_thor","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkajiih%2Frl_thor","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkajiih%2Frl_thor/lists"}