{"id":13457506,"url":"https://github.com/semitable/robotic-warehouse","last_synced_at":"2025-09-11T13:45:10.741Z","repository":{"id":41196687,"uuid":"216525761","full_name":"semitable/robotic-warehouse","owner":"semitable","description":"Multi-Robot Warehouse (RWARE): A multi-agent reinforcement learning environment","archived":false,"fork":false,"pushed_at":"2024-09-15T17:05:19.000Z","size":2578,"stargazers_count":378,"open_issues_count":15,"forks_count":82,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-09-04T20:02:03.174Z","etag":null,"topics":["environment","multi-agent","multi-agent-reinforcement-learning","multi-agent-systems","reinforcement-learning","robotics","simulation"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/semitable.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2019-10-21T09:12:39.000Z","updated_at":"2025-08-25T18:49:30.000Z","dependencies_parsed_at":"2024-06-19T17:40:37.364Z","dependency_job_id":"6b24265c-400f-4603-8867-4d8b71b4dc53","html_url":"https://github.com/semitable/robotic-warehouse","commit_stats":{"total_commits":160,"total_committers":9,"mean_commits":17.77777777777778,"dds":0.63125,"last_synced_commit":"7b83fe04ec5b57523bc4d6b6c1c179c48849a3f7"},"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/semitable/robotic-warehouse","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/semitable%2Frobotic-warehouse","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/semitable%2Frobotic-warehouse/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/semitable%2Frobotic-warehouse/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/semitable%2Frobotic-warehouse/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/semitable","download_url":"https://codeload.github.com/semitable/robotic-warehouse/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/semitable%2Frobotic-warehouse/sbom","scorecard":{"id":810836,"data":{"date":"2025-08-11","repo":{"name":"github.com/semitable/robotic-warehouse","commit":"96fbc64e3eae5fee915e0d390f864fa06ddccd47"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":4.1,"checks":[{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Code-Review","score":2,"reason":"Found 2/9 approved changesets -- score normalized to 2","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Token-Permissions","score":0,"reason":"detected GitHub workflow tokens with excessive permissions","details":["Warn: no topLevel permission defined: .github/workflows/python-publish.yml:1","Info: no jobLevel write permissions found"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: MIT License: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Pinned-Dependencies","score":2,"reason":"dependency not pinned by hash detected -- score normalized to 2","details":["Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/python-publish.yml:21: update your workflow using https://app.stepsecurity.io/secureworkflow/semitable/robotic-warehouse/python-publish.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/python-publish.yml:23: update your workflow using https://app.stepsecurity.io/secureworkflow/semitable/robotic-warehouse/python-publish.yml/master?enable=pin","Warn: pipCommand not pinned by hash: .github/workflows/python-publish.yml:28","Warn: pipCommand not pinned by hash: .github/workflows/python-publish.yml:29","Info:   0 out of   2 GitHub-owned GitHubAction dependencies pinned","Info:   1 out of   1 third-party GitHubAction dependencies pinned","Info:   0 out of   2 pipCommand dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":-1,"reason":"internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration","details":null,"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 24 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-23T13:08:26.564Z","repository_id":41196687,"created_at":"2025-08-23T13:08:26.564Z","updated_at":"2025-08-23T13:08:26.564Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":274648319,"owners_count":25324299,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-11T02:00:13.660Z","response_time":74,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["environment","multi-agent","multi-agent-reinforcement-learning","multi-agent-systems","reinforcement-learning","robotics","simulation"],"created_at":"2024-07-31T09:00:27.666Z","updated_at":"2025-09-11T13:45:10.694Z","avatar_url":"https://github.com/semitable.png","language":"Python","funding_links":[],"categories":["Other reading material (blogs, websites, videos)"],"sub_categories":["Opponent Modelling"],"readme":"\u003cp align=\"center\"\u003e\n \u003cimg width=\"350px\" src=\"docs/img/rware.png\" align=\"center\" alt=\"Multi-Robot Warehouse (RWARE)\" /\u003e\n \u003cp align=\"center\"\u003eA multi-agent reinforcement learning environment\u003c/p\u003e\n\u003c/p\u003e\n\n[![Maintenance](https://img.shields.io/badge/Maintained%3F-yes-green.svg)](https://GitHub.com/Naereen/StrapDown.js/graphs/commit-activity)\n[![GitHub license](https://img.shields.io/github/license/Naereen/StrapDown.js.svg)](https://github.com/Naereen/StrapDown.js/blob/master/LICENSE)\n\n\u003e [!CAUTION]\n\u003e The RWARE environment was updated to support the new [Gymnasium](https://gymnasium.farama.org/) interface in replacement of the deprecated `gym=0.21` dependency (many thanks @LukasSchaefer). For backwards compatibility, please see [Gymnasium compatibility documentation](https://gymnasium.farama.org/content/gym_compatibility/) or use version v1.0.3 of the repository. The main changes to the interface are as follows:\n\u003e - `obss = env.reset()` --\u003e `obss, info = env.reset()`\n\u003e - `obss, rewards, dones, info = env.step(actions)` --\u003e `obss, rewards, done, truncated, info = env.step(actions)`\n\u003e - `done` is given as single boolean value instead of one `bool` value per agent\n\u003e - You can give the reset function a particular seed with `obss, info = env.reset(seed=42)` to initialise a particular episode.\n\n\n\u003ch1\u003eTable of Contents\u003c/h1\u003e\n\n- [Environment Description](#environment-description)\n  - [What does it look like?](#what-does-it-look-like)\n  - [Action Space](#action-space)\n  - [Observation Space](#observation-space)\n  - [Dynamics: Collisions](#dynamics-collisions)\n  - [Rewards](#rewards)\n- [Environment Parameters](#environment-parameters)\n  - [Naming Scheme](#naming-scheme)\n  - [Custom layout](#custom-layout)\n- [Installation](#installation)\n- [Getting Started](#getting-started)\n- [Human Play](#human-play)\n- [Please Cite](#please-cite)\n\n\n# Environment Description\n\nThe multi-robot warehouse (RWARE) environment simulates a warehouse with robots moving and delivering requested goods. The simulator is inspired by real-world applications, in which robots pick-up shelves and deliver them to a workstation. Humans access the content of a shelf, and then robots can return them to empty shelf locations.\n\nThe environment is configurable: it allows for different sizes (difficulty), number of agents, communication capabilities, and reward settings (cooperative/individual). Of course, the parameters used in each experiment must be clearly reported to allow for fair comparisons between algorithms.\n\n## What does it look like?\n\nBelow is an illustration of a small (10x20) warehouse with four trained agents. Agents have been trained with the SEAC algorithm [[2](#please-cite)]. This visualisation can be achieved using the `env.render()` function as described later.\n\n\u003cp align=\"center\"\u003e\n \u003cimg width=\"450px\" src=\"docs/img/rware.gif\" align=\"center\" alt=\"Multi-Robot Warehouse (RWARE) illustration\" /\u003e\n\u003c/p\u003e\n\n\n## Action Space\nIn this simulation, robots have the following discrete action space:\n\nA={ Turn Left, Turn Right, Forward, Load/Unload Shelf }\n\nThe first three actions allow each robot only to rotate and move forward. Loading/Unloading only works when an agent is beneath a shelf on one of the predesignated locations.\n\n## Observation Space\nThe observation of an agent is partially observable and consists of a 3x3 (configurable) square centred on the agent. Inside this limited grid, all entities are observable:\n- The location, the rotation and whether the agent is carrying a shelf.\n- The location and rotation of other robots.\n- Shelves and whether they are currently in the request queue.\n\n## Dynamics: Collisions\nThe dynamics of the environment are also of particular interest. Like a real, 3-dimensional warehouse, the robots can move beneath the shelves. Of course, when the robots are loaded, they must use the corridors, avoiding any standing shelves.\n\nAny collisions are resolved in a way that allows for maximum mobility. When two or more agents attempt to move to the same location, we prioritise the one that also blocks others. Otherwise, the selection is done arbitrarily. The visuals below demonstrate the resolution of various collisions.\n\n Example 1                 |   Example 2               | Example 3\n:-------------------------:|:-------------------------:|:-------------------------:\n![](docs/img/collision1.gif)  |  ![](docs/img/collision2.gif)  |  ![](docs/img/collision3.gif)\n\n## Rewards\nAt each time a set number of shelves R is requested. When a requested shelf is brought to a goal location, another shelf is uniformly sampled and added to the current requests. Agents are rewarded for successfully delivering a requested shelf to a goal location, with a reward of 1. A significant challenge in these environments is for agents to deliver requested shelves but also finding an empty location to return the previously delivered shelf. Having multiple steps between deliveries leads a very sparse reward signal.\n\n# Environment Parameters\n\nThe multi-robot warehouse task is parameterised by:\n\n- The size of the warehouse which is preset to either tiny (10x11), small (10x20), medium (16x20), or large (16x29).\n- The number of agents N.\n- The number of requested shelves R. By default R=N, but easy and hard variations of the environment use R = 2N and R = N/2, respectively.\n\nNote that R directly affects the difficulty of the environment. A small R, especially on a larger grid, dramatically affects the sparsity of the reward and thus exploration: randomly bringing the correct shelf becomes increasingly improbable.\n\n## Naming Scheme\n\nWhile RWARE allows fine tuning of multiple parameters when using the Warehouse class, it also registers multiple default environments with Gymnasium for simplicity.\n\nThe registered names look like `rware-tiny-2ag-v1` and might cryptic in the beginning, but it is not actually complicated. Every name always starts with rware. Next, the map size is appended as -tiny, -small, -medium, or -large. The number of robots in the map is selected as Xag with X being a number larger than one (e.g. -4ag for 4 agents). A difficulty modifier is optionally appended in the form of -easy or -hard, making requested shelves twice or half the number of agents (see section Rewards). Finally -v2 is the version as required for Gymnasium. In the time of writing all environments are v1, but we will increase it during changes or bugfixes.\n\nA few examples:\n```python\nimport gymnasium as gym\nimport rware\n\nenv = gym.make(\"rware-tiny-2ag-v2\")\nenv = gym.make(\"rware-small-4ag-v2\")\nenv = gym.make(\"rware-medium-6ag-hard-v2\")\n```\n\n\nOf course, more settings are available, but have to be changed during environment creation. For example:\n```python\nimport gymnasium as gym\nimport rware\n\nenv = gym.make(\"rware-tiny-2ag-v2\", sensor_range=3, request_queue_size=6)\n```\n\n## Custom layout\nYou can design a custom warehouse layout with the following:\n```python\nimport gymnasium as gym\nimport rware\n\nlayout = \"\"\"\n.......\n...x...\n..x.x..\n.x...x.\n..x.x..\n...x...\n.g...g.\n\"\"\"\nenv = gym.make(\"rware:rware-tiny-2ag-v2\", layout=layout)\n```\nThis will transform \"X\"s to shelves and \"G\"s to goal locations with a result like the one below:\n\u003cp align=\"center\"\u003e\n \u003cimg width=\"300px\" src=\"docs/img/rware-round.png\" align=\"center\" alt=\"Multi-Robot Warehouse (RWARE) illustration\" /\u003e\n\u003c/p\u003e\n\n\n\nA detailed explanation of all parameters can be found [here](https://github.com/semitable/robotic-warehouse/blob/4307b1fe3afa26de4ca4003fd04ab1319879832a/robotic_warehouse/warehouse.py#L132)\n\n# Installation\n\nAssuming you have Python3 (preferably on a virtual environment: venv or Anaconda) installed, you can use PyPI:\n```sh\npip install rware\n```\n\nIf you prefer to have the code available and be able to edit it, you can use Git to download and install it:\n```sh\ngit clone git@github.com:uoe-agents/robotic-warehouse.git\ncd robotic-warehouse\npip install -e .\n```\n\n# Getting Started\n\nRWARE was designed to be compatible with Open AI's Gym framework.\n\nCreating the environment is done exactly as one would create a Gym environment:\n\n```python\nimport gymnasium as gym\nimport rware\nenv = gym.make(\"rware-tiny-2ag-v2\")\n```\n\nYou can even bypass the `import` statement with Gym, and directly use:\n```python\nimport gymnasium as gym\nenv = gym.make(\"rware:rware-tiny-2ag-v2\")\n```\nThe `rware:` in the beginning of the environment name tells Gym to import the respective package.\n\nThe number of agents, the observation space, and the action space are accessed using:\n```python\nenv.n_agents  # 2\nenv.action_space  # Tuple(Discrete(5), Discrete(5))\nenv.observation_space  # Tuple(Box(XX,), Box(XX,))\n```\n\nThe returned spaces are from the Gym library (`gym.spaces`) Each element of the tuple corresponds to an agent, meaning that `len(env.action_space) == env.n_agents` and `len(env.observation_space) == env.n_agents` are always true.\n\nThe reset and step functions again are identical to Gym:\n\n```python\nobs = env.reset()  # a tuple of observations\n\nactions = env.action_space.sample()  # the action space can be sampled\nprint(actions)  # (1, 0)\nn_obs, reward, done, info = env.step(actions)\nprint(done)    # False\nprint(reward)  # [0.0, 0.0]\n```\nwhich leaves as to the only difference with Gym: the rewards and the done flag are lists, and each element corresponds to the respective agent.\n\nFinally, the environment can be rendered for debugging purposes:\n```python\nenv.render()\n```\nand should be closed before terminating:\n```python\nenv.close()\n```\n\n\n# Human Play\n\nWe also provide a simple script that allows you to play the environment as a human. This is useful for debugging and understanding the environment dynamics. To play the environment, run the following command:\n```sh\npython human_play.py --env \u003cenv_name\u003e\n```\nwhere `\u003cenv_name\u003e` is the name of the environment you want to play. For example, to play the tiny environment with 2 agents, run:\n```sh\npython human_play.py --env rware-tiny-2ag-v2\n```\n\nWithin the script, you can control a single agent at the time using the following keys:\n- Up Arrow keys: move current agent forward\n- Left/ Right Arrow keys: rotate current agent left/ right\n- P/ L: pickup/ drop shelf\n- SPACE: do nothing\n- TAB: change the current agent (rotates through all agents)\n- R: reset the environment and start a new episode\n- H: show help\n- D: display agent info (at every time step)\n- ESC: exit\n\n\n# Please Cite\nIf you use this environment, consider citing\n1. A comperative evaluation of MARL algorithms that includes this environment\n```\n@inproceedings{papoudakis2021benchmarking,\n   title={Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Tasks},\n   author={Georgios Papoudakis and Filippos Christianos and Lukas Schäfer and Stefano V. Albrecht},\n   booktitle = {Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks (NeurIPS)},\n   year={2021},\n   url = {http://arxiv.org/abs/2006.07869},\n   openreview = {https://openreview.net/forum?id=cIrPX-Sn5n},\n   code = {https://github.com/uoe-agents/epymarl},\n}\n```\n2. A method that achieves state-of-the-art performance in the robotic warehouse task\n```\n@inproceedings{christianos2020shared,\n author = {Christianos, Filippos and Sch\\\"{a}fer, Lukas and Albrecht, Stefano},\n booktitle = {Advances in Neural Information Processing Systems},\n editor = {H. Larochelle and M. Ranzato and R. Hadsell and M. F. Balcan and H. Lin},\n pages = {10707--10717},\n publisher = {Curran Associates, Inc.},\n title = {Shared Experience Actor-Critic for Multi-Agent Reinforcement Learning},\n url = {https://proceedings.neurips.cc/paper/2020/file/7967cc8e3ab559e68cc944c44b1cf3e8-Paper.pdf},\n volume = {33},\n year = {2020}\n}\n\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsemitable%2Frobotic-warehouse","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsemitable%2Frobotic-warehouse","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsemitable%2Frobotic-warehouse/lists"}