{"id":16403893,"url":"https://github.com/marcometer/endless-memory-gym","last_synced_at":"2025-04-30T17:43:08.874Z","repository":{"id":65761538,"uuid":"510644918","full_name":"MarcoMeter/endless-memory-gym","owner":"MarcoMeter","description":"Challenging Memory-based Deep Reinforcement Learning Agents","archived":false,"fork":false,"pushed_at":"2024-10-27T14:19:35.000Z","size":6597,"stargazers_count":97,"open_issues_count":1,"forks_count":3,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-04-19T19:13:43.505Z","etag":null,"topics":["benchmark","deep-reinforcement-learning","gru","gtrxl","gym-environment","imperfect-information","lstm","pomdp","recurrence","transformer","transformer-xl","trxl"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MarcoMeter.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-07-05T08:11:23.000Z","updated_at":"2025-04-12T10:55:15.000Z","dependencies_parsed_at":"2024-01-28T14:08:59.900Z","dependency_job_id":"0e5f7b8f-29a6-4cf0-be51-c9b32ad996b7","html_url":"https://github.com/MarcoMeter/endless-memory-gym","commit_stats":null,"previous_names":["marcometer/endless-memory-gym"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MarcoMeter%2Fendless-memory-gym","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MarcoMeter%2Fendless-memory-gym/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MarcoMeter%2Fendless-memory-gym/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MarcoMeter%2Fendless-memory-gym/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MarcoMeter","download_url":"https://codeload.github.com/MarcoMeter/endless-memory-gym/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251753353,"owners_count":21638289,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["benchmark","deep-reinforcement-learning","gru","gtrxl","gym-environment","imperfect-information","lstm","pomdp","recurrence","transformer","transformer-xl","trxl"],"created_at":"2024-10-11T05:50:41.606Z","updated_at":"2025-04-30T17:43:08.834Z","avatar_url":"https://github.com/MarcoMeter.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"[[Paper](https://arxiv.org/abs/2309.17207)] [[Installation](#installation)] [[Usage](#usage)] [[Mortar Mayhem](#mortar-mayhem)] [[Endless Mortar Mayhem](#endless-mortar-mayhem)] [[Mystery Path](#mystery-path)] [[Endless Mystery Path](#endless-mystery-path)] [[Searing Spotlights](#searing-spotlights)] [[Endless Searing Spotlights](#endless-searing-spotlights)] [[Training](#training)]\n\n# Memory Gym: Towards Endless Tasks to Benchmark Memory Capabilities of Agents\n\n\u003ctable align=\"center\"\u003e\n \u003ctr\u003e\n \u003ctd\u003e\u003c/td\u003e\n \u003ctd\u003eEndless Mortar Mayhem\u003c/td\u003e\n \u003ctd\u003eEndless Mystery Path\u003c/td\u003e\n \u003ctd\u003eEndless Searing Spotlights\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003eAgent Observation\u003c/td\u003e\n \u003ctd\u003e\u003cimg src=\"docs/assets/emm_0.gif\" width=180\u003e\u003c/td\u003e\n \u003ctd\u003e\u003cimg src=\"docs/assets/emp_0.gif\" width=180\u003e\u003c/td\u003e\n \u003ctd\u003e\u003cimg src=\"docs/assets/ess_0.gif\" width=180\u003e\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003eGround Truth\u003c/td\u003e\n \u003ctd\u003e\u003cimg src=\"docs/assets/emm_0_gt.gif\" width=180\u003e\u003c/td\u003e\n \u003ctd\u003e\u003cimg src=\"docs/assets/emp_0_gt.gif\" width=180\u003e\u003c/td\u003e\n \u003ctd\u003e\u003cimg src=\"docs/assets/ess_0_gt.gif\" width=180\u003e\u003c/td\u003e\n \u003c/tr\u003e\n\u003c/table\u003e\n\n\nMemory Gym features the environments **Mortar Mayhem**, **Mystery Path**, and **Searing Spotlights** that are inspired by some mini games of [Pummel Party](http://rebuiltgames.com/). These 2D environments benchmark the memory capabilities of agents.\nEspecially, these environments feature endless task variants. As the agent's policy improves, the task goes on. The cumulative memory game \"I packed my bag ...\" inspired this dynamic concept, which allows for examining levels of effectiveness instead of just sample efficiency.\nInteractive videos, based on selected agent behavios, can be found here: [https://marcometer.github.io/](https://marcometer.github.io/jmlr_2024.github.io/)\n\n## Citation\n\nPreprint Journal Paper (under review)\n```bibtex\n@misc{pleines2024memory,\n title={Memory Gym: Towards Endless Tasks to Benchmark Memory Capabilities of Agents}, \n author={Marco Pleines and Matthias Pallasch and Frank Zimmer and Mike Preuss},\n year={2024},\n eprint={2309.17207},\n archivePrefix={arXiv},\n primaryClass={cs.LG}\n}\n```\n\nICLR Paper\n```bibtex\n@inproceedings{pleines2023memory,\n title={Memory Gym: Partially Observable Challenges to Memory-Based Agents},\n author={Marco Pleines and Matthias Pallasch and Frank Zimmer and Mike Preuss},\n booktitle={International Conference on Learning Representations},\n year={2023},\n url={https://openreview.net/forum?id=jHc8dCx6DDr}\n}\n```\n\n## Installation\n\nMajor dependencies:\n- gymnasium==0.29.0\n- PyGame==2.4.0\n\n```console\nconda create -n memory-gym python=3.11 --yes\nconda activate memory-gym\npip install memory-gym\n```\n\nor\n\n```console\nconda create -n memory-gym python=3.11 --yes\nconda activate memory-gym\ngit clone https://github.com/MarcoMeter/drl-memory-gym.git\ncd drl-memory-gym\npip install -e .\n```\n\n\n## Usage\n[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1LjlUOEer8vjGrz0rLM8pP5UyeNCsURkY?usp=sharing)\n\nExecuting the environment using random actions:\n```python\nimport memory_gym\nimport gymnasium as gym\n\nenv = gym.make(\"Endless-SearingSpotlights-v0\")\n# env = gym.make(\"SearingSpotlights-v0\")\n# env = gym.make(\"Endless-MortarMayhem-v0\")\n# env = gym.make(\"MortarMayhem-v0\")\n# env = gym.make(\"MortarMayhem-Grid-v0\")\n# env = gym.make(\"MortarMayhemB-v0\")\n# env = gym.make(\"MortarMayhemB-Grid-v0\")\n# env = gym.make(\"Endless-MysteryPath-v0\")\n# env = gym.make(\"MysteryPath-v0\")\n# env = gym.make(\"MysteryPath-Grid-v0\")\n\n# Pass reset parameters to the environment\noptions = {\"agent_scale\": 0.25}\n\nobs, info = env.reset(seed=1, options=options)\ndone = False\nwhile not done:\n obs, reward, done, truncation, info = env.step(env.action_space.sample())\n\nprint(info)\n```\n\nManually play the environments using the console scripts (works only using an anaconda environment):\n```console\nmortar_mayhem\n# MMAct\nmortar_mayhem_b\n# MMGrid\nmortar_mayhem_grid\n# MMAct Grid\nmortar_mayhem_b_grid\nmystery_path\nmystery_path_grid\nsearing_spotlights\n\n# Endless Environments\nendless_mortar_mayhem\nendless_mystery_path\nendless_searing_spotlights\n```\n\nYou can also execute the python scripts directly, for example:\n```\npython ./memory_gym/mortar_mayhem.py\n```\n\nControls:\n- WASD or Arrow Keys to move or rotate\n- Page Up / Page Down to increment / decrement environment seeds\n\n## Mortar Mayhem\n\n\u003ctable align=\"center\"\u003e\n \u003ctr\u003e\n \u003ctd\u003eAgent Observation\u003c/td\u003e\n \u003ctd\u003eGround Truth\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\u003cimg src=\"docs/assets/mortar_mayhem_0.gif\" width=180\u003e\u003c/td\u003e\n \u003ctd\u003e\u003cimg src=\"docs/assets/mortar_mayhem_0_gt.gif\" width=180\u003e\u003c/td\u003e\n \u003c/tr\u003e\n\u003c/table\u003e\n\nMortar Mayhem challenges the agent with a sequence of commands that the agent has to memorize and execute in the right order. During the beginning of the episode, each command is visualized one by one. Mortar Mayhem can be reduced to solely executing commands. In this case, the command sequence is always available as vector observation (one-hot encoded) and, therefore, is not visualized.\n\nThe max length of an episode can be calculated as follows:\n\n```\nmax episode length = (command_show_duration + command_show_delay) * command_count + (explosion_delay + explosion_duration) * command_count - 2\n```\n\n![Mortar Mayhem Environment](/docs/assets/mm.jpg)\n\n### Reset Parameters\n\n| Parameter | Default | Description |\n|------------------------|--------:|---------------------------------------------------------------------------------------------------------------------------------------------------|\n| agent_scale | 0.25 | The dimensions of the agent. |\n| agent_speed | 3.0 | The speed of the agent. |\n| arena_size | 5 | The grid dimension of the arena (min: 2, max: 6) |\n| allowed_commands | 9 | Available commands: right, down, left, up, stay, right down, right up, left down, left up. If set to five, the first five commands are available. |\n| command_count | [10] | The number of commands that are asked to be executed by the agent. This is a list that the environment samples from. |\n| command_show_duration | [3] | The number of steps that one command is shown. This is a list that the environment samples from. |\n| command_show_delay | [1] | The number of steps between showing one command. This is a list that the environment samples from. |\n| explosion_duration | [6] | The number of steps that an agent has to stay on the commanded tile. This is a list that the environment samples form. |\n| explosion_delay | [18] | The entire duration in steps that the agent has to execute the current command. This is a list that the environments samples from. |\n| visual_feedback | True | Whether to turn off the visualization of the feedback. Upon command evaluation, the wrong tiles are rendered red. |\n| reward_command_failure | 0.0 | What reward to signal upon failing at the current command. |\n| reward_command_success | 0.1 | What reward to signal upon succeeding at the current command. |\n| reward_episode_success | 0.0 | What reward to signal if the entire command sequence is successfully solved by the agent. |\n\n## Endless Mortar Mayhem\n\nTo extend the core concept of Mortar Mayhem to Endless Mortar Mayhem, we introduce an ever-growing command sequence. The phases of displaying and executing commands are alternated. Only one command is shown before command execution, while the agent must execute all previously displayed commands in the underlying episode. To accommodate a potentially infinite command sequence, the arena undergoes a screen wrap, behaving like a torus.\n\n### Reset Parameters\n\n| Parameter | Default | Description |\n|------------------------|--------:|---------------------------------------------------------------------------------------------------------------------------------------------------|\n| max_steps | -1 | Maximum number of steps that an episode may last. If less than 1, the episode length is not limited by this reset parameter. |\n| agent_scale | 0.25 | The dimensions of the agent. |\n| agent_speed | 3.0 | The speed of the agent. |\n| allowed_commands | 9 | Available commands: right, down, left, up, stay, right down, right up, left down, left up. If set to five, the first five commands are available. |\n| initial_command_count | 1 | Specifies the number of commands that are initially shown. |\n| command_show_duration | [3] | The number of steps that one command is shown. This is a list that the environment samples from. |\n| command_show_delay | [1] | The number of steps between showing one command. This is a list that the environment samples from. |\n| explosion_duration | [6] | The number of steps that an agent has to stay on the commanded tile. This is a list that the environment samples form. |\n| explosion_delay | [18] | The entire duration in steps that the agent has to execute the current command. This is a list that the environments samples from. |\n| visual_feedback | True | Whether to turn off the visualization of the feedback. Upon command evaluation, the wrong tiles are rendered red. |\n| reward_command_failure | 0.0 | What reward to signal upon failing at the current command. |\n| reward_command_success | 0.1 | What reward to signal upon succeeding at the current command (dense reward setting). |\n| reward_new_command_success | 0.0 | What reward to signal upon completing all commands of the current list of commands (sparse reward setting). |\n\n## Mystery Path\n\n\u003ctable align=\"center\"\u003e\n \u003ctr\u003e\n \u003ctd\u003eAgent Observation\u003c/td\u003e\n \u003ctd\u003eGround Truth\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\u003cimg src=\"docs/assets/mystery_path_0.gif\" width=180\u003e\u003c/td\u003e\n \u003ctd\u003e\u003cimg src=\"docs/assets/mystery_path_0_gt.gif\" width=180\u003e\u003c/td\u003e\n \u003c/tr\u003e\n\u003c/table\u003e\n\nMystery Path procedurally generates an invisible path for the agent to cross from the origin to the goal. Per default, only the origin of the path is visible. Upon falling off the path, the agent has to restart from the origin. Note that the episode is not terminated by falling off. Hence, the agent has to memorize where it fell off and where it did not.\n\n![Mystery Path Environment](/docs/assets/mp.jpg)\n\n### Reset Parameters\n\n| Parameter | Default | Explanation |\n|------------------------|-------------:|-----------------------------------------------------------------------------------------------------------------------------|\n| max_steps | 512 | The maximum number of steps for the agent to play one episode. |\n| agent_scale | 0.25 | The dimensions of the agent. |\n| agent_speed | 3.0 | The speed of the agent. |\n| cardinal_origin_choice | [0, 1, 2, 3] | Allowed cardinal directions for the path generation to place the origin. This is a list that the environment samples from. |\n| show_origin | False | Whether to hide or show the origin tile of the generated path. |\n| show_goal | False | Whether to hide or show the goal tile of the generated path. |\n| visual_feedback | True | Whether to visualize that the agent is off the path. A red cross is rendered on top of the agent. |\n| reward_goal | 1.0 | What reward to signal when reaching the goal tile. |\n| reward_fall_off | 0.0 | What reward to signal when falling off. |\n| reward_path_progress | 0.0 | What reward to signal when making progress on the path. This is only signaled for reaching another tile for the first time. |\n| reward_step | 0.0 | What reward to signal for each step. |\n\n## Endless Mystery Path\n\n\u003cp align=center\u003e\n\u003cimg src=\"docs/assets/emp_path.png\" width=420\u003e\n\u003c/p\u003e\n\nIn Endless Mystery Path, a never-ending path is generated by exploiting the path generation of Mystery Path, which concatenates path segments. The terminal conditions of an episode need to be varied to accommodate the design of short episodes without making progress. The episode terminates if the agent fails to make progress within a few steps. Termination also occurs if the agent falls off before reaching its farthest progress, and the agent cannot fall off at the same location twice.\n\n### Reset Parameters\n\n| Parameter | Default | Explanation |\n|------------------------|-------------:|-----------------------------------------------------------------------------------------------------------------------------|\n| max_steps | -1 | The maximum number of steps for the agent to play one episode. If smaller than 1, the episode is not affected by this reset parameter. |\n| agent_scale | 0.25 | The dimensions of the agent. |\n| agent_speed | 3.0 | The speed of the agent. |\n| show_origin | False | Whether to hide or show the origin tile of the generated path. |\n| show_past_path | True | Whether to hide or show the path behing the agent. |\n| show_background | False | Whether to hide or show a tiled background. |\n| show_stamina | False | Whether to hide or show a stamina bar indicating the remaining time to make progress on the path. |\n| visual_feedback | True | Whether to visualize that the agent is off the path. A red cross is rendered on top of the agent. |\n| camera_offset_scale | 5.0 | Offset of the camera's X position. Decreasing this value will hide more of the path behind the agennt. |\n| stamina_level | 20 | Number of steps that the agent has time to touch on the next path tile leading to progress. |\n| reward_fall_off | 0.0 | What reward to signal when falling off. |\n| reward_path_progress | 0.1 | Reward signaled whenever the agent reaches a new tile that it has not visited before (sparse reward setting). |\n| reward_path_progress_dense | 0.0 | Reward signaled whenever the agent reaches the next tile (dense reward setting). |\n| reward_step | 0.0 | What reward to signal for each step. |\n\n## Searing Spotlights\n\n\u003ctable align=\"center\"\u003e\n \u003ctr\u003e\n \u003ctd\u003eAgent Observation\u003c/td\u003e\n \u003ctd\u003eGround Truth\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\u003cimg src=\"docs/assets/searing_spotlights_0.gif\" width=180\u003e\u003c/td\u003e\n \u003ctd\u003e\u003cimg src=\"docs/assets/searing_spotlights_0_gt.gif\" width=180\u003e\u003c/td\u003e\n \u003c/tr\u003e\n\u003c/table\u003e\n\nSearing Spotlights is a pitch black surrounding to the agent. The environment is initially fully observable but the light is dimmed untill off during the first few frames. Only randomly moving spotlights unveil information on the environment's ground truth, while posing a threat to the agent. If spotted by spotlight, the agent looses health points. While the agent must avoid closing in spotlights, it further has to collect coins. After collecting all coins, the agent has to take the environment's exit.\n\n![Searing Spotlights Environment](/docs/assets/spots.jpg)\n\n### Reset Parameters\n\n| Parameter | Default | Explanation |\n|--------------------------|--------:|-----------------------------------------------------------------------------------------------------------------|\n| max_steps | -1 | The maximum number of steps for the agent to play one episode. If smaller than 1, the episode is not affected by this reset parameter. |\n| steps_per_coin | 160 | Number of steps that the agent has to collect a newly spawned coin. |\n| agent_scale | 0.25 | The dimensions of the agent. |\n| agent_speed | 3.0 | The speed of the agent. |\n| agent_health | 5 | The initial health points of the agent. |\n| agent_visible | False | Whether to make the agent permanently visible. |\n| sample_agent_position | True | Whether to hide or show the goal tile of the generated path. |\n| num_coins | [1] | The number of coins that are spawned. This is a list that the environment samples from. |\n| coin_scale | 0.375 | The scale of the coins. |\n| coins_visible | False | Whether to make the coins permanently visible. |\n| use_exit | True | Whether to spawn and use the exit task. The exit is accessible by the agent after collecting all coins. |\n| exit_scale | 0.0 | The scale of the exit. |\n| exit_visible | False | Whether to make the exit permanently visible. |\n| initial_spawns | 3 | The number of spotlights that are initially spawned. |\n| spawn_interval | 50 | Number of steps to spawn a new spotlight. |\n| spot_min_radius | 7.5 | The minimum radius of the spotlights. The radius is sampled from the range min to max. |\n| spot_max_radius | 13.75 | The maximum radius of the spotlights. The radius is sampled from the range min to max. |\n| spot_min_speed | 0.0025 | The minimum speed of the spotlights. The speed is sampled from the range min to max. |\n| spot_max_speed | 0.0075 | The maximum speed of the spotlights. The speed is sampled from the range min to max. |\n| spot_damage | 1.0 | Damage per step while the agent is spotted by one spotlight. |\n| light_dim_off_duration | 6 | The number of steps to dim off the global light. |\n| light_threshold | 255 | The threshold for dimming the global light. A value of 255 indicates that the light will dimmed of completely. |\n| visual_feedback | True | Whether to render the tiled background red if the agent is spotted. |\n| black_background | False | Whether to render the environments background black, while the spotlights are rendered as white circumferences. |\n| hide_chessboard | False | Whether to hide the chessboard background. This renders the background of the environment white. |\n| show_last_action | True | Whether to encode and render the previouss action to the visual observation. |\n| show_last_positive_reward | True | Whether to render if the agent received a positive reward on the previous step. |\n| reward_inside_spotlight | 0.0 | What reward to signal for each step while being inside a spotlight. |\n| reward_outside_spotlight | 0.0 | What reward to signal for each step while being outside of a spotlight. |\n| reward_death | 0.0 | What reward to signal upon losing all health points. |\n| reward_coin | 0.25 | What reward to signal upon collecting one coin. |\n\n## Endless Searing Spotlights\n\nEndless Searing Spotlights solely revolves around a coin collection task, with no consideration of an exit task leading to episode termination. Upon collecting the only coin present, a new one is immediately spawned. The agent operates under a limited time budget to collect the newly spawned coin.\n\n### Reset Parameters\n\n| Parameter | Default | Explanation |\n|--------------------------|--------:|-----------------------------------------------------------------------------------------------------------------|\n| max_steps | -1 | The maximum number of steps for the agent to play one episode. |\n| agent_scale | 0.25 | The dimensions of the agent. |\n| agent_speed | 3.0 | The speed of the agent. |\n| agent_health | 10 | The initial health points of the agent. |\n| agent_visible | False | Whether to make the agent permanently visible. |\n| sample_agent_position | True | Whether to hide or show the goal tile of the generated path. |\n| coin_enabled | True | Whether the coin collection task is enabled or disabled |\n| coin_show_duration | 6 | How many steps to make the coin visible to the agent unill its hidden behind the dark. |\n| coin_scale | 0.375 | The scale of the coins. |\n| coins_visible | False | Whether to make the coins permanently visible. |\n| steps_per_coin | 160 | Time budget to collect a single coin. |\n| initial_spawns | 3 | The number of spotlights that are initially spawned. |\n| spawn_interval | 50 | The number of steps until the next spotlight is spawned. |\n| spot_min_radius | 7.5 | The minimum radius of the spotlights. The radius is sampled from the range min to max. |\n| spot_max_radius | 13.75 | The maximum radius of the spotlights. The radius is sampled from the range min to max. |\n| spot_min_speed | 0.0025 | The minimum speed of the spotlights. The speed is sampled from the range min to max. |\n| spot_max_speed | 0.0075 | The maximum speed of the spotlights. The speed is sampled from the range min to max. |\n| spot_damage | 1.0 | Damage per step while the agent is spotted by one spotlight. |\n| light_dim_off_duration | 6 | The number of steps to dim off the global light. |\n| light_threshold | 255 | The threshold for dimming the global light. A value of 255 indicates that the light will dimmed of completely. |\n| visual_feedback | True | Whether to render the tiled background red if the agent is spotted. |\n| black_background | False | Whether to render the environments background black, while the spotlights are rendered as white circumferences. |\n| hide_chessboard | False | Whether to hide the chessboard background. This renders the background of the environment white. |\n| show_last_action | True | Whether to encode and render the previouss action to the visual observation. |\n| show_last_positive_reward | True | Whether to render if the agent received a positive reward on the previous step. |\n| reward_inside_spotlight | 0.0 | What reward to signal for each step while being inside a spotlight. |\n| reward_outside_spotlight | 0.0 | What reward to signal for each step while being outside of a spotlight. |\n| reward_death | 0.0 | What reward to signal upon losing all health points. |\n| reward_coin | 0.25 | What reward to signal upon collecting one coin. |\n\n## Training\n\nBaseline results are avaible via these repositories.\n\n[Recurrence + PPO](https://github.com/MarcoMeter/recurrent-ppo-truncated-bptt)\n\n[TransformerXL + PPO](https://github.com/MarcoMeter/episodic-transformer-memory-ppo)\n\n## Changelog\n\nv1.0.0\n\nImprovements\n- All environment concepts are extrapolated to endless episodes!\n - Endless Mortar Mayhem\n - Endless Mystery Path\n - Endless Searing Spotlights\n- Improved simulation speed by using already rotated sprites and not rotating the character's surface every frame\n- Mystery Path: A* obstacle walls are also placed now on the environments boundary to mitigate trivial paths\n- All endless environments feature a ground truth space. As specified by this space ground truth information is added to the info dictionary\n- Searing Spotlights may also visualize whether a positive reward was signaled on the previous frame\n\nBreaking Changes\n- Refactored the info key \"exit_success\" in Searing Spotlights to \"success\"\n\nBug Fixes\n- Fixed the speed of character controller, because moving downwards was slower than moving upwards due to float truncation\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarcometer%2Fendless-memory-gym","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmarcometer%2Fendless-memory-gym","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarcometer%2Fendless-memory-gym/lists"}