{"id":25878856,"url":"https://github.com/epignatelli/helx","last_synced_at":"2025-08-08T21:19:30.271Z","repository":{"id":64691894,"uuid":"323293223","full_name":"epignatelli/helx","owner":"epignatelli","description":"Interoperating between (Deep) Reiforcement Learning libraries","archived":false,"fork":false,"pushed_at":"2024-03-06T15:29:43.000Z","size":624,"stargazers_count":10,"open_issues_count":10,"forks_count":6,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-21T06:35:34.640Z","etag":null,"topics":["deep-learning","flax","jax","reinforcement-learning","reinforcement-learning-algorithms","reinforcement-learning-environments","rl","rl-environments"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/epignatelli.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":"AUTHORS","dei":null}},"created_at":"2020-12-21T09:48:08.000Z","updated_at":"2024-02-09T16:24:41.000Z","dependencies_parsed_at":"2023-12-19T15:58:29.317Z","dependency_job_id":"aec181a7-b266-4a71-993f-e356d9b0a144","html_url":"https://github.com/epignatelli/helx","commit_stats":{"total_commits":587,"total_committers":3,"mean_commits":"195.66666666666666","dds":0.04258943781942082,"last_synced_commit":"135be1f59d5401e4928e8e255d1ecad37b5e9bd4"},"previous_names":["epignatelli/helx-base"],"tags_count":27,"template":false,"template_full_name":"epignatelli/python-library","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epignatelli%2Fhelx","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epignatelli%2Fhelx/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epignatelli%2Fhelx/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epignatelli%2Fhelx/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/epignatelli","download_url":"https://codeload.github.com/epignatelli/helx/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241509593,"owners_count":19974070,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","flax","jax","reinforcement-learning","reinforcement-learning-algorithms","reinforcement-learning-environments","rl","rl-environments"],"created_at":"2025-03-02T12:39:54.673Z","updated_at":"2025-03-02T12:39:55.234Z","avatar_url":"https://github.com/epignatelli.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# HELX: The RL experiments framework\n\n[![Project Status: WIP – Initial development is in progress, but there has not yet been a stable, usable release suitable for the public.](https://www.repostatus.org/badges/latest/wip.svg)](https://www.repostatus.org/#wip)\n[![CI](https://github.com/epignatelli/helx/actions/workflows/CI.yml/badge.svg)](https://github.com/epignatelli/helx/actions/workflows/CI.yml)\n[![CD](https://github.com/epignatelli/helx/actions/workflows/CD.yml/badge.svg)](https://github.com/epignatelli/helx/actions/workflows/CD.yml)\n![GitHub release (latest by date)](https://img.shields.io/github/v/release/epignatelli/helx?color=%23216477\u0026label=Release)\n\n**[Quickstart](#what-is-helx)** | **[Why helx?](#why-helx)** | **[Installation](#installation)** | **[Examples](#examples)** | **[Cite](#cite)**\n\n## What is HELX?\n\nHELX is a JAX-based ecosystem that provides a standardised framework to run Reinforcement Learning experiments.\nWith HELX you easily can:\n- Use the `helx.envs` namespace to use the most common RL environments (gym, gymnax, dm_env, atari, ...)\n- Use the `helx.agents` namespace to use the most common RL agents (DQN, PPO, SAC, ...)\n- Use the `helx.experiment` namespace to run experiments on your local machine, on a cluster, or on the cloud\n- Use the `helx.base` namespace to access the most common RL data structures and functions (e.g., a Ring buffer)\n\n## Why HELX?\nHELX is designed to be easy to use, easy to extend, and easy to read.\n- No 2000 lines of code files\n- No multiple inheritance hierarchies where behaviours get lost in the middle\n- No complex abstractions that hide the underlying code\n\nEach namespace provides a single, standardised interface to all agents, environments and experiment runners.\n\n\n## Installation\n\n- ### Stable\nInstall the stable version of `helx` and its dependencies with:\n```bash\npip install helx\n```\n\n- ### Nightly\nOr, if you prefer to install the latest version from source:\n```bash\npip install git+https://github.com/epignatelli/helx\n```\n\n\n\n## Examples\n\nA typical use case is to design an agent, and toy-test it on `catch` before evaluating it on more complex environments, such as atari, procgen or mujoco.\n\n```python\nimport bsuite\nimport gym\n\nimport helx.environment\nimport helx.experiment\nimport helx.agents\n\n# create the enviornment in you favourite way\nenv = bsuite.load_from_id(\"catch/0\")\n# convert it to an helx environment\nenv = helx.environment.to_helx(env)\n# create the agent\nhparams = helx.agents.Hparams(env.obs_space(), env.action_space())\nagent = helx.agents.Random(hparams)\n\n# run the experiment\nhelx.experiment.run(env, agent, episodes=100)\n```\n\n\nSwitching to a different environment is as simple as changing the `env` variable.\n\n\n```diff\nimport bsuite\nimport gym\n\nimport helx.environment\nimport helx.experiment\nimport helx.agents\n\n# create the enviornment in you favourite way\n-env = bsuite.load_from_id(\"catch/0\")\n+env = gym.make(\"procgen:procgen-coinrun-v0\")\n# convert it to an helx environment\nenv = helx.environment.to_helx(env)\n# create the agent\nhparams = helx.agents.Hparams(env.obs_space(), env.action_space())\nagent = helx.agents.Random(hparams)\n\n# run the experiment\nhelx.experiment.run(env, agent, episodes=100)\n```\n\n\n\n## Joining development\n\n### Adding a new agent (`helx.agents.Agent`)\n\nAn `helx` agent interface is designed as the minimal set of functions necessary to *(i)* interact with an environment and *(ii)* reinforcement learn.\n\n```python\nfrom typing import Any\nfrom jax import Array\n\nfrom helx.base import Timestep\nfrom helx.agents import Agent\n\n\nclass NewAgent(helx.agents.Agent):\n    \"\"\"A new RL agent.\"\"\"\n    def create(self, hparams: Any) -\u003e None:\n        \"\"\"Initialises the agent's internal state (knowledge), such as a table,\n        or some function parameters, e.g., the parameters of a neural network.\"\"\"\n        # implement me\n\n    def init(self, key: KeyArray, timestep: Timestep) -\u003e None:\n        \"\"\"Initialises the agent's internal state (knowledge), such as a table,\n        or some function parameters, e.g., the parameters of a neural network.\"\"\"\n        # implement me\n\n    def sample_action(\n        self, agent_state: AgentState, obs: Array, *, key: KeyArray, eval: bool = False\n    ):\n        \"\"\"Applies the agent's policy to the current timestep to sample an action.\"\"\"\n        # implement me\n\n    def update(self, timestep: Timestep) -\u003e Any:\n        \"\"\"Updates the agent's internal state (knowledge), such as a table,\n        or some function parameters, e.g., the parameters of a neural network.\"\"\"\n        # implement me\n```\n\n\n## Adding a new environment library (`helx.environment.Environment`)\n\nTo add a new library requires three steps:\n1. Implement the `helx.environment.Environment` interface for the new library.\nSee the [dm_env](helx/environment/dm_env.py) implementation for an example.\n1. Implement serialisation (to `helx`) of the following objects:\n    - `helx.environment.Timestep`\n    - `helx.spaces.Discrete`\n    - `helx.spaces.Continuous`\n2. Add the new library to the [`helx.environment.to_helx`](helx/environment/interop.py#L16) function to tell `helx` about the new protocol.\n\n---\n## Cite\nIf you use `helx` please consider citing it as:\n\n```bibtex\n@misc{helx,\n  author = {Pignatelli, Eduardo},\n  title = {Helx: Interoperating between Reinforcement Learning Experimental Protocols},\n  year = {2021},\n  publisher = {GitHub},\n  journal = {GitHub repository},\n  howpublished = {\\url{https://github.com/epignatelli/helx}}\n  }\n```\n\n\n## A note on maintainance\nThis repository was born as the recipient of personal research code that was developed over the years.\nIts maintainance is limited by the time and the resources of a research project resourced with a single person.\nEven if I would like to automate many actions, I do not have the time to maintain the whole body of automation that a well maintained package deserves.\nThis is the reason of the WIP badge, which I do not plan to remove soon.\nMaintainance will prioritise the code functionality over documentation and automation.\n\nAny help is very welcome.\nA quick guide to interacting with this repository:\n- If you find a bug, please open an issue, and I will fix it as soon as I can.\n- If you want to request a new feature, please open an issue, and I will consider it as soon as I can.\n- If you want to contribute yourself, please open an issue first, let's discuss objective, plan a proposal, and open a pull request to act on it.\n\nIf you would like to be involved further in the development of this repository, please contact me directly at: `edu dot pignatelli at gmail dot com`.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fepignatelli%2Fhelx","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fepignatelli%2Fhelx","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fepignatelli%2Fhelx/lists"}