{"id":50510321,"url":"https://github.com/huggingface/OpenEnv","last_synced_at":"2026-06-05T14:00:54.088Z","repository":{"id":318948848,"uuid":"1067949357","full_name":"huggingface/OpenEnv","owner":"huggingface","description":"An interface library for RL post training with environments. ","archived":false,"fork":false,"pushed_at":"2026-05-30T19:27:51.000Z","size":76998,"stargazers_count":1902,"open_issues_count":82,"forks_count":358,"subscribers_count":15,"default_branch":"main","last_synced_at":"2026-05-30T21:07:34.830Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"http://huggingface.github.io/OpenEnv/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/huggingface.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-10-01T16:13:38.000Z","updated_at":"2026-05-30T01:00:56.000Z","dependencies_parsed_at":"2025-10-31T02:14:23.320Z","dependency_job_id":null,"html_url":"https://github.com/huggingface/OpenEnv","commit_stats":null,"previous_names":["meta-pytorch/openenv","huggingface/openenv"],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/huggingface/OpenEnv","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/huggingface%2FOpenEnv","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/huggingface%2FOpenEnv/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/huggingface%2FOpenEnv/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/huggingface%2FOpenEnv/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/huggingface","download_url":"https://codeload.github.com/huggingface/OpenEnv/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/huggingface%2FOpenEnv/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33944671,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-05T02:00:06.157Z","response_time":120,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-06-02T20:00:22.174Z","updated_at":"2026-06-05T14:00:54.081Z","avatar_url":"https://github.com/huggingface.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# \u003cimg width=\"35\" height=\"35\" alt=\"image\" src=\"https://github.com/user-attachments/assets/2700a971-e5d6-4036-b03f-2f89c9791609\" /\u003e OpenEnv: Agentic Execution Environments\n\nAn e2e framework for creating, deploying and using isolated execution environments for agentic RL training, built using Gymnasium style simple APIs.\n\n\u003cp align=\"center\"\u003e\n    \u003ca href=\"https://pypi.org/project/openenv/\"\u003e\u003cimg alt=\"PyPI\" src=\"https://img.shields.io/pypi/v/openenv?color=blue\"/\u003e\u003c/a\u003e\n    \u003ca href=\"https://github.com/huggingface/OpenEnv/blob/main/LICENSE\"\u003e\u003cimg alt=\"License\" src=\"https://img.shields.io/badge/License-BSD%203--Clause-blue.svg\"/\u003e\u003c/a\u003e\n    \u003ca href=\"https://huggingface.co/docs/openenv\"\u003e\u003cimg alt=\"Docs\" src=\"https://img.shields.io/badge/Docs-Explore-blue?logo=readthedocs\u0026logoColor=white\"/\u003e\u003c/a\u003e\n    \u003ca href=\"https://huggingface.co/openenv\"\u003e\u003cimg alt=\"Hugging Face\" src=\"https://img.shields.io/badge/🤗%20Hugging%20Face-OpenEnv-yellow\"/\u003e\u003c/a\u003e\n    \u003ca href=\"https://discord.gg/YsTYBh6PD9\"\u003e\u003cimg alt=\"Discord\" src=\"https://img.shields.io/badge/Discord-OpenEnv-7289da?style=flat\u0026logo=discord\u0026logoColor=white\"/\u003e\u003c/a\u003e\n    \u003ca href=\"https://colab.research.google.com/github/huggingface/OpenEnv/blob/main/examples/OpenEnv_Tutorial.ipynb\"\u003e\u003cimg alt=\"Open In Colab\" src=\"https://colab.research.google.com/assets/colab-badge.svg\"/\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n---\n\n**Featured Example:** Train LLMs to play BlackJack using [torchforge](https://meta-pytorch.org/torchforge/) (PyTorch's agentic RL framework): [`examples/grpo_blackjack/`](examples/grpo_blackjack/)\n\n**Zero to Hero Tutorial:** End to end tutorial from our [GPU Mode](tutorial/README.md) lecture and other hackathons.\n\n## Quick Start\n\nInstall the OpenEnv package:\n\n```bash\npip install openenv\n```\n\nInstall an environment client (e.g., Echo):\n\n```bash\npip install git+https://huggingface.co/spaces/openenv/echo_env\n```\n\nThen use the environment:\n\n```python\nimport asyncio\nfrom echo_env import CallToolAction, EchoEnv\n\nasync def main():\n    # Connect to a running Space (async context manager)\n    async with EchoEnv(base_url=\"https://openenv-echo-env.hf.space\") as client:\n        # Reset the environment\n        result = await client.reset()\n        print(result.observation.echoed_message)  # \"Echo environment ready!\"\n\n        # Send messages\n        result = await client.step(\n            CallToolAction(\n                tool_name=\"echo_message\",\n                arguments={\"message\": \"Hello, World!\"},\n            )\n        )\n        print(result.observation.result)  # \"Hello, World!\"\n        print(result.reward)\n\nasyncio.run(main())\n```\n\n**Synchronous usage** is also supported via the `.sync()` wrapper:\n\n```python\nfrom echo_env import CallToolAction, EchoEnv\n\n# Use .sync() for synchronous context manager\nwith EchoEnv(base_url=\"https://openenv-echo-env.hf.space\").sync() as client:\n    result = client.reset()\n    result = client.step(\n        CallToolAction(\n            tool_name=\"echo_message\",\n            arguments={\"message\": \"Hello, World!\"},\n        )\n    )\n    print(result.observation.result)\n```\n\nFor a detailed quick start, check out the [docs page](https://huggingface.co/docs/openenv/getting-started).\n\n## Overview\n\nOpenEnv provides a standard for interacting with agentic execution environments via simple Gymnasium style APIs - `step()`, `reset()`, `state()`. Users of agentic execution environments can interact with the environment during RL training loops using these simple APIs.\n\nIn addition to making it easier for researchers and RL framework writers, we also provide tools for environment creators making it easier for them to create richer environments and make them available over familiar protocols like HTTP and packaged using canonical technologies like docker. Environment creators can use the OpenEnv framework to create environments that are isolated, secure, and easy to deploy and use.\n\nThe OpenEnv CLI (`openenv`) provides commands to initialize new environments and deploy them to Hugging Face Spaces.\n\n\u003e ⚠️ **Early Development Warning** OpenEnv is currently in an experimental\n\u003e stage. You should expect bugs, incomplete features, and APIs that may change\n\u003e in future versions. The project welcomes bugfixes, but significant changes\n\u003e should be discussed before implementation so the technical committee and\n\u003e community can coordinate scope, compatibility, and release timing. It's\n\u003e recommended that you signal your intention to contribute in the issue tracker,\n\u003e either by filing a new issue or by claiming an existing one.\n\n### RFCs\n\nBelow is a list of active and historical RFCs for OpenEnv. RFCs are proposals for major changes or features. Please review and contribute!\n\n- [RFC 001: Baseline API and Interface Specifications](https://github.com/huggingface/OpenEnv/pull/26)\n- [RFC 002: Discoverability of environment tools by agents](https://github.com/huggingface/OpenEnv/pull/32)\n- [RFC 003: Add MCP (Model Context Protocol) support](https://github.com/huggingface/OpenEnv/pull/224)\n- [RFC 004: Add delayed rewards support for trajectory-based scoring](https://github.com/huggingface/OpenEnv/pull/337)\n- [RFC 005: Agentic Harness Integration](https://github.com/huggingface/OpenEnv/pull/387)\n\n## Architecture\n\n### Component Overview\n\n```\n┌─────────────────────────────────────────────────────────┐\n│                    Client Application                   │\n│  ┌────────────────┐              ┌──────────────────┐   │\n│  │  EchoEnv       │              │  CodingEnv       │   │\n│  │  (EnvClient)   │              │   (EnvClient)    │   │\n│  └────────┬───────┘              └────────┬─────────┘   │\n└───────────┼───────────────────────────────┼─────────────┘\n            │ WebSocket                     │ WebSocket\n            │ (reset, step, state)          │\n┌───────────▼───────────────────────────────▼─────────────┐\n│              Docker Containers (Isolated)               │\n│  ┌──────────────────────┐    ┌──────────────────────┐   │\n│  │ FastAPI Server       │    │ FastAPI Server       │   │\n│  │   EchoEnvironment    │    │ PythonCodeActEnv     │   │\n│  │ (Environment base)   │    │ (Environment base)   │   │\n│  └──────────────────────┘    └──────────────────────┘   │\n└─────────────────────────────────────────────────────────┘\n```\n\n### Core Components\n\n#### 1. Web Interface\n\nOpenEnv includes a built-in web interface for interactive environment exploration and debugging. The web interface provides:\n\n- **Two-Pane Layout**: HumanAgent interaction on the left, state observation on the right\n- **Real-time Updates**: WebSocket-based live updates without page refresh\n- **Dynamic Forms**: Automatically generated action forms based on environment Action types\n- **Action History**: Complete log of all actions taken and their results\n\nThe web interface is **conditionally enabled** based on environment variables:\n\n- **Local Development**: Disabled by default for lightweight development\n- **Manual Override**: Enable with `ENABLE_WEB_INTERFACE=true`\n\nTo use the web interface:\n\n```python\nfrom openenv.core.env_server import create_web_interface_app\nfrom your_env.models import YourAction, YourObservation\nfrom your_env.server.your_environment import YourEnvironment\n\nenv = YourEnvironment()\napp = create_web_interface_app(env, YourAction, YourObservation)\n```\n\nWhen enabled, open `http://localhost:8000/web` in your browser to interact with the environment.\n\n#### 2. Environment (Server-Side)\nBase class for implementing environment logic:\n- **`reset()`**: Initialize a new episode, returns initial `Observation`\n- **`step(action)`**: Execute an `Action`, returns resulting `Observation`\n- **`state()`**: Access episode metadata (`State` with episode_id, step_count, etc.)\n\n#### 3. EnvClient (Client-Side)\nBase class for environment communication:\n- **Async by default**: Use `async with` and `await` for all operations\n- **Sync wrapper**: Call `.sync()` to get a `SyncEnvClient` for synchronous usage\n- Handles WebSocket connections to environment server\n- Contains a utility to spin up a docker container locally for the corresponding environment\n- Type-safe action/observation parsing\n\n#### 4. Container Providers\nManage container deployment:\n- `LocalDockerProvider`: Run containers on local Docker daemon\n- `DockerSwarmProvider`: Deploy to Docker Swarm clusters\n- `KubernetesProvider`: Deploy to Kubernetes clusters\n- `UVProvider`, `DaytonaProvider`: Additional runtime providers\n\n#### 5. Models\nType-safe data structures:\n- `Action`: Base class for environment actions\n- `Observation`: Base class for environment observations\n- `State`: Episode state tracking\n- `StepResult`: Combines observation, reward, done flag\n\n## Project Structure\n\n### For Environment Creators\n\nUse the CLI to quickly scaffold a new environment:\n\n```bash\nopenenv init my_env\n```\n\nThis creates the following structure:\n\n```\nmy_env/\n├── .dockerignore        # Docker build exclusions\n├── __init__.py           # Export YourAction, YourObservation, YourEnv\n├── models.py             # Define Action, Observation, State dataclasses\n├── client.py             # Implement YourEnv(EnvClient)\n├── README.md             # Document your environment\n├── openenv.yaml          # Environment manifest\n├── pyproject.toml        # Dependencies and package configuration\n├── outputs/              # Runtime outputs (logs, evals) - gitignored\n│   ├── logs/\n│   └── evals/\n└── server/\n    ├── your_environment.py  # Implement YourEnvironment(Environment)\n    ├── app.py               # Create FastAPI app\n    ├── requirements.txt     # Dependencies for Docker (can be generated)\n    └── Dockerfile           # Define container image\n```\n\n#### Dependency Management\n\nOpenEnv uses `pyproject.toml` as the primary dependency specification:\n\n- **Environment-level `pyproject.toml`**: Each environment defines its own dependencies\n- **Root-level `pyproject.toml`**: Contains shared core dependencies (fastapi, pydantic, uvicorn)\n- **Server `requirements.txt`**: Can be auto-generated from `pyproject.toml` for Docker builds\n\n**Development Workflow:**\n\n```bash\n# Install environment in editable mode\ncd my_env\npip install -e .\n\n# Or using uv (faster)\nuv pip install -e .\n\n# Run server locally without Docker\nuv run server --host 0.0.0.0 --port 8000\n```\n\nSee [`envs/README.md`](envs/README.md) for a complete guide on building environments.\n\n### For Environment Users\n\nTo use an environment:\n1. Install the client: `pip install git+https://huggingface.co/spaces/openenv/echo_env`\n2. Import: `from echo_env import CallToolAction, EchoEnv`\n3. Use async (recommended) or sync API:\n\n**Async (recommended):**\n```python\nasync with EchoEnv(base_url=\"...\") as client:\n    result = await client.reset()\n    result = await client.step(action)\n```\n\n**Sync (via `.sync()` wrapper):**\n```python\nwith EchoEnv(base_url=\"...\").sync() as client:\n    result = client.reset()\n    result = client.step(action)\n```\n\nSee example scripts in `examples/` directory.\n\n## CLI Commands\n\nThe OpenEnv CLI provides commands to manage environments:\n\n- **`openenv init \u003cenv_name\u003e`** - Initialize a new environment from template\n- **`openenv push [--repo-id \u003crepo\u003e] [--private]`** - Deploy environment to Hugging Face Spaces\n- **`openenv serve`** - Serve an environment locally with optional auto-reload\n- **`openenv build`** - Build the Docker image for an environment\n- **`openenv fork \u003cspace-id\u003e`** - Fork a Space from HF Hub to your account\n- **`openenv validate`** - Validate an environment configuration\n\n### Quick Start\n\n```bash\n# Create a new environment\nopenenv init my_game_env\n\n# Deploy to Hugging Face (will prompt for login if needed)\ncd my_game_env\nopenenv push\n```\n\nFor detailed options run any command with `--help`.\n\n## Development\n\n### Installation\n\n```bash\n# Clone the repository\ngit clone https://github.com/huggingface/OpenEnv.git\ncd OpenEnv\n\n# Install core package in editable mode\npip install -e .\n# Or using uv (faster)\nuv pip install -e .\n```\n\n### Running Tests\n\nOpenEnv uses a modular dependency structure: the core package is minimal, and each environment has its own dependencies. This means some tests require environment-specific packages.\n\n```bash\n# Install pytest (required for running tests)\nuv pip install pytest\n\n# Run all tests (skips tests requiring uninstalled dependencies)\nPYTHONPATH=src:envs uv run pytest tests/ -v --tb=short\n\n# Run a specific test file\nPYTHONPATH=src:envs uv run pytest tests/envs/test_echo_environment.py -v\n```\n\n**To run environment-specific tests**, install that environment's dependencies:\n\n```bash\n# Example: Install coding_env with dev dependencies (includes smolagents + pytest)\nuv pip install -e \"envs/coding_env[dev]\"\n\n# Then run coding_env tests\nPYTHONPATH=src:envs uv run pytest tests/envs/test_python_codeact_rewards.py -v\n```\n\nTests will be automatically skipped if their required dependencies aren't installed.\n\n## Integrations\n\nOpenEnv works with a growing ecosystem of RL frameworks and platforms. If your project supports OpenEnv, open a PR to add it here.\n\n### TRL\nSee the [TRL example](https://huggingface.co/docs/trl/openenv) on how to integrate OpenEnv environments with GRPO training.\n\n### torchforge\nSee GRPO BlackJack training example: [`examples/grpo_blackjack/`](examples/grpo_blackjack/)\n\n### Unsloth\nSee the 2048 game example based on gpt-oss: [Colab notebook](https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/OpenEnv_gpt_oss_(20B)_Reinforcement_Learning_2048_Game.ipynb)\n\n### SkyRL\nSee the [SkyRL example](https://skyrl.readthedocs.io/en/latest/examples/openenv.html) on how to train on OpenEnv environments with SkyRL.\n\n### ART\nSee the [ART example](https://art.openpipe.ai/integrations/openenv-integration) on how OpenEnv environments can be used to train models with ART.\n\n### Oumi\nSee the [Oumi example](https://github.com/oumi-ai/oumi/blob/main/notebooks/Oumi%20-%20OpenEnv%20GRPO%20with%20trl.ipynb) on how OpenEnv environments can be used to train models with Oumi.\n\n### Lightning AI\n[Lightning AI templates](https://lightning.ai/templates?section=featured\u0026query=openenv)\n\n## Example Environments\n\n| Environment | Description |\n|---|---|\n| [Echo Environment](envs/echo_env/README.md) | Echoes back messages with metadata. Ideal for testing HTTP server infrastructure, learning framework basics, and verifying container deployment. |\n| [Coding Environment](envs/coding_env/README.md) | Sandboxed Python code execution via smolagents. Captures stdout/stderr/exit codes, supports persistent episode context, and provides detailed error handling. |\n| [Chess Environment](envs/chess_env/README.md) | Chess RL environment with configurable opponents and full rules support. |\n| [Atari Environment](envs/atari_env/README.md) | Classic Arcade Learning Environment tasks for RL benchmarking. |\n| [FinRL Environment](envs/finrl_env/README.md) | Financial market simulations for algorithmic trading experiments. |\n\n\u003e Browse the full catalog of community environments at [huggingface.co/docs/openenv/environments](https://huggingface.co/docs/openenv/environments).\n\n## Community Support \u0026 Acknowledgments\n\nOpenEnv is governed by a technical committee that coordinates project direction, major technical decisions, RFCs, and release planning through the public issue tracker, pull requests, and RFC process. Current committee members: Hugging Face, Unsloth, Reflection, Meta PyTorch, Modal, and Prime Intellect.\n\nThe project is also supported by a broader community of organizations. If you would like to add your project or organization here, please open a pull request for maintainer review.\n\nSupporters include: [Meta-PyTorch](https://github.com/meta-pytorch), [Hugging Face](https://huggingface.co), [Scaler AI Labs](https://scalerailabs.com), [Patronus AI](https://patronus.ai), [Surge AI](https://surgehq.ai), [LastMile AI](https://www.lastmileai.dev), [Unsloth](https://unsloth.ai), [Reflection](https://reflection.ai), [vLLM](https://vllm.ai), [SkyRL](https://skyrl.readthedocs.io) (UC-Berkeley), [Lightning AI](https://lightning.ai), [Axolotl AI](https://github.com/axolotl-ai-cloud/axolotl), [Stanford Scaling Intelligence Lab](https://scalingintelligence.stanford.edu/), [Mithril](https://mithril.ai), [OpenMined](https://openmined.org/), [Fleet AI](https://fleetai.com), [Halluminate](https://halluminate.ai/), [Turing](https://www.turing.com/), [Scale AI](https://scale.com/), [Scorecard](https://www.scorecard.io/)\n\nAnd we'd also like to acknowledge the team at Farama Foundation as the OpenEnv API was heavily inspired by the work you all have done on Gymnasium. Cheers!\n\n## License\n\nBSD 3-Clause License (see [LICENSE](./LICENSE) file)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhuggingface%2FOpenEnv","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhuggingface%2FOpenEnv","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhuggingface%2FOpenEnv/lists"}