{"id":15018001,"url":"https://github.com/stillonearth/bevy_rl","last_synced_at":"2025-10-30T21:17:23.191Z","repository":{"id":38739557,"uuid":"485399391","full_name":"stillonearth/bevy_rl","owner":"stillonearth","description":" Reinforcement Learning environments with Bevy","archived":false,"fork":false,"pushed_at":"2024-12-06T08:38:18.000Z","size":1176,"stargazers_count":81,"open_issues_count":0,"forks_count":3,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-03-27T09:05:23.999Z","etag":null,"topics":["bevy","gym","rl"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/stillonearth.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-04-25T14:13:14.000Z","updated_at":"2025-02-13T04:34:18.000Z","dependencies_parsed_at":"2024-11-16T18:41:28.164Z","dependency_job_id":"85c8e430-bed4-4d26-a62f-53108fc5868a","html_url":"https://github.com/stillonearth/bevy_rl","commit_stats":{"total_commits":78,"total_committers":2,"mean_commits":39.0,"dds":"0.23076923076923073","last_synced_commit":"d81ebd25b4a867bd89d0e0215c7d8acaa463e3e0"},"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stillonearth%2Fbevy_rl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stillonearth%2Fbevy_rl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stillonearth%2Fbevy_rl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stillonearth%2Fbevy_rl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/stillonearth","download_url":"https://codeload.github.com/stillonearth/bevy_rl/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247142043,"owners_count":20890652,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bevy","gym","rl"],"created_at":"2024-09-24T19:51:18.351Z","updated_at":"2025-10-30T21:17:17.943Z","avatar_url":"https://github.com/stillonearth.png","language":"Rust","funding_links":[],"categories":["Reinforcement Learning"],"sub_categories":[],"readme":"# 🏋️‍♀️ bevy_rl\r\n\r\n[![Crates.io](https://img.shields.io/crates/v/bevy_rl.svg)](https://crates.io/crates/bevy_rl)\r\n[![MIT/Apache 2.0](https://img.shields.io/badge/license-MIT%2FApache-blue.svg)](https://github.com/bevyengine/bevy#license)\r\n[![Crates.io](https://img.shields.io/crates/d/bevy_rl.svg)](https://crates.io/crates/bevy_rl)\r\n[![Rust](https://github.com/stillonearth/bevy_rl/workflows/CI/badge.svg)](https://github.com/stillonearth/bevy_rl/actions)\r\n\r\n![image](https://github.com/stillonearth/bevy_rl/blob/main/img/dog.gif?raw=true)\r\n\r\n## Reinforcement Learning for Bevy Engine\r\n\r\nBuild Reinforcement Learning [Gym](https://gym.openai.com/) environments with [Bevy](https://bevyengine.org/) engine to train AI agents that can learn from screen pixels or defined obeservation state.\r\n\r\n## 📝Features\r\n\r\n- Set of APIs to implement OpenAI Gym interface, such as `reset`, `step`, `render` and associated simulator states\r\n- Multi-Agent support\r\n- Rendering screen pixels to RAM buffer — for training agents with raw pixels\r\n- REST API to control agents\r\n\r\n## 👩‍💻 Usage\r\n\r\n### 1. Define Action and Obeservation Space\r\n\r\nObservation space needs to be `Serializable` for REST API to work.\r\n\r\n```rust\r\n// Action space\r\n#[derive(Default)]\r\npub struct Actions {\r\n    // actuator_signals: [f32; 3],\r\n}\r\n\r\n// Observation space\r\n#[derive(Default, Serialize, Clone)]\r\npub struct Observations {\r\n    // agent_coords: [(f32, f32); 16],\r\n}\r\n```\r\n\r\n### 2. Enable AI Gym Plugin\r\n\r\nWidth and height should exceed 256, otherwise wgpu will panic.\r\n\r\n```rust\r\n// Setup bevy_rl\r\nlet ai_gym_state = AIGymState::\u003cActions, State\u003e::new(AIGymSettings {\r\n    width: u32,              // Width and height of the screen\r\n    height: u32,             // ...\r\n    num_agents: 1,           // Number of agents — each will get a camera handle\r\n    render_to_buffer: false, // You can disable rendering to buffer\r\n    pause_interval: 0.01,    // 100 Hz\r\n    ..default()\r\n});\r\napp.insert_resource(ai_gym_state)\r\n    .add_plugins(AIGymPlugin::\u003cActions, Observations\u003e::default());\r\n```\r\n\r\n### 2.1 (Optional) Enable Rendering to Buffer\r\n\r\nIf your environment wants to export raw pixels, you will need to attach a render target to each camera you want to export them from. Render targets are copied each frame from GPU memory to RAM buffers so that they can be accessed with REST API.\r\n\r\n```rust\r\npub(crate) fn spawn_cameras(\r\n    ai_gym_state: Res\u003cAIGymState\u003cActions, Observations\u003e\u003e,\r\n) {\r\n    let mut ai_gym_state = ai_gym_state.lock().unwrap();\r\n    let ai_gym_settings = ai_gym_state.settings.clone();\r\n\r\n    for i in 0..ai_gym_settings.num_agents {\r\n        let render_image_handle = ai_gym_state.render_image_handles[i as usize].clone();\r\n        let render_target = RenderTarget::Image(render_image_handle);\r\n        let camera_bundle = Camera3dBundle {\r\n            camera: Camera {\r\n                target: render_target,  // Render target is baked in bevy_rl and used to export pixels\r\n                priority: -1,           // set to -1 to render at the firstmost pass\r\n                ..default()\r\n            },\r\n            ..default()\r\n        };\r\n        commands.spawn(camera_bundle);\r\n    }\r\n}\r\n```\r\n\r\n### 4. Handle bevy_rl events\r\n\r\n`bevy_rl` will communicate with your environment through events. Those events initiate from REST API or `bevy_rl.SimulationPauseTimer` that pauses the simulation with given interval (`AIGymSettings.pause_interval`).\r\n\r\n| Event          | Description                        | Usage                                                                                      |\r\n| -------------- | ---------------------------------- | ------------------------------------------------------------------------------------------ |\r\n| `EventReset`   | Reset environment to initial state | You should rebuild your evnironment here                                                   |\r\n| `EventControl` | Switch to control state            | You should recieve actions here and apply them to your environment (and resume simulation) |\r\n| `EventPause`   | Pause environment execution        | Pause physics engine or game clock and take snapshot of your game state                    |\r\n\r\nHere's example of how to handle those events:\r\n\r\n```rust\r\n// EventPauseResume\r\nfn bevy_rl_pause_request(\r\n    mut pause_event_reader: EventReader\u003cEventPauseResume\u003e,\r\n    ai_gym_state: Res\u003cAIGymState\u003cActions, State\u003e\u003e,\r\n) {\r\n    for _ in pause_event_reader.iter() {\r\n        // Pause simulation (physics engine)\r\n        // ...\r\n        // Collect state into serializable struct\r\n        let env_state = Observations(...);\r\n        // Set bevy_rl gym state\r\n        let mut ai_gym_state = ai_gym_state.lock().unwrap();\r\n        ai_gym_state.set_env_state(env_state);\r\n    }\r\n}\r\n\r\n// EventControl\r\nfn bevy_rl_control_request(\r\n    mut pause_event_reader: EventReader\u003cEventControl\u003e,\r\n    mut simulation_state: ResMut\u003cState\u003cSimulationState\u003e\u003e,\r\n) {\r\n    for control in pause_event_reader.iter() {\r\n        let unparsed_actions = \u0026control.0;\r\n        for i in 0..unparsed_actions.len() {\r\n            if let Some(unparsed_action) = unparsed_actions[i].clone() {\r\n                let action: Vec\u003cf64\u003e = serde_json::from_str(\u0026unparsed_action).unwrap();\r\n                // Pass control inputs to your agents\r\n                // ...\r\n            }\r\n        }\r\n        // Resume simulation (physics engine)\r\n        // ...\r\n        // Return to running state; note that it uses pop/push to avoid\r\n        // entering `SystemSet::on_enter(SimulationState::Running)` which initialized game world anew\r\n        simulation_state.pop().unwrap();\r\n    }\r\n}\r\n\r\n/// Handle bevy_rl::EventReset\r\npub(crate) fn bevy_rl_reset_request(\r\n    mut reset_event_reader: EventReader\u003cEventReset\u003e,\r\n    mut commands: Commands,\r\n    mut walls: Query\u003cEntity, \u0026Wall\u003e,\r\n    mut players: Query\u003c(Entity, \u0026Actor)\u003e,\r\n    mut simulation_state: ResMut\u003cState\u003cSimulationState\u003e\u003e,\r\n    ai_gym_state: Res\u003cAIGymState\u003cActions, Observations\u003e\u003e,\r\n) {\r\n    if reset_event_reader.iter().count() == 0 {\r\n        return;\r\n    }\r\n\r\n    // Reset envrionment state here\r\n\r\n    // Return simulation in Running state\r\n    simulation_state.set(SimulationState::Running).unwrap();\r\n\r\n    // tell bevy_rl that environment is reset and return response to REST API\r\n    let ai_gym_state = ai_gym_state.lock().unwrap();\r\n    ai_gym_state.send_reset_result(true);\r\n}\r\n```\r\n\r\nRegister systems to handle bevy_rl events.\r\n\r\n```rust\r\napp.add_systems(\r\n    Update,\r\n    (\r\n        .with_system(bevy_rl_control_request)\r\n        .with_system(bevy_rl_reset_request)\r\n        .with_system(bevy_rl_pause_request),\r\n    ).in_set(SimulationState::PausedForControl)\r\n);\r\n```\r\n\r\n## 💻 AIGymState API\r\n\r\nThose methods are available on `AIGymState` resource. You should use them to alter bevy_rl internal state.\r\n\r\n| Method                                             | Description                         | Usage                                                                                        |\r\n| -------------------------------------------------- | ----------------------------------- | -------------------------------------------------------------------------------------------- |\r\n| `set_reward(agent_index: usize, score: f32)`       | Set reward for an agent             | When a certain event happens, you can set reward for an agent.                               |\r\n| `set_terminated(agent_index: usize, result: bool)` | Set termination status for an agent | Once your agent is killed, you should set it's status to `true`. Useful for Multi-agent.     |\r\n| `reset()`                                          | Reset bevy_rl state                 | You should call this method when you reset your environment to clear exported state history  |\r\n| `set_env_state(state: State)`                      | Set current environment state       | When you serialize your environment state, you should set it here.                           |\r\n| `send_reset_result(result: bool)`                  | Send reset result to REST API       | You should call this method when you have reset your environment to sychronize with REST API |\r\n\r\n## 🌐 REST API\r\n\r\nAccessing `bevy_rl`-enabled environment is possible through REST API. Here's a list of available endpoints:\r\n\r\n| Method            | Verb    | bevy_rl version                             |\r\n| ----------------- | ------- | ------------------------------------------- |\r\n| Camera Pixels     | **GET** | `http://localhost:7878/visual_observations` |\r\n| State             | **GET** | `http://localhost:7878/state`               |\r\n| Reset Environment | **GET** | `http://localhost:7878/reset`               |\r\n| Step              | **GET** | `http://localhost:7878/step?payload=ACTION` |\r\n\r\n[bevy_rl_shooter](https://github.com/stillonearth/bevy_rl_shooter) implements an example Python wrapper.\r\n\r\n## ✍️ Examples\r\n\r\n- [bevy_rl_shooter](https://github.com/stillonearth/bevy_rl_shooter) — example FPS project\r\n- [bevy_quadruped_neural_control](https://github.com/stillonearth/bevy_quadruped_neural_control) — quadruped locomotion with bevy_mujoco and bevy_rl\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstillonearth%2Fbevy_rl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fstillonearth%2Fbevy_rl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstillonearth%2Fbevy_rl/lists"}