{"id":13720222,"url":"https://github.com/utiasDSL/safe-control-gym","last_synced_at":"2025-05-07T12:31:01.258Z","repository":{"id":37024031,"uuid":"392837160","full_name":"utiasDSL/safe-control-gym","owner":"utiasDSL","description":"PyBullet CartPole and Quadrotor environments—with CasADi symbolic a priori dynamics—for learning-based control and RL","archived":false,"fork":false,"pushed_at":"2025-05-01T22:15:24.000Z","size":217838,"stargazers_count":712,"open_issues_count":11,"forks_count":140,"subscribers_count":12,"default_branch":"main","last_synced_at":"2025-05-01T23:23:07.256Z","etag":null,"topics":["cartpole","casadi","control","gym","pybullet","quadcopter","quadrotor","reinforcement-learning","robotics","robustness","safety","symbolic"],"latest_commit_sha":null,"homepage":"https://www.dynsyslab.org/safe-robot-learning/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/utiasDSL.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2021-08-04T22:14:15.000Z","updated_at":"2025-05-01T22:15:29.000Z","dependencies_parsed_at":"2023-10-13T09:34:51.647Z","dependency_job_id":"0dc757c0-79dd-4633-b075-751420b4ac94","html_url":"https://github.com/utiasDSL/safe-control-gym","commit_stats":{"total_commits":435,"total_committers":17,"mean_commits":25.58823529411765,"dds":0.6643678160919539,"last_synced_commit":"5f20bac6d642a00c81fca9823bcd0b05fa118275"},"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/utiasDSL%2Fsafe-control-gym","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/utiasDSL%2Fsafe-control-gym/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/utiasDSL%2Fsafe-control-gym/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/utiasDSL%2Fsafe-control-gym/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/utiasDSL","download_url":"https://codeload.github.com/utiasDSL/safe-control-gym/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252876321,"owners_count":21818163,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cartpole","casadi","control","gym","pybullet","quadcopter","quadrotor","reinforcement-learning","robotics","robustness","safety","symbolic"],"created_at":"2024-08-03T01:01:01.238Z","updated_at":"2025-05-07T12:30:56.241Z","avatar_url":"https://github.com/utiasDSL.png","language":"Python","funding_links":[],"categories":["Reinforcement Learning (RL) and Deep Reinforcement Learning (DRL)","Simulation"],"sub_categories":["RL/DRL Environments","Version Control"],"readme":"# safe-control-gym\n\nPhysics-based CartPole and Quadrotor [Gym](https://gym.openai.com) environments (using [PyBullet](https://pybullet.org/wordpress/)) with symbolic *a priori* dynamics (using [CasADi](https://web.casadi.org)) for **learning-based control**, and model-free and model-based **reinforcement learning** (RL).\n\nThese environments include (and evaluate) symbolic safety constraints and implement input, parameter, and dynamics disturbances to test the robustness and generalizability of control approaches. [[PDF]](https://arxiv.org/pdf/2108.06266.pdf)\n\n\u003cimg src=\"figures/problem_illustration.jpg\" alt=\"problem illustration\" width=\"800\"\u003e\n\n```bibtex\n@article{brunke2021safe,\n         title={Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning},\n         author={Lukas Brunke and Melissa Greeff and Adam W. Hall and Zhaocong Yuan and Siqi Zhou and Jacopo Panerati and Angela P. Schoellig},\n         journal = {Annual Review of Control, Robotics, and Autonomous Systems},\n         year={2021},\n         url = {https://arxiv.org/abs/2108.06266}}\n```\n\nTo reproduce the results in the article, see [branch `ar`](https://github.com/utiasDSL/safe-control-gym/tree/ar).\n\n```bibtex\n@article{yuan2021safecontrolgym,\n  author={Yuan, Zhaocong and Hall, Adam W. and Zhou, Siqi and Brunke, Lukas and Greeff, Melissa and Panerati, Jacopo and Schoellig, Angela P.},\n  journal={IEEE Robotics and Automation Letters},\n  title={Safe-Control-Gym: A Unified Benchmark Suite for Safe Learning-Based Control and Reinforcement Learning in Robotics},\n  year={2022},\n  volume={7},\n  number={4},\n  pages={11142-11149},\n  doi={10.1109/LRA.2022.3196132}}\n```\n\nTo reproduce the results in the article, see [branch `submission`](https://github.com/utiasDSL/safe-control-gym/tree/submission).\n\n\u003c!-- ![baselines](./figures/baselines.png) --\u003e\n\n## Install on Ubuntu/macOS\n\n### Clone repo\n\n```bash\ngit clone https://github.com/utiasDSL/safe-control-gym.git\ncd safe-control-gym\n```\n\n### (optional) Create a `conda` environment\n\nCreate and access a Python 3.10 environment using\n[`conda`](https://docs.conda.io/projects/conda/en/latest/user-guide/install/index.html)\n\n```bash\nconda create -n safe python=3.10\nconda activate safe\n```\n\n### Install\n\nInstall the `safe-control-gym` repository\n\n```bash\npython -m pip install --upgrade pip\npython -m pip install -e .\n```\n\n#### Note\n\nYou may need to separately install `gmp`, a dependency of `pycddlib`:\n\n ```bash\nconda install -c anaconda gmp\n ```\n\nor\n\n  ```bash\n sudo apt-get install libgmp-dev\n ```\n\n #### (optional) Additional requirements for MPC\n\nYou may need to separately install [`acados`](https://github.com/acados/acados) for fast MPC implementations.\n\n- To build and install acados, see their [installation guide](https://docs.acados.org/installation/index.html).\n- To set up the acados python interface, check out [these installtion steps](https://docs.acados.org/python_interface/index.html).\n\n## Architecture\n\nOverview of [`safe-control-gym`](https://arxiv.org/abs/2109.06325)'s API:\n\n\u003cimg src=\"figures/block.png\" alt=\"block diagram\" width=\"800\"\u003e\n\n## Configuration\n\n\u003cimg src=\"figures/config.png\" alt=\"config\" width=\"800\"\u003e\n\n## Getting Started\n\nFamiliarize with APIs and environments with the scripts in [`examples/`](https://github.com/utiasDSL/safe-control-gym/tree/main/examples)\n\n### 3D Quadrotor Lemniscate Trajectory Tracking with PID\n\n```bash\ncd ./examples/   # Navigate to the examples folder\npython3 pid/pid_experiment.py \\\n    --algo pid \\\n    --task quadrotor \\\n    --overrides \\\n        ./pid/config_overrides/quadrotor_3D/quadrotor_3D_tracking.yaml\n```\n\n\u003cimg src=\"figures/systems.png\" alt=\"systems\" width=\"450\"\u003e \u003cimg src=\"figures/figure8.gif\" alt=\"trajectory\" width=\"350\"\u003e\n\n### Cartpole Stabilization with LQR\n\n```bash\ncd ./examples/   # Navigate to the examples folder\npython3 lqr/lqr_experiment.py \\\n    --algo lqr \\\n    --task cartpole \\\n    --overrides \\\n        ./lqr/config_overrides/cartpole/cartpole_stabilization.yaml \\\n        ./lqr/config_overrides/cartpole/lqr_cartpole_stabilization.yaml\n```\n\n### 2D Quadrotor Trajectory Tracking with PPO\n\n```bash\ncd ./examples/rl/   # Navigate to the RL examples folder\npython3 rl_experiment.py \\\n    --algo ppo \\\n    --task quadrotor \\\n    --overrides \\\n        ./config_overrides/quadrotor_2D/quadrotor_2D_track.yaml \\\n        ./config_overrides/quadrotor_2D/ppo_quadrotor_2D.yaml \\\n    --kv_overrides \\\n        algo_config.training=False\n```\n\n### Verbose API Example\n\n```bash\ncd ./examples/   # Navigate to the examples folder\npython3 no_controller/verbose_api.py \\\n    --task cartpole \\\n    --overrides no_controller/verbose_api.yaml\n```\n\n\u003cimg src=\"figures/prints.png\" al=\"prints\" width=\"800\"\u003e\n\n## List of Implemented Controllers\n\n- [PID](https://github.com/utiasDSL/safe-control-gym/blob/main/safe_control_gym/controllers/pid/pid.py)\n- [LQR](https://github.com/utiasDSL/safe-control-gym/blob/main/safe_control_gym/controllers/lqr/lqr.py)\n- [iLQR](https://github.com/utiasDSL/safe-control-gym/blob/main/safe_control_gym/controllers/lqr/ilqr.py)\n- [Linear MPC](https://github.com/utiasDSL/safe-control-gym/blob/main/safe_control_gym/controllers/mpc/linear_mpc.py)\n- [GP-MPC](https://github.com/utiasDSL/safe-control-gym/blob/main/safe_control_gym/controllers/mpc/gp_mpc.py)\n- [SAC](https://github.com/utiasDSL/safe-control-gym/blob/main/safe_control_gym/controllers/sac/sac.py)\n- [PPO](https://github.com/utiasDSL/safe-control-gym/blob/main/safe_control_gym/controllers/ppo/ppo.py)\n- [DDPG](https://github.com/utiasDSL/safe-control-gym/blob/main/safe_control_gym/controllers/ddpg/ddpg.py)\n- [Safety Layer](https://github.com/utiasDSL/safe-control-gym/tree/main/safe_control_gym/controllers/safe_explorer)\n- [RARL](https://github.com/utiasDSL/safe-control-gym/blob/main/safe_control_gym/controllers/rarl/rarl.py)\n- [RAP](https://github.com/utiasDSL/safe-control-gym/blob/main/safe_control_gym/controllers/rarl/rap.py)\n\n## List of Implemented Safety Filters\n\n- [MPSC](https://github.com/utiasDSL/safe-control-gym/blob/main/safe_control_gym/safety_filters/mpsc/linear_mpsc.py)\n- [CBF](https://github.com/utiasDSL/safe-control-gym/blob/main/safe_control_gym/safety_filters/cbf/cbf.py)\n- [Neural Network CBF](https://github.com/utiasDSL/safe-control-gym/blob/main/safe_control_gym/safety_filters/cbf/cbf_nn.py)\n\n## Performance\n\nWe compare the sample efficiency of `safe-control-gym` with the original [OpenAI Cartpole][001] and [PyBullet Gym's Inverted Pendulum][002], as well as [`gym-pybullet-drones`][003].\nWe choose the default physic simulation integration step of each project.\nWe report performance results for open-loop, random action inputs.\nNote that the Bullet engine frequency reported for `safe-control-gym` is typically much finer grained for improved fidelity.\n`safe-control-gym` quadrotor environment is not as light-weight as [`gym-pybullet-drones`][003] but provides the same order of magnitude speed-up and several more safety features/symbolic models.\n\n| Environment                | GUI    | Control Freq.  | PyBullet Freq.  | Constraints \u0026 Disturbances^       | Speed-Up^^      |\n| :------------------------: | :----: | :------------: | :-------------: | :-------------------------------: | :-------------: |\n| [Gym cartpole][001]        | True   | 50Hz           | N/A             | No                                | 1.16x           |\n| [InvPenPyBulletEnv][002]   | False  | 60Hz           | 60Hz            | No                                | 158.29x         |\n| [cartpole][004]            | True   | 50Hz           | 50Hz            | No                                | 0.85x           |\n| [cartpole][004]            | False  | 50Hz           | 1000Hz          | No                                | 24.73x          |\n| [cartpole][004]            | False  | 50Hz           | 1000Hz          | Yes                               | 22.39x          |\n| | | | | | |\n| [gym-pyb-drones][003]      | True   | 48Hz           | 240Hz           | No                                | 2.43x           |\n| [gym-pyb-drones][003]      | False  | 50Hz           | 1000Hz          | No                                | 21.50x          |\n| [quadrotor][005]           | True   | 60Hz           | 240Hz           | No                                | 0.74x           |\n| [quadrotor][005]           | False  | 50Hz           | 1000Hz          | No                                | 9.28x           |\n| [quadrotor][005]           | False  | 50Hz           | 1000Hz          | Yes                               | 7.62x           |\n\n\u003e ^ Whether the environment includes a default set of constraints and disturbances\n\u003e\n\u003e ^^ Speed-up = Elapsed Simulation Time / Elapsed Wall Clock Time; on a 2.30GHz Quad-Core i7-1068NG7 with 32GB 3733MHz LPDDR4X; no GPU\n\n[001]: https://github.com/openai/gym/blob/master/gym/envs/classic_control/cartpole.py\n[002]: https://github.com/benelot/pybullet-gym/blob/master/pybulletgym/envs/mujoco/envs/pendulum/inverted_pendulum_env.py\n[003]: https://github.com/utiasDSL/gym-pybullet-drones\n\n[004]: https://github.com/utiasDSL/safe-control-gym/blob/main/safe_control_gym/envs/gym_control/cartpole.py\n[005]: https://github.com/utiasDSL/safe-control-gym/blob/main/safe_control_gym/envs/gym_pybullet_drones/quadrotor.py\n\n## Run Tests and Linting\nTests can be run locally by executing:\n```bash\npython3 -m pytest ./tests/  # Run all tests\n```\n\nLinting can be run locally with:\n```bash\npre-commit install  # Install the pre-commit hooks\npre-commit autoupdate  # Auto-update the version of the hooks\npre-commit run --all  # Run the hooks on all files\n```\n\n## References\n\n- Brunke, L., Greeff, M., Hall, A. W., Yuan, Z., Zhou, S., Panerati, J., \u0026 Schoellig, A. P. (2022). [Safe learning in robotics: From learning-based control to safe reinforcement learning](https://www.annualreviews.org/doi/abs/10.1146/annurev-control-042920-020211). Annual Review of Control, Robotics, and Autonomous Systems, 5, 411-444.\n- Yuan, Z., Hall, A. W., Zhou, S., Brunke, L., Greeff, M., Panerati, J., \u0026 Schoellig, A. P. (2022). [safe-control-gym: A unified benchmark suite for safe learning-based control and reinforcement learning in robotics](https://ieeexplore.ieee.org/abstract/document/9849119). IEEE Robotics and Automation Letters, 7(4), 11142-11149.\n\n## Related Open-source Projects\n\n- [`gym-pybullet-drones`](https://github.com/utiasDSL/gym-pybullet-drones): single and multi-quadrotor environments\n- [`stable-baselines3`](https://github.com/DLR-RM/stable-baselines3): PyTorch reinforcement learning algorithms\n- [`bullet3`](https://github.com/bulletphysics/bullet3): multi-physics simulation engine\n- [`gym`](https://github.com/openai/gym): OpenAI reinforcement learning toolkit\n- [`casadi`](https://github.com/casadi/casadi): symbolic framework for numeric optimization\n- [`safety-gym`](https://github.com/openai/safety-gym): environments for safe exploration in RL\n- [`realworldrl_suite`](https://github.com/google-research/realworldrl_suite): real-world RL challenge framework\n- [`gym-marl-reconnaissance`](https://github.com/JacopoPan/gym-marl-reconnaissance): multi-agent heterogeneous (UAV/UGV) environments\n\n-----\n\u003e University of Toronto's [Dynamic Systems Lab](https://github.com/utiasDSL) / [Vector Institute for Artificial Intelligence](https://github.com/VectorInstitute)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FutiasDSL%2Fsafe-control-gym","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FutiasDSL%2Fsafe-control-gym","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FutiasDSL%2Fsafe-control-gym/lists"}