Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/utiasDSL/safe-control-gym
PyBullet CartPole and Quadrotor environments—with CasADi symbolic a priori dynamics—for learning-based control and RL
https://github.com/utiasDSL/safe-control-gym
cartpole casadi control gym pybullet quadcopter quadrotor reinforcement-learning robotics robustness safety symbolic
Last synced: about 1 month ago
JSON representation
PyBullet CartPole and Quadrotor environments—with CasADi symbolic a priori dynamics—for learning-based control and RL
- Host: GitHub
- URL: https://github.com/utiasDSL/safe-control-gym
- Owner: utiasDSL
- License: mit
- Created: 2021-08-04T22:14:15.000Z (over 3 years ago)
- Default Branch: main
- Last Pushed: 2024-11-05T15:28:12.000Z (about 2 months ago)
- Last Synced: 2024-11-05T16:37:27.174Z (about 2 months ago)
- Topics: cartpole, casadi, control, gym, pybullet, quadcopter, quadrotor, reinforcement-learning, robotics, robustness, safety, symbolic
- Language: Python
- Homepage: https://www.dynsyslab.org/safe-robot-learning/
- Size: 208 MB
- Stars: 619
- Watchers: 11
- Forks: 132
- Open Issues: 7
-
Metadata Files:
- Readme: README.md
- License: LICENSE
- Citation: CITATION.cff
Awesome Lists containing this project
- awesome-robotic-tooling - safe-control-gym - PyBullet-based CartPole and Quadrotor environments—with CasADi symbolic dynamics and constraints—for safe and robust learning-based control. (Simulation / Version Control)
README
# safe-control-gym
Physics-based CartPole and Quadrotor [Gym](https://gym.openai.com) environments (using [PyBullet](https://pybullet.org/wordpress/)) with symbolic *a priori* dynamics (using [CasADi](https://web.casadi.org)) for **learning-based control**, and model-free and model-based **reinforcement learning** (RL).
These environments include (and evaluate) symbolic safety constraints and implement input, parameter, and dynamics disturbances to test the robustness and generalizability of control approaches. [[PDF]](https://arxiv.org/pdf/2108.06266.pdf)
```bibtex
@article{brunke2021safe,
title={Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning},
author={Lukas Brunke and Melissa Greeff and Adam W. Hall and Zhaocong Yuan and Siqi Zhou and Jacopo Panerati and Angela P. Schoellig},
journal = {Annual Review of Control, Robotics, and Autonomous Systems},
year={2021},
url = {https://arxiv.org/abs/2108.06266}}
```
To reproduce the results in the article, see [branch `ar`](https://github.com/utiasDSL/safe-control-gym/tree/ar).
```bibtex
@article{yuan2021safecontrolgym,
author={Yuan, Zhaocong and Hall, Adam W. and Zhou, Siqi and Brunke, Lukas and Greeff, Melissa and Panerati, Jacopo and Schoellig, Angela P.},
journal={IEEE Robotics and Automation Letters},
title={Safe-Control-Gym: A Unified Benchmark Suite for Safe Learning-Based Control and Reinforcement Learning in Robotics},
year={2022},
volume={7},
number={4},
pages={11142-11149},
doi={10.1109/LRA.2022.3196132}}
```
To reproduce the results in the article, see [branch `submission`](https://github.com/utiasDSL/safe-control-gym/tree/submission).
## Install on Ubuntu/macOS
### Clone repo
```bash
git clone https://github.com/utiasDSL/safe-control-gym.git
cd safe-control-gym
```
### (optional) Create a `conda` environment
Create and access a Python 3.10 environment using
[`conda`](https://docs.conda.io/projects/conda/en/latest/user-guide/install/index.html)
```bash
conda create -n safe python=3.10
conda activate safe
```
### Install
Install the `safe-control-gym` repository
```bash
python -m pip install --upgrade pip
python -m pip install -e .
```
#### Note
You may need to separately install `gmp`, a dependency of `pycddlib`:
```bash
conda install -c anaconda gmp
```
or
```bash
sudo apt-get install libgmp-dev
```
#### (optional) Additional requirements for MPC
You may need to separately install [`acados`](https://github.com/acados/acados) for fast MPC implementations.
- To build and install acados, see their [installation guide](https://docs.acados.org/installation/index.html).
- To set up the acados python interface, check out [these installtion steps](https://docs.acados.org/python_interface/index.html).
## Architecture
Overview of [`safe-control-gym`](https://arxiv.org/abs/2109.06325)'s API:
## Configuration
## Getting Started
Familiarize with APIs and environments with the scripts in [`examples/`](https://github.com/utiasDSL/safe-control-gym/tree/main/examples)
### 3D Quadrotor Lemniscate Trajectory Tracking with PID
```bash
cd ./examples/ # Navigate to the examples folder
python3 pid/pid_experiment.py \
--algo pid \
--task quadrotor \
--overrides \
./pid/config_overrides/quadrotor_3D/quadrotor_3D_tracking.yaml
```
### Cartpole Stabilization with LQR
```bash
cd ./examples/ # Navigate to the examples folder
python3 lqr/lqr_experiment.py \
--algo lqr \
--task cartpole \
--overrides \
./lqr/config_overrides/cartpole/cartpole_stabilization.yaml \
./lqr/config_overrides/cartpole/lqr_cartpole_stabilization.yaml
```
### 2D Quadrotor Trajectory Tracking with PPO
```bash
cd ./examples/rl/ # Navigate to the RL examples folder
python3 rl_experiment.py \
--algo ppo \
--task quadrotor \
--overrides \
./config_overrides/quadrotor_2D/quadrotor_2D_track.yaml \
./config_overrides/quadrotor_2D/ppo_quadrotor_2D.yaml \
--kv_overrides \
algo_config.training=False
```
### Verbose API Example
```bash
cd ./examples/ # Navigate to the examples folder
python3 no_controller/verbose_api.py \
--task cartpole \
--overrides no_controller/verbose_api.yaml
```
## List of Implemented Controllers
- [PID](https://github.com/utiasDSL/safe-control-gym/blob/main/safe_control_gym/controllers/pid/pid.py)
- [LQR](https://github.com/utiasDSL/safe-control-gym/blob/main/safe_control_gym/controllers/lqr/lqr.py)
- [iLQR](https://github.com/utiasDSL/safe-control-gym/blob/main/safe_control_gym/controllers/lqr/ilqr.py)
- [Linear MPC](https://github.com/utiasDSL/safe-control-gym/blob/main/safe_control_gym/controllers/mpc/linear_mpc.py)
- [GP-MPC](https://github.com/utiasDSL/safe-control-gym/blob/main/safe_control_gym/controllers/mpc/gp_mpc.py)
- [SAC](https://github.com/utiasDSL/safe-control-gym/blob/main/safe_control_gym/controllers/sac/sac.py)
- [PPO](https://github.com/utiasDSL/safe-control-gym/blob/main/safe_control_gym/controllers/ppo/ppo.py)
- [DDPG](https://github.com/utiasDSL/safe-control-gym/blob/main/safe_control_gym/controllers/ddpg/ddpg.py)
- [Safety Layer](https://github.com/utiasDSL/safe-control-gym/tree/main/safe_control_gym/controllers/safe_explorer)
- [RARL](https://github.com/utiasDSL/safe-control-gym/blob/main/safe_control_gym/controllers/rarl/rarl.py)
- [RAP](https://github.com/utiasDSL/safe-control-gym/blob/main/safe_control_gym/controllers/rarl/rap.py)
## List of Implemented Safety Filters
- [MPSC](https://github.com/utiasDSL/safe-control-gym/blob/main/safe_control_gym/safety_filters/mpsc/linear_mpsc.py)
- [CBF](https://github.com/utiasDSL/safe-control-gym/blob/main/safe_control_gym/safety_filters/cbf/cbf.py)
- [Neural Network CBF](https://github.com/utiasDSL/safe-control-gym/blob/main/safe_control_gym/safety_filters/cbf/cbf_nn.py)
## Performance
We compare the sample efficiency of `safe-control-gym` with the original [OpenAI Cartpole][001] and [PyBullet Gym's Inverted Pendulum][002], as well as [`gym-pybullet-drones`][003].
We choose the default physic simulation integration step of each project.
We report performance results for open-loop, random action inputs.
Note that the Bullet engine frequency reported for `safe-control-gym` is typically much finer grained for improved fidelity.
`safe-control-gym` quadrotor environment is not as light-weight as [`gym-pybullet-drones`][003] but provides the same order of magnitude speed-up and several more safety features/symbolic models.
| Environment | GUI | Control Freq. | PyBullet Freq. | Constraints & Disturbances^ | Speed-Up^^ |
| :------------------------: | :----: | :------------: | :-------------: | :-------------------------------: | :-------------: |
| [Gym cartpole][001] | True | 50Hz | N/A | No | 1.16x |
| [InvPenPyBulletEnv][002] | False | 60Hz | 60Hz | No | 158.29x |
| [cartpole][004] | True | 50Hz | 50Hz | No | 0.85x |
| [cartpole][004] | False | 50Hz | 1000Hz | No | 24.73x |
| [cartpole][004] | False | 50Hz | 1000Hz | Yes | 22.39x |
| | | | | | |
| [gym-pyb-drones][003] | True | 48Hz | 240Hz | No | 2.43x |
| [gym-pyb-drones][003] | False | 50Hz | 1000Hz | No | 21.50x |
| [quadrotor][005] | True | 60Hz | 240Hz | No | 0.74x |
| [quadrotor][005] | False | 50Hz | 1000Hz | No | 9.28x |
| [quadrotor][005] | False | 50Hz | 1000Hz | Yes | 7.62x |
> ^ Whether the environment includes a default set of constraints and disturbances
>
> ^^ Speed-up = Elapsed Simulation Time / Elapsed Wall Clock Time; on a 2.30GHz Quad-Core i7-1068NG7 with 32GB 3733MHz LPDDR4X; no GPU
[001]: https://github.com/openai/gym/blob/master/gym/envs/classic_control/cartpole.py
[002]: https://github.com/benelot/pybullet-gym/blob/master/pybulletgym/envs/mujoco/envs/pendulum/inverted_pendulum_env.py
[003]: https://github.com/utiasDSL/gym-pybullet-drones
[004]: https://github.com/utiasDSL/safe-control-gym/blob/main/safe_control_gym/envs/gym_control/cartpole.py
[005]: https://github.com/utiasDSL/safe-control-gym/blob/main/safe_control_gym/envs/gym_pybullet_drones/quadrotor.py
## Run Tests and Linting
Tests can be run locally by executing:
```bash
python3 -m pytest ./tests/ # Run all tests
```
Linting can be run locally with:
```bash
pre-commit install # Install the pre-commit hooks
pre-commit autoupdate # Auto-update the version of the hooks
pre-commit run --all # Run the hooks on all files
```
## References
- Brunke, L., Greeff, M., Hall, A. W., Yuan, Z., Zhou, S., Panerati, J., & Schoellig, A. P. (2022). [Safe learning in robotics: From learning-based control to safe reinforcement learning](https://www.annualreviews.org/doi/abs/10.1146/annurev-control-042920-020211). Annual Review of Control, Robotics, and Autonomous Systems, 5, 411-444.
- Yuan, Z., Hall, A. W., Zhou, S., Brunke, L., Greeff, M., Panerati, J., & Schoellig, A. P. (2022). [safe-control-gym: A unified benchmark suite for safe learning-based control and reinforcement learning in robotics](https://ieeexplore.ieee.org/abstract/document/9849119). IEEE Robotics and Automation Letters, 7(4), 11142-11149.
## Related Open-source Projects
- [`gym-pybullet-drones`](https://github.com/utiasDSL/gym-pybullet-drones): single and multi-quadrotor environments
- [`stable-baselines3`](https://github.com/DLR-RM/stable-baselines3): PyTorch reinforcement learning algorithms
- [`bullet3`](https://github.com/bulletphysics/bullet3): multi-physics simulation engine
- [`gym`](https://github.com/openai/gym): OpenAI reinforcement learning toolkit
- [`casadi`](https://github.com/casadi/casadi): symbolic framework for numeric optimization
- [`safety-gym`](https://github.com/openai/safety-gym): environments for safe exploration in RL
- [`realworldrl_suite`](https://github.com/google-research/realworldrl_suite): real-world RL challenge framework
- [`gym-marl-reconnaissance`](https://github.com/JacopoPan/gym-marl-reconnaissance): multi-agent heterogeneous (UAV/UGV) environments
-----
> University of Toronto's [Dynamic Systems Lab](https://github.com/utiasDSL) / [Vector Institute for Artificial Intelligence](https://github.com/VectorInstitute)