https://github.com/leggedrobotics/rsl_rl
A fast and simple implementation of learning algorithms for robotics.
https://github.com/leggedrobotics/rsl_rl
Last synced: 4 months ago
JSON representation
A fast and simple implementation of learning algorithms for robotics.
- Host: GitHub
- URL: https://github.com/leggedrobotics/rsl_rl
- Owner: leggedrobotics
- License: other
- Created: 2021-10-18T13:00:35.000Z (over 4 years ago)
- Default Branch: main
- Last Pushed: 2026-02-17T13:33:14.000Z (5 months ago)
- Last Synced: 2026-02-17T18:27:01.036Z (5 months ago)
- Language: Python
- Homepage: https://pypi.org/project/rsl-rl-lib/
- Size: 232 KB
- Stars: 2,234
- Watchers: 37
- Forks: 519
- Open Issues: 13
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Citation: CITATION.cff
Awesome Lists containing this project
- awesome-isaac-gym - RSL RL
- awesome-deep-reinforcement-learning - leggedrobotics/rsl_rl - commit/leggedrobotics/rsl_rl?label=last%20update) (Reinforcement Learning (RL) and Deep Reinforcement Learning (DRL) / RL/DRL Algorithm Implementations and Software Frameworks)
README
# RSL RL
Fast and simple implementation of RL algorithms, designed to run fully on GPU.
This code is an evolution of `rl-pytorch` provided with NVIDIA's Isaac GYM.
Environment repositories using the framework:
* **`Isaac Lab`** (built on top of NVIDIA Isaac Sim): https://github.com/isaac-sim/IsaacLab
* **`Legged-Gym`** (built on top of NVIDIA Isaac Gym): https://leggedrobotics.github.io/legged_gym/
The main branch supports **PPO** and **Student-Teacher Distillation** with additional features from our research. These include:
* [Random Network Distillation (RND)](https://proceedings.mlr.press/v229/schwarke23a.html) - Encourages exploration by adding
a curiosity driven intrinsic reward.
* [Symmetry-based Augmentation](https://arxiv.org/abs/2403.04359) - Makes the learned behaviors more symmetrical.
We welcome contributions from the community. Please check our contribution guidelines for more
information.
**Maintainer**: Mayank Mittal and Clemens Schwarke
**Affiliation**: Robotic Systems Lab, ETH Zurich & NVIDIA
**Contact**: cschwarke@ethz.ch
> **Note:** The `algorithms` branch supports additional algorithms (SAC, DDPG, DSAC, and more). However, it isn't currently actively maintained.
## Setup
The package can be installed via PyPI with:
```bash
pip install rsl-rl-lib
```
or by cloning this repository and installing it with:
```bash
git clone https://github.com/leggedrobotics/rsl_rl
cd rsl_rl
pip install -e .
```
The package supports the following logging frameworks which can be configured through `logger`:
* Tensorboard: https://www.tensorflow.org/tensorboard/
* Weights & Biases: https://wandb.ai/site
* Neptune: https://docs.neptune.ai/
For a demo configuration of PPO, please check the [dummy_config.yaml](config/dummy_config.yaml) file.
## Contribution Guidelines
For documentation, we adopt the [Google Style Guide](https://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_google.html) for docstrings. Please make sure that your code is well-documented and follows the guidelines.
We use the following tools for maintaining code quality:
- [pre-commit](https://pre-commit.com/): Runs a list of formatters and linters over the codebase.
- [black](https://black.readthedocs.io/en/stable/): The uncompromising code formatter.
- [flake8](https://flake8.pycqa.org/en/latest/): A wrapper around PyFlakes, pycodestyle, and McCabe complexity checker.
Please check [here](https://pre-commit.com/#install) for instructions to set these up. To run over the entire repository, please execute the following command in the terminal:
```bash
# for installation (only once)
pre-commit install
# for running
pre-commit run --all-files
```
## Citing
**We are working on writing a white paper for this library.** Until then, please cite the following work
if you use this library for your research:
```text
@InProceedings{rudin2022learning,
title = {Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning},
author = {Rudin, Nikita and Hoeller, David and Reist, Philipp and Hutter, Marco},
booktitle = {Proceedings of the 5th Conference on Robot Learning},
pages = {91--100},
year = {2022},
volume = {164},
series = {Proceedings of Machine Learning Research},
publisher = {PMLR},
url = {https://proceedings.mlr.press/v164/rudin22a.html},
}
```
If you use the library with curiosity-driven exploration (random network distillation), please cite:
```text
@InProceedings{schwarke2023curiosity,
title = {Curiosity-Driven Learning of Joint Locomotion and Manipulation Tasks},
author = {Schwarke, Clemens and Klemm, Victor and Boon, Matthijs van der and Bjelonic, Marko and Hutter, Marco},
booktitle = {Proceedings of The 7th Conference on Robot Learning},
pages = {2594--2610},
year = {2023},
volume = {229},
series = {Proceedings of Machine Learning Research},
publisher = {PMLR},
url = {https://proceedings.mlr.press/v229/schwarke23a.html},
}
```
If you use the library with symmetry augmentation, please cite:
```text
@InProceedings{mittal2024symmetry,
author={Mittal, Mayank and Rudin, Nikita and Klemm, Victor and Allshire, Arthur and Hutter, Marco},
booktitle={2024 IEEE International Conference on Robotics and Automation (ICRA)},
title={Symmetry Considerations for Learning Task Symmetric Robot Policies},
year={2024},
pages={7433-7439},
doi={10.1109/ICRA57147.2024.10611493}
}
```