Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/ahmedheakl/multi-level-rl-for-robotics
A library for training robots using RL under the scheme of multi-level RL.
https://github.com/ahmedheakl/multi-level-rl-for-robotics
continuous-integration deep-reinforcement-learning gym-environment pyglet python reinforcement-learning robotics unittest
Last synced: about 1 month ago
JSON representation
A library for training robots using RL under the scheme of multi-level RL.
- Host: GitHub
- URL: https://github.com/ahmedheakl/multi-level-rl-for-robotics
- Owner: ahmedheakl
- License: mit
- Created: 2022-08-09T17:49:12.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2023-04-20T00:20:01.000Z (over 1 year ago)
- Last Synced: 2024-10-08T09:57:41.158Z (3 months ago)
- Topics: continuous-integration, deep-reinforcement-learning, gym-environment, pyglet, python, reinforcement-learning, robotics, unittest
- Language: Python
- Homepage:
- Size: 6.63 MB
- Stars: 8
- Watchers: 2
- Forks: 0
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
[](https://github.com/ahmedheakl/Mutli_Level_RL_Robotics/actions)
# Highrl: Multi-level Reinforcement Learning for Robotics Navigation
Highrl is a library for training robots using RL under the scheme of multi-level RL. The library has numerous features from generating random environment, training agents to generate curriculum learning schemes, or train robots in pre-defined environments.
The robot can explore a synthetic [gym](https://www.gymlibrary.dev/) environment using lidar vision (either in flat or rings format). The robot is trained to reach the goal with obstacles in between to hinder its movement and simulate the real-world environment. A teacher (an RL agent) will be trained to synthesize the perfect curriculum learning for the robot, so that the robot will solve maps with certain difficulties in minimal time.
The robot model is implemented with a [CNN](https://en.wikipedia.org/wiki/Convolutional_neural_network) for the feature extractor and an [MLP](https://en.wikipedia.org/wiki/Multilayer_perceptron) for both the value and policy networks. The teacher model is implemented with an LSTM network for the feature extractor and an MLP for value/policy network. The robot model is fed with the lidar data and outputs the velocity ```(vx, vy)``` of the robot. The teacher model is fed with data of the last session for the robot that the teacher is training, and outputs the configurations for the next environment to train the robot. At each step of the teacher, a new robot will be generated with probability of 10% and will be trained for a fixed number of steps. You can find the models in ```src/highrl/policy```.
## Installation
Please note that the library is **only tested on Linux distributions**. If you want to install it on Windows, you can use [WSL](https://learn.microsoft.com/en-us/windows/wsl/install).
Use the package manager [pip](https://pip.pypa.io/en/stable/) to install higrl library.
```bash
pip install highrl
```
## Usage
```bash
highrl -h # get available arguments
```
### Configurations
**--robot-config**: path of configuration file of robot environment (relative path).
**--teacher-config**: path of configuration file of teacher environment (relative path).
**--mode**: choose train or test mode.
**--env-mode**: choose whether to train the robot alone or in the presence of a teacher to generate curriculum learning.
**--render-each**: the frequency of rendering for robot environment (integer).
**--output-path**: relative path to output results for robot mode.
**--lidar-mode**: mode to process lidar flat=1D, rings=2D.
### Example
```bash
highrl --render-each=50 --output-dir=~/Desktop
```
## Contributing
Pull requests are welcome. For major changes, please open an issue first
to discuss what you would like to change.
Please make sure to update tests as appropriate.
## License
[MIT](https://choosealicense.com/licenses/mit/)