Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/pockerman/cuberl
Library for reinforcement learning with c++
https://github.com/pockerman/cuberl
cpp pytorch reinforcement-learning reinforcement-learning-algorithms
Last synced: 17 days ago
JSON representation
Library for reinforcement learning with c++
- Host: GitHub
- URL: https://github.com/pockerman/cuberl
- Owner: pockerman
- Created: 2021-11-06T09:28:52.000Z (about 3 years ago)
- Default Branch: master
- Last Pushed: 2024-07-07T11:27:11.000Z (7 months ago)
- Last Synced: 2024-07-07T12:40:06.728Z (7 months ago)
- Topics: cpp, pytorch, reinforcement-learning, reinforcement-learning-algorithms
- Language: C++
- Homepage: https://pockerman-py-cubeai.readthedocs.io/en/latest/
- Size: 1.65 MB
- Stars: 3
- Watchers: 1
- Forks: 1
- Open Issues: 30
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# cuberl
_cuberl_ is a C++ library containing implementations of various reinforcement learning, filtering and planning algorithms.
The following is an indicative list of examples.
## Examples
### Introductory
- Monte Carlo integration
- Using PyTorch C++ API Part 1
- Using PyTorch C++ API Part 2
- Using PyTorch C++ API Part 3
- Using ```TensorboardServer``` class
- Toy Markov chain
- Importance sampling
- Vector addition with CUDA
### Reinforcement learning
- DummyAgent on ```MountainCar-v0```
- Multi-armed bandit with epsilon-greedy policy
- Multi-armed bandit with soft-max policy
- Advertisement placement
- Iterative policy evaluation on ```FrozenLake-v0```
- Policy iteration on ```FrozenLake-v0```
- Value iteration on ```FrozenLake-v0```
- SARSA on ```CliffWalking-v0```
- Q-learning on ```CliffWalking-v0```
- A2C on ```CartPole-v1```
- DQN on ```Gridworld```
- DQN on ```Gridworld``` with experience replay
- REINFORCE algorithm on ```CartPole-v0```
- Expected SARSA on ```CliffWalking-v0```
- Approximate Monte Carlo on ```MountainCar-v0```
- Monte Carlo tree search on ```Taxi-v3```
- Double Q-learning on ```CartPole-v0```
- First visit Monte Carlo on ```FrozenLake-v0```
### Filtering
- Kalman filtering
- Extended Kalman filter for diff-drive system
### Path planning
- Path planning with rapidly-exploring random trees (TODO)
- Path planning with dynamic windows (TODO)
- A* search on a road network from Open Street Map data
## Installation
The cubeai library has a host of dependencies:
- A compiler that supports C++20 e.g. g++-11
- Boost C++
- CMake >= 3.6
- Eigen
- Gtest (if configured with tests)
In addition, the library also incorporates, see ```(include/cubeai/extern)```, the following libraries (you don't need to install these):
- HTTPRequest
- better-enums
- nlohmann/json
### Enabling PyTorch and CUDA
_cuberl_ can be complied with CUDA and/or PyTorch support. If PyTorch has been compiled using CUDA support, then
you need to enable CUDA as well. In order to do so set the flag _USE_CUDA_ in the _CMakeLists.txt_ to _ON_.
_cuberl_ assumes that PyTorch is compiled with the C++11 ABI.
### Documentation dependencies
There are extra dependencies if you want to generate the documentation. Namely,
- Doxygen
- Sphinx
- sphinx_rtd_theme
- breathe
- m2r2
Note that if Doxygen is not found on your system CMake will skip this. On a Ubuntu/Debian based machine, you can install
Doxygen using
```bash
sudo apt-get install doxygen
```
Similarly, install ```sphinx_rtd_theme``` using
```bash
pip install sphinx_rtd_theme
```
Install ```breathe``` using
```bash
pip install breathe
```
Install ```m2r2``` using
```bash
pip install m2r2
```
## Issues
#### undefined reference to ```[email protected]```.
You may want to check with ```nvidia-msi``` your CUDA Version and make sure it is compatible with the PyTorch library you are linking against
#### TypeError: Descriptors cannot be created directly.
This issue may be occur when using the TensorBoardServer in _cuberl_.
This issue is related with an issue with _protobuf_. See: https://stackoverflow.com/questions/72441758/typeerror-descriptors-cannot-not-be-created-directly for
possible solutions.