https://github.com/barisyazici/deep-rl-grasping
Train deep reinforcement learning model for robotics grasping. Choose from different perception layers raw Depth, RGBD and autoencoder. Test the learned models in different scenes and object datasets
https://github.com/barisyazici/deep-rl-grasping
deep-learning kuka machine-learning open-ai pybullet reinfrocement-learning robotics
Last synced: 3 months ago
JSON representation
Train deep reinforcement learning model for robotics grasping. Choose from different perception layers raw Depth, RGBD and autoencoder. Test the learned models in different scenes and object datasets
- Host: GitHub
- URL: https://github.com/barisyazici/deep-rl-grasping
- Owner: BarisYazici
- License: mit
- Created: 2020-10-08T17:09:01.000Z (over 4 years ago)
- Default Branch: master
- Last Pushed: 2022-07-10T19:40:36.000Z (almost 3 years ago)
- Last Synced: 2023-11-07T17:04:23.484Z (over 1 year ago)
- Topics: deep-learning, kuka, machine-learning, open-ai, pybullet, reinfrocement-learning, robotics
- Language: Python
- Homepage:
- Size: 100 MB
- Stars: 161
- Watchers: 2
- Forks: 40
- Open Issues: 5
-
Metadata Files:
- Readme: README.md
- License: LICENSE.md
Awesome Lists containing this project
README
# Deep Reinforcement Learning on Robotics Grasping
Train robotics model with integrated curriculum learning-based gripper environment. Choose from different perception layers depth, RGB-D. Run pretrained models with SAC, BDQ and DQN algorithms. Test trained algorithms in different scenes and domains.
Master's thesis [PDF](https://github.com/BarisYazici/masters_thesis/blob/master/final_report.pdf)
### Prerequisites (CPU)
Install anaconda. Start a clean conda environment.
```
conda create -n grasp_env python=3.6
conda activate grasp_env
```
python manipulation_main/training/train_stable_baselines.py train --config config/gripper_grasp.yaml --algo SAC --model_dir trained_models/SAC_full --timestep 100000 -v
### Prerequisites (GPU)
```
conda create -n grasp_env python=3.6
conda activate grasp_env
conda install -c conda-forge cudatoolkit=10.0 cudnn=7.6.5
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$CONDA_PREFIX/lib/
```
In [setup.py](setup.py) change:
```
'tensorflow==1.14.0',
to
'tensorflow_gpu==1.14.0',
```
## Installation
Use pip to install the dependencies.
```
pip install -e .
```
If using GPU you can check if it was successful with:
´´´
python -c "import tensorflow as tf; print(tf.config.experimental.list_physical_devices('GPU'))"
´´´
## Run Models
train_stable_baselines script provides the functionality of **running** and **training** models.
For running models *'manipulation_main/training/train_stable_baselines.py'* takes the following arguments
* --model - trained model file e.g trained_models/SAC_full_depth_1mbuffer/best_model/best_model.zip
* -t - use test dataset if not given runs on training dataset
* -v - visualize the model (faster without the -v option)
* -s - run stochastic model if not deterministic
For running functionality *run* sub-parser needs to be passed to the script.
```
python manipulation_main/training/train_stable_baselines.py run --model trained_models/SAC_full_depth_1mbuffer/best_model/best_model.zip -v -t
```
## Train models
For training models *'manipulation_main/training/train_stable_baselines.py'* takes the following arguments
* --config - config file (e.g *'config/simplified_object_picking.yaml'* or *'config/gripper_grasp.yaml'*)
* --algo - algorithm to use(e.g BDQ, DQN, SAC, TRPO)
* --model_dir - name of the folder to host the trained model logs and best performing model on validation set.
* -sh - use shaped reward function (Only makes sense for Full Environment version)
* -v - visualize the model
For training functionality *train* sub-parser needs to be passed to the script.
```
python manipulation_main/training/train_stable_baselines.py train --config config/gripper_grasp.yaml --algo SAC --model_dir trained_models/SAC_full --timestep 100000 -v
```
## Running the tests
To run the gripperEnv related test use
```
pytest tests_gripper
```
* **Domain and Scene Transfer**
* **Different Perception Layers**
* **Ablation Studies**
* **Training Environment**
* **Domain transfer performance**
## Authors
* **Baris Yazici** - *Initial work* - [BarisYazici](https://github.com/BarisYazici)
## Citing the Project
To cite the master's thesis:
```
@MastersThesis{Yazici2020,
author = {Yazici Baris},
title = {{Branch Dueling Deep Q-Networks for Robotics Applications}},
school = {Technical University of Munich},
year = {2020},
howpublished = {\url{https://github.com/BarisYazici/tum_masters_thesis}}
}
```
## License
This project is licensed under the MIT License - see the [LICENSE.md](LICENSE.md) file for details
## Acknowledgments
* Mahmoud Akl (supervisor)
* Breyer Michel (author of https://arxiv.org/abs/1803.04996)
* https://github.com/atavakol/action-branching-agents