Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/yunjaechoi/reinforment-implementation-on-a-quadruped
Reinforment Implementation on a Quadruped using DDPG(tensorflow), ROS, Gazebo, real quadruped robot.
https://github.com/yunjaechoi/reinforment-implementation-on-a-quadruped
python reinforcement-learning ros-kinetic tensorflow
Last synced: about 5 hours ago
JSON representation
Reinforment Implementation on a Quadruped using DDPG(tensorflow), ROS, Gazebo, real quadruped robot.
- Host: GitHub
- URL: https://github.com/yunjaechoi/reinforment-implementation-on-a-quadruped
- Owner: YunjaeChoi
- License: mit
- Created: 2018-07-02T11:32:05.000Z (over 6 years ago)
- Default Branch: master
- Last Pushed: 2023-08-21T09:12:24.000Z (about 1 year ago)
- Last Synced: 2023-10-20T23:58:19.109Z (about 1 year ago)
- Topics: python, reinforcement-learning, ros-kinetic, tensorflow
- Language: OpenEdge ABL
- Size: 37 MB
- Stars: 60
- Watchers: 3
- Forks: 10
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Reinforcement Learning Implementation on a Quadruped
My project in Korea University.
# Overview
This project is a reinforment learning environment for a quadruped. It uses python 2.7, ROS (robot operating system) Kinetic Kame, Tensorflow, Gazebo simulation. The robot interacts with both Gazebo simulation and the real world using ROS. It is trained to walk forward in Gazebo simulation and can be deployed in a real robot. Deep Deterministic Policy Gradient(DDPG) is used for the robot.
# Files
#### quadruped folder
This folder contains files for reinforment learning environment.
- config/quadruped_control.yaml : joint controller config for Gazebo
- launch/quadruped_control.launch : Spawn quadruped model with controller in Gazebo
- launch/quadruped_gazebo.launch : Spawn quadruped model only in Gazebo
- launch/rvis_display.launch : display quadruped model in Rvis
- rvis/urdf.rvis : rvis config
- src/model/ : tensorflow model files of learned agent
- src/agent_mem.p : agent Replay buffer memory
- src/eps_rewards.npy : agent episode rewards
- src/ddpg.py : Deep Deterministic Policy Gradient agent written in tensorflow
- src/quadruped_env.py : quadruped environment for gazebo simulation
- src/quadruped_learn.py : learn a ddpg agent
- src/quadruped_gazebo_act.py : learned agent takes actions in Gazebo
- src/quadruped_model.py : quadruped environment for real world (no learning, only actions)
- src/quadruped_act.py : learned agent takes actions in a real quadruped
- src/plot_eps_rewards.py : plot episode rewards from eps_rewards.npy
- urdf/materials.xacro : colors etc. for quadruped urdf
- urdf/quadruped_model.xacro : xacro file for quadruped urdf
- worlds/quadruped.world : Gazebo world file
- CMakeLists.txt : ROS CMakeLists file
- package.xml : ROS package file
#### quadruped_imu_and_servo folder
This folder contains files for a real robot implementation. ROS should be installed on the real robot. (Running confirmed on Raspberry pi model B + Ubuntu MATE)
#### quadruped_imu_publisher
- launch/quadruped_imu_publisher_launch.launch : publish IMU readings from adafruit BNO055 9DOF IMU
- src/quadruped_imu_publisher.py : publisher src file
- CMakeLists.txt : ROS CMakeLists file
- package.xml : ROS package file
#### quadruped_servo_subscriber
- launch/quadruped_servo_subscriber_launch.launch : control servo position(PWM) from published servo position messages
- src/quadruped_servo_subscriber.py : subscriber src file
- CMakeLists.txt : ROS CMakeLists file
- package.xml : ROS package file
# Prerequisites
- python 2.7
- ROS Kinetic Kame
- Tensorflow >= 1.4
- Gazebo simulation
### additional installations for simulation
- ROS control
```
sudo apt-get install ros-kinetic-ros-control
sudo apt-get install ros-kinetic-ros-controllers
sudo apt-get install ros-kinetic-gazebo-ros-control
```
- Gazebo simulation update -> update to 7.x (7.0 causes error)
### additional installations for real robot
-
# How to
- config ROS packages (quadruped, quadruped_imu_publisher, quadruped_servo_subscriber)
#### Learning an agent
```
roslaunch quadruped quadruped_control.launch
python quadruped_learn.py
```
#### simulate learned agent
```
roslaunch quadruped quadruped_control.launch
python quadruped_gazebo_act.py
```
#### learned agent on real quadruped
- setup networking for ROS if needed (run roscore on master)
- on quadruped
```
roslaunch quadruped_imu_publisher quadruped_imu_publisher_launch.launch
roslaunch quadruped_servo_subscriber quadruped_servo_subscriber_launch.launch
```
- on computer running tensorflow
```
python quadruped_act.py
```
# Tips
#### always
```
source ~/catkin_ws/devel/setup.bash
```
#### ROS core
roscore
#### ROS via network (local LAN)
- MASTER Setting -> get master_ip_addr via ifconfig (master runs roscore)
```
export ROS_MASTER_URI=http://master_ip_addr:11311
export ROS_IP=master_ip_addr
export ROS_MASTER_URI=http://192.168.0.5:11311
```
- SECOND Setting (from other computer)
```
ssh second_ip_addr -p(port_num)
export ROS_MASTER_URI=http://master_ip_addr:11311
export ROS_IP=second_ip_addr
```
- exiting ssh
```
exit
```
#### xacro to URDF file
```
rosrun xacro xacro --inorder quadruped_model.xacro > model1.urdf
```
# Videos
- simulation
https://youtu.be/jo47bkJQrjU
- real quadruped
https://youtu.be/T0lQ6aDYJkE