{"id":28563809,"url":"https://github.com/geonhee-lee/ur-reaching-reinforcement-learning","last_synced_at":"2025-06-10T13:08:39.694Z","repository":{"id":104698750,"uuid":"205483744","full_name":"Geonhee-LEE/ur-reaching-reinforcement-learning","owner":"Geonhee-LEE","description":"Reinforcement learning using rlkit, UR5, Robotiq gripper on ROS(Robot Operating System)","archived":false,"fork":false,"pushed_at":"2021-06-23T04:18:18.000Z","size":51584,"stargazers_count":94,"open_issues_count":4,"forks_count":21,"subscribers_count":4,"default_branch":"master","last_synced_at":"2023-10-19T23:41:44.764Z","etag":null,"topics":["gazebo","manipulator-robotics","moveit","moveit-api","reaching-task","reinforcement-learning","rlkit","ur5"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Geonhee-LEE.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2019-08-31T02:24:29.000Z","updated_at":"2023-10-20T19:58:43.074Z","dependencies_parsed_at":null,"dependency_job_id":"16978f29-99bf-4dbe-8a32-b32ee0c8b7d8","html_url":"https://github.com/Geonhee-LEE/ur-reaching-reinforcement-learning","commit_stats":null,"previous_names":[],"tags_count":0,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Geonhee-LEE%2Fur-reaching-reinforcement-learning","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Geonhee-LEE%2Fur-reaching-reinforcement-learning/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Geonhee-LEE%2Fur-reaching-reinforcement-learning/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Geonhee-LEE%2Fur-reaching-reinforcement-learning/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Geonhee-LEE","download_url":"https://codeload.github.com/Geonhee-LEE/ur-reaching-reinforcement-learning/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Geonhee-LEE%2Fur-reaching-reinforcement-learning/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":259081012,"owners_count":22802404,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["gazebo","manipulator-robotics","moveit","moveit-api","reaching-task","reinforcement-learning","rlkit","ur5"],"created_at":"2025-06-10T13:08:22.858Z","updated_at":"2025-06-10T13:08:39.680Z","avatar_url":"https://github.com/Geonhee-LEE.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Object tracking video using Reinforcement learning  \n\n[![Object tracking video using Reinforcement learning](http://img.youtube.com/vi/A3mYJT2g-us/0.jpg)](https://youtu.be/A3mYJT2g-us?t=0s)  \n\n# How to launch original env\n First launch the simulator\n  \n  ``` \n  roslaunch ur_robotiq_gazebo gym.launch\n  ```\n \n And run the training launch\n  ```\n  roslaunch ur_training default.launch\n  ```\n\n## Conveyer GAZEBO env\n\nFirst launch the gazebo and gym interface and node publishing block point.\n ```\n roslaunch ur_robotiq_gazebo conveyer_gym.launch --screen\n ```\n \n Run the RL algorithms and unpause the GAZEBO\n  ```\n  roslaunch ur_training default.launch\n  ```\n \n\n\u003e Latest block's point:\n``` \nrostopic echo /target_blocks_pose\n```\n\n\u003e Total block's points:\n``` \nrostopic echo /blocks_poses \n```\n\n\n# How to launch REINFORCE algorithm\n First launch the simulator\n  \n  ``` \nroslaunch ur_robotiq_gazebo conveyer_gym.launch controller:=vel --screen gui:=false\n  ```\n \n And load the parameters and launch python file for reset\n  ```\nroslaunch ur_reaching reinforcement.launch\n  ```\n\n And start the learning algorithm \n  ```\npython reinforcement_main.py \n  ```\n\n And unpause physics of GAZEBO simulator\n ```\n rosservice call /gazebo/unpause_physics \"{}\"\n ```\n\n\n\n# How to launch PPO+GAE algorithm\n\nFirst launch the simulator including loading the parameters and GAZEBO Excution func\n```\nroslaunch ur_robotiq_gazebo conveyer_gym.launch --screen gui:=false\n```\n\n And start the learning algorithm \n ```\n  python ppo_gae_main.py\n ```\n\n\n# How to use the RLkit \n[RLkit](https://github.com/vitchyr/rlkit) is reinforcement learning framework based on [rllab](https://github.com/rll/rllab)\n\n## Run GAZEBO simulator and gazebo_execution \nFirst launch the simulator including loading the parameters and GAZEBO Excution func\n```\nroslaunch ur_robotiq_gazebo conveyer_gym.launch --screen gui:=false\n```\n\n## Training\n And start the SAC learning algorithm based on RLkit\n ```\n  python rlkit_sac_main.py\n ```\n\n And unpause physics of GAZEBO simulator\n ```\n rosservice call /gazebo/unpause_physics \"{}\"\n ```\n \n After training, you may find the pickled files on the rlkit/data folder.\n \n you can easily see the results through selecting the generated folder about training like follwing:\n \n ```\n python viskit/frontend.py ../rlkit/data/SAC/SAC_2019_10_14_08_27_55_0000--s-0/\n```\n\n## Evaluation\n\nIf you want to evaluate the trained weight, you can run like following:\n```\npython rlkit/scripts/run_policy.py rlkit/data/SAC/SAC_2019_10_14_08_27_55_0000--s-0/params.pkl \n```\n\n\n## Visualization\n\n- Rviz\n```\nroslaunch ur_robotiq_moveit_config  moveit_rviz.launch \n```\n\n- rviz_visual_tools\n```\nrosrun visual_tools rviz_visual_tools_demo\n```\n\n![Visualization](../images/rviz_visual_tools.png)\n\n- moveit_visual_tools\n```\nrosrun visual_tools moveit_visual_tools_demo\n```\n\n\n\n\n## Services\n\n- _/gazebo/unpause_physics_ ([Empty](http://docs.ros.org/melodic/api/std_srvs/html/srv/Empty.html))\n  - Unpause GAZEBO simulator for starting\n  ```\n  rosservice call /gazebo/unpause_physics \"{}\"\n  ```\n- _/set_velocity_controller_ ([SetBool](http://docs.ros.org/melodic/api/std_srvs/html/srv/SetBool.html))\n  -  Set velocity controllers including 6 velocity controllers after stoppint velocity_controller/JointTrajectoryController\n- _/set_trajectory_velocity_controller_ ([SetBool](http://docs.ros.org/melodic/api/std_srvs/html/srv/SetBool.html))\n  -  Set velocity trajectory controller including Joint trajectory controllers after stopping velocity_controller/JointVelocityController\n- _/stop_training_ ([SetBool](http://docs.ros.org/melodic/api/std_srvs/html/srv/SetBool.html))\n  -  Stop training process\n- _/start_training_ ([SetBool](http://docs.ros.org/melodic/api/std_srvs/html/srv/SetBool.html))\n  -  Start training process\n\n\n## Helpful function\nWe can check the controller manager status using following:\n```\nrosrun rqt_controller_manager rqt_controller_manager\n```\n\n\n![controller manager](../images/controller_manager.png)\n\n# Universal Robot\n\n\n__Usage with real Hardware__  \nThere are launch files available to bringup a real robot - either UR5 or UR10.  \nIn the following the commands for the UR5 are given. For the UR10, simply replace the prefix accordingly.\n\nDon't forget to source the correct setup shell files and use a new terminal for each command!   \n\nTo bring up the real robot, run:\n\n```roslaunch ur_modern_driver ur5_bringup.launch robot_ip:=192.168.0.3```\n\n\n__MoveIt! with real Hardware and gripper__  \nTo run the MoveIt with gripper, run\n\n```roslaunch total_moveit_config total_moveit_planning_execution.launch ```\n\nMoveIt API Python execution!\n\n```rosrun ur_robotiq_moveit_api ur_robotiq_moveit.py  ```\n\nA simple test script that moves the robot to predefined positions can be executed like this:\n\n```rosrun ur_driver test_move.py```\n\n\nCAUTION:  \nRemember that you should always have your hands on the big red button in case there is something in the way or anything unexpected happens.\n\n\n__Usage with Gazebo Simulation__  \nThere are launch files available to bringup a simulated robot - either UR5 or UR10.  \nIn the following the commands for the UR5 are given. For the UR10, simply replace the prefix accordingly.\n\nDon't forget to source the correct setup shell files and use a new terminal for each command!   \n\nTo bring up the simulated robot in Gazebo, run:\n\n```roslaunch ur_gazebo ur5.launch```\n\n\n__MoveIt! with a simulated robot__  \nAgain, you can use MoveIt! to control the simulated robot.  \n\nFor setting up the MoveIt! nodes to allow motion planning run:\n\n```roslaunch ur5_moveit_config ur5_moveit_planning_execution.launch sim:=true```\n\nFor starting up RViz with a configuration including the MoveIt! Motion Planning plugin run:\n\n```roslaunch ur5_moveit_config moveit_rviz.launch config:=true```\n\n\nNOTE:  \nAs MoveIt! seems to have difficulties with finding plans for the UR with full joint limits [-2pi, 2pi], there is a joint_limited version using joint limits restricted to [-pi,pi]. In order to use this joint limited version, simply use the launch file arguments 'limited', i.e.:  \n\n```roslaunch ur_gazebo ur5.launch limited:=true```\n\n```roslaunch ur5_moveit_config ur5_moveit_planning_execution.launch sim:=true limited:=true```\n\n```roslaunch ur5_moveit_config moveit_rviz.launch config:=true```\n\n\n\n---\n\n\n__MoveIt! with real Hardware__  \nAdditionally, you can use MoveIt! to control the robot.  \nThere exist MoveIt! configuration packages for both robots.  \n\nFor setting up the MoveIt! nodes to allow motion planning run:\n\n```roslaunch ur5_moveit_config ur5_moveit_planning_execution.launch```\n\nFor starting up RViz with a configuration including the MoveIt! Motion Planning plugin run:\n\n```roslaunch ur5_moveit_config moveit_rviz.launch config:=true```\n\n\nNOTE:  \nAs MoveIt! seems to have difficulties with finding plans for the UR with full joint limits [-2pi, 2pi], there is a joint_limited version using joint limits restricted to [-pi,pi]. In order to use this joint limited version, simply use the launch file arguments 'limited', i.e.:  \n\n```roslaunch ur_bringup ur5_bringup.launch limited:=true robot_ip:=IP_OF_THE_ROBOT [reverse_port:=REVERSE_PORT]```\n\n```roslaunch ur5_moveit_config ur5_moveit_planning_execution.launch limited:=true```\n\n```roslaunch ur5_moveit_config moveit_rviz.launch config:=true```\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgeonhee-lee%2Fur-reaching-reinforcement-learning","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgeonhee-lee%2Fur-reaching-reinforcement-learning","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgeonhee-lee%2Fur-reaching-reinforcement-learning/lists"}