{"id":23174077,"url":"https://github.com/zebradevs/fetch_pbd","last_synced_at":"2026-01-19T03:33:05.504Z","repository":{"id":89252014,"uuid":"67728278","full_name":"ZebraDevs/fetch_pbd","owner":"ZebraDevs","description":"Programming by Demonstration for Fetch","archived":false,"fork":false,"pushed_at":"2017-08-08T20:52:04.000Z","size":1828,"stargazers_count":15,"open_issues_count":5,"forks_count":6,"subscribers_count":24,"default_branch":"master","last_synced_at":"2025-04-05T00:29:57.163Z","etag":null,"topics":["indigo","open-source"],"latest_commit_sha":null,"homepage":null,"language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ZebraDevs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-09-08T18:11:59.000Z","updated_at":"2024-11-02T02:53:30.000Z","dependencies_parsed_at":null,"dependency_job_id":"ad28a52f-4c4f-48d3-8abd-4334b3b4c9c6","html_url":"https://github.com/ZebraDevs/fetch_pbd","commit_stats":null,"previous_names":[],"tags_count":8,"template":false,"template_full_name":null,"purl":"pkg:github/ZebraDevs/fetch_pbd","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ZebraDevs%2Ffetch_pbd","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ZebraDevs%2Ffetch_pbd/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ZebraDevs%2Ffetch_pbd/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ZebraDevs%2Ffetch_pbd/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ZebraDevs","download_url":"https://codeload.github.com/ZebraDevs/fetch_pbd/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ZebraDevs%2Ffetch_pbd/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28560363,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-19T03:31:16.861Z","status":"ssl_error","status_checked_at":"2026-01-19T03:31:15.069Z","response_time":67,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["indigo","open-source"],"created_at":"2024-12-18T05:19:06.225Z","updated_at":"2026-01-19T03:33:05.491Z","avatar_url":"https://github.com/ZebraDevs.png","language":"HTML","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Fetch Programming by Demonstration [![Build Status](https://api.travis-ci.org/fetchrobotics/fetch_pbd.png)](https://travis-ci.org/fetchrobotics/fetch_pbd)\n\nTODO: Travis build status and coveralls testing?\n\nThis repository is based on PR2 Programming by Demonstration. This version is for the Fetch Robot.\n\nThe original [PR2 Programming by Demonstration](https://github.com/PR2/pr2_pbd) was done by [Maya Cakmak](http://www.mayacakmak.com/) and the [Human-Centered Robotics Lab](https://hcrlab.cs.washington.edu/) at the University of Washington.\n\n## System Requirements\nThis PbD is designed for Ubuntu 14.04 and ROS Indigo.\n\n## Installing from Source\nClone this repository and build on the robot:\n```bash\ncd ~/catkin_ws/src\ngit clone https://github.com/fetchrobotics/fetch_pbd.git\ncd ~/catkin_ws\ncatkin_make\n```\nTo make sure all dependencies are installed:\n```bash\ncd ~/catkin_ws\nsource ~/devel/setup.bash\nrosdep update\nrosdep install --from-paths src --ignore-src --rosdistro=indigo -y\n```\n## Making Changes to the Web Interface\nTo make changes to the web interface, you will want to install all of the web dependencies:\n```bash\nroscd fetch_pbd_interaction\ncd web_interface/fetch-pbd-gui\n./install_web_dependencies.sh\n```\nNote that the above adds the following paths to the end of your `~/.bashrc`:\n```bash\nexport NVM_DIR=\"/home/USERNAME/.nvm\"\n[ -s \"$NVM_DIR/nvm.sh\" ] \u0026\u0026 . \"$NVM_DIR/nvm.sh\"  # This loads nvm\n```\nYou will want to source your `~/.bashrc` file again before running the software.\nNow all of the dependencies are installed. After you make changes to the web interface, to build you can run:\n```bash \nroscd fetch_pbd_interaction\ncd web_interface/fetch-pbd-gui\n./build_frontend.sh\n```\nNow when you roslaunch this software it will run using your newly built changes!\n\n## Running\n### Commands on Fetch Robot\n#### Terminal #1\n```bash\nsource ~/catkin_ws/devel/setup.bash\nroslaunch fetch_pbd_interaction pbd.launch\n```\nYou can run the backend without the \"social gaze\" head movements or without the sounds by passing arguments to the launch file:\n```bash\nsource ~/catkin_ws/devel/setup.bash\nroslaunch fetch_pbd_interaction pbd.launch social_gaze:=false play_sound:=false\n```\n\nYou can also pass arguments to the launch file to save your actions to a json file or load them from a json file.\nThis behaviour is a bit complicated. It is recommended that you specify the full path to files or else it will look in your .ros folder.\nIf you specify a from_file then actions will be loaded from that file. They will replace the ones in your session database.\nWhatever was in your session database will get stored in a timestamped file in your .ros folder (not overwritten).\nIf you specify a to_file then the session you are starting will be saved to that file.\n```bash\nsource ~/catkin_ws/devel/setup.bash\nroslaunch fetch_pbd_interaction pbd.launch from_file:=/full/path/from.json to_file:=/full/path/to.json\n```\n\n### Using the GUI\nIn your browser go to ROBOT_HOSTNAME:8080 in your browser to use the GUI. There is a \"mobile\" interface, which is the same but without the visualization. This can be useful for just saving/deleting primitives. \n\nThe main page lists all the available actions.\n![Main page](https://cloud.githubusercontent.com/assets/1470402/17989388/c71a3da2-6ae1-11e6-9d2f-894a67e508ca.png)\nYou can directly run/copy/delete actions from the main page. Or hit the \"Edit\" button to see more information on that action.\n\nOn the \"Current Action\" screen, most of the buttons are pretty self-explanatory. You can execute the entire action using the \"Run\" button at the bottom of the screen. This will execute all of the primitives in the order they appear in the Primitive List. You can click on a specific primitive (either the marker or the list item), to highlight the primitive.\n![](https://cloud.githubusercontent.com/assets/1470402/17989398/d0602b2e-6ae1-11e6-8add-edcedf6285b6.png)\n\nYou can show/hide the markers for each primitive by clicking the marker icon for the primitive in the Primitive List.\n![](https://cloud.githubusercontent.com/assets/1470402/17989394/d05bf02c-6ae1-11e6-9446-9847bbd419ea.png)\n\nYou can change the order of the primitives by dragging them to a new position in the list.\n![](https://cloud.githubusercontent.com/assets/1470402/17989397/d0608290-6ae1-11e6-98a4-bbb1049e1185.png)\n\nYou can edit the position and orientation of certain primitives by clicking the edit icon or by moving the interactive marker.\n![](https://cloud.githubusercontent.com/assets/1470402/17989393/d05b87ea-6ae1-11e6-85d7-922c6dc4844a.png)\n\nYou can change the frame that certain primitives are relative to by right-clicking the marker.\n![](https://user-images.githubusercontent.com/1470402/28805009-d7e45398-761b-11e7-88e2-57898dde0a99.png)\n\nYou can also change the name of the action.\n![](https://cloud.githubusercontent.com/assets/1470402/17989396/d05f44d4-6ae1-11e6-9363-f242c5ea15b6.png)\n\nYou can record objects in the environment using the \"Record Objects\" button. The objects will appear in the viewer. Poses can be relative to these objects in the environment.\n![](https://user-images.githubusercontent.com/1470402/28805010-d7e6513e-761b-11e7-9fc8-b63c0e517571.png)\n![](https://user-images.githubusercontent.com/1470402/28805011-d7edb910-761b-11e7-81a9-6e1f9c2f0be8.png)\n\n### Grasping in Fetch PbD\nYou can now (optionally) run Fetch PbD with a grasp suggestion service. This is a service that takes in a point cloud for an object and returns a PoseArray of possible grasps for that object. Fetch Pbd provides feedback by publishing which grasp was chosen by the user. Any service that adheres to the .srv interface defined by [SuggestGrasps.srv](https://github.com/GT-RAIL/rail_manipulation_msgs/blob/develop/srv/SuggestGrasps.srv) and optionally the feedback .msg interface, [GraspFeedback.msg](https://github.com/GT-RAIL/rail_manipulation_msgs/blob/develop/msg/GraspFeedback.msg), can be used. If you want to start Fetch PbD with a grasp suggestion service:\n```bash\nsource ~/catkin_ws/devel/setup.bash\nroslaunch fetch_pbd_interaction pbd.launch grasp_suggestion_service:=grasp_service_name grasp_feedback_topic:=grasp_feedback\n```\nThe grasp_feedback_topic is optional and can be excluded. Then in the interface you can right-click on objects in the scene and add a grasp for that object. Initially a grasp primitive is generated that is just a placeholder and does not specify any poses. To use the grasp suggestion service to generate grasps, you then right click the blue placeholder box and select \"Generate grasps\". The service will return a list of grasp poses. Fetch PbD sets a pre-grasp that is 15 cm away from the grasp. The first grasp generated is shown and you can switch to other grasp options by right-clicking the grasp marker and selecting one from the list. \n\n![](https://user-images.githubusercontent.com/1470402/28805083-48681a96-761c-11e7-9921-2e9b7e07e318.png)\n![](https://user-images.githubusercontent.com/1470402/28805082-4866964e-761c-11e7-80bd-b67ebd9cc4b2.png)\n![](https://user-images.githubusercontent.com/1470402/28805081-48664284-761c-11e7-91b2-89958b14eab3.png)\n\nGrasps are executed like any other primitve and can be added to actions in combination with other primitives.\n\n### Code Interface\nYou can also access the actions you've programmed through code. You still need to run pbd_backend.launch. \n\n#### Commands on Fetch\n```bash\nsource ~/catkin_ws/devel/setup.bash\nrosrun fetch_pbd_interaction demo.py\n```\n\n## System Overview\n### Interaction Node\nThe pbd_interaction_node.py handles the interaction between speech/GUI and the rest of the system. Changes happen through the update loop in interaction.py and also through the callbacks from speech/GUI commands. interaction.py also subscribes to updates from the pbd_world_node.py, which notifies it of changes in objects in the world. Through callbacks and the update loop, interaction.py hooks in to session.py. session.py handles creating actions and primitives and saving them to the database.\n\n### Arm Control Node\nThe pbd_arm_control_node.py is how the robot's arm is controlled to execute actions/primitives. It provides a lower level service interface to move the arm. The interaction node interacts with this through the interface in robot.py.\n\n### World Node\nThe pbd_world_node.py handles the robot's perception of the world. Other nodes ask the world node about the state of the world and can both send and subscribe to updates to the world. Its main function is to provide a list of objects currently in the scene.\n\n### Social Gaze Node\nThe social_gaze_server.py handles the movements of the robot's head. This is also controlled through the robot.py interface. The sounds are also provided through this interface.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzebradevs%2Ffetch_pbd","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzebradevs%2Ffetch_pbd","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzebradevs%2Ffetch_pbd/lists"}