https://github.com/catrinlam/gesture_localisation_robot
https://github.com/catrinlam/gesture_localisation_robot
Last synced: about 1 year ago
JSON representation
- Host: GitHub
- URL: https://github.com/catrinlam/gesture_localisation_robot
- Owner: catrinlam
- Created: 2023-11-10T11:37:14.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2023-12-04T21:06:00.000Z (over 2 years ago)
- Last Synced: 2025-02-14T05:34:55.882Z (over 1 year ago)
- Language: JavaScript
- Homepage: http://catrin.is-a.dev/gesture_localisation_robot/
- Size: 41.3 MB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# GestureCommand: A Simulated Camera-Based Gesture Recognition System for Autonomous Table-Specific Delivery Robot
`Gesture-Localisaton-Robot` is a package of camera based hand gesture robot control system. This is the official repository of the final team assignment of Intelligent Robotics Module at University of Birmingham.
A demonstration can be viewed on our [Project website](https://winter7eaf.github.io/gesture_localisation_robot/).
| Robot going Table 5 | Ordering robot to Table 5 using mediapipe |
|:--------------------------------:|:-----------------------------------------:|
|  |  |
## Contributor
Chit Lee ([Github](https://github.com/chit-uob))
Juni Katsu ([Github](https://github.com/JuniJoo))
Cheuk Yu Lam ([Github](https://github.com/winter7eaf))
Abbas Mandasorwala ([Github](https://github.com/abbas-119))
Kozerenko Elizaveta ([Github](https://github.com/Lizzzzzok))
## Motivation
Imagining you are a barista in a coffee shop. You just made a cup of coffee, and you want it delivered to a customer at a certain table. You have a robot, and you put the cup of coffee on top of it. Your hands are not clean, so it will be problematic to press a touchscreen. How can you tell the robot where to go?
Introducing GestureCommand, a camera-based gesture recognition system for autonomous table-specific delivery robot. You can gesture the robot where to go, and it will go there.
## Installation
### Ideal Working Environment
- Ubuntu 20.04
- [ROS Noetic](http://wiki.ros.org/noetic/Installation/Ubuntu)
(desktop-full)
- Python 3.8
### Install ROS Noetic
[Install ROS Noetic](http://wiki.ros.org/ROS/Installation/TwoLineInstall/).
[Set up your catkin workspace](https://wiki.ros.org/catkin/Tutorials/create_a_workspace).
### Install dependencies
Before install everything, run the upgrade command so your system stays update to date.
- `sudo apt upgrade && sudo apt update`
- `sudo apt install ros-$ROS_DISTRO-pr2-teleop ros-$ROS_DISTRO-map-server`.
This package mainly relies on two libraries: [Mediapipe Machine
Learning library](https://github.com/google/mediapipe) developed by Google and [OpenCV](https://github.com/opencv/opencv) Open
source Computer Vision library (for real time hand detection).
- `pip install mediapipe`
- `pip install opencv-python`
### Clone our repository and build
Git clone this repo to your `/src`
Run `catkin_make`
### Compile laser_trace
* Compile laser_trace.cpp (provides laser ray tracing) as follows **if you are not using arm system(windows, unix...)**:
cd /src/gesture_localisation_robot/src/laser_trace
./compile.sh #You may have to '''chmod +x compile.sh'''
* replace `./compile.sh` with `./compilearm.sh` **if you are using arm system(m1 chip mac)**:
If correctly compiled, you should find `laser_trace.so` in the directory `/src/gesture_localisation_robot/src/pf_localisation`.
If the ***code does not compile*** you need to install PythonBoost from https://github.com/boostorg/python. This requires the download and compiling of Boost and installation of Faber.
### Make scripts executable
You may need to make the varies scripts executable by running `chmod +x {filename}` in the directory `/src/gesture_localisation_robot/scripts`.
## Running the Code
Run `roslaunch Gesture-Localisation-Robot everything.launch`
This should start
- the map server
- the robot simulator stage_ros
- rviz for visualisation
- pf_localisation for particle filter localisation
- move_to_coords.py for the local planner which moves the robot to the goal coordinates
- hand_track_control.py for the hand tracking and gesture recognition
- initial_pose_publisher.py for publishing the initial pose of the robot
This will allow you to input hand gesture from 0 to 5. 0 is corresponding the Till, and 1 to 5 to Tables respectfully. Hold you hand still about 3 seconds, and the robot should start heading to the ordered table number.