Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/sensorsini/dextra-roshambo-python
https://github.com/sensorsini/dextra-roshambo-python
Last synced: about 1 month ago
JSON representation
- Host: GitHub
- URL: https://github.com/sensorsini/dextra-roshambo-python
- Owner: SensorsINI
- Created: 2020-11-19T15:47:56.000Z (about 4 years ago)
- Default Branch: master
- Last Pushed: 2024-10-24T09:53:38.000Z (2 months ago)
- Last Synced: 2024-10-25T06:58:16.614Z (2 months ago)
- Language: Python
- Size: 6.97 MB
- Stars: 2
- Watchers: 9
- Forks: 1
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
The Dextra rock-scissor-paper robot perception pipeline redone in python using pyaer and tensorflow.
See also
* The Dextra tendon driven hand paper and design https://sensorsini.github.io/dextra-robot-hand/
* The Arduino firmware [https://github.com/SensorsINI/DextraRoshamboHand](https://github.com/SensorsINI/Dextra-robot-hand-firmware)
The pretrained network is a 16-bit quantized weight and state CNN.
## Requirements
- OS: Fully tested on Ubuntu 18.04 and 22.04
* Python 3.9 **Important, necessary for Tensorflow 2.5.0 required to run the roshambo CNN**
* Tensorflow 2.5.0
* CUDA 10.2+ or whatever comes with tensorflow install
- Keras: 2.5.0
- pyaer https://github.com/duguyue100/pyaer
* sensors DAVIS camera
* Robot hand with Arduino control via USB serial port bytes if you want to see the hand move; see project https://github.com/SensorsINI/DextraRoshamboHand
## Setup
Project includes pycharm _.idea/_ folder and vscode _.vscode/_ folder.
### System level prerequisites
1. *Install libcaer.* Can be installed with sudo apt-get install libcaer-dev, otherewise see https://gitlab.com/inivation/dv/libcaer.
### Make a conda environment
Create the environment, activate it, then in it install the libraries. We recommend you use conda because it will download the necessary python version 3.9. (That
is the last python version to have tensorflow 2.5.0 which this project uses.)
``` bash
conda create -n roshambo python=3.9
conda activate roshambo
```
### Other requirements
See requirements.txt for libraries needed.
Install them to the new conda env from the conda prompt with
``` bash
conda activate roshambo # probably already activate
pip install -r requirements.txt
```
If you have trouble with pyaer, see https://github.com/duguyue100/pyaer. It should work for linux and mac OS intel silicon machines. Windows probably will not work natively, but you can run the code in a WSL2 Ubuntu virtual machine using https://github.com/dorssel/usbipd-win to map the USB port to WSL2 Ubuntu.
### USB/pyaer on Windows
Recommand to use WSL2 and the vscode plugin https://marketplace.visualstudio.com/items?itemName=thecreativedodo.usbip-connect to bind the USB port to WSL2 Ubuntu VM
# Running Dextra
Run _roshambo_; it uses multiprocessing to launch 2 subprocessees, _producer_ and _consumer_. (You can run these separately for testing.)
Run two processes, producer and consumer.
1. connect hardware: DAVIS to USB and Arduino to USB.
2. Find out which serial port device the hand controller Arduino appears on. You can use dmesg on linux. You can put the serial port into _globals_and_utils.py_ to avoid adding as argument.
2. In terminal run producer
```bash
python -m roshambo
```