Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/leo9344/Implementation-of-GelSight
An implementation of Gelsight Wedge.
https://github.com/leo9344/Implementation-of-GelSight
gelsight sensor tactile-sensor
Last synced: about 2 months ago
JSON representation
An implementation of Gelsight Wedge.
- Host: GitHub
- URL: https://github.com/leo9344/Implementation-of-GelSight
- Owner: leo9344
- License: gpl-3.0
- Created: 2022-01-24T09:02:38.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2022-02-24T15:19:39.000Z (over 2 years ago)
- Last Synced: 2024-06-20T04:14:41.905Z (3 months ago)
- Topics: gelsight, sensor, tactile-sensor
- Language: Python
- Homepage:
- Size: 29.4 MB
- Stars: 14
- Watchers: 1
- Forks: 3
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-tactile-sensing - Implementation of Gelsight Wedge
README
# Implementation-of-GelSight
An implementation of [GelSight Wedge](https://arxiv.org/abs/2106.08851).
Test passed on Ubuntu 18.04 and Windows 10. 🚩
## Requirements
python>=3.7
opencv-python
opencv4.x (C++ Version)
pybind11
pyyaml
argparse
glob
numpy
matplotlib
scipy
skimage
open3d
Visual Studio 2022
## Hardware
🔨 🔨
### Camera: RaspberryPi Zero Camera Module (22Pin)
We connect the camera module to a raspberrypi zero for image capture.
### Illumination: Three colors of LED arrays (RGB)
We used surface mounted LEDs with 120 degree angle (in RGB colors) as the light source. Three LED arrays (each contains 2 LEDs) are circly distributted under the silicone.
### Silicone: Composed of transparent base, black markers and the reflective membrane
#### Transparent base:
1. We 3D printed the mould (with resin) and laser-cut the acrylic sheet (placed at the bottom of the mould) for transparent silicone base manufacturing.
2. we use Solaris (part A and part B) with Shore A 15 and Slacker (used to increase softness) from vendor Smooth-on® to produce the transparent elastomeric base. A ratio of 1:1:3 for each component has proven to be ideal for making an elastomeric base with the appropriate hardness. The mixture is then degassed and cured for 12 hours.
#### Black markers:
We painted markers on top surface of the transparent base with Silc-Pig (black colorant). The distance between each marker is around 1 mm.
#### Reflective membrane:
1. We dip a small amount of aluminum powder and spread it evenly upon the black markers.
2. we use aluminum powder, Psycho Paint (part A and part B) and Novocs Matte (silicone diluter) to produce the reflective membrane. A ratio of 1:5:5:30 for each component has proven to be ideal for making a moderate membrane. The mixture is degassed and then sprayed on top of the transparent base surface. The membrane is cured for 4 hours.
## Software
### Step -1: set `config.yaml`
`camid` for `cv2.VideoCapture(camid)`.
`sample: from` for `{data_path}/sample_{sample_from}.jpg`.
`sample: to` for `{data_path}/sample_{sample_to}.jpg`.
`data_path` for `{data_path}`
### Step 0: run `pref_ref_and_sample.py`
python pref_ref_and_sample.py -r -s
`-r` or `--ref` for capturing `{data_path}/ref.jpg`.
Click `left button` to take `ref.jpg`, you can click more than once until you are satiesfied. Then press `q` to exit or continue.
`-s` or `--ref` for capturing `{data_path}/sample_xx.jpg`.
Click `left button` to take `sample_xx.jpg`. If `sample_to - sample_from` pictures are captured, it will terminate automatically. Or you can manually press `q` to exit in advance or continue.
### Step 1: run `camera_config.py`
### Step 2: Get prepared for running `calibration.py`
#### 2.1 Measure `self.BallRad` eg: 3
#### 2.2 Measure `self.Pixmm` = $\frac{Length\ (mm)}{Pixel}$
##### 2.2.1 Capture & save `Pixmm.jpg`
Eg:
##### 2.2.2 Get measurement of `mm`
Eg: mm = 3.40(mm)
##### 2.2.3 Use `mesaure_Pixmm.py` to select 2 keypoints and calculate their distance (in `pixel`).
Click once on the first keypoint, then click once on the other keypoint, you will see an arrow linking 2 keypoints with their distance.
Eg: distance = 103.07764
##### 2.2.4 Calculate the Pixmm and fill it into `calibration.py`
In `line 13` of `calibration.py`:
`self.Pixmm` = $\frac{Length\ (mm)}{Pixel} = \frac{3.40}{103.0776}=0.03298$
#### 2.3 Capture & save 30 x `sample_xx.jpg`, from `sample_1.jpg` to `sample_30,jpg`. eg:
#### 2.4 run `calibration.py`
### Step 3: run `Test0Cable.py`
Get reconstruction result.
### Step 4: run `tracking.py`
Get tracking result.
## Improvements
🔨Coming soon!🔨
Thin plate spline for inpaint of markers?
Use model to map color to gradients.
## References
We used https://github.com/siyuandong16/gelsight_heightmap_reconstruction for calibration and heightmap reconstruction and https://github.com/GelSight/tracking for tracking.
## Acknowledgements
https://arxiv.org/abs/2106.08851
https://github.com/siyuandong16/gelsight_heightmap_reconstruction
https://github.com/GelSight/tracking
https://tutorial.cytron.io/2020/12/29/raspberry-pi-zero-usb-webcam/