https://github.com/Robo-Touch/YCB-Sight
Visuo-tactile dataset with GelSight and depth camera for YCB objects.
https://github.com/Robo-Touch/YCB-Sight
manipulation robotics tactile-perception
Last synced: 6 days ago
JSON representation
Visuo-tactile dataset with GelSight and depth camera for YCB objects.
- Host: GitHub
- URL: https://github.com/Robo-Touch/YCB-Sight
- Owner: Robo-Touch
- License: cc-by-sa-4.0
- Created: 2021-10-11T20:51:04.000Z (almost 5 years ago)
- Default Branch: main
- Last Pushed: 2021-10-20T15:12:46.000Z (almost 5 years ago)
- Last Synced: 2024-01-06T20:32:51.454Z (over 2 years ago)
- Topics: manipulation, robotics, tactile-perception
- Language: Python
- Homepage:
- Size: 7.74 MB
- Stars: 10
- Watchers: 3
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- Awesome-Touch - YCB-Sight
README
# **YCB-Sight**: A visuo-tactile dataset for object understanding
[![CC BY-SA 4.0][cc-by-sa-shield]][cc-by-sa] [](https://opensource.org/licenses/MIT) 
[cc-by-sa]: http://creativecommons.org/licenses/by-sa/4.0/
[cc-by-sa-shield]: https://img.shields.io/badge/License-CC%20BY--SA%204.0-lightgrey.svg
YCB-Sight is a visuo-tactile dataset including the simulated and real data from a GelSight tactile sensor and Kinect Azure RGB-D camera on the [YCB dataset](https://www.ycbbenchmarks.com).
## Dataset
You can find the whole dataset [here](https://drive.google.com/drive/folders/17BPST4biGzduVtoCUBswOmkISqNh1srI?usp=sharing), or download partial data below
### YCBSight-Sim
Simulated tactile and depth data with [Taxim](https://github.com/CMURoboTouch/Taxim) and [pyrender](https://github.com/mmatl/pyrender)
| Object Name | Size (MB) | Link |
| -- | -- | -- |
| 002_master_chef_can | 64.3 | [[Link]](https://drive.google.com/file/d/1XD3nrd4gKf74ljJKt_StGrUCads2OKom/view?usp=sharing) |
| 003_cracker_box | 63.2 | [[Link]](https://drive.google.com/file/d/1nFePHk4CCHT0o31aNIivueqM8zWoQXXW/view?usp=sharing) |
| 004_sugar_box | 61.2 | [[Link]](https://drive.google.com/file/d/1-l7U06Puh47WBL6FvDtOvttBF0qkkCJm/view?usp=sharing) |
| 005_tomato_soup_can | 63.8 | [[Link]](https://drive.google.com/file/d/1WcSEomKsUNSQMOscxRkC14opf2PA9dxO/view?usp=sharing) |
| 006_mustard_bottle | 63.8 | [[Link]](https://drive.google.com/file/d/1o-qZh8bU65cHKcBY18iFXlMyUiPulkyN/view?usp=sharing) |
| 007_tuna_fish_can | 63.2 | [[Link]](https://drive.google.com/file/d/1qI1RufCRG_UkepDflnStQVg085nFzZNJ/view?usp=sharing) |
| 008_pudding_box | 61.5 | [[Link]](https://drive.google.com/file/d/1AWhCIf2WYIVVnxlL2eL0DXRP60IHRTKh/view?usp=sharing) |
| 009_gelatin_box | 60.3 | [[Link]](https://drive.google.com/file/d/1eRBESy4wxIKbGD1d-SZrCuYBlh9Ukfdf/view?usp=sharing) |
| 010_potted_meat_can | 62.8 | [[Link]](https://drive.google.com/file/d/1UtiJM-PtXa5OGErGIC4M60zpHlb4XM5d/view?usp=sharing) |
| 011_banana | 63.7 | [[Link]](https://drive.google.com/file/d/1pSNc9f_b7akubCMHkn572qloV4FnU9Fd/view?usp=sharing) |
| 012_strawberry | 64.2 | [[Link]](https://drive.google.com/file/d/10bQiES46hooZrPw5MaUMXvfeaSQxcVva/view?usp=sharing) |
| 013_apple | 63.4 | [[Link]](https://drive.google.com/file/d/1OSt_RFXQo-ad5jAIxaHeDeUxRBhO-hNw/view?usp=sharing) |
| 014_lemon | 63.4 | [[Link]](https://drive.google.com/file/d/1_XL0YKCwQUQWrFt9XMjdNcaqh9ss6Fdp/view?usp=sharing) |
| 017_orange | 63.2 | [[Link]](https://drive.google.com/file/d/17YrIXvepLBlh9RQSexNdDwR3L78NoG39/view?usp=sharing) |
| 019_pitcher_base | 64.5 | [[Link]](https://drive.google.com/file/d/1YYlTtBVNQ8uEUtAhxH_wTkokimK-SRQK/view?usp=sharing) |
| 021_bleach_cleanser | 62.6 | [[Link]](https://drive.google.com/file/d/1FwlqJG9prxv7qsaAWJk7SZsDRx_OjL0l/view?usp=sharing) |
| 024_bowl | 65.1 | [[Link]](https://drive.google.com/file/d/1w1ybTkEPAnq5UTbMzwXGZBqikMbggIoK/view?usp=sharing) |
| 025_mug | 64.2 | [[Link]](https://drive.google.com/file/d/1dqXYqCcImpFzxox9XKA_LONVJz7gswIZ/view?usp=sharing) |
| 029_plate | 66.3 | [[Link]](https://drive.google.com/file/d/1KKdWX4z0HGvt_bfcLM_peBqCtowbsyVh/view?usp=sharing) |
| 035_power_drill | 64.7 | [[Link]](https://drive.google.com/file/d/1gOOzuZIgbgprzOQYZz_pNilHN0miO3xT/view?usp=sharing) |
| 036_wood_block | 60.1 | [[Link]](https://drive.google.com/file/d/1dZjTRx-gB_mUYZeS0juJ3e6nnwjor7Or/view?usp=sharing) |
| 037_scissors | 64.2 | [[Link]](https://drive.google.com/file/d/1KDOxdqPMDH3_JXenwkZLntulV8yOIZxN/view?usp=sharing) |
| 042_adjustable_wrench | 64.7 | [[Link]](https://drive.google.com/file/d/1Vv7qCiT0Ac8wBEW67FTF836sB3ZO9jGw/view?usp=sharing) |
| 043_phillips_screwdriver | 63.9 | [[Link]](https://drive.google.com/file/d/13HdgVl17iEu78ZNgOyPy4QnXH4uS6UAA/view?usp=sharing) |
| 048_hammer | 64.1 | [[Link]](https://drive.google.com/file/d/1pgEufbUhlcKOA7-LhjICdL1dEsrwtJyB/view?usp=sharing) |
| 055_baseball | 63.5 | [[Link]](https://drive.google.com/file/d/1x01Jh_7WQNr5o2oUfhvnoLzK91r0wd2c/view?usp=sharing) |
| 056_tennis_ball | 63.4 | [[Link]](https://drive.google.com/file/d/12Jp_A-uHZ9DwXuxR03uAuZyOlXX7E59T/view?usp=sharing) |
| 072-a_toy_airplane | 65.5 | [[Link]](https://drive.google.com/file/d/14IWMKCpcBp7dA_oruiRmVD8DQ5hiSl1S/view?usp=sharing) |
| 072-b_toy_airplane | 63.8 | [[Link]](https://drive.google.com/file/d/1HcbQCpZllofE_VSqhyQscUq74335nWz7/view?usp=sharing) |
| 077_rubiks_cube | 61.3 | [[Link]](https://drive.google.com/file/d/1kpvxRrcCvKx_SaBcJWVS9KghLbQEkdxZ/view?usp=sharing) |
### YCBSight-Real
Collected tactile and depth data from real world experiments
| Object Name | Size (GB) | Link |
| -- | -- | -- |
| 002_master_chef_can | 0.97 | [[Link]](https://drive.google.com/file/d/1ZfwuXom_ngccnyox-ud4b-pOa1zMw-Wr/view?usp=sharing) |
| 004_sugar_box | 1.15 | [[Link]](https://drive.google.com/file/d/1ZAZ4y2pCI7YWOSx5VI7tNlJ2EJpZy1F2/view?usp=sharing) |
| 005_tomato_soup_can | 1.09 | [[Link]](https://drive.google.com/file/d/1tCoHd7qf4AAXWMfLCRecpnimJcz52sRv/view?usp=sharing) |
| 010_potted_meat_can | 1.09 | [[Link]](https://drive.google.com/file/d/1VNQyOW1ooBClp4QogJJqh7SHN2zGCq1E/view?usp=sharing) |
| 021_bleach_cleanser | 1.23 | [[Link]](https://drive.google.com/file/d/1HVTtGYX_Doj_xk_TIyFEd82njQ1DhA8a/view?usp=sharing) |
| 036_wood_block | 1.02 | [[Link]](https://drive.google.com/file/d/1Hdab1i4UHrkUAJ5RtDigpMHfx868aOtD/view?usp=sharing) |
### Data directory format
```bash
YCBSight-Sim
├── obj1
│ ├── gt_contact_mask
│ │ ├── .npy
│ │ └── ...
│ ├── gt_height_map
│ │ ├── .npy
│ │ └── ...
│ ├── gelsight
│ │ ├── .jpg
│ │ └── ...
│ ├── pose.txt
│ ├── depthCam.npy
│ └── depthCam.pdf
├── obj2
└── ...
```
```bash
YCBSight-Real
├── obj1
│ ├── gelsight
│ │ ├── gelsight__.jpg
│ │ └── ...
│ ├── depth
│ │ └── depth_0_.tif
│ ├── pc
│ │ └── pc_0_.npy
│ ├── rgb
│ │ ├── rgb__.jpg
│ │ └── ...
│ ├── robot.csv
│ ├── tf.json
│ └── obj1.mp4
├── obj2
└── ...
```
## Dependencies
The visualization and data processing are implemented in python3 and require numpy, scipy, matplotlib, cv2.
To install dependencies: `pip install -r requirements.txt`.
## Data Visualization
- `scripts/lookup_mapping/lookup.py` reconstructs the height maps from the tactile readings. Here are several parameters to set:
- `path2model`: the path to the directory storing the YCBSight-Real and/or YCBSight-Sim
- `sim`: True/False, visualize whether the simulated data or real data
- `obj`: specify a certain object's data to visualize, or set to None to visualize all the data
## Local Shape Reconstruction from Touch with Lookup Table
- `scripts/data_visualization/data_visualizer.py` visualize the data in YCB-Sight dataset.
## Local Shape Reconstruction from Touch with FCRN network
Please refer to [this repo](https://github.com/XPFly1989/FCRN) (pytorch version) and [this repo](https://github.com/iro-cp/FCRN-DepthPrediction) (tensorflow version).
## License
This dataset is licensed under a
[Creative Commons Attribution-ShareAlike 4.0 International License][cc-by-sa], with the accompanying processing code licensed under the [MIT License](https://opensource.org/licenses/MIT).
## Citation
If you use YCB-Sight dataset in your research, please cite:
```BibTeX
@article{suresh2021efficient,
title={Efficient shape mapping through dense touch and vision},
author={Suresh, Sudharshan and Si, Zilin and Mangelson, Joshua G and Yuan, Wenzhen and Kaess, Michael},
journal={arXiv preprint arXiv:2109.09884},
year={2021}
}
```