https://github.com/probcomp/durablevs
DURableVS: Data-efficient Unsupervised Recalibrating Visual Servoing via online learning in a structured generative model
https://github.com/probcomp/durablevs
Last synced: 11 months ago
JSON representation
DURableVS: Data-efficient Unsupervised Recalibrating Visual Servoing via online learning in a structured generative model
- Host: GitHub
- URL: https://github.com/probcomp/durablevs
- Owner: probcomp
- Created: 2022-05-03T05:46:48.000Z (almost 4 years ago)
- Default Branch: main
- Last Pushed: 2022-05-06T02:20:08.000Z (almost 4 years ago)
- Last Synced: 2025-05-08T22:58:11.930Z (11 months ago)
- Language: Jupyter Notebook
- Size: 7.47 MB
- Stars: 1
- Watchers: 5
- Forks: 2
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# DURableVS: Data-efficient Unsupervised Recalibrating Visual Servoing via online learning in a structured generative model
Nishad Gothoskar, Miguel Lázaro-Gredilla, Yasemin Bekiroglu, Abhishek Agarwal, Joshua B. Tenenbaum, Vikash K. Mansinghka, Dileep George
This is the official implementation of our ICRA 2022 paper "DURableVS: Data-efficient Unsupervised Recalibrating Visual Servoing via online learning in a structured generative model".
DurableVS is our a method for unsupervised learning of visual servoing which does not require any prior calibration and is extremely data-efficient. Our key insight is that visual servoing does not depend on identifying the veridical kinematic and camera parameters, but instead only on an accurate generative model of image feature observations from the joint positions of the robot. With our model architecture and learning algorithm, we can consistently learn accurate models from less than 50 training samples (which amounts to less than 1 min of unsupervised data collection) and then use these to enable accurate visual servoing. We can also continue learning online, which enables the robotic system to recover from calibration errors and to detect and quickly adapt to possibly unexpected changes in the robot-camera system (e.g. bumped camera, new objects).
## Installation
To install Python dependencies and the `durablevs` package:
```setup
pip install --upgrade pip
pip install -r requirements.txt
python setup.py develop
```
## Demo
The jupyter notebook in `notebooks/demo.ipynb` runs data collection, learning, and servoing given desired image feature locations for the 6DoF PyBullet robot.
## Citation
```
@article{gothoskar2022durablevs,
title={DURableVS: Data-efficient Unsupervised Recalibrating Visual Servoing via online learning in a structured generative model},
author={Gothoskar, Nishad and L{\'a}zaro-Gredilla, Miguel and Bekiroglu, Yasemin and Agarwal, Abhishek and Tenenbaum, Joshua B and Mansinghka, Vikash K and George, Dileep},
journal={arXiv preprint arXiv:2202.03697},
year={2022}
}
```
## Contact
If you have any questions, please email NishadG Gothoskar at nishad@mit.edu.