Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/ZhihaoAIRobotic/awesome-robot-visual-imitation-learning

A collection of papers, codes and talks of visual imitation learning/imitation learning from video for robotics.
https://github.com/ZhihaoAIRobotic/awesome-robot-visual-imitation-learning

List: awesome-robot-visual-imitation-learning

awesome-list imitation imitation-from-observation robotics

Last synced: 16 days ago
JSON representation

A collection of papers, codes and talks of visual imitation learning/imitation learning from video for robotics.

Host: GitHub
URL: https://github.com/ZhihaoAIRobotic/awesome-robot-visual-imitation-learning
Owner: ZhihaoAIRobotic
Created: 2022-09-09T09:09:26.000Z (over 2 years ago)
Default Branch: main
Last Pushed: 2022-12-10T01:47:33.000Z (about 2 years ago)
Last Synced: 2024-05-19T22:43:01.524Z (7 months ago)
Topics: awesome-list, imitation, imitation-from-observation, robotics
Homepage:
Size: 35.2 KB
Stars: 43
Watchers: 2
Forks: 3
Open Issues: 0
Metadata Files:
- Readme: README.md

Awesome Lists containing this project

ultimate-awesome - awesome-robot-visual-imitation-learning - A collection of papers, codes and talks of visual imitation learning/imitation learning from video for robotics. (Other Lists / Monkey C Lists)

README

        # awesome-robot-imitation-learning-from-video[![Awesome](https://cdn.rawgit.com/sindresorhus/awesome/d7305f38d29fed78fa85652e3a63e154dd8e8829/media/badge.svg)](https://github.com/sindresorhus/awesome)

A collection of papers, codes and resources of learning from video in robotics.

## Papers

### Survey/Datasets

[Recent Advances in Imitation Learning from Observation](https://arxiv.org/pdf/1905.13566.pdf) IJCAI 2019

[Recent advances in robot learning from demonstration](https://www.annualreviews.org/doi/abs/10.1146/annurev-control-100819-063206?casa_token=aHxQSEEqTm8AAAAA:_xq0aRAFtAsb2CiASkQOOK4uouEUqvK8-4vgwEF1EkOnnG2gi21d3HW1Rqo61UYqKbuv6aCHrzJjLlWf) Annual Review of Control, Robotics, and Autonomous Systems 2020

[What Matters in Learning from Offline Human Demonstrations for Robot Manipulation](https://arxiv.org/pdf/2108.03298.pdf) arxiv2021 | [website](https://robomimic.github.io/) |[code](https://github.com/ARISE-Initiative/robomimic)

[RoboTube: Learning Household Manipulation from Human Videos with Simulated Twin Environments](https://openreview.net/pdf?id=SYUEnQtK85o)  RSS2022 | [video](https://www.youtube.com/watch?v=oTnsZs7yOVc) 

[Challenges and Opportunities in Offline Reinforcement Learning from Visual Observations](https://openreview.net/pdf?id=bPOBIKaqLba) RSS 2022

### Visual Representation Learning

[R3M: A Universal Visual Representation for Robot Manipulation ](https://arxiv.org/pdf/2203.12601.pdf) arXiv 2022 | [code](https://github.com/facebookresearch/r3m)

[Adversarial Skill Networks: Unsupervised Robot Skill Learning from Videos](https://arxiv.org/pdf/1910.09430.pdf) ICRA 2020 | [website](http://robotskills.cs.uni-freiburg.de/#code)

[Reinforcement Learning with Action-Free Pre-Training from Videos](https://proceedings.mlr.press/v162/seo22a/seo22a.pdf) ICML2022 | [code](https://github.com/younggyoseo/apv) 

[Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videos](https://arxiv.org/pdf/2206.11795.pdf) arXiv 2022

[BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning](https://proceedings.mlr.press/v164/jang22a/jang22a.pdf) CORL 2022 | [website](https://sites.google.com/view/bc-z/home)

[VIP: TOWARDS UNIVERSAL VISUAL REWARD AND REPRESENTATION VIA VALUE-IMPLICIT PRETRAINING](https://arxiv.org/pdf/2210.00030.pdf) arXiv 2022

### Estimated Pose/Key Point

[DexVIP: Learning Dexterous Grasping with Human Hand Pose Priors from Video](https://proceedings.mlr.press/v164/mandikal22a/mandikal22a.pdf) CORL 2021

[DexMV: Imitation Learning for Dexterous Manipulation from Human Videos](https://arxiv.org/pdf/2108.05877.pdf) arxiv2022 | [website](https://yzqin.github.io/dexmv/) | [code](https://github.com/yzqin/dexmv-sim)

[Learning Agile Robotic Locomotion Skills by Imitating Animals](https://xbpeng.github.io/projects/Robotic_Imitation/2020_Robotic_Imitation.pdf) RSS 2020 | [website](https://xbpeng.github.io/projects/Robotic_Imitation/) | [code](https://github.com/erwincoumans/motion_imitation)

[SFV: Reinforcement Learning of Physical Skills from Videos](https://xbpeng.github.io/projects/SFV/2018_TOG_SFV.pdf) ACM Transactions on Graphics 2018 | [code](https://github.com/akanazawa/motion_reconstruction)

[Graph-Structured Visual Imitation](http://proceedings.mlr.press/v100/sieb20a/sieb20a.pdf) CoRL 2019 

[Learning to Manipulate Tools by Aligning Simulation to Video Demonstration](https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=9612073&casa_token=20714ek6aaQAAAAA:F05cTWri23dQ1ijDqgFU-QOey6ZeDROrFNDtYsRudUE9JoXUPEb5_ljXlv7_7uXGAfM3zeb7lQ) RAL 2022

[Learning Periodic Tasks from Human Demonstrations](https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=9812402&casa_token=jd7QDSj6HtoAAAAA:VdNXiKamIsexSD_mmUDusYF14rvfrfgmhDP8sHSZO7Wom_GJykTvaOFhz0Xk39oVBLYhkw977w&tag=1) arxiv2021

[Robotic Telekinesis: Learning a Robotic Hand Imitator by Watching Humans on YouTube](https://openreview.net/pdf?id=Z9L-Jde0iPx) RSS 2022

[You Only Demonstrate Once: Category-Level Manipulation from Single Visual Demonstration](https://www.researchgate.net/profile/Bowen-Wen/publication/358259887_You_Only_Demonstrate_Once_Category-Level_Manipulation_from_Single_Visual_Demonstration/links/62753457973bbb29cc67a93d/You-Only-Demonstrate-Once-Category-Level-Manipulation-from-Single-Visual-Demonstration.pdf) arxiv2022

[VideoDex: Learning Dexterity from Internet Videos](https://arxiv.org/pdf/2212.04498.pdf) arxiv2022

[Robotic Telekinesis: Learning a Robotic Hand Imitator by Watching Humans on YouTube](https://arxiv.org/pdf/2202.10448.pdf) arxiv2022

[Graph Inverse Reinforcement Learning from Diverse Videos](https://arxiv.org/pdf/2207.14299.pdf) arxiv2022

### Image Translation

[Third-Person Visual Imitation Learning via Decoupled Hierarchical Controller](https://proceedings.neurips.cc/paper/2019/file/8a146f1a3da4700cbf03cdc55e2daae6-Paper.pdf) NeurIPS 2019| [website](https://pathak22.github.io/hierarchical-imitation/) | [code](https://github.com/pathak22/hierarchical-imitation/)

[Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translation](https://arxiv.org/pdf/1707.03374.pdf) ICML 2016 | [code](https://github.com/wyndwarrior/imitation_from_observation?utm_source=catalyzex.com)

[Learning by Watching: Physical Imitation of Manipulation Skills from Human Videos](https://arxiv.org/pdf/2101.07241.pdf) RSS 2021 | [website](https://www.pair.toronto.edu/lbw-kp/)

### Meta Learning

[One-Shot Hierarchical Imitation Learning of Compound Visuomotor Tasks](https://arxiv.org/pdf/1810.11043.pdf) arxiv 2018 

[One-shot imitation from observing humans via domain-adaptive meta-learning](https://arxiv.org/pdf/1802.01557.pdf) RSS 2018 

## Talks

[Visual Imitation Learning: Generalization, Perceptual Grounding, and Abstraction](https://www.youtube.com/watch?v=1TJHuO5TAfo) RSS Workshop 2020 

[Imitation Learning Tutorial ICML 2018](https://www.youtube.com/watch?v=WjFdD7PDGw0)

[Model-Based Visual Imitation Learning](https://www.youtube.com/watch?v=aiJp9GWdJnw) CVPR 2021 - Franziska Meier

[Workshop on Learning from Diverse, Offline Data](https://www.youtube.com/watch?v=lHXp6j6YrY4&t=13s) RSS2022