Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/ZhihaoAIRobotic/awesome-robot-visual-imitation-learning
A collection of papers, codes and talks of visual imitation learning/imitation learning from video for robotics.
https://github.com/ZhihaoAIRobotic/awesome-robot-visual-imitation-learning
List: awesome-robot-visual-imitation-learning
awesome-list imitation imitation-from-observation robotics
Last synced: about 1 month ago
JSON representation
A collection of papers, codes and talks of visual imitation learning/imitation learning from video for robotics.
- Host: GitHub
- URL: https://github.com/ZhihaoAIRobotic/awesome-robot-visual-imitation-learning
- Owner: ZhihaoAIRobotic
- Created: 2022-09-09T09:09:26.000Z (about 2 years ago)
- Default Branch: main
- Last Pushed: 2022-12-10T01:47:33.000Z (almost 2 years ago)
- Last Synced: 2024-05-19T22:43:01.524Z (4 months ago)
- Topics: awesome-list, imitation, imitation-from-observation, robotics
- Homepage:
- Size: 35.2 KB
- Stars: 43
- Watchers: 2
- Forks: 3
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
- ultimate-awesome - awesome-robot-visual-imitation-learning - A collection of papers, codes and talks of visual imitation learning/imitation learning from video for robotics. (Other Lists / PowerShell Lists)
README
# awesome-robot-imitation-learning-from-video[](https://github.com/sindresorhus/awesome)
A collection of papers, codes and resources of learning from video in robotics.
## Papers
### Survey/Datasets
[Recent Advances in Imitation Learning from Observation](https://arxiv.org/pdf/1905.13566.pdf) IJCAI 2019
[Recent advances in robot learning from demonstration](https://www.annualreviews.org/doi/abs/10.1146/annurev-control-100819-063206?casa_token=aHxQSEEqTm8AAAAA:_xq0aRAFtAsb2CiASkQOOK4uouEUqvK8-4vgwEF1EkOnnG2gi21d3HW1Rqo61UYqKbuv6aCHrzJjLlWf) Annual Review of Control, Robotics, and Autonomous Systems 2020
[What Matters in Learning from Offline Human Demonstrations for Robot Manipulation](https://arxiv.org/pdf/2108.03298.pdf) arxiv2021 | [website](https://robomimic.github.io/) |[code](https://github.com/ARISE-Initiative/robomimic)
[RoboTube: Learning Household Manipulation from Human Videos with Simulated Twin Environments](https://openreview.net/pdf?id=SYUEnQtK85o) RSS2022 | [video](https://www.youtube.com/watch?v=oTnsZs7yOVc)
[Challenges and Opportunities in Offline Reinforcement Learning from Visual Observations](https://openreview.net/pdf?id=bPOBIKaqLba) RSS 2022
### Visual Representation Learning
[R3M: A Universal Visual Representation for Robot Manipulation ](https://arxiv.org/pdf/2203.12601.pdf) arXiv 2022 | [code](https://github.com/facebookresearch/r3m)
[Adversarial Skill Networks: Unsupervised Robot Skill Learning from Videos](https://arxiv.org/pdf/1910.09430.pdf) ICRA 2020 | [website](http://robotskills.cs.uni-freiburg.de/#code)
[Reinforcement Learning with Action-Free Pre-Training from Videos](https://proceedings.mlr.press/v162/seo22a/seo22a.pdf) ICML2022 | [code](https://github.com/younggyoseo/apv)
[Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videos](https://arxiv.org/pdf/2206.11795.pdf) arXiv 2022
[BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning](https://proceedings.mlr.press/v164/jang22a/jang22a.pdf) CORL 2022 | [website](https://sites.google.com/view/bc-z/home)
[VIP: TOWARDS UNIVERSAL VISUAL REWARD AND REPRESENTATION VIA VALUE-IMPLICIT PRETRAINING](https://arxiv.org/pdf/2210.00030.pdf) arXiv 2022
### Estimated Pose/Key Point
[DexVIP: Learning Dexterous Grasping with Human Hand Pose Priors from Video](https://proceedings.mlr.press/v164/mandikal22a/mandikal22a.pdf) CORL 2021
[DexMV: Imitation Learning for Dexterous Manipulation from Human Videos](https://arxiv.org/pdf/2108.05877.pdf) arxiv2022 | [website](https://yzqin.github.io/dexmv/) | [code](https://github.com/yzqin/dexmv-sim)
[Learning Agile Robotic Locomotion Skills by Imitating Animals](https://xbpeng.github.io/projects/Robotic_Imitation/2020_Robotic_Imitation.pdf) RSS 2020 | [website](https://xbpeng.github.io/projects/Robotic_Imitation/) | [code](https://github.com/erwincoumans/motion_imitation)
[SFV: Reinforcement Learning of Physical Skills from Videos](https://xbpeng.github.io/projects/SFV/2018_TOG_SFV.pdf) ACM Transactions on Graphics 2018 | [code](https://github.com/akanazawa/motion_reconstruction)
[Graph-Structured Visual Imitation](http://proceedings.mlr.press/v100/sieb20a/sieb20a.pdf) CoRL 2019
[Learning to Manipulate Tools by Aligning Simulation to Video Demonstration](https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=9612073&casa_token=20714ek6aaQAAAAA:F05cTWri23dQ1ijDqgFU-QOey6ZeDROrFNDtYsRudUE9JoXUPEb5_ljXlv7_7uXGAfM3zeb7lQ) RAL 2022
[Learning Periodic Tasks from Human Demonstrations](https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=9812402&casa_token=jd7QDSj6HtoAAAAA:VdNXiKamIsexSD_mmUDusYF14rvfrfgmhDP8sHSZO7Wom_GJykTvaOFhz0Xk39oVBLYhkw977w&tag=1) arxiv2021
[Robotic Telekinesis: Learning a Robotic Hand Imitator by Watching Humans on YouTube](https://openreview.net/pdf?id=Z9L-Jde0iPx) RSS 2022
[You Only Demonstrate Once: Category-Level Manipulation from Single Visual Demonstration](https://www.researchgate.net/profile/Bowen-Wen/publication/358259887_You_Only_Demonstrate_Once_Category-Level_Manipulation_from_Single_Visual_Demonstration/links/62753457973bbb29cc67a93d/You-Only-Demonstrate-Once-Category-Level-Manipulation-from-Single-Visual-Demonstration.pdf) arxiv2022
[VideoDex: Learning Dexterity from Internet Videos](https://arxiv.org/pdf/2212.04498.pdf) arxiv2022
[Robotic Telekinesis: Learning a Robotic Hand Imitator by Watching Humans on YouTube](https://arxiv.org/pdf/2202.10448.pdf) arxiv2022
[Graph Inverse Reinforcement Learning from Diverse Videos](https://arxiv.org/pdf/2207.14299.pdf) arxiv2022
### Image Translation
[Third-Person Visual Imitation Learning via Decoupled Hierarchical Controller](https://proceedings.neurips.cc/paper/2019/file/8a146f1a3da4700cbf03cdc55e2daae6-Paper.pdf) NeurIPS 2019| [website](https://pathak22.github.io/hierarchical-imitation/) | [code](https://github.com/pathak22/hierarchical-imitation/)
[Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translation](https://arxiv.org/pdf/1707.03374.pdf) ICML 2016 | [code](https://github.com/wyndwarrior/imitation_from_observation?utm_source=catalyzex.com)
[Learning by Watching: Physical Imitation of Manipulation Skills from Human Videos](https://arxiv.org/pdf/2101.07241.pdf) RSS 2021 | [website](https://www.pair.toronto.edu/lbw-kp/)
### Meta Learning
[One-Shot Hierarchical Imitation Learning of Compound Visuomotor Tasks](https://arxiv.org/pdf/1810.11043.pdf) arxiv 2018
[One-shot imitation from observing humans via domain-adaptive meta-learning](https://arxiv.org/pdf/1802.01557.pdf) RSS 2018
## Talks
[Visual Imitation Learning: Generalization, Perceptual Grounding, and Abstraction](https://www.youtube.com/watch?v=1TJHuO5TAfo) RSS Workshop 2020
[Imitation Learning Tutorial ICML 2018](https://www.youtube.com/watch?v=WjFdD7PDGw0)
[Model-Based Visual Imitation Learning](https://www.youtube.com/watch?v=aiJp9GWdJnw) CVPR 2021 - Franziska Meier
[Workshop on Learning from Diverse, Offline Data](https://www.youtube.com/watch?v=lHXp6j6YrY4&t=13s) RSS2022