Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.
Awesome Lists | Featured Topics | Projects
https://github.com/wangcongrobot/awesome-isaac-gym

A curated list of awesome NVIDIA Issac Gym frameworks, papers, software, and resources
https://github.com/wangcongrobot/awesome-isaac-gym
List: awesome-isaac-gym
isaac-gym openai-gym reinforcement-learning robot-learning robotics
Last synced: about 2 months ago
JSON representation
A curated list of awesome NVIDIA Issac Gym frameworks, papers, software, and resources
Host: GitHub
URL: https://github.com/wangcongrobot/awesome-isaac-gym
Owner: wangcongrobot
Created: 2021-08-07T01:00:37.000Z (about 3 years ago)
Default Branch: main
Last Pushed: 2023-04-05T05:57:16.000Z (over 1 year ago)
Last Synced: 2024-05-21T17:07:34.306Z (4 months ago)
Topics: isaac-gym, openai-gym, reinforcement-learning, robot-learning, robotics
Homepage:
Size: 7.91 MB
Stars: 637
Watchers: 22
Forks: 56
Open Issues: 2
Metadata Files:
- Readme: README.md
Awesome Lists containing this project

awesome-legged-locomotion-learning - awesome-isaac-gym
ultimate-awesome - awesome-isaac-gym - A curated list of awesome NVIDIA Issac Gym frameworks, papers, software, and resources. (Other Lists / PowerShell Lists)
README

        # Awesome NVIDIA Isaac Gym

Collect some related resources of NVIDIA Isaac Gym

## News

- [PhysX 5](https://github.com/NVIDIA-Omniverse/PhysX): NVIDIA PhysX 5 SDK

- [02/07/2022] Isaac Gym Preview 4 (1.3.0) is available

- [23/03/2022] GTC2022: [Isaac Gym: The Next Generation — High-performance Reinforcement Learning in Omniverse](https://www.nvidia.com/gtc/session-catalog/?search=Isaac#/session/1638331324610001KvlV)

- [29/10/2021] Isaac Gym Preview 3 is available

- [NVIDIA Isaac Sim on Omniverse Now Available in Open Beta 21/06/2021](https://developer.nvidia.com/blog/nvidia-isaac-sim-on-omniverse-now-available-in-open-beta/)

- [Isaac Gym](https://www.nvidia.com/en-us/on-demand/session/gtcsiliconvalley2019-s9918/)

- [Isaac Gym: End-to-End GPU-Accelerated Reinforcement Learning](https://www.nvidia.com/en-us/on-demand/session/gtcspring21-s32037/)

## Official resource

- [OmniIsaacGymEnvs](https://github.com/NVIDIA-Omniverse/OmniIsaacGymEnvs)

- [Isaac Gym](https://developer.nvidia.com/isaac-gym)

- [Isaac SDK](https://docs.nvidia.com/isaac/isaac/doc/index.html)

- [Isaac gym forum](https://forums.developer.nvidia.com/c/agx-autonomous-machines/isaac/isaac-gym/322)

- [Isaac Sim GTC 2021, sim-to-real](https://www.nvidia.com/en-us/on-demand/session/gtcspring21-s31824/)

- [Isaac Sim video toturials](https://www.youtube.com/playlist?list=PL3jK4xNnlCVf1SzxjCm7ZxDBNl9QYyV8X)

- [Training Your JetBot in NVIDIA Isaac Sim](https://developer.nvidia.com/blog/training-your-jetbot-in-isaac-sim/)

- [Training Your NVIDIA JetBot to Avoid Collisions Using NVIDIA Isaac Sim](https://developer.nvidia.com/blog/training-your-nvidia-jetbot-to-avoid-collisions-using-nvidia-isaac-sim/)

- [Introducing NVIDIA Isaac Gym: End-to-End Reinforcement Learning for Robotics](https://developer.nvidia.com/blog/introducing-isaac-gym-rl-for-robotics/)

- [Accelerating Robotics Simulation with NVIDIA Omniverse Isaac Sim](https://developer.nvidia.com/blog/accelerating-robotics-simulation-with-nvidia-omniverse-isaac-sim/)

- [Developing Robotics Applications in Python with NVIDIA Isaac SDK](https://developer.nvidia.com/blog/developing-robotics-applications-in-python-with-isaac-sdk/)

- [Building an Intelligent Robot Dog with the NVIDIA Isaac SDK](https://developer.nvidia.com/blog/building-intelligent-robot-dog-with-isaac-sdk/)

- [youtube video NVIDIAOmniverse](https://www.youtube.com/c/NVIDIAOmniverse/videos?&ab_channel=NVIDIAOmniverse)

## GTC

- [Isaac Gym and Omniverse: High Performance Reinforcement Learning Evolved [A31118]](https://events.rainfocus.com/widget/nvidia/nvidiagtc/sessioncatalog?search=A31118)

- [Learning Challenging Tasks For Quadrupedal Robots: From Simulation To Reality [A31308]](https://events.rainfocus.com/widget/nvidia/nvidiagtc/sessioncatalog?search=A31308)

- [Sim-to-Real in Isaac Sim](https://www.nvidia.com/en-us/on-demand/session/gtcspring21-s31824/)

- [Isaac Gym: End-to-End GPU-Accelerated Reinforcement Learning](https://www.nvidia.com/en-us/on-demand/session/gtcspring21-s32037/)

- [Bridging Sim2Real Gap: Simulation Tuning for Training Deep Learning Robotic Perception Models](https://www.nvidia.com/en-us/on-demand/session/gtcspring21-s31649/)

- [Reinforcement Learning and Intralogistics: Soft Actor Critic for Maples Navigation in Warehouses](https://www.nvidia.com/en-us/on-demand/session/gtcspring21-e31467/)

- [Building Robotics Applications Using NVIDIA Isaac SDK](https://www.nvidia.com/en-us/on-demand/session/gtcfall20-a21856/)

- [NVIDIA Isaac SIM — Amazing Robot Models and Tasks Simulated in Isaac Sim 2020.1](https://www.nvidia.com/en-us/on-demand/session/gtcsj20-d2s43/)

- [Building Robotics Applications Using NVIDIA Isaac SDK](https://www.nvidia.com/en-us/on-demand/session/gtcfall20-a21856/)

- [Sim-to-Real in Isaac Sim](https://www.nvidia.com/en-us/on-demand/session/gtcspring21-s31824/)

- [Omniverse View 2021.2 - Application Tour](https://www.nvidia.com/en-us/on-demand/session/omniverse2020-om1315/)

- [ISAAC SIM Introduction and Live Demo](https://www.nvidia.com/en-us/on-demand/session/omniverse2020-om1314/)

- [NVIDIA on-demand ISAAC SIM](https://www.nvidia.com/en-us/on-demand/search/?facet.mimetype[]=event%20session&layout=list&page=1&q=isaac%20sim&sort=relevance)

## Papers

### Manipulation

- [RLAfford](https://github.com/hyperplane-lab/RLAfford): Official Implementation of "RLAfford: End-to-end Affordance Learning with Reinforcement Learning" ICRA 2023.

- [mvp](https://github.com/ir413/mvp): Masked Visual Pre-training for Robotics

- [RSS2022] Factory: Fast contact for robotic assembly: [paper](http://doi.acm.org/10.1145/3450626.3459670), [project](https://sites.google.com/nvidia.com/factory), [code](https://github.com/NVIDIA-Omniverse/IsaacGymEnvs)

- [SIGGRAPH2022] ASE: Large-scale Reusable Adversarial Skill Embeddings for Physically Simulated Characters: [paper](https://arxiv.org/abs/2205.01906), [project](https://nv-tlabs.github.io/ASE/), [code](https://github.com/NVIDIA-Omniverse/IsaacGymEnvs)

- [arxiv2021] Data-Driven Operational Space Control for Adaptative and Robust Robot Manipulation: [project](https://cremebrule.github.io/oscar-web/), [paper](https://arxiv.org/abs/2110.00704), [code](https://github.com/nvlabs/oscar)

- [RSS2021@DO-Sim] DO-Sim: Workshop on Deformable Object Simulation in Robotics: [link](https://sites.google.com/nvidia.com/do-sim/home)

- [ICRA2021] Causal Reasoning in Simulationfor Structure and Transfer Learning of Robot Manipulation Policies: [paper](https://arxiv.org/pdf/2103.16772.pdf), [project](https://sites.google.com/view/crest-causal-struct-xfer-manip)

- [RSS2021 DO-Sim workshop] DefGraspSim: Simulation-based grasping of 3D deformable objects: [paper](https://arxiv.org/pdf/2107.05778.pdf), [project](https://sites.google.com/nvidia.com/defgraspsim), [video](https://youtu.be/Caj0AtsKKVI), [**code**](https://github.com/NVlabs/deformable_object_grasping)

- [arxiv2021] Search-Based Task Planning with Learned Skill Effect Models for Lifelong Robotic Manipulation: [paper](https://arxiv.org/pdf/2109.08771.pdf), [project](https://sites.google.com/view/sem-for-lifelong-manipulation)

- [2021] Deformation-Aware Data-Driven Grasp Synthesis: [paper](https://arxiv.org/pdf/2109.05320.pdf)

- [2021] Transferring Dexterous Manipulation from GPU Simulation to a Remote Real-World TriFinger: [project](https://s2r2-ig.github.io/), [paper](https://arxiv.org/pdf/2108.09779.pdf), [**code**](https://github.com/pairlab/leibnizgym/)

- [ICRA2021] In-Hand Object Pose Tracking via Contact Feedback and GPU-Accelerated Robotic Simulation: [paper](https://arxiv.org/pdf/2002.12160.pdf), [project](https://sites.google.com/view/in-hand-object-pose-tracking/)

- [IROS2021] Reactive Long Horizon Task Execution via Visual Skill and Precondition Models: [paper](https://arxiv.org/pdf/2011.08694.pdf), [video](https://www.youtube.com/playlist?list=PL-oD0xHUngeLfQmpngYkGFZarstfPOXqX)

- [CoRL2021] STORM: An Integrated Framework for Fast Joint-Space Model-Predictive Control for Reactive Manipulation: [paper](https://arxiv.org/pdf/2104.13542.pdf), [project](https://sites.google.com/view/manipulation-mpc), [**code**](https://github.com/NVlabs/storm)

- [ICRA2021] Sim-to-Real for Robotic Tactile Sensing via Physics-Based Simulation and Learned Latent Projections: [paper](https://arxiv.org/pdf/2103.16747.pdf)

- [2021] DeformerNet: A Deep Learning Approach to 3D Deformable Object Manipulation: [paper](https://arxiv.org/pdf/2107.08067.pdf)

- [RSS2021_VLRR] A Simple Method for Complex In-Hand Manipulation: [paper](https://rssvlrr.github.io/papers/13_CameraReady_RSS2021_VLRR.pdf), [project](https://sites.google.com/view/in-hand-reorientation)

### Localization

- [CoRL2021] Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning: [paper](https://arxiv.org/pdf/2109.11978.pdf), [openreview](https://openreview.net/forum?id=wK2fDDJ5VcF), [**code**](https://github.com/leggedrobotics/legged_gym), [project](https://leggedrobotics.github.io/legged_gym/)

  

- [ICRA2021] Dynamics Randomization Revisited:A Case Study for Quadrupedal Locomotion: [project](https://www.pair.toronto.edu/understanding-dr/), [paper](https://arxiv.org/abs/2011.02404), [video](https://youtu.be/ckdHWWpfSpk)

- [2021] GLiDE: Generalizable Quadrupedal Locomotion in Diverse Environments with a Centroidal Model: [project](https://www.pair.toronto.edu/glide-quadruped/), [paper](https://arxiv.org/abs/2104.09771)

- [CoRL2020] Learning a Contact-Adaptive Controller for Robust, Efficient Legged Locomotion: [paper](https://arxiv.org/abs/2009.10019), [video](https://youtu.be/JJOmFZKpYTo), [project](https://sites.google.com/view/learn-contact-controller/home), [blog](https://developer.nvidia.com/blog/contact-adaptive-controller-locomotion/)

- [RAL2021] Learning a State Representation and Navigation in Cluttered and Dynamic Environments: [paper](https://arxiv.org/pdf/2103.04351.pdf)

- [CoRL2020] Learning to Compose Hierarchical Object-Centric Controllers for Robotic Manipulation: [paper](https://arxiv.org/pdf/2011.04627.pdf), [project](https://sites.google.com/view/compositional-object-control/)

### Others

- [arxiv2021] BayesSimIG: Scalable Parameter Inference for Adaptive Domain Randomization with Isaac Gym: [paper](https://arxiv.org/pdf/2107.04527.pdf), [**code**](https://github.com/NVlabs/bayes-sim-ig)

- [2021] Articulated Object Interaction in Unknown Scenes with Whole-Body Mobile Manipulation: [paper](http://arxiv.org/abs/2103.10534), [video](https://youtu.be/Mwer50_fdCU), [project](https://www.pair.toronto.edu/articulated-mm/)

- [NeurIPS2021] Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning: [project](https://sites.google.com/view/isaacgym-nvidia), [paper](https://arxiv.org/abs/2108.10470), [openreview](https://openreview.net/forum?id=fgFBtYgJQX_)

- [RSS2020] Learning Active Task-Oriented Exploration Policies for Bridging the Sim-to-Real Gap: [paper](https://arxiv.org/pdf/2006.01952.pdf), [project](https://sites.google.com/view/task-oriented-exploration/)

- [ICRA2019] Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experience: [paper](https://arxiv.org/abs/1810.05687), [video](https://youtu.be/nilcJY5Kdt8), [project](https://sites.google.com/view/simopt)

- [CoRL2018] GPU-Accelerated Robotics Simulation for Distributed Reinforcement Learning: [paper](https://arxiv.org/pdf/1810.05762.pdf), [project](https://sites.google.com/view/accelerated-gpu-simulation/home)

## RL library 

These RL libraries can support the training with Isaac Gym.

- [VRKitchen2.0-IndoorKit](https://github.com/yizhouzhao/VRKitchen2.0-IndoorKit): Omniverse IndoorKit Extension

- [rl_games](https://github.com/Denys88/rl_games): rl algorithms with isaac gym

- [ElegantRL](https://github.com/AI4Finance-Foundation/ElegantRL)

- [skrl](https://github.com/Toni-SM/skrl), [paper](https://arxiv.org/abs/2202.03825)

## Related GitHub Repos

- [IsaacGymEnvs](https://github.com/NVIDIA-Omniverse/IsaacGymEnvs)

- [isaacgym_hammering](https://github.com/LiCHOTHU/isaacgym_hammering)

- [isaacgym-utils](https://github.com/iamlab-cmu/isaacgym-utils): Developed by the CMU Intelligent Autonomous Manipulation Lab

- [isaacgym_sandbox](https://github.com/kploeger/isaacgym_sandbox)

- [thormang3-gogoro-PPO](https://github.com/guichristmann/thormang3-gogoro-PPO): Steering-based control of a two-wheeled vehicle using RL-PPO and NVIDIA Isaac Gym

- [dvrk_IssacGym](https://github.com/baotruyenthach/dvrk_IssacGym), [link](https://github.com/baotruyenthach/dvrk_grasp_pipeline_isaacgym)

- [dvrk-shape_servoing](https://github.com/Utah-ARMLab/shape_servoing)

- [codebase_thesis](https://github.com/sivva-dev/codebase_thesis)

- [Bittle_URDF](https://github.com/AIWintermuteAI/Bittle_URDF)

- [SceneCollisionNet](https://github.com/NVlabs/SceneCollisionNet)

- [TD3 applied to the Bittle robot in Isaac Sim](https://github.com/Sentdex/TD3-Bittle)

- [Isaac-ManipulaRL](https://github.com/cypypccpy/Isaac-ManipulaRL) Deep Reinforcement Learning Framework for Manipulator based on NVIDIA's Isaac-gym, Additional add SAC2019 and Reinforcement Learning from Demonstration Algorithm.

- [legged_gym_isaac](https://github.com/chengxuxin/legged_gym_isaac)

- [minimal-isaac-gym](https://github.com/lorenmt/minimal-isaac-gym)

- [DexterousHands](https://github.com/PKU-MARL/DexterousHands): This is a library that provides dual dexterous hand manipulation tasks through Isaac Gym.

- [Safe Multi-Agent Isaac Gym Benchmark](https://github.com/chauncygu/Safe-Multi-Agent-Isaac-Gym): Safe Multi-Agent Isaac Gym benchmark for safe multi-agent reinforcement learning research.

- [Bez_IsaacGym](https://github.com/utra-robosoccer/Bez_IsaacGym): Isaac Gym Reinforcement Learning Environments for humanoid robot Bez

- [Bimanual_offlineRL](https://github.com/ZaneZh/Bimanual_offlineRL)

- [isaac_rover_mars_gym](https://github.com/AAU-RoboticsAutomationGroup/isaac_rover_mars_gym)

- [isaac_rover_2.0](https://github.com/abmoRobotics/isaac_rover_2.0)

- [Rapid Locomotion](https://github.com/Improbable-AI/rapid-locomotion-rl)

- [walk-these-ways](https://github.com/Improbable-AI/walk-these-ways): Go1 Sim-to-Real Locomotion Starter Kit

- [walk-these-ways](https://github.com/Junfeng-Long/walk-these-ways): A1 Sim-to-Real Locomotion Starter Kit

- [shifu](https://github.com/42jaylonw/shifu): Lightweight Isaac Gym Environment Builder for Any Robot

- [Dofbot Reacher](https://github.com/j3soon/OmniIsaacGymEnvs-DofbotReacher): Dofbot Reacher Reinforcement Learning Sim2Real Environment for Omniverse Isaac Gym/Sim

- [UR10Reacher](https://github.com/j3soon/OmniIsaacGymEnvs-UR10Reacher): UR10 Reacher Reinforcement Learning Sim2Real Environment for Omniverse Isaac Gym/Sim

- [minimal-isaac-gym](https://github.com/lorenmt/minimal-isaac-gym): A Minimal Example of Isaac Gym with DQN and PPO.

- [TimeChamber](https://github.com/inspirai/TimeChamber): A Massively Parallel Large Scale Self-Play Framework.

- [Rofunc](https://github.com/Skylark0924/Rofunc): The Full Process Python Package for Robot Learning from Demonstration.

- [rl-mpc-locomotion](https://github.com/silvery107/rl-mpc-locomotion): Deep RL for MPC control of Quadruped Robot Locomotion.

## Tutorials & Videos

### RSS2021 Workshop (https://sites.google.com/view/isaacgym/home)

- [Isaac Gym Part 1: Introduction and Getting Started](https://youtu.be/nleDq-oJjGk)

- [Isaac Gym Part 2: Environments, Training, and Tips](https://youtu.be/1RSugmJ4_gs)

- [Isaac Gym Part 3A: Academic Labs - University of Toronto](https://youtu.be/nXM5_mwUFOI)

- [Isaac Gym Part 3B: Academic Labs - IMLab](https://youtu.be/VrTVUpDM7K8)

- [Isaac Gym Part 3C: Academic Labs - Stanford University](https://youtu.be/RhjRrUK2abs))

- [Isaac Gym Part 3D: Academic Labs - Soft-Body Simulation](https://youtu.be/i4fGVc6lImo)

- [Isaac Gym Part 3E: Academic Labs - Eth Zurich](https://youtu.be/Afi17BnSuBM)

- [Isaac Gym Part 4: New Frontiers in End-to-End GPU Accelerated Reinforcement Learning](https://youtu.be/WhaybakLTXE)

### Videos

- [How to Import Your Robot Into Isaac Sim in NVIDIA Omniverse](https://youtu.be/pxPFr58gHmQ?list=PL3jK4xNnlCVf1SzxjCm7ZxDBNl9QYyV8X)

- [Youtube NVIDIA Omniverse](https://www.youtube.com/channel/UCSKUoczbGAcMld7HjpCR8OA)

- [Basic Demo of the NVIDIA Isaac Simulator (Part 1)](https://www.youtube.com/watch?v=b12M_kCW82o)

- [Basic Demo of the NVIDIA Isaac Simulator (Part 2)](https://youtu.be/XcvMCs9NJfM)

- [Introduction and Live Demo in Isaac Sim - Community Stream](https://youtu.be/vpHR0qiH-GY)

- [From Point Clouds to Material Graphs: Explore the Latest in Omniverse Create 2021.3](https://youtu.be/t9nVWhnOgbE)

- [Robot Autonomy with the Digital Twin in Isaac Sim](https://youtu.be/vOEdzxR-_Iw)

- [Can we simulate a real robot?](https://youtu.be/phTnbmXM06g) A journey through trying to find a high quality physics simulator for a robot dog/quadruped (using the Petoi Bittle in this case).

- [Teaching Robots to Walk w/ Reinforcement Learning](https://youtu.be/6qbW7Ki9NUc) Robot sim adventure video part two, covering my attempts to get some reinforcement learning to work with the Bittle robot in the Isaac sim. 

- [Robot Dog Learns to Walk - Bittle Reinforcement Learning p.3](https://youtu.be/A0tPe7-R8z0) Further progress with using reinforcement learning to train robot dogs/quadrupeds to walk

### Blogs

- [A brief introduction to Nvidia Omniverse](https://zhuanlan.zhihu.com/p/462305733)