Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/real-stanford/xskill
[CoRL 2023] XSkill: cross embodiment skill discovery
https://github.com/real-stanford/xskill
robotics
Last synced: 2 months ago
JSON representation
[CoRL 2023] XSkill: cross embodiment skill discovery
- Host: GitHub
- URL: https://github.com/real-stanford/xskill
- Owner: real-stanford
- License: mit
- Created: 2023-09-27T14:27:09.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2023-10-27T15:05:43.000Z (about 1 year ago)
- Last Synced: 2023-10-27T17:26:10.115Z (about 1 year ago)
- Topics: robotics
- Language: Python
- Homepage: https://xskill.cs.columbia.edu/
- Size: 11.6 MB
- Stars: 3
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# XSkill: Cross Embodiment Skill Discovery
1,2[Mengda Xu](https://mengdaxu.github.io/), 1,[Zhenjia Xu](https://www.zhenjiaxu.com/), 1[Cheng Chi](https://cheng-chi.github.io/), 2,3[Manuela Veloso](https://www.cs.cmu.edu/~mmv/), 1[Shuran Song](https://shurans.github.io/)
1Columbia University, 2JP Morgan AI Research,3CMU
**CoRL 2023**
[Project Page](https://xskill.cs.columbia.edu/)|[arxiv](https://arxiv.org/pdf/2307.09955.pdf)
This repository contains code for training and evaluating XSkill in both simulation and real-world settings.
## 🚀 Installation
Follow these steps to install `XSkill`:
1. Create and activate the conda environment:
```bash
cd xskill
conda env create -f environment.yml
conda activate xskill
pip install -e .
```
## 📦 Simulation Dataset
To set up the simulation dataset:
1. Create a new directory for datasets under XSkill:
```bash
mkdir datasets
cd datasets
wget https://xskill.cs.columbia.edu/data/kitchen.zip
unzip kitchen.zip
```
2. Set the `base_dev_dir` config/simulation/create_kitchen_datase.yaml to your working directory. Run the following command to generate the cross-embodiment kitchen data and the training mask:
```bash
cd scripts
python create_all_kitchen_dataset.py
python extract_kitchen_info.py
```
## 🌐 Real World Dataset
To Download the real world kitchen dataset:
```bash
mkdir datasets
cd datasets
wget https://xskill.cs.columbia.edu/data/real_kitchen_data.zip
```
## 🚴♂️ Training
### Simulation
1. Run the skill discovery script:
```bash
python scripts/skill_discovery.py
```
2. Label the dataset using the learned prototype by the trained model.
```bash
python scripts/label_sim_kitchen_dataset.py
```
3. Execute the skill transfer and composing script. Replace the pretrain_path and pretrain_ckpt in cfg/simulation/skill_transfer_composing.yaml
```
python scripts/skill_transfer_composing.py
```
### Real World
1. Execute the real-world skill discovery script:
```bash
python scripts/realworld/skill_discovery.py
```
2. Label the real-world dataset:
```bash
python scripts/realworld/label_real_kitchen_dataset.py
```
3. 📊 Visualization
Open the provided Jupyter notebook `viz_real.ipynb` to visualize the learned prototypes:
### BibTeX
```bash
@inproceedings{
xu2023xskill,
title={{XS}kill: Cross Embodiment Skill Discovery},
author={Mengda Xu and Zhenjia Xu and Cheng Chi and Manuela Veloso and Shuran Song},
booktitle={7th Annual Conference on Robot Learning},
year={2023},
url={https://openreview.net/forum?id=8L6pHd9aS6w}
}
```
### License
This repository is released under the MIT license.
### Acknowledgement
* Diffusion Policy is adapted from [Diffusion Policy](https://github.com/real-stanford/diffusion_policy)
* Many useful utilies are adapted from [XIRL](https://x-irl.github.io/).