https://github.com/real-stanford/dgdm
[CoRL'24] Dynamics-Guided Diffusion Model for Robot Manipulator Design
https://github.com/real-stanford/dgdm
machine-learning robotics
Last synced: 11 months ago
JSON representation
[CoRL'24] Dynamics-Guided Diffusion Model for Robot Manipulator Design
- Host: GitHub
- URL: https://github.com/real-stanford/dgdm
- Owner: real-stanford
- Created: 2024-02-23T17:23:14.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2024-07-03T02:04:37.000Z (almost 2 years ago)
- Last Synced: 2025-04-09T18:12:16.419Z (about 1 year ago)
- Topics: machine-learning, robotics
- Language: Python
- Homepage: https://dgdm-robot.github.io/
- Size: 55.7 KB
- Stars: 39
- Watchers: 3
- Forks: 2
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Dynamics-Guided Diffusion Model for Robot Manipulator Design
### [Paper](https://arxiv.org/abs/2402.15038) | [Website](https://dgdm-robot.github.io) | [Video](https://www.youtube.com/watch?v=0m5nTWgHULg)
[Xiaomeng Xu](https://xxm19.github.io/), [Huy Ha](https://www.cs.columbia.edu/~huy/), [Shuran Song](https://shurans.github.io/)
### Dependencies
Required packages can be installed by:
```
pip install -r requirements.txt
```
## Data Preparation
### Download object dataset
#### 2D objects
Download 2D object icons from [Icons50 dataset](https://www.kaggle.com/datasets/danhendrycks/icons50).
#### 3D objects
Download 3D object meshes from [MuJoCo scanned object dataset](https://github.com/kevinzakka/mujoco_scanned_objects).
### Generate simulation data
Replace ```OBJECT_DIR``` in ```sim/sim_2d.py``` and ```sim/sim_3d.py``` with the directory to object dataset.
Install [v-hacd](https://github.com/kmammou/v-hacd).
#### 2D
```
bash sim/run_sim_2d.sh
```
#### 3D
```
bash sim/run_sim_3d.sh
```
Note for data generation: Sometimes the sampled objects or manipulators may have weird shapes and thus lead to qhull error when doing convex decomposition. And ray is used to parallelize cpu-based data generation, which sometimes may lead to timeout issues. Therefore, it is expected if you see some error message and the data for some object-manipulator pairs is not generated, but it should be fine as long as you see most data is being generated.
## Training
[Download pretrained model checkpoints](https://drive.google.com/drive/folders/1jjC6G5Qv_ZkJwTjk2mCBkSyXkZu_w5EB?usp=sharing)
### Train Dynamics Model
#### 2D
```
bash dynamics/train_dynamics_2d.sh
```
#### 3D
```
bash dynamics/train_dynamics_3d.sh
```
### Train Diffusion Model
#### 2D
```
bash generator/train_diffusion_2d.sh
```
#### 3D
```
bash generator/train_diffusion_3d.sh
```
## Inference
### Generate Task-Specific Manipulators
#### 2D
```
bash generator/guided_sample_2d.sh
```
#### 3D
```
bash generator/guided_sample_3d.sh
```
## Citation
If you find DGDM useful for your work, please cite:
```
@misc{xu2024dynamicsguided,
title={Dynamics-Guided Diffusion Model for Robot Manipulator Design},
author={Xiaomeng Xu and Huy Ha and Shuran Song},
year={2024},
eprint={2402.15038},
archivePrefix={arXiv},
primaryClass={cs.RO}
}
```
## Contact
If you have any questions, please feel free to contact Xiaomeng Xu (xuxm@stanford.edu)