https://github.com/lucidrains/diffusion-policy

Implementation of Diffusion Policy, Toyota Research's supposed breakthrough in leveraging DDPMs for learning policies for real-world Robotics
https://github.com/lucidrains/diffusion-policy

artificial-intelligence attention-mechanisms deep-learning denoising-diffusion robotics transformers

Last synced: about 11 hours ago
JSON representation

Implementation of Diffusion Policy, Toyota Research's supposed breakthrough in leveraging DDPMs for learning policies for real-world Robotics

• Host: GitHub
• URL: https://github.com/lucidrains/diffusion-policy
• Owner: lucidrains
• License: mit
• Created: 2023-09-20T20:25:32.000Z (about 1 year ago)
• Default Branch: main
• Last Pushed: 2024-07-06T15:12:02.000Z (3 months ago)
• Last Synced: 2024-07-06T16:29:19.919Z (3 months ago)
• Topics: artificial-intelligence, attention-mechanisms, deep-learning, denoising-diffusion, robotics, transformers
• Language: Python
• Homepage:
• Size: 1.02 MB
• Stars: 70
• Watchers: 4
• Forks: 1
• Open Issues: 2
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        


## Diffusion Policy (wip)

Implementation of Diffusion Policy, Toyota Research's supposed breakthrough in leveraging DDPMs for learning policies for real-world Robotics

What seemed to have happened is that a research group at Columbia adapted the popular SOTA text-to-image models (complete with denoising diffusion with cross attention conditioning) to policy generation (predicting robot actions conditioned on observations). Toyota research then validated this at a certain scale for imitation learning with real world robotic demonstrations. It is hard to know how much of a breakthrough this is given corporate press is prone to exaggerations, but let me try to get a clean implementation out, just in the case that it is.

The great thing is, if this really works, all the advances being made in text-to-image space can translate to robotics. Yes, this includes stuff like dreambooth.

Discord

## Todo

- [ ] add rlhf

- [ ] add adversarial distillation

## Citations

```bibtex

@article{Chi2023DiffusionPV,

    title   = {Diffusion Policy: Visuomotor Policy Learning via Action Diffusion},

    author  = {Cheng Chi and Siyuan Feng and Yilun Du and Zhenjia Xu and Eric A. Cousineau and Benjamin Burchfiel and Shuran Song},

    journal = {ArXiv},

    year    = {2023},

    volume  = {abs/2303.04137},

    url     = {https://api.semanticscholar.org/CorpusID:257378658}

}

```

```bibtex

@article{Sauer2023AdversarialDD,

    title   = {Adversarial Diffusion Distillation},

    author  = {Axel Sauer and Dominik Lorenz and A. Blattmann and Robin Rombach},

    journal = {ArXiv},

    year    = {2023},

    volume  = {abs/2311.17042},

    url     = {https://api.semanticscholar.org/CorpusID:265466173}

}

```