https://github.com/learning-at-home/dalle-hivemind

Training a model similar to OpenAI DALL-E with volunteers from all over the Internet using hivemind and dalle-pytorch (NeurIPS 2021 demo)
https://github.com/learning-at-home/dalle-hivemind

Last synced: 6 days ago
JSON representation

Training a model similar to OpenAI DALL-E with volunteers from all over the Internet using hivemind and dalle-pytorch (NeurIPS 2021 demo)

• Host: GitHub
• URL: https://github.com/learning-at-home/dalle-hivemind
• Owner: learning-at-home
• License: mit
• Created: 2021-11-02T18:38:02.000Z (about 3 years ago)
• Default Branch: main
• Last Pushed: 2023-05-29T11:57:17.000Z (over 1 year ago)
• Last Synced: 2024-08-01T13:30:24.505Z (3 months ago)
• Language: Python
• Homepage: https://training-transformers-together.github.io/
• Size: 58.6 KB
• Stars: 27
• Watchers: 5
• Forks: 1
• Open Issues: 2
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
## Training DALL-E with volunteers from all over the Internet



This repository is a part of the NeurIPS 2021 demonstration "Training Transformers Together".

  



In this demo, we train a model similar to OpenAI DALL-E —

a Transformer "language model" that generates images from text descriptions.

Training happens collaboratively — volunteers from all over the Internet contribute to the training using hardware available to them.

We use LAION-400M,

the world's largest openly available image-text-pair dataset with 400 million samples. Our model is based on

the dalle‑pytorch implementation

by Phil Wang with a few tweaks to make it communication-efficient.





See details about how to join and how it works on our website.