https://github.com/microsoft/DeepSpeed

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
https://github.com/microsoft/DeepSpeed

billion-parameters compression data-parallelism deep-learning gpu inference machine-learning mixture-of-experts model-parallelism pipeline-parallelism pytorch trillion-parameters zero

Last synced: 8 days ago
JSON representation

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

• Host: GitHub
• URL: https://github.com/microsoft/DeepSpeed
• Owner: microsoft
• License: apache-2.0
• Created: 2020-01-23T18:35:18.000Z (almost 5 years ago)
• Default Branch: master
• Last Pushed: 2024-04-14T05:36:34.000Z (7 months ago)
• Last Synced: 2024-04-14T09:57:49.543Z (7 months ago)
• Topics: billion-parameters, compression, data-parallelism, deep-learning, gpu, inference, machine-learning, mixture-of-experts, model-parallelism, pipeline-parallelism, pytorch, trillion-parameters, zero
• Language: Python
• Homepage: https://www.deepspeed.ai/
• Size: 203 MB
• Stars: 32,440
• Watchers: 324
• Forks: 3,820
• Open Issues: 1,035
• Metadata Files:
  • Readme: README.md
  • Contributing: CONTRIBUTING.md
  • License: LICENSE
  • Code of conduct: CODE_OF_CONDUCT.md
  • Codeowners: CODEOWNERS
  • Security: SECURITY.md

Awesome Lists containing this project

• awesome - microsoft/DeepSpeed - DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective. (Python)
• awesome-local-ai - DeepSpeed - DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective. (Training)
• awesome-azure-openai-llm - DeepSpeed
• awesome-high-performance-computing - DeepSpeed - An easy-to-use deep learning optimization software suite that enables unprecedented scale and speed for Deep Learning Training and Inference (Software / Trends)
• awesome-open-gpt - DeepSpeed Chat🔥
• awesome-llmops - DeepSpeed -  - A deep learning optimization library that makes distributed training and inference easy, efficient, and effective. (Tools (GitHub) / LLMOps vs MLOps)
• Awesome_Multimodel_LLM - DeepSpeed - DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective. (LLM Training Frameworks)
• awesome-lm-system - DeepSpeed - parallelism | (Frameworks / Survey)
• my-awesome-starred - microsoft/DeepSpeed - DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective. (Python)
• awesome - microsoft/DeepSpeed - DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective. (Python)
• awesome-mlvid - DeepSpeed - DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective. (MLVU Frameworks / Weakly-supervised learning)
• awesome-llmops - DeepSpeed - square) | (Training / Frameworks for Training)
• awesome-mojo - DeepSpeed
• awesome-llm-eval - DeepSpeed - DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective. (Frameworks-for-Training / Popular-LLM)
• awesome-starts - microsoft/DeepSpeed - DeepSpeed is a deep learning optimization library that makes distributed training easy, efficient, and effective. (Python)
• AiTreasureBox - microsoft/DeepSpeed - 11-02_35295_3](https://img.shields.io/github/stars/microsoft/DeepSpeed.svg) |A deep learning optimization library that makes distributed training and inference easy, efficient, and effective| (Repos)
• awesome-list - DeepSpeed - An easy-to-use deep learning optimization software suite that enables unprecedented scale and speed for Deep Learning Training and Inference. (Deep Learning Framework / Deployment & Distribution)
• StarryDivineSky - microsoft/DeepSpeed
• awesome-mixture-of-experts - DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale
• Awesome-LLM - DeepSpeed - DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective. (LLM Training Frameworks)
• awesome-mlops - DeepSpeed - Deep learning optimization library that makes distributed training easy, efficient, and effective. (Optimization Tools)
• awesome-gpt - DeepSpeed - DeepSpeed empowers ChatGPT-like model training with a single click, offering 15x speedup over SOTA RLHF systems with unprecedented cost reduction at all scales. (Projects and Tools / LLM (Large Language Model))
• awesome-python-machine-learning-resources - GitHub - 48% open · ⏱️ 25.08.2022): (分布式机器学习)
• awesome-hacking-lists - microsoft/DeepSpeed - DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective. (Python)
• awesome-production-machine-learning - DeepSpeed - A deep learning optimization library (lightweight PyTorch wrapper) that makes distributed training easy, efficient, and effective. (Computation Load Distribution)
• Awesome-ChatGPT - DeepSpeed Chat🔥
• awesome-huggingface - Deepspeed - Deepspeed-ZeRO - scales any model size with zero to no changes to the model. [Integrated with HF Trainer](https://huggingface.co/docs/transformers/master/main_classes/deepspeed). (🌗 Model Scalability)
• awesome-ai-papers - [DeepSpeed - us/research/blog/zero-deepspeed-new-system-optimizations-enable-training-models-with-over-100-billion-parameters/)\] (NLP / 3. Pretraining)
• awesome-ai-papers - [DeepSpeed - us/research/blog/zero-deepspeed-new-system-optimizations-enable-training-models-with-over-100-billion-parameters/)\] (NLP / 3. Pretraining)