https://github.com/microsoft/Megatron-DeepSpeed

Ongoing research training transformer language models at scale, including: BERT & GPT-2
https://github.com/microsoft/Megatron-DeepSpeed

Last synced: about 2 months ago
JSON representation

Ongoing research training transformer language models at scale, including: BERT & GPT-2

• Host: GitHub
• URL: https://github.com/microsoft/Megatron-DeepSpeed
• Owner: microsoft
• License: other
• Fork: true (NVIDIA/Megatron-LM)
• Created: 2021-06-21T22:26:38.000Z (over 3 years ago)
• Default Branch: main
• Last Pushed: 2024-10-18T10:31:05.000Z (2 months ago)
• Last Synced: 2024-10-20T06:27:10.912Z (2 months ago)
• Language: Python
• Homepage:
• Size: 7.77 MB
• Stars: 1,867
• Watchers: 24
• Forks: 343
• Open Issues: 146
• Metadata Files:
  • Readme: README.md
  • License: LICENSE
  • Codeowners: CODEOWNERS
  • Security: SECURITY.md

Awesome Lists containing this project

• Awesome_Multimodel_LLM - Megatron-DeepSpeed - DeepSpeed version of NVIDIA's Megatron-LM that adds additional support for several features such as MoE model training, Curriculum Learning, 3D Parallelism, and others. (LLM Training Frameworks)
• awesome-llm - Megatron-DeepSpeed - NVIDIA Megatron-LM的DeepSpeed版本，增强了对MoE模型训练、课程学习、3D并行等特性的支持。 (LLM训练框架 / LLM 评估工具)
• awesome-llm - Megatron-DeepSpeed - NVIDIA Megatron-LM的DeepSpeed版本，增强了对MoE模型训练、课程学习、3D并行等特性的支持。 (LLM训练框架 / LLM 评估工具)
• awesome-lm-system - Megatron-DeepSpeed
• awesome-mlvid - Megatron-DeepSpeed - DeepSpeed version of NVIDIA's Megatron-LM that adds additional support for several features such as MoE model training, Curriculum Learning, 3D Parallelism, and others. (MLVU Frameworks / Weakly-supervised learning)
• Awesome-LLM - Megatron-DeepSpeed - DeepSpeed version of NVIDIA's Megatron-LM that adds additional support for several features such as MoE model training, Curriculum Learning, 3D Parallelism, and others. (LLM Training Frameworks)