Projects in Awesome Lists tagged with megatron-lm
A curated list of projects in awesome lists tagged with megatron-lm .
https://github.com/alibaba/Megatron-LLaMA
Best practice for training LLaMA models in Megatron-LM
deepspeed distributed-training llama llm megatron-lm pretraining pytorch
Last synced: 27 Mar 2025
https://github.com/shreyansh26/annotated-ml-papers
Annotations of the interesting ML papers I read
annotated-paper bert deep-learning gpt gpt-2 machine-learning megatron-lm nlp papers-annotations research-paper transformers xlnet
Last synced: 03 Mar 2025
https://github.com/xrsrke/pipegoose
Large scale 4D parallelism pre-training for 🤗 transformers in Mixture of Experts *(still work in progress)*
3d-parallelism data-parallelism distributed-optimizers huggingface-transformers large-scale-language-modeling megatron megatron-lm mixture-of-experts model-parallelism moe pipeline-parallelism sequence-parallelism tensor-parallelism transformers zero-1
Last synced: 20 Nov 2024
https://github.com/feifeibear/odysseus-transformer
Odysseus: Playground of LLM Sequence Parallelism
Last synced: 22 Nov 2024
https://github.com/mofheka/llama-megatron
A LLaMA1/LLaMA12 Megatron implement.
llama llama2 llm llm-training megatron megatron-lm pytorch
Last synced: 23 Apr 2025
https://github.com/googlecloudplatform/nvidia-nemo-on-gke
Training NVIDIA NeMo Megatron Large Language Model (LLM) using NeMo Framework on Google Kubernetes Engine
gke megatron-lm nvidia nvidia-gpu nvidia-nemo
Last synced: 05 Feb 2025
https://github.com/janelu9/easyllm
Running Large Language Model easily.
deepseek deepspeed fine-tuning llama megatron-lm npu pretrain qwen qwen-vl
Last synced: 24 Apr 2025
https://github.com/janelu9/flash-finetuning
Running Large Language Model easily.
deepspeed fine-tuning llama3 llm megatron-lm pretrain qwen2-vl vlm
Last synced: 22 Dec 2024
https://github.com/beomi/megatronlm_dataset_autotokenizer
Megatron-LM/GPT-NeoX compatible Text Encoder with 🤗Transformers AutoTokenizer.
gpt-neox megatron-lm tokenizers transformers
Last synced: 07 May 2025
