Projects in Awesome Lists tagged with tensor-parallelism
A curated list of projects in awesome lists tagged with tensor-parallelism .
https://github.com/bigscience-workshop/petals
🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
bloom chatbot deep-learning distributed-systems falcon gpt guanaco language-models large-language-models llama machine-learning mixtral neural-networks nlp pipeline-parallelism pretrained-models pytorch tensor-parallelism transformer volunteer-computing
Last synced: 13 May 2025
https://github.com/internlm/internevo
InternEvo is an open-sourced lightweight training framework aims to support model pre-training without the need for extensive dependencies.
910b deepspeed-ulysses flash-attention gemma internlm internlm2 llama3 llava llm-framework llm-training multi-modal pipeline-parallelism pytorch ring-attention sequence-parallelism tensor-parallelism transformers-models zero3
Last synced: 14 Apr 2025
https://github.com/InternLM/InternEvo
InternEvo is an open-sourced lightweight training framework aims to support model pre-training without the need for extensive dependencies.
910b deepspeed-ulysses flash-attention gemma internlm internlm2 llama3 llava llm-framework llm-training multi-modal pipeline-parallelism pytorch ring-attention sequence-parallelism tensor-parallelism transformers-models zero3
Last synced: 27 Mar 2025
https://github.com/ai-decentralized/bloombee
Decentralized LLMs fine-tuning and inference with offloading
deep-learning distributed-systems llama machine-learning pipeline-parallelism pytorch tensor-parallelism
Last synced: 07 Apr 2025
https://github.com/yottalabsai/bloombee
Decentralized LLMs fine-tuning and inference with offloading
deep-learning distributed-systems llama machine-learning pipeline-parallelism pytorch tensor-parallelism
Last synced: 07 Apr 2025
https://github.com/xrsrke/pipegoose
Large scale 4D parallelism pre-training for 🤗 transformers in Mixture of Experts *(still work in progress)*
3d-parallelism data-parallelism distributed-optimizers huggingface-transformers large-scale-language-modeling megatron megatron-lm mixture-of-experts model-parallelism moe pipeline-parallelism sequence-parallelism tensor-parallelism transformers zero-1
Last synced: 20 Nov 2024
