Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
Projects in Awesome Lists tagged with pipeline-parallelism
A curated list of projects in awesome lists tagged with pipeline-parallelism .
https://github.com/hpcaitech/ColossalAI
Making large AI models cheaper, faster and more accessible
ai big-model data-parallelism deep-learning distributed-computing foundation-models heterogeneous-training hpc inference large-scale model-parallelism pipeline-parallelism
Last synced: 31 Jul 2024
https://github.com/hpcaitech/colossalai
Making large AI models cheaper, faster and more accessible
ai big-model data-parallelism deep-learning distributed-computing foundation-models heterogeneous-training hpc inference large-scale model-parallelism pipeline-parallelism
Last synced: 02 Aug 2024
https://github.com/microsoft/deepspeed
DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
billion-parameters compression data-parallelism deep-learning gpu inference machine-learning mixture-of-experts model-parallelism pipeline-parallelism pytorch trillion-parameters zero
Last synced: 29 Sep 2024
https://github.com/microsoft/DeepSpeed
DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
billion-parameters compression data-parallelism deep-learning gpu inference machine-learning mixture-of-experts model-parallelism pipeline-parallelism pytorch trillion-parameters zero
Last synced: 31 Jul 2024
https://github.com/bigscience-workshop/petals
🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
bloom chatbot deep-learning distributed-systems falcon gpt guanaco language-models large-language-models llama machine-learning mixtral neural-networks nlp pipeline-parallelism pretrained-models pytorch tensor-parallelism transformer volunteer-computing
Last synced: 29 Sep 2024
https://github.com/kakaobrain/torchgpipe
A GPipe implementation in PyTorch
checkpointing deep-learning gpipe model-parallelism parallelism pipeline-parallelism pytorch
Last synced: 01 Oct 2024
https://github.com/Oneflow-Inc/libai
LiBai(李白): A Toolbox for Large-Scale Distributed Parallel Training
data-parallelism deep-learning distributed-training large-scale model-parallelism nlp oneflow pipeline-parallelism self-supervised-learning transformer vision-transformer
Last synced: 03 Aug 2024
https://github.com/alibaba/EasyParallelLibrary
Easy Parallel Library (EPL) is a general and efficient deep learning framework for distributed model training.
data-parallelism deep-learning distributed-training gpu memory-efficient model-parallelism pipeline-parallelism
Last synced: 01 Aug 2024
https://github.com/torchpipe/torchpipe
Serving Inside Pytorch With Multi-threads
deployment inference llm-serving pipeline-parallelism pytorch ray serve serving tensorrt torch2trt triton-inference-server
Last synced: 31 Jul 2024
https://github.com/AlibabaPAI/DAPPLE
An Efficient Pipelined Data Parallel Approach for Training Large Model
distribution-strategy-planner hybrid-parallelism pipeline-parallelism
Last synced: 01 Aug 2024
https://github.com/saareliad/FTPipe
FTPipe and related pipeline model parallelism research.
deep-neural-networks distributed-training fine-tuning nlp pipeline-parallelism t5
Last synced: 01 Aug 2024
https://github.com/Shigangli/Chimera
Chimera: Efficiently Training Large-Scale Neural Networks with Bidirectional Pipelines.
distributed-deep-learning pipeline-parallelism transformers
Last synced: 01 Aug 2024
