https://github.com/ray-project/distml
Distributed ML Optimizer
https://github.com/ray-project/distml
Last synced: over 1 year ago
JSON representation
Distributed ML Optimizer
- Host: GitHub
- URL: https://github.com/ray-project/distml
- Owner: ray-project
- Created: 2021-04-12T20:21:11.000Z (over 5 years ago)
- Default Branch: master
- Last Pushed: 2021-07-28T23:09:51.000Z (almost 5 years ago)
- Last Synced: 2025-04-13T00:45:22.706Z (over 1 year ago)
- Language: Python
- Size: 3.12 MB
- Stars: 31
- Watchers: 7
- Forks: 1
- Open Issues: 9
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Introduction
*DistML* is a [Ray](https://github.com/ray-project/ray) extension library to support large-scale distributed ML training
on heterogeneous multi-node multi-GPU clusters. This library is under active development and we are adding more advanced
training strategies and auto-parallelization features.
DistML currently supports:
* Distributed training strategies
* Data parallelism
* AllReduce strategy
* Sharded parameter server strategy
* BytePS strategy
Pipeline parallleism
* Micro-batch pipeline parallelism
* DL Frameworks:
* PyTorch
* JAX
# Installation
### Install Dependencies
Depending on your CUDA version, install cupy following https://docs.cupy.dev/en/stable/install.html.
### Install from source for dev
```python
pip install -e .
```