Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/pytorch/torchx
TorchX is a universal job launcher for PyTorch applications. TorchX is designed to have fast iteration time for training/research and support for E2E production ML pipelines when you're ready.
https://github.com/pytorch/torchx
airflow aws-batch components deep-learning distributed-training kubernetes machine-learning pipelines python pytorch ray slurm
Last synced: about 1 month ago
JSON representation
TorchX is a universal job launcher for PyTorch applications. TorchX is designed to have fast iteration time for training/research and support for E2E production ML pipelines when you're ready.
- Host: GitHub
- URL: https://github.com/pytorch/torchx
- Owner: pytorch
- License: other
- Created: 2021-05-04T18:06:53.000Z (over 3 years ago)
- Default Branch: main
- Last Pushed: 2024-09-23T15:57:52.000Z (about 2 months ago)
- Last Synced: 2024-09-30T15:23:18.077Z (about 1 month ago)
- Topics: airflow, aws-batch, components, deep-learning, distributed-training, kubernetes, machine-learning, pipelines, python, pytorch, ray, slurm
- Language: Python
- Homepage: https://pytorch.org/torchx
- Size: 33.6 MB
- Stars: 328
- Watchers: 21
- Forks: 109
- Open Issues: 87
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
Awesome Lists containing this project
README
[](https://pypi.org/project/torchx/)
[](https://github.com/pytorch/torchx/blob/main/LICENSE)
[](https://codecov.io/gh/pytorch/torchx)
# TorchX
TorchX is a universal job launcher for PyTorch applications.
TorchX is designed to have fast iteration time for training/research and support
for E2E production ML pipelines when you're ready.
TorchX currently supports:
* Kubernetes (EKS, GKE, AKS, etc)
* Slurm
* AWS Batch
* Docker
* Local
* Ray (prototype)
* GCP Batch (prototype)
Need a scheduler not listed? [Let us know!](https://github.com/pytorch/torchx/issues?q=is%3Aopen+is%3Aissue+label%3Ascheduler-request)
## Quickstart
See the [quickstart guide](https://pytorch.org/torchx/latest/quickstart.html).
## Documentation
* [Stable Documentation](https://pytorch.org/torchx/latest/)
* [Nightly Documentation](https://pytorch.org/torchx/main/)
## Requirements
torchx:
* python3 (3.8+)
* [PyTorch](https://pytorch.org/get-started/locally/)
* optional: [Docker](https://docs.docker.com/get-docker/) (needed for docker based schedulers)
Certain schedulers may require scheduler specific requirements. See installation
for info.
## Installation
### Stable
```bash
# install torchx sdk and CLI -- minimum dependencies
pip install torchx
# install torchx sdk and CLI -- all dependencies
pip install "torchx[dev]"
# install torchx kubeflow pipelines (kfp) support
pip install "torchx[kfp]"
# install torchx Kubernetes / Volcano support
pip install "torchx[kubernetes]"
# install torchx Ray support
pip install "torchx[ray]"
# install torchx GCP Batch support
pip install "torchx[gcp_batch]"
```
### Nightly
```bash
# install torchx sdk and CLI
pip install torchx-nightly[dev]
```
### Source
```bash
# install torchx sdk and CLI from source
$ pip install -e git+https://github.com/pytorch/torchx.git#egg=torchx
# install extra dependencies
$ pip install -e git+https://github.com/pytorch/torchx.git#egg=torchx[dev]
```
### Docker
TorchX provides a docker container for using as as part of a TorchX role.
See: https://github.com/pytorch/torchx/pkgs/container/torchx
## Contributing
We welcome PRs! See the [CONTRIBUTING](https://github.com/pytorch/torchx/blob/main/CONTRIBUTING.md) file.
## License
TorchX is BSD licensed, as found in the [LICENSE](https://github.com/pytorch/torchx/blob/main/LICENSE) file.