https://github.com/ler0ever/hpgo
Development of Project HPGO | Hybrid Parallelism Global Orchestration
https://github.com/ler0ever/hpgo
data-parallelism distributed-training gpipe machine-learning model-parallelism pipedream pipeline-parallelism pytorch rust tensorflow
Last synced: 2 months ago
JSON representation
Development of Project HPGO | Hybrid Parallelism Global Orchestration
- Host: GitHub
- URL: https://github.com/ler0ever/hpgo
- Owner: LER0ever
- Created: 2020-02-05T07:55:48.000Z (over 5 years ago)
- Default Branch: master
- Last Pushed: 2021-03-26T08:33:48.000Z (about 4 years ago)
- Last Synced: 2024-10-29T07:06:27.144Z (7 months ago)
- Topics: data-parallelism, distributed-training, gpipe, machine-learning, model-parallelism, pipedream, pipeline-parallelism, pytorch, rust, tensorflow
- Homepage: https://HPGO.rongyi.io
- Size: 5.29 MB
- Stars: 3
- Watchers: 3
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# HPGO
Hybrid Parallelism Global Orchestration
[](https://pypi.org/project/HPGO/)
[](https://crates.io/crates/HPGO)
[](https://dev.azure.com/rongyi-io/HPGO/_build/latest?definitionId=1&branchName=master)
[](https://gitlab.com/rongyi.io/work/alibaba/HPGO/pipelines)
[](https://renovatebot.com)
[](https://opensource.org/licenses/BSD-3-Clause)
_This project is in Alpha Stage: The functionality provided should be considered research grade and the name "HPGO" is expected to change upon stable release_
## Announcement
Repository temporarily moved to Lab: https://lab.rongyi.io/research/ml/HPGO until further notice. Previous archive can be downloaded [here](https://github.com/AlibabaPAI/DAPPLE) and [here](https://cdn.rongyi.io/bdist/research/ml/HPGO/).
## Get Started
### Install
The binary package of this library is provided for every release.
#### From Rust Cargo, as a Rust crate
(deprecated)
```bash
cargo install HPGO
```
#### From Python PyPI, as a Python3 package
(release only)
```bash
pip3 install HPGO
```
### Use
```python
# Import HPGO Python API from HPGO.so
import HPGO
# Construct the Conductor object
c = HPGO.conductor_from_torch_graph_and_seps("./profiles/xlnet/graph.txt", 64, 512, [8, 16])
res = c.py_orchestrate()
print(res)
```
## Contribution
## License
This project is open sourced under the terms of BSD-3-Clause, details of which can be found in the [`LICENSE`](LICENSE) file
If you apply this library to any project and research, please cite this code:
```
@misc{HPGO,
author = {Yi Rong},
title = {HPGO, [{Hybrid Parallelism} Global Orchestration]},
howpublished = {\url{https://github.com/LER0ever/HPGO}},
year = {2020}
}
```
The project contains source code from [PipeDream](https://github.com/msr-fiddle/pipedream), a Microsoft Research project licensed under the MIT License. Code taken from Pipedream includes a Rust file `src/input/torch_graph_py.rs`, several profiling data files under the `profiles` directory, and the `contrib` directory.
This project also contains profiling data provided by the [AlibabaPAI/DAPPLE](https://github.com/AlibabaPAI/DAPPLE) project. The development of the DAPPLE planner continues here.