https://github.com/ag14774/diffdist

https://github.com/ag14774/diffdist

Last synced: about 2 months ago
JSON representation

• Host: GitHub
• URL: https://github.com/ag14774/diffdist
• Owner: ag14774
• License: gpl-3.0
• Created: 2019-07-10T15:18:23.000Z (over 5 years ago)
• Default Branch: master
• Last Pushed: 2020-06-18T08:06:07.000Z (over 4 years ago)
• Last Synced: 2024-04-26T06:21:56.300Z (8 months ago)
• Language: Python
• Size: 27.3 KB
• Stars: 60
• Watchers: 1
• Forks: 6
• Open Issues: 1
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

• Awesome-pytorch-list-CNVersion - diffdist
• Awesome-pytorch-list - diffdist

README

        
# diffdist

## NOTE: PyTorch now includes the distributed RPC framework (https://pytorch.org/docs/stable/rpc.html). Most tasks can now be achieved using RPC. If you need a simpler MPI-like alternative, `diffdist` is for you!

`diffdist` is a python library for pytorch. It extends the default functionality of `torch.autograd` and adds support for differentiable communication between processes. This enables backpropagation to work in distributed settings and makes it super easy to use distributed model parallelism! `diffdist` achieves that by simply implementing the backward passes for most common communication primitives of `torch.distributed`. Processes that communicate during the forward pass, will automatically communicate during the backward pass to exchange gradients.

### Installation

After installing `pytorch` install simply using:

```sh

$ pip install diffdist

```

### Examples

You can find examples on how to use `diffdist` in `diffdist/testing.py`

License

----

GNU GPLv3