https://github.com/shigangli/wagma-sgd
WAGMA-SGD is a decentralized asynchronous SGD based on wait-avoiding group model averaging. The synchronization is relaxed by making the collectives externally-triggerable, namely, a collective can be initiated without requiring that all the processes enter it. It partially reduces the data within non-overlapping groups of process, improving the parallel scalability.
https://github.com/shigangli/wagma-sgd
distributed-deep-learning model-averaging partial-allreduce
Last synced: 11 months ago
JSON representation
WAGMA-SGD is a decentralized asynchronous SGD based on wait-avoiding group model averaging. The synchronization is relaxed by making the collectives externally-triggerable, namely, a collective can be initiated without requiring that all the processes enter it. It partially reduces the data within non-overlapping groups of process, improving the parallel scalability.
- Host: GitHub
- URL: https://github.com/shigangli/wagma-sgd
- Owner: Shigangli
- License: gpl-3.0
- Created: 2021-06-28T11:06:59.000Z (almost 5 years ago)
- Default Branch: main
- Last Pushed: 2021-06-30T08:29:23.000Z (almost 5 years ago)
- Last Synced: 2023-03-06T18:10:54.199Z (over 3 years ago)
- Topics: distributed-deep-learning, model-averaging, partial-allreduce
- Language: Python
- Homepage:
- Size: 1.11 MB
- Stars: 6
- Watchers: 2
- Forks: 0
- Open Issues: 0