{"id":19066267,"url":"https://github.com/epfml/quasi-global-momentum","last_synced_at":"2025-04-28T12:27:28.558Z","repository":{"id":48241436,"uuid":"374711358","full_name":"epfml/quasi-global-momentum","owner":"epfml","description":null,"archived":false,"fork":false,"pushed_at":"2022-12-23T10:01:11.000Z","size":145,"stargazers_count":11,"open_issues_count":1,"forks_count":3,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-04-18T16:15:58.838Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/epfml.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-06-07T15:17:55.000Z","updated_at":"2025-02-04T19:44:22.000Z","dependencies_parsed_at":"2023-01-30T18:16:20.338Z","dependency_job_id":null,"html_url":"https://github.com/epfml/quasi-global-momentum","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epfml%2Fquasi-global-momentum","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epfml%2Fquasi-global-momentum/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epfml%2Fquasi-global-momentum/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epfml%2Fquasi-global-momentum/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/epfml","download_url":"https://codeload.github.com/epfml/quasi-global-momentum/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251312867,"owners_count":21569310,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-09T00:55:41.593Z","updated_at":"2025-04-28T12:27:28.466Z","avatar_url":"https://github.com/epfml.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Quasi-Global Momentum\nThis repository is the official implementation of the paper: [Quasi-Global Momentum: Accelerating Decentralized Deep Learning on Heterogeneous Data](https://arxiv.org/abs/2102.04761), appeared in ICML 2021.\n\n## Abstract\nDecentralized training of deep learning models\nis a key element for enabling data privacy and on-device learning over networks.\nIn realistic learning scenarios,\nthe presence of heterogeneity across different clients' local datasets\nposes an optimization challenge and may severely deteriorate the generalization performance.\n\nIn this paper, we investigate and identify the limitation of several decentralized optimization algorithms\nfor different degrees of data heterogeneity.\nWe propose a novel momentum-based method\nto mitigate this decentralized training difficulty.\nWe show in extensive empirical experiments\non various CV/NLP datasets (CIFAR-10, ImageNet, and AG News)\nand several network topologies (Ring and Social Network) that\nour method is much more robust to the heterogeneity of clients' data than other existing methods,\nby a significant improvement in test performance (1% - 20%).\n\n\n## References\nIf you use the code, please cite the following paper:\n\n```\n@inproceedings{lin2021quasi,\n  title     = {Quasi-global momentum: Accelerating decentralized deep learning on heterogeneous data},\n  author    = {Lin, Tao and Karimireddy, Sai Praneeth and Stich, Sebastian U and Jaggi, Martin},\n  booktitle = {International Conference on Machine Learning (ICML)},\n  url       = {https://arxiv.org/abs/2102.04761},\n  year      = {2021}\n}\n```\n\n## Examples\nPlease refer to [`code/README.md`](code/README.md)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fepfml%2Fquasi-global-momentum","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fepfml%2Fquasi-global-momentum","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fepfml%2Fquasi-global-momentum/lists"}