{"id":19066160,"url":"https://github.com/epfml/topology-in-decentralized-learning","last_synced_at":"2026-03-06T21:33:32.274Z","repository":{"id":52486858,"uuid":"500737873","full_name":"epfml/topology-in-decentralized-learning","owner":"epfml","description":"Code related to ’Beyond spectral gap: The role of the topology in decentralized learning‘.","archived":false,"fork":false,"pushed_at":"2022-06-07T08:02:27.000Z","size":57,"stargazers_count":13,"open_issues_count":0,"forks_count":4,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-04-28T12:39:20.252Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/epfml.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-06-07T07:38:14.000Z","updated_at":"2025-01-18T09:13:31.000Z","dependencies_parsed_at":"2022-09-18T21:38:52.343Z","dependency_job_id":null,"html_url":"https://github.com/epfml/topology-in-decentralized-learning","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/epfml/topology-in-decentralized-learning","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epfml%2Ftopology-in-decentralized-learning","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epfml%2Ftopology-in-decentralized-learning/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epfml%2Ftopology-in-decentralized-learning/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epfml%2Ftopology-in-decentralized-learning/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/epfml","download_url":"https://codeload.github.com/epfml/topology-in-decentralized-learning/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epfml%2Ftopology-in-decentralized-learning/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30198661,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-06T19:07:06.838Z","status":"ssl_error","status_checked_at":"2026-03-06T18:57:34.882Z","response_time":250,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-09T00:54:59.110Z","updated_at":"2026-03-06T21:33:32.246Z","avatar_url":"https://github.com/epfml.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Beyond spectral gap: The role of the topology in decentralized learning\n\nAbstract: In data-parallel optimization of machine learning models, workers collaborate to improve their estimates of the model: more accurate gradients allow them to use larger learning rates and optimize faster.\nWe consider the setting in which all workers sample from the same dataset,\nand communicate over a sparse graph (decentralized).\nIn this setting, current theory fails to capture important aspects of real-world behavior.\nFirst, the ‘spectral gap’ of the communication graph is not predictive of its empirical performance in (deep) learning.\nSecond, current theory does not explain that collaboration enables *larger* learning rates than training alone.\nIn fact, it prescribes *smaller* learning rates, which further decrease as graphs become larger, failing to explain convergence in infinite graphs.\nThis paper aims to paint an accurate picture of sparsely-connected distributed optimization when workers share the same data distribution.\nWe quantify how the graph topology influences convergence in a quadratic toy problem and provide theoretical results for general smooth and (strongly) convex objectives.\nOur theory matches empirical observations in deep learning, and accurately describes the relative merits of different graph topologies.\n\n- [Paper (pre-print)](#)\n\n## Reference\nIf you use this code, please cite the following paper\n\n```\n@article{vogels2022bsg,\n  author    = {Thijs Vogels and Hadrien Hendrikx and Martin Jaggi},\n  title     = {Beyond spectral gap: The role of the topology in decentralized learning},\n  journal   = {CoRR},\n  volume    = {abs/TODO},\n  year      = {2022},\n  ee        = {http://arxiv.org/abs/TODO},\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fepfml%2Ftopology-in-decentralized-learning","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fepfml%2Ftopology-in-decentralized-learning","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fepfml%2Ftopology-in-decentralized-learning/lists"}