{"id":19066157,"url":"https://github.com/epfml/byzantine-robust-optimizer","last_synced_at":"2025-04-28T12:25:13.853Z","repository":{"id":109046255,"uuid":"375657234","full_name":"epfml/byzantine-robust-optimizer","owner":"epfml","description":"Learning from history for Byzantine Robustness","archived":false,"fork":false,"pushed_at":"2021-06-11T21:00:43.000Z","size":7355,"stargazers_count":22,"open_issues_count":0,"forks_count":2,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-04-18T16:16:38.567Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/epfml.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2021-06-10T10:22:22.000Z","updated_at":"2024-12-29T12:40:19.000Z","dependencies_parsed_at":"2023-03-13T14:19:12.157Z","dependency_job_id":null,"html_url":"https://github.com/epfml/byzantine-robust-optimizer","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epfml%2Fbyzantine-robust-optimizer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epfml%2Fbyzantine-robust-optimizer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epfml%2Fbyzantine-robust-optimizer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epfml%2Fbyzantine-robust-optimizer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/epfml","download_url":"https://codeload.github.com/epfml/byzantine-robust-optimizer/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251312300,"owners_count":21569209,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-09T00:54:46.883Z","updated_at":"2025-04-28T12:25:13.847Z","avatar_url":"https://github.com/epfml.png","language":"Jupyter Notebook","readme":"# Learning from History for Byzantine Robust Optimization\n\nThis repository contains research code for our [Byzantine robust optimization paper](https://arxiv.org/abs/2012.10333).\n\n\nSummary:\nWe study the problem of federated and distributed learning in the presence of untrusted workers who may try to derail the training process. We first describe a simple new aggregator based on iterative **centered clipping** which has significantly stronger theoretical guarantees than previous methods. This aggregator is especially interesting since, unlike most preceding methods, it is very scalable requiring only *O(n)* computation and communication per round. Further, it is also compatible with other strategies such as [asynchronous updates](https://arxiv.org/abs/1604.00981) and [secure aggregation](https://eprint.iacr.org/2017/281.pdf), both of which are crucial for real world applications. Secondly, we show that the time coupled attacks can easily be overcome by using **worker momentum**. \n\n# Code organization\nThe structure of the repository is as follows:\n- `codes/`\n  - Source code.\n- `momentum-robustness/`\n  - Launcher scripts.\n- `outputs/`\n  - Store the output of the launcher scripts.\n- `notebooks`\n  - Process the outputs and make plots.\n\n### Reproduction\n\nTo reproduce the results in the paper, do the following steps\n\n1. Add `codes/` to environment variable `PYTHONPATH`\n2. Install the dependencies: `pip install -r requirements.txt`\n3. Enter `momentum-robusntess/`\n  - Run `bash cifar10-all.sh` and select option 2 / 3 / 4 / 5.\n  - Run `bash cifar10-CC-HP-explore.sh` and select option 2.\n4. The output will be saved to the corresponding folders under `outputs`\n\nNote that if the GPU memory is small (e.g. less than 16 GB), then running the previous commands may raise insufficient exception. In this case, one can decrease the level parallelism in the script by changing the order of loops and reduce the number of parallel processes. \n\n### Example results\n![](./notebooks/exp3_transpose.png)\n\n\n# Reference\n\nIf you use this code, please cite the following [paper](https://arxiv.org/abs/2012.10333)\n\n    @inproceedings{karimireddy2020learning,\n      author = {Karimireddy, Sai Praneeth and He, Lie and Jaggi, Martin},\n      title = \"{Learning from History for Byzantine Robust Optimization}\",\n      booktitle = {ICML 2021 - Proceedings of International Conference on Machine Learning},\n      year = 2021,\n      url = {https://arxiv.org/abs/2012.10333}\n    }\n\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fepfml%2Fbyzantine-robust-optimizer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fepfml%2Fbyzantine-robust-optimizer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fepfml%2Fbyzantine-robust-optimizer/lists"}