{"id":24213699,"url":"https://github.com/sarodyatawatta/federated-pytorch-test","last_synced_at":"2025-09-09T14:21:41.882Z","repository":{"id":167816647,"uuid":"201751443","full_name":"SarodYatawatta/federated-pytorch-test","owner":"SarodYatawatta","description":"Federated learning with PyTorch (federated averaging and consensus optimization): with  'reduced' bandwidth","archived":false,"fork":false,"pushed_at":"2024-04-30T11:30:43.000Z","size":308,"stargazers_count":40,"open_issues_count":0,"forks_count":7,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-04-30T13:05:08.359Z","etag":null,"topics":["clustering","consensus-model","consensus-optimization","deep-learning","federated-learning","lbfgs","pytorch","sagecal","variational-autoencoder"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/SarodYatawatta.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-08-11T10:40:21.000Z","updated_at":"2024-04-30T11:30:46.000Z","dependencies_parsed_at":null,"dependency_job_id":"a8553257-6dab-46fb-8005-9b11a054f9f6","html_url":"https://github.com/SarodYatawatta/federated-pytorch-test","commit_stats":null,"previous_names":["sarodyatawatta/federated-pytorch-test"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SarodYatawatta%2Ffederated-pytorch-test","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SarodYatawatta%2Ffederated-pytorch-test/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SarodYatawatta%2Ffederated-pytorch-test/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SarodYatawatta%2Ffederated-pytorch-test/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/SarodYatawatta","download_url":"https://codeload.github.com/SarodYatawatta/federated-pytorch-test/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":233837045,"owners_count":18737909,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["clustering","consensus-model","consensus-optimization","deep-learning","federated-learning","lbfgs","pytorch","sagecal","variational-autoencoder"],"created_at":"2025-01-14T03:17:44.728Z","updated_at":"2025-01-14T03:17:46.100Z","avatar_url":"https://github.com/SarodYatawatta.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# federated-pytorch-test\nWe train CNN models __without__ having access to the full dataset. The CIFAR10 dataset is used in all examples. The CNN models can be chosen from simpler models similar to PyTorch or Tensorflow demos and also ResNet18.  In all cases, we use only 1/K (where K is user defined) of the data for training each CNN model. We also compare the performance of federated averaging and consensus optimization in training the K models, without sharing the training data between models. Note that we only pass __a subset of parameters__ between the models, unlike in normal federated averaging or consensus. This __reduces the bandwidth__ required enormously! \n\nThe stochastic LBFGS optimizer is provided with the code. Further details are given [in this paper](https://ieeexplore.ieee.org/document/8755567). Also see [this introduction](http://sagecal.sourceforge.net/pytorch/index.html).\n\nGPU acceleration is enabled when available, set ```use_cuda=True```.\nFiles included are:\n\n``` lbfgsnew.py ```: New LBFGS optimizer\n\n``` simple_models.py ```: Relatively simple CNN models for CIFAR10, derived from PyTorch/Tensorflow demos, also ResNet18\n\n``` no_consensus_multi.py ```: Train K models using 1/K of the training data for each model\n\n``` federated_multi.py ```: Train K models using 1/K of the data, with federated averaging, K can be varied\n\n``` fedprox_multi.py ```: Train K models using 1/K of the data, with federated proximal algorithm, K can be varied, based on [this paper](https://arxiv.org/abs/1812.06127)\n\n``` consensus_multi.py ```: Train K models using 1/K of the data, with consensus optimization (adaptive) ADMM, K can be varied\n\n``` federated_vae.py ```: Train K variational autoencoders, using federated averaging\n\n``` federated_vae_cl.py ```: Train K variational autoencoders for clustering, using federated averaging, based on [this paper](https://arxiv.org/abs/2005.04613)\n\n``` federated_cpc.py ```: Train K models using contrastive predictive coding, using LOFAR data, based on [this paper](https://arxiv.org/abs/1905.09272)\n\n\n\u003cimg src=\"comparison.png\" alt=\"test accuracy for training K=10 models\" width=\"700\"/\u003e\n\nThis images compares training K=1 and K=10 models, stad alone training using ```no_consensus_multi.py```, with consensus optimization ```consensus_multi.py``` and with federated averaging ```federated_multi.py```. The upper bound is using the full dataset for training ( K=1 ) while using 1/K of the data gives the lower bound.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsarodyatawatta%2Ffederated-pytorch-test","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsarodyatawatta%2Ffederated-pytorch-test","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsarodyatawatta%2Ffederated-pytorch-test/lists"}