{"id":20094980,"url":"https://github.com/wnma3mz/flearn","last_synced_at":"2025-05-06T05:31:07.131Z","repository":{"id":41083138,"uuid":"405372205","full_name":"wnma3mz/flearn","owner":"wnma3mz","description":"Federated Learning Framework","archived":false,"fork":false,"pushed_at":"2022-10-18T12:06:45.000Z","size":1717,"stargazers_count":21,"open_issues_count":0,"forks_count":2,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-09T08:44:32.773Z","etag":null,"topics":["deep-learning","ensemble-learning","federated-learning","framework","privacy-protection","pytorch"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/wnma3mz.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-09-11T12:23:05.000Z","updated_at":"2024-11-07T07:51:57.000Z","dependencies_parsed_at":"2023-01-19T22:05:11.484Z","dependency_job_id":null,"html_url":"https://github.com/wnma3mz/flearn","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wnma3mz%2Fflearn","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wnma3mz%2Fflearn/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wnma3mz%2Fflearn/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wnma3mz%2Fflearn/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/wnma3mz","download_url":"https://codeload.github.com/wnma3mz/flearn/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252628795,"owners_count":21779082,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","ensemble-learning","federated-learning","framework","privacy-protection","pytorch"],"created_at":"2024-11-13T16:53:23.005Z","updated_at":"2025-05-06T05:31:06.379Z","avatar_url":"https://github.com/wnma3mz.png","language":"Python","funding_links":[],"categories":["Projects"],"sub_categories":["2022"],"readme":"## 联邦学习框架\n\n![Pypi](https://img.shields.io/pypi/v/cfl)\n\n### Quickstart\n\n1. 下载最新的[release版本](https://github.com/wnma3mz/flearn/releases/latest) 并使用pip安装。或手动下载源码，在当前目录进行编译 `python setup.py sdist bdist_wheel`。\n2. 切换至运行目录 `cd example/mnist_cifar/`\n3. 运行 `python main.py --strategy_name avg --dataset_name mnist dataset_fpath 数据集路径`\n\n详细解释见 `example/mnist_cifar`中的[README.md](https://github.com/wnma3mz/flearn/tree/master/example/mnist_cifar)\n\n### 进阶1——复现LG-FedAVG\n\n- 修改 `Client.py`，以及如何配置共享层\n\n见[README.md](https://github.com/wnma3mz/flearn/tree/master/example/LG_reproduction)\n\n### 进阶2——复现FedProx\n\n- 修改训练器，以运用至更多任务与模型\n\n见[README.md](https://github.com/wnma3mz/flearn/tree/master/example/Prox)\n\n### 进阶3——复现FedPAV\n\n- 修改客户端以及服务器端，以适用于FedPAV策略\n\n见[README.md](https://github.com/wnma3mz/flearn/tree/master/example/PAV_reproduction)\n\n### 支持策略\n\n- 2016\n  - [X] [FedSGD/FedAVG](), Google, [Communication-Efficient Learning of Deep Networksfrom Decentralized Data](https://arxiv.org/pdf/1602.05629), PMLR\n- 2018\n  - [X] [FedProx](https://github.com/litian96/FedProx), CMU, [Federated Optimization in Heterogeneous Networks](https://arxiv.org/abs/1812.06127), MLSys\n- 2019\n  - [X] [FedAVGM](), MIT/Google, [Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification](https://arxiv.org/pdf/1909.06335)\n  - [X] [FedMD](https://github.com/diogenes0319/FedMD_clean), Harvard, [FedMD: Heterogenous Federated Learning via Model Distillation](https://arxiv.org/pdf/1910.03581), NIPS\n- 2020\n  - [X] [SCAFFOLD](), Google, [SCAFFOLD: Stochastic Controlled Averaging for Federated Learning](https://arxiv.org/pdf/1910.06378), ICML\n  - [X] [LG-FedAVG](https://github.com/pliang279/LG-FedAvg), CMU, [Think Locally, Act Globally: Federated Learning with Local and Global Representations](https://arxiv.org/pdf/2001.01523), NIPS WorkShop\n  - [X] [FedOPT](https://github.com/adap/flower/blob/main/src/py/flwr/server/strategy/fedadagrad.py)(非官方), Google, [Adaptive Federated Optimization](https://arxiv.org/pdf/2003.00295), ICLR\n  - [X] [FedPAV](https://github.com/cap-ntu/FedReID), NTU, [Performance Optimization for Federated Person Re-identification via Benchmark Analysis](https://arxiv.org/pdf/2008.11560), ACM MM\n  - [X] [FedDistill](https://github.com/zhuangdizhu/FedGen)(非官方), Oulu, [Federated Knowledge Distillation](https://arxiv.org/pdf/2011.02367)\n  - [X] [FedMutual](), ZJU, [Federated Mutual Learning](https://arxiv.org/pdf/2006.16765)\n  - [X] [FedDF(Ensemble Distillation)](https://github.com/epfml/federated-learning-public-code/), EPFL, [Ensemble Distillation for Robust Model Fusion in Federated Learning](https://arxiv.org/pdf/2006.07242), NIPS\n  - [X] [FedPer](), Adobe Research, [Federated learning with personalization layers](https://arxiv.org/pdf/1912.00818)\n  - [ ] [FedBE](https://github.com/hongyouc/FedBE), Ohio State, [FedBE: Making Bayesian Model Ensemble Applicable to Federated Learning](https://arxiv.org/abs/2009.01974), ICLR\n  - [ ] [FedNova](), CMU, [Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization](https://arxiv.org/pdf/2007.07481), NIPS\n- 2021\n  - [X] [FedBN](https://github.com/med-air/FedBN), Princeton, [FedBN: Federated Learning on Non-IID Features via Local Batch Normalization](https://arxiv.org/pdf/2102.07623), ICLR\n  - [X] [FedDyn](https://github.com/AntixK/FedDyn), Boston, [FEDERATED LEARNING BASED ON DYNAMIC REGULARIZATION](https://arxiv.org/pdf/2111.04263), ICLR\n  - [X] [MOON](https://github.com/QinbinLi/MOON), NUS/Berkeley, [Model-Contrastive Federated Learning](https://arxiv.org/pdf/2103.16257.pdf), CVPR\n  - [X] [CCVR](), NUS/Huawei, [No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Data](https://arxiv.org/pdf/2106.05001), NIPS\n  - [X] [FedGen](https://github.com/zhuangdizhu/FedGen), Michigan State, [Data-Free Knowledge Distillation for Heterogeneous Federated Learning](https://arxiv.org/pdf/2105.10056), ICML\n  - [ ] [FedDist](), Grenoble Alpes, [A Federated Learning Aggregation Algorithm for Pervasive Computing: Evaluation and Comparison](https://arxiv.org/pdf/2110.10223), PerCom\n  - [X] [FedRep](https://github.com/lgcollins/FedRep/), University of Texas at Austin, [Exploiting Shared Representations for Personalized Federated Learning](https://arxiv.org/pdf/2102.07078), ICML\n- 2022\n  - [ ] [FedKD](https://github.com/wuch15/FedKD), Tsinghua University/Microsoft Research Asia, [Communication-efficient federated learning via knowledge distillation](https://www.nature.com/articles/s41467-022-29763-x), Nature Communications\n  - [ ] [FedProto](https://github.com/yuetan031/fedproto), Australian Artificial Intelligence Institute, [FedProto: Federated Prototype Learning across Heterogeneous Clients](https://www.aaai.org/AAAI22Papers/AAAI-6846.YueT.pdf), AAAI\n\nsplit-learning可见[README.md](https://github.com/wnma3mz/flearn/tree/master/example/split_learning)，尚存在loss爆炸问题。\n\n### TODO\n\n- [ ] IDA聚合方式。[Inverse Distance Aggregation for Federated Learning with Non-IID Data](https://arxiv.org/pdf/2008.07665)\n- [ ] FedKD有待更新，测试\n- [ ] 添加FedLSD论文\n\n### 框架图\n\n![CFL](./imgs/CFL.png)\n\n- 对于active_client当前是随机选择客户端，可以使用更加先进的客户端选择算法\n- 对于Server端的evaluate，每种算法的目标大致可以分为两种，1) 获得一个模型，这个模型在某个数据集上能够取得很好的性能；2) 每个客户端获得不同的模型，每个客户端的模型在对应的数据集上能够取得很好的性能。因此这里分两部分处理\n  - 仅在服务器端对单个模型在测试集上进行测试，直接获得测试结果（为区分，在log中会对这部分测试结果加上[Server]）\n  - 分别在所有/指定客户端中对客户端上的测试集上进行测试，在服务器端对结果取平均\n  - 两种测试均进行\n- 对于辅助类\n  - Distiller是为了进行知识蒸馏的操作，注：这里主要是对蒸馏方式进行整理，损失函数采用默认的KL散度\n  - Encrypt是为了对通信参数进行加密，注：这里仅作base编解码操作\n  - Logger是为了输出日志文件的数据\n- 对于Trainer，部分算法是修改loss函数，因此这里通过增加函数fed_loss，方便快速修改\n- 对于Strategy，部分算法需要修改上传以及下载参数（不仅上传模型参数，还有可能有其他信息），所以将Strategy处理为三个函数\n  - client（仅对客户端有效）：处理上传参数\n  - server（仅对服务器端有效）：服务器端处理所有客户端上传的参数\n  - client_revice（仅对客户端有效）：接收服务器端的参数并进行处理\n\n### 工作流\n\n1. 服务器(Server)发送**训练指令**至各个客户端(Client)进行训练 (Server-\u003eComm(S)-\u003eComm(C)-\u003eClient)；模拟实验时，(Server-\u003eClient)\n2. Client根据配置好的训练器(Trainer)进行训练，训练完成后返回指令至Server\n3. Server发送**上传指令**至Client，Client根据配置好的策略(Strategy)，准备好上传的参数并进行上传，即Server发送指令后收到Client上传的参数\n4. Server根据预先配好的Strategy对参数进行聚合\n5. Server发送**接收指令**至Client，此时把参数发回至Client，Client根据配置好的Strategy进行接收\n6. 若Server继续发送**测试指令**至Client，Client还要对更新后的模型进行验证，并返回验证后的结果至Server。否则，Server直接进行验证\n\nP.S.\n\n- Trainer中一般是需要配置联邦的损失函数 `fed_loss`，主要作用是为了防止灾难性遗忘\n- Distiller可以看作 `fed_loss`，也可以看作聚合策略的一种，所以可能会在 `Strategy`进行调用优化模型参数\n- Strategy其实可以分为Server和Client两个部分，其中Client有两个函数（上传和接收）。此处是将策略看作一个整体，即Client和Server都调用同一个Strategy\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwnma3mz%2Fflearn","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwnma3mz%2Fflearn","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwnma3mz%2Fflearn/lists"}