Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.
https://github.com/wnma3mz/flearn

Federated Learning Framework
https://github.com/wnma3mz/flearn
deep-learning ensemble-learning federated-learning framework privacy-protection pytorch
Last synced: about 1 month ago
JSON representation
Federated Learning Framework
Host: GitHub
URL: https://github.com/wnma3mz/flearn
Owner: wnma3mz
License: apache-2.0
Created: 2021-09-11T12:23:05.000Z (almost 3 years ago)
Default Branch: master
Last Pushed: 2022-10-18T12:06:45.000Z (over 1 year ago)
Last Synced: 2024-02-08T16:09:31.519Z (5 months ago)
Topics: deep-learning, ensemble-learning, federated-learning, framework, privacy-protection, pytorch
Language: Python
Homepage:
Size: 1.64 MB
Stars: 20
Watchers: 2
Forks: 2
Open Issues: 0
Metadata Files:
- Readme: README.md
- License: LICENSE
Lists

awesome-Federated-Learning - wnma3mz/flearn
README

        ## 联邦学习框架

![Pypi](https://img.shields.io/pypi/v/cfl)

### Quickstart

1. 下载最新的[release版本](https://github.com/wnma3mz/flearn/releases/latest) 并使用pip安装。或手动下载源码，在当前目录进行编译 `python setup.py sdist bdist_wheel`。

2. 切换至运行目录 `cd example/mnist_cifar/`

3. 运行 `python main.py --strategy_name avg --dataset_name mnist dataset_fpath 数据集路径`

详细解释见 `example/mnist_cifar`中的[README.md](https://github.com/wnma3mz/flearn/tree/master/example/mnist_cifar)

### 进阶1——复现LG-FedAVG

- 修改 `Client.py`，以及如何配置共享层

见[README.md](https://github.com/wnma3mz/flearn/tree/master/example/LG_reproduction)

### 进阶2——复现FedProx

- 修改训练器，以运用至更多任务与模型

见[README.md](https://github.com/wnma3mz/flearn/tree/master/example/Prox)

### 进阶3——复现FedPAV

- 修改客户端以及服务器端，以适用于FedPAV策略

见[README.md](https://github.com/wnma3mz/flearn/tree/master/example/PAV_reproduction)

### 支持策略

- 2016

  - [X] [FedSGD/FedAVG](), Google, [Communication-Efficient Learning of Deep Networksfrom Decentralized Data](https://arxiv.org/pdf/1602.05629), PMLR

- 2018

  - [X] [FedProx](https://github.com/litian96/FedProx), CMU, [Federated Optimization in Heterogeneous Networks](https://arxiv.org/abs/1812.06127), MLSys

- 2019

  - [X] [FedAVGM](), MIT/Google, [Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification](https://arxiv.org/pdf/1909.06335)

  - [X] [FedMD](https://github.com/diogenes0319/FedMD_clean), Harvard, [FedMD: Heterogenous Federated Learning via Model Distillation](https://arxiv.org/pdf/1910.03581), NIPS

- 2020

  - [X] [SCAFFOLD](), Google, [SCAFFOLD: Stochastic Controlled Averaging for Federated Learning](https://arxiv.org/pdf/1910.06378), ICML

  - [X] [LG-FedAVG](https://github.com/pliang279/LG-FedAvg), CMU, [Think Locally, Act Globally: Federated Learning with Local and Global Representations](https://arxiv.org/pdf/2001.01523), NIPS WorkShop

  - [X] [FedOPT](https://github.com/adap/flower/blob/main/src/py/flwr/server/strategy/fedadagrad.py)(非官方), Google, [Adaptive Federated Optimization](https://arxiv.org/pdf/2003.00295), ICLR

  - [X] [FedPAV](https://github.com/cap-ntu/FedReID), NTU, [Performance Optimization for Federated Person Re-identification via Benchmark Analysis](https://arxiv.org/pdf/2008.11560), ACM MM

  - [X] [FedDistill](https://github.com/zhuangdizhu/FedGen)(非官方), Oulu, [Federated Knowledge Distillation](https://arxiv.org/pdf/2011.02367)

  - [X] [FedMutual](), ZJU, [Federated Mutual Learning](https://arxiv.org/pdf/2006.16765)

  - [X] [FedDF(Ensemble Distillation)](https://github.com/epfml/federated-learning-public-code/), EPFL, [Ensemble Distillation for Robust Model Fusion in Federated Learning](https://arxiv.org/pdf/2006.07242), NIPS

  - [X] [FedPer](), Adobe Research, [Federated learning with personalization layers](https://arxiv.org/pdf/1912.00818)

  - [ ] [FedBE](https://github.com/hongyouc/FedBE), Ohio State, [FedBE: Making Bayesian Model Ensemble Applicable to Federated Learning](https://arxiv.org/abs/2009.01974), ICLR

  - [ ] [FedNova](), CMU, [Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization](https://arxiv.org/pdf/2007.07481), NIPS

- 2021

  - [X] [FedBN](https://github.com/med-air/FedBN), Princeton, [FedBN: Federated Learning on Non-IID Features via Local Batch Normalization](https://arxiv.org/pdf/2102.07623), ICLR

  - [X] [FedDyn](https://github.com/AntixK/FedDyn), Boston, [FEDERATED LEARNING BASED ON DYNAMIC REGULARIZATION](https://arxiv.org/pdf/2111.04263), ICLR

  - [X] [MOON](https://github.com/QinbinLi/MOON), NUS/Berkeley, [Model-Contrastive Federated Learning](https://arxiv.org/pdf/2103.16257.pdf), CVPR

  - [X] [CCVR](), NUS/Huawei, [No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Data](https://arxiv.org/pdf/2106.05001), NIPS

  - [X] [FedGen](https://github.com/zhuangdizhu/FedGen), Michigan State, [Data-Free Knowledge Distillation for Heterogeneous Federated Learning](https://arxiv.org/pdf/2105.10056), ICML

  - [ ] [FedDist](), Grenoble Alpes, [A Federated Learning Aggregation Algorithm for Pervasive Computing: Evaluation and Comparison](https://arxiv.org/pdf/2110.10223), PerCom

  - [X] [FedRep](https://github.com/lgcollins/FedRep/), University of Texas at Austin, [Exploiting Shared Representations for Personalized Federated Learning](https://arxiv.org/pdf/2102.07078), ICML

- 2022

  - [ ] [FedKD](https://github.com/wuch15/FedKD), Tsinghua University/Microsoft Research Asia, [Communication-efficient federated learning via knowledge distillation](https://www.nature.com/articles/s41467-022-29763-x), Nature Communications

  - [ ] [FedProto](https://github.com/yuetan031/fedproto), Australian Artificial Intelligence Institute, [FedProto: Federated Prototype Learning across Heterogeneous Clients](https://www.aaai.org/AAAI22Papers/AAAI-6846.YueT.pdf), AAAI

split-learning可见[README.md](https://github.com/wnma3mz/flearn/tree/master/example/split_learning)，尚存在loss爆炸问题。

### TODO

- [ ] IDA聚合方式。[Inverse Distance Aggregation for Federated Learning with Non-IID Data](https://arxiv.org/pdf/2008.07665)

- [ ] FedKD有待更新，测试

- [ ] 添加FedLSD论文

### 框架图

![CFL](./imgs/CFL.png)

- 对于active_client当前是随机选择客户端，可以使用更加先进的客户端选择算法

- 对于Server端的evaluate，每种算法的目标大致可以分为两种，1) 获得一个模型，这个模型在某个数据集上能够取得很好的性能；2) 每个客户端获得不同的模型，每个客户端的模型在对应的数据集上能够取得很好的性能。因此这里分两部分处理

  - 仅在服务器端对单个模型在测试集上进行测试，直接获得测试结果（为区分，在log中会对这部分测试结果加上[Server]）

  - 分别在所有/指定客户端中对客户端上的测试集上进行测试，在服务器端对结果取平均

  - 两种测试均进行

- 对于辅助类

  - Distiller是为了进行知识蒸馏的操作，注：这里主要是对蒸馏方式进行整理，损失函数采用默认的KL散度

  - Encrypt是为了对通信参数进行加密，注：这里仅作base编解码操作

  - Logger是为了输出日志文件的数据

- 对于Trainer，部分算法是修改loss函数，因此这里通过增加函数fed_loss，方便快速修改

- 对于Strategy，部分算法需要修改上传以及下载参数（不仅上传模型参数，还有可能有其他信息），所以将Strategy处理为三个函数

  - client（仅对客户端有效）：处理上传参数

  - server（仅对服务器端有效）：服务器端处理所有客户端上传的参数

  - client_revice（仅对客户端有效）：接收服务器端的参数并进行处理

### 工作流

1. 服务器(Server)发送**训练指令**至各个客户端(Client)进行训练 (Server->Comm(S)->Comm(C)->Client)；模拟实验时，(Server->Client)

2. Client根据配置好的训练器(Trainer)进行训练，训练完成后返回指令至Server

3. Server发送**上传指令**至Client，Client根据配置好的策略(Strategy)，准备好上传的参数并进行上传，即Server发送指令后收到Client上传的参数

4. Server根据预先配好的Strategy对参数进行聚合

5. Server发送**接收指令**至Client，此时把参数发回至Client，Client根据配置好的Strategy进行接收

6. 若Server继续发送**测试指令**至Client，Client还要对更新后的模型进行验证，并返回验证后的结果至Server。否则，Server直接进行验证

P.S.

- Trainer中一般是需要配置联邦的损失函数 `fed_loss`，主要作用是为了防止灾难性遗忘

- Distiller可以看作 `fed_loss`，也可以看作聚合策略的一种，所以可能会在 `Strategy`进行调用优化模型参数

- Strategy其实可以分为Server和Client两个部分，其中Client有两个函数（上传和接收）。此处是将策略看作一个整体，即Client和Server都调用同一个Strategy