{"id":28458760,"url":"https://github.com/openmined/fl_aggregator","last_synced_at":"2025-07-02T09:31:53.700Z","repository":{"id":262960922,"uuid":"888906701","full_name":"OpenMined/fl_aggregator","owner":"OpenMined","description":null,"archived":false,"fork":false,"pushed_at":"2025-05-30T20:30:45.000Z","size":3065,"stargazers_count":1,"open_issues_count":3,"forks_count":4,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-06-25T06:47:32.897Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/OpenMined.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":"openmined","open_collective":"openmined"}},"created_at":"2024-11-15T08:32:19.000Z","updated_at":"2025-04-08T13:48:20.000Z","dependencies_parsed_at":"2025-01-29T16:35:48.715Z","dependency_job_id":null,"html_url":"https://github.com/OpenMined/fl_aggregator","commit_stats":null,"previous_names":["openmined/fl_aggregator"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/OpenMined/fl_aggregator","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenMined%2Ffl_aggregator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenMined%2Ffl_aggregator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenMined%2Ffl_aggregator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenMined%2Ffl_aggregator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/OpenMined","download_url":"https://codeload.github.com/OpenMined/fl_aggregator/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenMined%2Ffl_aggregator/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263111474,"owners_count":23415464,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-06-07T00:39:43.531Z","updated_at":"2025-07-02T09:31:53.653Z","avatar_url":"https://github.com/OpenMined.png","language":"Python","funding_links":["https://github.com/sponsors/openmined","https://opencollective.com/openmined"],"categories":[],"sub_categories":[],"readme":"# Fl Aggregator\n\nFederated Learning (FL) Aggregator [SyftBox](https://syftbox-documentation.openmined.org) API.\n\nThe `fl_agregator` API is featured in the tutorial **Getting Started with FL on SyftBox**. \nRead more [here](https://syftbox-documentation.openmined.org/tutorials/federated-learning/getting-started/).\n\n## Usage\n\n**1. Install the API**\n\n```bash\ngit clone https://github.com/OpenMined/fl_aggregator\n\ncp -r ./fl_aggregator \u003cSYFTBOX_DATADIR\u003e/apis/  # default: ~/SyftBox\n```\n\n**Note**: `\u003cSYFTBOX_DATADIR\u003e` refers to the SyftBox data directory, according to your\nSyftBox installation (default: `$HOME/SyftBox`).\n\n**2. Agree on roles in the FL flow**\n\nFor example: Aggregator: `\u003ca@openmined.org\u003e`; Clients: `\u003cb@openmined.org\u003e`,`\u003cc@openmined.org\u003e`\n\n- Aggregator must decide and share the model architecture with the clients.\n- Aggregator will also provide a seed model weights which will be fine-tuned by each participant.\n\n**3. Setup the FL config**\n\nCreate a `fl_config.json` configuration file including the following information:\n\n- `\"project_name\"`: name assigned to the FL experiment.\n- `\"aggregator\"`: designated aggregator datasite.\n- `\"participants\"`: list of the designated client datasites.\n- `\"rounds\"`: number of FL training rounds.\n- `\"model_arch\"`: the Python module containing the ML model implementation.\n- `\"model_weight\"`: model parameters file\n- `\"epoch\"`: number of training epochs to run on each clients,\n- `\"learning_rate\"`: learning rate of the optimizer.\n\nPlease see `samples/launch_config/fl_config.json` for an example configuration file:\n\n```json\n{\n    \"project_name\": \"MNIST_FL\",\n    \"aggregator\": \"a@openmined.org\",\n    \"participants\": [\"b@openmined.org\",\"c@openmined.org\"],\n    \"rounds\": 3,\n    \"model_arch\": \"model.py\",\n    \"model_weight\": \"global_model_weight.pt\",\n    \"epoch\": 10,\n    \"learning_rate\": 0.1\n}\n```\n\n**4. Start the FL experiment**\n\n1. To start the FL experiment, the following files needs to be copied in \n`\u003cSYFTBOX_DATADIR\u003e/datasites/\u003caggregator_email\u003e/api_data/fl_aggregator/launch` directory:\n\n- `fl_config.json`\n- `global_model_weight.pt`\n- `model.py`\n\n2. Copy the test dataset to `\u003cSYFTBOX_DATADIR\u003e/datasites/\u003caggregator_email\u003e/private/fl_aggregator` \n\nAn example test dataset could be found in `./samples/test_data`.\n\nIf this directory isn't available, either run the syftbox client with fl_aggregator API installed OR create it manually.\n\nOnce the files are in the `launch` folder, the API will create a folder named after the `project_name` specified in the `file_config.json` in the `running` folder.\n\nInside this folder the API will create a folder for each client datasite, where received updates in each round are gathered. \nEach of these folders will have a `._syftperm` file containing the appropriate permissions granting read/write access to the folder.\n\nFinally, the API send a request to each participant to join the FL flow.\n\nPlease see below an example of the resulting folders structure during the FL execution.\n\n```plaintext\napi_data\n└── fl_aggregator\n    ├── launch\n    │   ├── config.json\n    │   ├── model_arch.py\n    │   ├── global_model_weights.py\n    └── running\n        └── my_cool_fl_proj\n            ├── fl_clients \n            │   ├── a@openmined.org\n            │   ├── b@openmined.org\n            │   ├── c@openmined.org\n            ├── agg_weights  # to store aggregator's weights for each round\n            ├── config.json  # moved from the launch folder after the app start\n            ├── global_model_weights.pt\n            ├── model_arch.py  # moved from the launch folder\n            └── state.json\n    └── done\n        └── my_cool_fl_proj\n            └── aggregated_model_weights.pt\n```\n\n**5. Monitoring the FL experiment**\n\n- Visit the aggregator's dashboard to monitor the FL progress\nwhich is available at `http://server_url/datasites/\u003caggregator_email\u003e/fl/\u003cproject_name\u003e/`, with `\u003cproject_name\u003e` matches the name of the FL experiment setup in the `fl_config.json` file.\n\nThe server_url depends on which the server the client is running on.\n- `syftbox.openmined.org` (for the public server)\n- `localhost:5001` (when running locally, with default configuration)\n\n## Running in dev mode\n\n\u003e ⚠️ Make sure you have the latest version of the SyftBox repository cloned on your local machine:\n\u003e\n\u003e ```bash\n\u003e git clone https://github.com/OpenMined/syft.git\n\u003e ```\n\u003e Also, make sure you have [just](https://github.com/casey/just) installed.\n\n**Note**: The following commands need to be run within the `syft` directory (the repository root), unless otherwise indicated.\n\n### **Set up**: \n\nFirst launch a local SyftBox server, and clients for all datasites participating in the FL flow:\n\n- `just rs`: run SyftBox local dev server on port `5001`\n- `just rc a`: creates a SyftBox client for `a@openmined.org`. (Repeat this for all the clients)\n\nEach of those commands need to be run in a separate terminal session, or within the same terminal\nusing tools like [`tmux`](https://github.com/tmux/tmux/wiki) (recommended!).\n\n\n### **Install the aggregator and the client APIs**: \n\nAggregator:\n- `git clone https://github.com/OpenMined/fl_aggregator` \n- `cp -R ./fl_aggregator ./.clients/a@openmined.org/apis` \n\nClient(s):\n\n- `git clone https://github.com/OpenMined/fl_client` \n- `cp -R ./fl_client ./.clients/b@openmined.org/apis`\n- `cp -R ./fl_client ./.clients/c@openmined.org/apis`\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenmined%2Ffl_aggregator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fopenmined%2Ffl_aggregator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenmined%2Ffl_aggregator/lists"}