{"id":20446351,"url":"https://github.com/ray-project/rayfed","last_synced_at":"2025-04-05T23:12:07.269Z","repository":{"id":65191146,"uuid":"567190910","full_name":"ray-project/rayfed","owner":"ray-project","description":"A multiple parties joint, distributed execution engine based on Ray, to help build your own federated learning frameworks in minutes.","archived":false,"fork":false,"pushed_at":"2024-08-28T09:03:26.000Z","size":1445,"stargazers_count":96,"open_issues_count":34,"forks_count":22,"subscribers_count":8,"default_branch":"main","last_synced_at":"2025-03-29T22:08:07.567Z","etag":null,"topics":["distributed-execution","federated-learning","privacy-preserving","python","ray"],"latest_commit_sha":null,"homepage":"https://rayfed.readthedocs.io","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ray-project.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-11-17T09:11:02.000Z","updated_at":"2025-02-16T08:30:23.000Z","dependencies_parsed_at":"2023-11-14T14:25:50.270Z","dependency_job_id":"c0877bba-9b44-4175-9c15-1440c9024e69","html_url":"https://github.com/ray-project/rayfed","commit_stats":{"total_commits":90,"total_committers":8,"mean_commits":11.25,"dds":0.4666666666666667,"last_synced_commit":"2575c7af175008913e7ac34f79aa0daa1a5f2b45"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ray-project%2Frayfed","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ray-project%2Frayfed/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ray-project%2Frayfed/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ray-project%2Frayfed/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ray-project","download_url":"https://codeload.github.com/ray-project/rayfed/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247411239,"owners_count":20934653,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["distributed-execution","federated-learning","privacy-preserving","python","ray"],"created_at":"2024-11-15T10:19:36.562Z","updated_at":"2025-04-05T23:12:07.250Z","avatar_url":"https://github.com/ray-project.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# RayFed\n![docs building](https://readthedocs.org/projects/rayfed/badge/?version=latest) ![test on many rays](https://github.com/ray-project/rayfed/actions/workflows/unit_tests_on_ray_matrix.yml/badge.svg) ![test on ray 1.13.0](https://github.com/ray-project/rayfed/actions/workflows/test_on_ray1.13.0.yml/badge.svg)\n\nA multiple parties joint, distributed execution engine based on Ray, to help build your own federated learning frameworks in minutes.\n\n## Overview\n**Note: This project is now in actively developing.**\n\nRayFed is a distributed computing framework for cross-parties federated learning.\nBuilt in the Ray ecosystem, RayFed provides a Ray native programming pattern for federated learning so that users can build a distributed program easily.\n\nIt provides users the role of \"party\", thus users can write code belonging to the specific party explicitly imposing more clear data perimeters. These codes will be restricted to execute within the party.\n\nAs for the code execution, RayFed introduces the multi-controller architecture:\nThe code view in each party is exactly the same, but the execution differs based on the declared party of code and the current party of executor. \n\n\n\n## Features\n- **Ray Native Programming Pattern**  \n  \n  Let you write your federated and distributed computing applications like a single-machine program.\n\n- **Multiple Controller Execution Mode**  \n  \n  The RayFed job can be run in the single-controller mode for developing and debugging and the multiple-controller mode for production without code change.\n  \n- **Very Restricted and Clear Data Perimeters**  \n  \n  Because of the PUSH-BASED data transferring mechanism and multiple controller execution mode, the data transmission authority is held by the data owner rather than the data demander.\n\n- **Very Large Scale Federated Computing and Training**  \n  \n  Powered by the scalabilities and the distributed abilities from Ray, large scale federated computing and training jobs are naturally supported.\n\n\n## Supported Ray Versions\nDue to Ray's aggressive release strategy, Rayfed only supports the last 5 Ray versions.\n| RayFed Versions | ray-1.13.0 | ray-2.31.0 | ray-2.32.0 | ray-2.33.0 | ray-2.34.0 | ray-2.35.0 |\n|:---------------:|:--------:|:--------:|:--------:|:--------:|:--------:|:--------:|\n| 0.1.0           |✅         | ✅       | ✅       | ✅       | ✅        | ✅       |\n| 0.2.0           |not released|not released|not released|not released|not released|not released|\n\n\n## Installation\nInstall it from pypi.\n\n```shell\npip install -U rayfed\n```\n\nInstall the nightly released version from pypi.\n\n```shell\npip install -U rayfed-nightly\n```\n## Quick Start\n\nThis example shows how to aggregate values across two participators.\n\n### Step 1: Write an Actor that Generates Value\nThe `MyActor` increment its value by `num`. \nThis actor will be executed within the explicitly declared party.\n\n```python\nimport sys\nimport ray\nimport fed\n\n@fed.remote\nclass MyActor:\n    def __init__(self, value):\n        self.value = value\n\n    def inc(self, num):\n        self.value = self.value + num\n        return self.value\n```\n### Step 2: Define Aggregation Function\nThe below function collects and aggragates values from two parties separately, and will also be executed within the declared party.\n\n```python\n@fed.remote\ndef aggregate(val1, val2):\n    return val1 + val2\n```\n\n### Step 3: Create the actor and call methods in a specific party\n\nThe creation code is similar with `Ray`, however, the difference is that in `RayFed` the actor must be explicitly created within a party:\n\n```python\nactor_alice = MyActor.party(\"alice\").remote(1)\nactor_bob = MyActor.party(\"bob\").remote(1)\n\nval_alice = actor_alice.inc.remote(1)\nval_bob = actor_bob.inc.remote(2)\n\nsum_val_obj = aggregate.party(\"bob\").remote(val_alice, val_bob)\n```\nThe above codes:\n1. Create two `MyActor`s separately in each party, i.e. 'alice' and 'bob';\n2. Increment by '1' in alice and '2' in 'bob';\n3. Execute the aggregation function in party 'bob'.\n\n### Step 4: Declare Cross-party Cluster \u0026 Init \n```python\ndef main(party):\n    ray.init(address='local', include_dashboard=False)\n\n    addresses = {\n        'alice': '127.0.0.1:11012',\n        'bob': '127.0.0.1:11011',\n    }\n    fed.init(addresses=addresses, party=party)\n```\nThis first declares a two-party cluster, whose addresses corresponding to '127.0.0.1:11012' in 'alice' and '127.0.0.1:11011' in 'bob'.\nAnd then, the `fed.init` create a cluster in the specified party.\nNote that `fed.init` should be called twice, passing in the different party each time.\n\nWhen executing codes in step 1~3, the 'alice' cluster will only execute functions whose \"party\" are also declared as 'alice'.\n\n### Put it together !\nSave below codes as `demo.py`: \n```python\nimport sys\nimport ray\nimport fed\n\n\n@fed.remote\nclass MyActor:\n    def __init__(self, value):\n        self.value = value\n\n    def inc(self, num):\n        self.value = self.value + num\n        return self.value\n\n\n@fed.remote\ndef aggregate(val1, val2):\n    return val1 + val2\n\n\ndef main(party):\n    ray.init(address='local', include_dashboard=False)\n\n    addresses = {\n        'alice': '127.0.0.1:11012',\n        'bob': '127.0.0.1:11011',\n    }\n    fed.init(addresses=addresses, party=party)\n\n    actor_alice = MyActor.party(\"alice\").remote(1)\n    actor_bob = MyActor.party(\"bob\").remote(1)\n\n    val_alice = actor_alice.inc.remote(1)\n    val_bob = actor_bob.inc.remote(2)\n\n    sum_val_obj = aggregate.party(\"bob\").remote(val_alice, val_bob)\n    result = fed.get(sum_val_obj)\n    print(f\"The result in party {party} is {result}\")\n\n    fed.shutdown()\n    ray.shutdown()\n\n\nif __name__ == \"__main__\":\n    assert len(sys.argv) == 2, 'Please run this script with party.'\n    main(sys.argv[1])\n\n```\n\n### Run The Code.\n\nOpen a terminal and run the code as `alice`. It's recommended to run the code with Ray TLS enabled (please refer to [Ray TLS](https://docs.ray.io/en/latest/ray-core/configure.html#tls-authentication))\n```shell\nRAY_USE_TLS=1 \\\nRAY_TLS_SERVER_CERT='/path/to/the/server/cert/file' \\\nRAY_TLS_SERVER_KEY='/path/to/the/server/key/file' \\\nRAY_TLS_CA_CERT='/path/to/the/ca/cert/file' \\\npython test.py alice\n```\n\nIn the mean time, open another terminal and run the code as `bob`.\n```shell\nRAY_USE_TLS=1 \\\nRAY_TLS_SERVER_CERT='/path/to/the/server/cert/file' \\\nRAY_TLS_SERVER_KEY='/path/to/the/server/key/file' \\\nRAY_TLS_CA_CERT='/path/to/the/ca/cert/file' \\\npython test.py bob\n```\n\nThen you will get `The result in party alice is 5` on the first terminal screen and `The result in party bob is 5` on the second terminal screen.\n\nFigure shows the execution under the hood:\n\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"https://s1.ax1x.com/2023/03/08/ppeH68x.png\" alt=\"Figure\" width=\"500\"\u003e  \n\u003c/div\u003e\n## Running untrusted codes\nAs a general rule: Always execute untrusted codes inside a sandbox (e.g., [nsjail](https://github.com/google/nsjail)).\n\n## Who use us\n\u003ca href=\"https://github.com/alipay/Antchain-MPC\"\u003e\n\u003cimg src=\"docs/images/morse-logo.png\" alt=\"Ant Chain Morse\" width=\"240\"\u003e  \n\u003c/a\u003e\n\u003ca href=\"https://github.com/secretflow/secretflow\"\u003e\n\u003cimg src=\"docs/images/secretflow-logo.png\" alt=\"SecretFlow\" width=\"240\"\u003e  \n\u003c/a\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fray-project%2Frayfed","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fray-project%2Frayfed","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fray-project%2Frayfed/lists"}