{"id":18554047,"url":"https://github.com/xuefeng-xu/fedps","last_synced_at":"2025-06-26T23:06:24.256Z","repository":{"id":216928669,"uuid":"740401864","full_name":"xuefeng-xu/fedps","owner":"xuefeng-xu","description":"Federated data Preprocessing via aggregated Statistics","archived":false,"fork":false,"pushed_at":"2024-10-20T10:14:24.000Z","size":206,"stargazers_count":4,"open_issues_count":0,"forks_count":1,"subscribers_count":6,"default_branch":"main","last_synced_at":"2024-11-06T21:20:37.754Z","etag":null,"topics":["data-preprocessing","federated-learning","python","scikit-learn","statistics"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/xuefeng-xu.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-01-08T09:19:13.000Z","updated_at":"2024-10-20T10:14:28.000Z","dependencies_parsed_at":"2024-01-16T13:38:33.108Z","dependency_job_id":"ed822640-c5d4-45aa-8226-75fff8370693","html_url":"https://github.com/xuefeng-xu/fedps","commit_stats":{"total_commits":8,"total_committers":1,"mean_commits":8.0,"dds":0.0,"last_synced_commit":"accd13d5223ad19fcf9e2ccc911a0a2bb82e1c79"},"previous_names":["primihub/fedps","xuefeng-xu/fedps"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xuefeng-xu%2Ffedps","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xuefeng-xu%2Ffedps/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xuefeng-xu%2Ffedps/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xuefeng-xu%2Ffedps/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/xuefeng-xu","download_url":"https://codeload.github.com/xuefeng-xu/fedps/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248129559,"owners_count":21052594,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-preprocessing","federated-learning","python","scikit-learn","statistics"],"created_at":"2024-11-06T21:19:31.306Z","updated_at":"2025-04-09T23:30:56.587Z","avatar_url":"https://github.com/xuefeng-xu.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# FedPS\n\nFedPS is a Python module designed for data preprocessing in Federated Learning, primarily leveraging aggregated statistics. The preprocessing workflow involves the following five steps:\n\n1. Local Statistics Estimation: Clients estimate local statistics from their local data.\n2. Aggregation: The server receives the local statistics and performs aggregation.\n3. Global Parameter Calculation: The server calculates the global preprocessing parameters.\n4. Parameter Distribution: The global parameters are then sent back to the clients.\n5. Data Preprocessing: Clients apply the preprocessing to their local data.\n\n\u003cdiv align=center\u003e\n    \u003cimg src=\"doc/overview.svg\", alt=\"Overview\", width=\"60%\"\u003e\n\u003c/div\u003e\n\n## Installation\n\n### Dependencies\n\n- Python (\u003e= 3.9)\n- Scikit-learn (~= 1.6)\n- NumPy (\u003e= 1.20)\n- DataSketches\n- PyZMQ\n\n### Building from source\n\n1. Create a Python env\n\n```bash\nconda create --name fedps python=3.9\nconda activate fedps\n```\n\n2. Clone this project\n\n```bash\ngit clone https://github.com/xuefeng-xu/fedps.git\n```\n\n3. Build the project\n\n```bash\ncd fedps\npip install .\n```\n\n## Usage\n\n1. Set up communication channels\n\n```python\n# Client1 channel\nfrom fedps.channel import ClientChannel\n\nchannel = ClientChannel(\n    local_ip=\"127.0.0.1\", local_port=5556,\n    remote_ip=\"127.0.0.1\", remote_port=5555,\n)\n```\n\n```python\n# Client2 channel\nfrom fedps.channel import ClientChannel\n\nchannel = ClientChannel(\n    local_ip=\"127.0.0.1\", local_port=5557,\n    remote_ip=\"127.0.0.1\", remote_port=5555,\n)\n```\n\n```python\n# Server channel\nfrom fedps.channel import ServerChannel\n\nchannel = ServerChannel(\n    local_ip=\"127.0.0.1\", local_port=5555,\n    remote_ip=[\"127.0.0.1\", \"127.0.0.1\"],\n    remote_port=[5556, 5557],\n)\n```\n\n2. Specify `FL_type` and `role` in the preprocessor\n\n- `FL_type`: \"H\" (Horizontal) or \"V\" (Vertical)\n\n- `role`: \"client\" or \"server\"\n\n```python\n# Client1 code example\nfrom fedps.preprocessing import MinMaxScaler\n\nX = [[-1, 2], [-0.5, 6]]\nest = MinMaxScaler(FL_type=\"H\", role=\"client\", channel=channel)\nXt = est.fit_transform(X)\nprint(Xt)\n```\n\n```python\n# Client2 code example\nfrom fedps.preprocessing import MinMaxScaler\n\nX = [[0, 10], [1, 18]]\nest = MinMaxScaler(FL_type=\"H\", role=\"client\", channel=channel)\nXt = est.fit_transform(X)\nprint(Xt)\n```\n\n```python\n# Server code example\nfrom fedps.preprocessing import MinMaxScaler\n\nest = MinMaxScaler(FL_type=\"H\", role=\"server\", channel=channel)\nest.fit()\n```\n\n3. Run the script\n\n```bash\n# Run in three terminals\npython client1.py\npython client2.py\npython server.py\n```\n\nPS: See more cases in the [example](example) folder.\n\n## Available preprocessing modules\n\n- Discretization\n  - [`KBinsDiscretizer`](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.KBinsDiscretizer.html)\n\n- Encoding\n  - [`LabelEncoder`](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.LabelEncoder.html)\n  - [`LabelBinarizer`](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.LabelBinarizer.html)\n  - [`MultiLabelBinarizer`](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.MultiLabelBinarizer.html)\n  - [`OneHotEncoder`](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.OneHotEncoder.html)\n  - [`OrdinalEncoder`](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.OrdinalEncoder.html)\n  - [`TargetEncoder`](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.TargetEncoder.html)\n\n- Scaling\n  - [`MaxAbsScaler`](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.MaxAbsScaler.html)\n  - [`MinMaxScaler`](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.MinMaxScaler.html)\n  - [`Normalizer`](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.Normalizer.html)\n  - [`RobustScaler`](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.RobustScaler.html)\n  - [`StandardScaler`](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.StandardScaler.html)\n\n- Transformation\n  - [`PowerTransformer`](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.PowerTransformer.html)\n  - [`QuantileTransformer`](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.QuantileTransformer.html)\n  - [`SplineTransformer`](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.SplineTransformer.html)\n\n- Imputation\n  - [`IterativeImputer`](https://scikit-learn.org/stable/modules/generated/sklearn.impute.IterativeImputer.html) (experimental)\n  - [`KNNImputer`](https://scikit-learn.org/stable/modules/generated/sklearn.impute.KNNImputer.html)\n  - [`SimpleImputer`](https://scikit-learn.org/stable/modules/generated/sklearn.impute.SimpleImputer.html)\n\n## Differences from Scikit-learn\n\n- Currently, this library does not support sparse data.\n- [`KBinsDiscretizer`](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.KBinsDiscretizer.html), [`StandardScaler`](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.StandardScaler.html), and [`SplineTransformer`](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.SplineTransformer.html) cannot set the `sample_weight` parameter in their fit methods.\n- [`IterativeImputer`](https://scikit-learn.org/stable/modules/generated/sklearn.impute.IterativeImputer.html) does not support the `sample_posterior` and `n_nearest_features` parameters.\n- [`KNNImputer`](https://scikit-learn.org/stable/modules/generated/sklearn.impute.KNNImputer.html) does not support custom weight funtion and distance metric.\n\n## Acknowledgement\n\nThis project is build on [Scikit-learn](https://github.com/scikit-learn/scikit-learn).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxuefeng-xu%2Ffedps","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fxuefeng-xu%2Ffedps","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxuefeng-xu%2Ffedps/lists"}