{"id":20484726,"url":"https://github.com/sap-samples/datasphere-fedml","last_synced_at":"2025-04-13T14:52:41.610Z","repository":{"id":41170287,"uuid":"421176349","full_name":"SAP-samples/datasphere-fedml","owner":"SAP-samples","description":"The publication is a collection of sample code to show how data from SAP and non-SAP systems can be made available for training in ANY hyperscaler machine learning service via several layers of abstraction from data connection to training using our FedML Python libraries. ","archived":false,"fork":false,"pushed_at":"2024-04-05T21:57:01.000Z","size":21889,"stargazers_count":11,"open_issues_count":6,"forks_count":11,"subscribers_count":13,"default_branch":"main","last_synced_at":"2024-04-06T19:47:50.426Z","etag":null,"topics":["3944","4200","btp-use-case-factory","data-federation","data-to-value","hyperscalers","machine-learning","sample","sample-code","sap-data-warehouse-cloud","sap-datasphere"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/SAP-samples.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2021-10-25T20:33:21.000Z","updated_at":"2024-04-15T23:43:53.273Z","dependencies_parsed_at":"2024-04-15T23:54:02.389Z","dependency_job_id":null,"html_url":"https://github.com/SAP-samples/datasphere-fedml","commit_stats":null,"previous_names":["sap-samples/datasphere-fedml","sap-samples/data-warehouse-cloud-fedml"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SAP-samples%2Fdatasphere-fedml","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SAP-samples%2Fdatasphere-fedml/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SAP-samples%2Fdatasphere-fedml/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SAP-samples%2Fdatasphere-fedml/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/SAP-samples","download_url":"https://codeload.github.com/SAP-samples/datasphere-fedml/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248732509,"owners_count":21152851,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3944","4200","btp-use-case-factory","data-federation","data-to-value","hyperscalers","machine-learning","sample","sample-code","sap-data-warehouse-cloud","sap-datasphere"],"created_at":"2024-11-15T16:25:19.736Z","updated_at":"2025-04-13T14:52:41.603Z","avatar_url":"https://github.com/SAP-samples.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![REUSE status](https://api.reuse.software/badge/github.com/SAP-samples/datasphere-fedml)](https://api.reuse.software/info/github.com/SAP-samples/datasphere-fedml)\n\n# FedML\n\n## Description \n \n\u003cbr\u003e\nThe SAP Federated ML Python libraries (FedML) applies the Data Federation architecture of SAP Datasphere for intelligently sourcing SAP as well as non-SAP data for Machine Learning experiments done \u003cb\u003eat any Machine Learning platform\u003c/b\u003e thereby removing the need for replicating or moving data.\nBy abstracting data connection, data loading (for all ML platforms), model training (with flexibility and support for user-provided training scripts), model deployment, and inferencing (for hyperscaler machine learning platforms), the FedML library offers end-to-end integration with just a few lines of code.\n\n ## What's New \n \n\u003cb\u003e1.\u003c/b\u003e The new version of FedML (available as \u003cb\u003efedml-dsp\u003c/b\u003e in PyPi, V1.0.0) :\n\u003cul\u003e\u003cli\u003eIs machine learning platform-independent. It can be used in all machine learning platforms\n\u003cli\u003eSupports NVIDIA RAPIDS™, CUDA cuDF and cuPy and hence can be used for training models in GPU environments.\n\u003cli\u003eSupports sourcing data from SAP Datasphere models directly into PySpark and cuPy (for GPU) dataframes.\n\u003cli\u003eSupports SAP AI Core Deployment  - Models that are trained in any ML Platform (and containerized independently) can now be deployed in SAP GenAI Hub's AI Core with couple lines of code.\n\u003cli\u003eSupports writing inferenced results back to SAP Datasphere.\n \u003c/ul\u003e\n \n ### Solution Architecture \n \n ![ARD](/FedMLNew.jpg)\n \n\u003cb\u003e2.\u003c/b\u003eFedML (Original, V2.0) for hyperscaler platforms [AWS, GCP, Azure and Databricks] :\u003c/font\u003e\n\u003cul\u003e\u003cli\u003eIs pip installable from PyPi for its respective hyperscaler platforms.\n\u003cli\u003eSupports model training and deployment to hyperscaler environment.\n\u003cli\u003eSupports deployment to SAP Business Technology Platform Kyma environment. \n\u003cli\u003eSupports inferencing with hyperscaler deployed as well as Kyma deployed models.\n\u003cli\u003eSupports writing inferenced results back to SAP Datasphere.\n\u003c/ul\u003e\u003c/ul\u003e\n\n## Requirements \n \n- SAP Datasphere tenant instance, with connectivity established to the remote data sources, and views exposed, that can be consumed by FedML. \n\n- Access to corresponding  Machine learning Platforms with appropriate configurations. See [Configuration](#configuration) section.\n\n\n## Download and Installation \n\n Try out examples from the **samples-notebooks** directory of corresponding library folders\n\n## Configuration \n- For FedML (platform-independent) library specific pre-requisites, configuration and documentation, [please refer here](Datasphere/fedml-dsp.md) \u003cbr\u003e\n- For AWS FedML library specific pre-requisites, configuration and documentation, [please refer here](AWS/fedml_aws.md) \u003cbr\u003e\n- For GCP FedML library specific pre-requisites, configuration and documentation, [please refer here](GCP/fedml_gcp.md)\u003cbr\u003e\n- For Azure FedML library specific pre-requisites, configuration and documentation, [please refer here](Azure/readme.md) \u003cbr\u003e\n- For Databricks FedML library specific pre-requisites, configuration and documentation, [please refer here](Databricks/README.md)\u003cbr\u003e\u003cbr\u003e\n\n## Limitations \n\nNone\n  \u003cbr\u003e\n\n## How to obtain support \n\nThis project is provided \"as-is\" with no expectation for major changes or support. \u003cbr\u003e\n[Create an issue](/issues) in this repository if you find a bug or have questions about the content. \u003cbr\u003e\nFor additional support, [ask a question](https://answers.sap.com/questions/ask.html) in SAP Community. \n   \u003cbr\u003e\u003cbr\u003e\n\n## Licensing \n \nCopyright (c) 2021 SAP SE or an SAP affiliate company. All rights reserved. This project is licensed under the Apache Software License, version 2.0 except as noted otherwise in the [LICENSE](LICENSES/Apache-2.0.txt) file.\n\u003cbr\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsap-samples%2Fdatasphere-fedml","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsap-samples%2Fdatasphere-fedml","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsap-samples%2Fdatasphere-fedml/lists"}