{"id":19888277,"url":"https://github.com/project-codeflare/serverless-distributed-dl-training","last_synced_at":"2026-01-29T03:03:28.855Z","repository":{"id":103618194,"uuid":"410952314","full_name":"project-codeflare/serverless-distributed-dl-training","owner":"project-codeflare","description":null,"archived":false,"fork":false,"pushed_at":"2021-11-22T21:06:19.000Z","size":13,"stargazers_count":1,"open_issues_count":0,"forks_count":1,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-05-31T11:31:45.709Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/project-codeflare.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2021-09-27T16:02:54.000Z","updated_at":"2022-02-16T07:47:14.000Z","dependencies_parsed_at":"2023-05-23T23:30:47.644Z","dependency_job_id":null,"html_url":"https://github.com/project-codeflare/serverless-distributed-dl-training","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/project-codeflare/serverless-distributed-dl-training","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/project-codeflare%2Fserverless-distributed-dl-training","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/project-codeflare%2Fserverless-distributed-dl-training/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/project-codeflare%2Fserverless-distributed-dl-training/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/project-codeflare%2Fserverless-distributed-dl-training/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/project-codeflare","download_url":"https://codeload.github.com/project-codeflare/serverless-distributed-dl-training/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/project-codeflare%2Fserverless-distributed-dl-training/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28861654,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-28T22:56:21.783Z","status":"online","status_checked_at":"2026-01-29T02:00:06.714Z","response_time":59,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-12T18:06:48.500Z","updated_at":"2026-01-29T03:03:28.849Z","avatar_url":"https://github.com/project-codeflare.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# Serverless Distributed Deep Learning Training with Code Engine\n\nThis tutorial walks through the steps to train a deep learning model in a ***serverless*** and ***distributed*** environment with IBM Code Engine.\n\n## Components\n\n- **Code Engine**: provides the underlying serverless environment.\n- **Ray**: provides a Ray cluster on Code Engine for easily running distributed applications.\n- **Horovod**: provides a distributed deep learning training framework for TensorFlow, Keras, PyTorch, and Apache MXNet.\n\nThe **serverless** and **distributed** deep learning training is achieved by **running Horovod code on a Ray cluster which launched in Code Engine**.\n\n## Steps\n\n1. Setup Code Engine. \n    1. Create a [Code Engine](https://cloud.ibm.com/catalog/services/codeengine) project in IBM Cloud.\n    2. Enable the project with Ray. Currently this can only be done by sharing the namespace id of your project with Code Engine team and ask their help to enable it. **Update: now you can do it yourself: https://www.ibm.com/cloud/blog/ray-on-ibm-cloud-code-engine**\n2. Prepare Ray Cluster file. Ray cluster is configured and provisioned by a single ray cluster yaml file. All cluster-related inforation (nodes spec, container image, resouce allocation, etc.) should be configured in this file. Here is a [template](examples/mnist/ray_cluster.yaml).\n3. Prepare training script. Modify your existing training script to use Horovod on Ray.\n    1. Modify for Horovod. The modification varies a little bit by your favorite framework, but is very easy and straightforward. You can find more information on [Horovod with Tensorflow](https://horovod.readthedocs.io/en/stable/tensorflow.html), [Horovod with Tensorflow Keras](https://horovod.readthedocs.io/en/stable/keras.html), [Horovod with PyTorch](https://horovod.readthedocs.io/en/stable/pytorch.html) and [Horovod on MXNet](https://horovod.readthedocs.io/en/stable/mxnet.html).\n    2. Modify for Ray integration. Script for running on Horovod-Ray is slightly different from running on Horovod alone, and a few more modifications for ray integration is required. This is as simple as wrapping your code from previous step in a train() function and execute it with RayExecutor. You can find the example [here](examples/mnist/ray_mnist.py)\n4. Run it! Running is as simple as a two-line script:\n    ```bash\n    # Launch Ray cluster\n    ray up -y --no-config-cache ray_cluster.yaml \n    # Submit your training\n    ray submit --no-config-cache ray_cluster.yaml ray_mnist.py\n    ```\n    You might have to wait couple minutes for the cluster launch to complete after running the first command, if you are launching it for the first time (as it takes time to pull the image and allocate the resources).\n    \n## Examples\nYou can find examples [here](examples).  \n[mnist](examples/mnist) contains an example of training Fashion MNIST model where data is consumed from COS and trained model is saved back to COS.  \n[mnist-transfer-learning](examples/mnist-transfer-learning) contains an example of doing transfer learning or fine-tuning or reusume-training where data is consumed from COS, pre-trained model is loaded from COS and trained model is saved back to COS.\n\nTo run the demo (e.g. mnist), just do:\n```bash\n# 1. Connect to your code engine project, e.g.\nexport KUBECONFIG=/Users/lchu/.bluemix/plugins/code-engine/horovod-5dc3ff50-23e7-46be-92b2-e2de2da9bd71.yaml\n# 2. Launch Ray cluster\nray up -y --no-config-cache ray_cluster.yaml \n# 3. Submit your training\ncd examples/mnist\n### Modify information as needed, e.g. change credentials in ray_mnist.py, change resouce allocation in ray_cluster.yaml, etc. ###\nray submit --no-config-cache ray_cluster.yaml ray_mnist.py\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fproject-codeflare%2Fserverless-distributed-dl-training","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fproject-codeflare%2Fserverless-distributed-dl-training","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fproject-codeflare%2Fserverless-distributed-dl-training/lists"}