{"id":19329758,"url":"https://github.com/outerbounds/metaflow-tensorflow","last_synced_at":"2026-05-18T05:11:32.295Z","repository":{"id":199362063,"uuid":"695726404","full_name":"outerbounds/metaflow-tensorflow","owner":"outerbounds","description":null,"archived":false,"fork":false,"pushed_at":"2024-09-24T18:30:58.000Z","size":27,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-01-06T08:17:45.882Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/outerbounds.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-09-24T02:57:02.000Z","updated_at":"2024-09-24T18:31:36.000Z","dependencies_parsed_at":"2023-11-03T00:23:02.892Z","dependency_job_id":null,"html_url":"https://github.com/outerbounds/metaflow-tensorflow","commit_stats":null,"previous_names":["outerbounds/metaflow-tensorflow"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/outerbounds%2Fmetaflow-tensorflow","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/outerbounds%2Fmetaflow-tensorflow/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/outerbounds%2Fmetaflow-tensorflow/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/outerbounds%2Fmetaflow-tensorflow/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/outerbounds","download_url":"https://codeload.github.com/outerbounds/metaflow-tensorflow/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240434233,"owners_count":19800548,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-10T02:29:47.681Z","updated_at":"2026-05-18T05:11:32.232Z","avatar_url":"https://github.com/outerbounds.png","language":"Python","funding_links":[],"categories":["Distributed Compute \u0026 Training"],"sub_categories":[],"readme":"# Metaflow TensorFlow decorator\nThe [`tf.distribute.Strategy`](https://www.tensorflow.org/api_docs/python/tf/distribute/Strategy) allows TensorFlow developers to distribute model training to multiple GPUs/TPUs and machines. This repository implements the Metaflow `@tensorflow` decorator, which sets up a multi-node Metaflow step to use this functionality.\n\n### Features\n\n### Installation\n\nInstall this experimental module:\n```\npip install metaflow-tensorflow\n```\n\n### Getting Started\nThis package will add a Metaflow extension to your already installed Metaflow, so you can use the `tensorflow` decorator.\n```\nfrom metaflow import FlowSpec, step, tensorflow, ...\n```\n\nThe rest of this `README.md` file describes how you can use TensorFlow with Metaflow in the single node and multi-node cases which require `@tensorflow`.\n\n# TensorFlow Distributed on Metaflow guide\nThe examples in this repository are based on the [original TensorFlow Examples](https://www.tensorflow.org/guide/distributed_training#examples_and_tutorials).\n\n### Examples and guides\n\n| Directory | TensorFlow script description |\n| :--- | ---: |\n| [MirroredStrategy](examples/single-node/README.md) | Synchronous distributed training on multiple GPUs on one machine. |  \n| [MultiWorkerMirroredStrategy](examples/multi-node/README.md) | Synchronous distributed training across multiple workers, each with potentially multiple GPUs. |  \n\n#### Parameter Server\nNot yet tested, please reach out to the Outerbounds team if you need help.\n\n#### Installing TensorFlow for GPU usage in Metaflow\n\u003e From [TensorFlow documentation](https://www.tensorflow.org/install/pip): Do not install TensorFlow with conda. It may not have the latest stable version. pip is recommended since TensorFlow is only officially released to PyPI.\n\nWe have found the easiest way to install TensorFlow for GPU is to use the pre-made Docker image `tensorflow/tensorflow:latest-gpu`.\n\n#### Fault Tolerance\nSee [TensorFlow documentation on this matter](https://www.tensorflow.org/tutorials/distribute/multi_worker_with_keras#fault_tolerance).\nThe TL;DR is to use a flavor of `tf.distribute.Strategy`, which implement mechanisms to handle worker failures gracefully.\n\n### License \n`metaflow-tensorflow` is distributed under the \u003cu\u003eApache License\u003c/u\u003e.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fouterbounds%2Fmetaflow-tensorflow","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fouterbounds%2Fmetaflow-tensorflow","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fouterbounds%2Fmetaflow-tensorflow/lists"}