{"id":13400229,"url":"https://github.com/tensorflow/transform","last_synced_at":"2025-05-13T20:09:56.230Z","repository":{"id":37276387,"uuid":"81509390","full_name":"tensorflow/transform","owner":"tensorflow","description":"Input pipeline framework","archived":false,"fork":false,"pushed_at":"2025-04-30T06:47:49.000Z","size":3811,"stargazers_count":986,"open_issues_count":40,"forks_count":217,"subscribers_count":56,"default_branch":"master","last_synced_at":"2025-05-08T00:09:46.299Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tensorflow.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2017-02-10T00:36:53.000Z","updated_at":"2025-05-04T20:26:10.000Z","dependencies_parsed_at":"2023-02-17T09:31:32.376Z","dependency_job_id":"182eae7a-09d9-470a-989a-71a97bf844ce","html_url":"https://github.com/tensorflow/transform","commit_stats":{"total_commits":894,"total_committers":29,"mean_commits":30.82758620689655,"dds":0.6364653243847875,"last_synced_commit":"af90f98817c9ee172e42a485abfd1d55730deb07"},"previous_names":[],"tags_count":51,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tensorflow%2Ftransform","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tensorflow%2Ftransform/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tensorflow%2Ftransform/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tensorflow%2Ftransform/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tensorflow","download_url":"https://codeload.github.com/tensorflow/transform/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254020606,"owners_count":22000753,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-30T19:00:49.765Z","updated_at":"2025-05-13T20:09:56.199Z","avatar_url":"https://github.com/tensorflow.png","language":"Python","readme":"\u003c!-- See: www.tensorflow.org/tfx/transform/ --\u003e\n\n# TensorFlow Transform\n\n[![Python](https://img.shields.io/badge/python%7C3.9%7C3.10%7C3.11-blue)](https://github.com/tensorflow/transform)\n[![PyPI](https://badge.fury.io/py/tensorflow-transform.svg)](https://badge.fury.io/py/tensorflow-transform)\n[![Documentation](https://img.shields.io/badge/api-reference-blue.svg)](https://www.tensorflow.org/tfx/transform/api_docs/python/tft)\n\n*TensorFlow Transform* is a library for preprocessing data with TensorFlow.\n`tf.Transform` is useful for data that requires a full-pass, such as:\n\n* Normalize an input value by mean and standard deviation.\n* Convert strings to integers by generating a vocabulary over all input values.\n* Convert floats to integers by assigning them to buckets based on the observed\n  data distribution.\n\nTensorFlow has built-in support for manipulations on a single example or a batch\nof examples. `tf.Transform` extends these capabilities to support full-passes\nover the example data.\n\nThe output of `tf.Transform` is exported as a\n[TensorFlow graph](http://tensorflow.org/guide/graphs) to use for training and\nserving. Using the same graph for both training and serving can prevent skew\nsince the same transformations are applied in both stages.\n\nFor an introduction to `tf.Transform`, see the `tf.Transform` section of the\nTFX Dev Summit talk on TFX\n([link](https://www.youtube.com/watch?v=vdG7uKQ2eKk\u0026feature=youtu.be\u0026t=199)).\n\n## Installation\n\nThe `tensorflow-transform`\n[PyPI package](https://pypi.org/project/tensorflow-transform/) is the\nrecommended way to install `tf.Transform`:\n\n```bash\npip install tensorflow-transform\n```\n\n### Build TFT from source\n\nTo build from source follow the following steps:\nCreate a virtual environment by running the commands\n\n```\npython3 -m venv \u003cvirtualenv_name\u003e\nsource \u003cvirtualenv_name\u003e/bin/activate\npip3 install setuptools wheel\ngit clone https://github.com/tensorflow/transform.git\ncd transform\npython3 setup.py bdist_wheel\n```\n\nThis will build the TFT wheel in the dist directory. To install the wheel from\ndist directory run the commands\n\n```\ncd dist\npip3 install tensorflow_transform-\u003cversion\u003e-py3-none-any.whl\n```\n\n### Nightly Packages\n\nTFT also hosts nightly packages at https://pypi-nightly.tensorflow.org on\nGoogle Cloud. To install the latest nightly package, please use the following\ncommand:\n\n```bash\npip install --extra-index-url https://pypi-nightly.tensorflow.org/simple tensorflow-transform\n```\n\nThis will install the nightly packages for the major dependencies of TFT such\nas TensorFlow Metadata (TFMD), TFX Basic Shared Libraries (TFX-BSL).\n\n### Notable Dependencies\n\nTensorFlow is required.\n\n[Apache Beam](https://beam.apache.org/) is required; it's the way that efficient\ndistributed computation is supported. By default, Apache Beam runs in local\nmode but can also run in distributed mode using\n[Google Cloud Dataflow](https://cloud.google.com/dataflow/) and other Apache\nBeam\n[runners](https://beam.apache.org/documentation/runners/capability-matrix/).\n\n[Apache Arrow](https://arrow.apache.org/) is also required. TFT uses Arrow to\nrepresent data internally in order to make use of vectorized numpy functions.\n\n## Compatible versions\n\nThe following table is the `tf.Transform` package versions that are\ncompatible with each other. This is determined by our testing framework, but\nother *untested* combinations may also work.\n\ntensorflow-transform                                                            | apache-beam[gcp] | pyarrow | tensorflow        | tensorflow-metadata | tfx-bsl |\n------------------------------------------------------------------------------- | -----------------| --------|-------------------|---------------------|---------|\n[GitHub master](https://github.com/tensorflow/transform/blob/master/RELEASE.md) | 2.60.0           | 10.0.1  | nightly (2.x)     | 1.16.1              | 1.16.1  |\n[1.16.0](https://github.com/tensorflow/transform/blob/v1.16.0/RELEASE.md)       | 2.60.0           | 10.0.1  | 2.16              | 1.16.1              | 1.16.1  |\n[1.15.0](https://github.com/tensorflow/transform/blob/v1.15.0/RELEASE.md)       | 2.47.0           | 10.0.0  | 2.15              | 1.15.0              | 1.15.1  |\n[1.14.0](https://github.com/tensorflow/transform/blob/v1.14.0/RELEASE.md)       | 2.47.0           | 10.0.0  | 2.13              | 1.14.0              | 1.14.0  |\n[1.13.0](https://github.com/tensorflow/transform/blob/v1.13.0/RELEASE.md)       | 2.41.0           | 6.0.0   | 2.12              | 1.13.1              | 1.13.0  |\n[1.12.0](https://github.com/tensorflow/transform/blob/v1.12.0/RELEASE.md)       | 2.41.0           | 6.0.0   | 2.11              | 1.12.0              | 1.12.0  |\n[1.11.0](https://github.com/tensorflow/transform/blob/v1.11.0/RELEASE.md)       | 2.41.0           | 6.0.0   | 1.15.5 / 2.10     | 1.11.0              | 1.11.0  |\n[1.10.0](https://github.com/tensorflow/transform/blob/v1.10.0/RELEASE.md)       | 2.40.0           | 6.0.0   | 1.15.5 / 2.9      | 1.10.0              | 1.10.0  |\n[1.9.0](https://github.com/tensorflow/transform/blob/v1.9.0/RELEASE.md)         | 2.38.0           | 5.0.0   | 1.15.5 / 2.9      | 1.9.0               | 1.9.0   |\n[1.8.0](https://github.com/tensorflow/transform/blob/v1.8.0/RELEASE.md)         | 2.38.0           | 5.0.0   | 1.15.5 / 2.8      | 1.8.0               | 1.8.0   |\n[1.7.0](https://github.com/tensorflow/transform/blob/v1.7.0/RELEASE.md)         | 2.36.0           | 5.0.0   | 1.15.5 / 2.8      | 1.7.0               | 1.7.0   |\n[1.6.1](https://github.com/tensorflow/transform/blob/v1.6.1/RELEASE.md)         | 2.35.0           | 5.0.0   | 1.15.5 / 2.8      | 1.6.0               | 1.6.0   |\n[1.6.0](https://github.com/tensorflow/transform/blob/v1.6.0/RELEASE.md)         | 2.35.0           | 5.0.0   | 1.15.5 / 2.7      | 1.6.0               | 1.6.0   |\n[1.5.0](https://github.com/tensorflow/transform/blob/v1.5.0/RELEASE.md)         | 2.34.0           | 5.0.0   | 1.15.2 / 2.7      | 1.5.0               | 1.5.0   |\n[1.4.1](https://github.com/tensorflow/transform/blob/v1.4.1/RELEASE.md)         | 2.33.0           | 4.0.1   | 1.15.2 / 2.6      | 1.4.0               | 1.4.0   |\n[1.4.0](https://github.com/tensorflow/transform/blob/v1.4.0/RELEASE.md)         | 2.33.0           | 4.0.1   | 1.15.2 / 2.6      | 1.4.0               | 1.4.0   |\n[1.3.0](https://github.com/tensorflow/transform/blob/v1.3.0/RELEASE.md)         | 2.31.0           | 2.0.0   | 1.15.2 / 2.6      | 1.2.0               | 1.3.0   |\n[1.2.0](https://github.com/tensorflow/transform/blob/v1.2.0/RELEASE.md)         | 2.31.0           | 2.0.0   | 1.15.2 / 2.5      | 1.2.0               | 1.2.0   |\n[1.1.1](https://github.com/tensorflow/transform/blob/v1.1.1/RELEASE.md)         | 2.29.0           | 2.0.0   | 1.15.2 / 2.5      | 1.1.0               | 1.1.1   |\n[1.1.0](https://github.com/tensorflow/transform/blob/v1.1.0/RELEASE.md)         | 2.29.0           | 2.0.0   | 1.15.2 / 2.5      | 1.1.0               | 1.1.0   |\n[1.0.0](https://github.com/tensorflow/transform/blob/v1.0.0/RELEASE.md)         | 2.29.0           | 2.0.0   | 1.15 / 2.5        | 1.0.0               | 1.0.0   |\n[0.30.0](https://github.com/tensorflow/transform/blob/v0.30.0/RELEASE.md)       | 2.28.0           | 2.0.0   | 1.15 / 2.4        | 0.30.0              | 0.30.0  |\n[0.29.0](https://github.com/tensorflow/transform/blob/v0.29.0/RELEASE.md)       | 2.28.0           | 2.0.0   | 1.15 / 2.4        | 0.29.0              | 0.29.0  |\n[0.28.0](https://github.com/tensorflow/transform/blob/v0.28.0/RELEASE.md)       | 2.28.0           | 2.0.0   | 1.15 / 2.4        | 0.28.0              | 0.28.1  |\n[0.27.0](https://github.com/tensorflow/transform/blob/v0.27.0/RELEASE.md)       | 2.27.0           | 2.0.0   | 1.15 / 2.4        | 0.27.0              | 0.27.0  |\n[0.26.0](https://github.com/tensorflow/transform/blob/v0.26.0/RELEASE.md)       | 2.25.0           | 0.17.0  | 1.15 / 2.3        | 0.26.0              | 0.26.0  |\n[0.25.0](https://github.com/tensorflow/transform/blob/v0.25.0/RELEASE.md)       | 2.25.0           | 0.17.0  | 1.15 / 2.3        | 0.25.0              | 0.25.0  |\n[0.24.1](https://github.com/tensorflow/transform/blob/v0.24.1/RELEASE.md)       | 2.24.0           | 0.17.0  | 1.15 / 2.3        | 0.24.0              | 0.24.1  |\n[0.24.0](https://github.com/tensorflow/transform/blob/v0.24.0/RELEASE.md)       | 2.23.0           | 0.17.0  | 1.15 / 2.3        | 0.24.0              | 0.24.0  |\n[0.23.0](https://github.com/tensorflow/transform/blob/v0.23.0/RELEASE.md)       | 2.23.0           | 0.17.0  | 1.15 / 2.3        | 0.23.0              | 0.23.0  |\n[0.22.0](https://github.com/tensorflow/transform/blob/v0.22.0/RELEASE.md)       | 2.20.0           | 0.16.0  | 1.15 / 2.2        | 0.22.0              | 0.22.0  |\n[0.21.2](https://github.com/tensorflow/transform/blob/v0.21.2/RELEASE.md)       | 2.17.0           | 0.15.0  | 1.15 / 2.1        | 0.21.0              | 0.21.3  |\n[0.21.0](https://github.com/tensorflow/transform/blob/v0.21.0/RELEASE.md)       | 2.17.0           | 0.15.0  | 1.15 / 2.1        | 0.21.0              | 0.21.0  |\n[0.15.0](https://github.com/tensorflow/transform/blob/v0.15.0/RELEASE.md)       | 2.16.0           | 0.14.0  | 1.15 / 2.0        | 0.15.0              | 0.15.0  |\n[0.14.0](https://github.com/tensorflow/transform/blob/v0.14.0/RELEASE.md)       | 2.14.0           | 0.14.0  | 1.14              | 0.14.0              | n/a     |\n[0.13.0](https://github.com/tensorflow/transform/blob/v0.13.0/RELEASE.md)       | 2.11.0           | n/a     | 1.13              | 0.12.1              | n/a     |\n[0.12.0](https://github.com/tensorflow/transform/blob/v0.12.0/RELEASE.md)       | 2.10.0           | n/a     | 1.12              | 0.12.0              | n/a     |\n[0.11.0](https://github.com/tensorflow/transform/blob/v0.11.0/RELEASE.md)       | 2.8.0            | n/a     | 1.11              | 0.9.0               | n/a     |\n[0.9.0](https://github.com/tensorflow/transform/blob/v0.9.0/RELEASE.md)         | 2.6.0            | n/a     | 1.9               | 0.9.0               | n/a     |\n[0.8.0](https://github.com/tensorflow/transform/blob/v0.8.0/RELEASE.md)         | 2.5.0            | n/a     | 1.8               | n/a                 | n/a     |\n[0.6.0](https://github.com/tensorflow/transform/blob/v0.6.0/RELEASE.md)         | 2.4.0            | n/a     | 1.6               | n/a                 | n/a     |\n[0.5.0](https://github.com/tensorflow/transform/blob/v0.5.0/RELEASE.md)         | 2.3.0            | n/a     | 1.5               | n/a                 | n/a     |\n[0.4.0](https://github.com/tensorflow/transform/blob/v0.4.0/RELEASE.md)         | 2.2.0            | n/a     | 1.4               | n/a                 | n/a     |\n[0.3.1](https://github.com/tensorflow/transform/blob/v0.3.1/RELEASE.md)         | 2.1.1            | n/a     | 1.3               | n/a                 | n/a     |\n[0.3.0](https://github.com/tensorflow/transform/blob/v0.3.0/RELEASE.md)         | 2.1.1            | n/a     | 1.3               | n/a                 | n/a     |\n[0.1.10](https://github.com/tensorflow/transform/blob/v0.1.10/RELEASE.md)       | 2.0.0            | n/a     | 1.0               | n/a                 | n/a     |\n\n## Questions\n\nPlease direct any questions about working with `tf.Transform` to\n[Stack Overflow](https://stackoverflow.com) using the\n[tensorflow-transform](https://stackoverflow.com/questions/tagged/tensorflow-transform)\ntag.\n","funding_links":[],"categories":["Deep Learning Framework","Tensorflow实用程序","Python","其他_机器学习与深度学习"],"sub_categories":["High-Level DL APIs"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftensorflow%2Ftransform","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftensorflow%2Ftransform","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftensorflow%2Ftransform/lists"}