{"id":13487102,"url":"https://github.com/riga/tfdeploy","last_synced_at":"2025-04-04T10:08:48.674Z","repository":{"id":57474802,"uuid":"53326300","full_name":"riga/tfdeploy","owner":"riga","description":"Deploy tensorflow graphs for fast evaluation and export to tensorflow-less environments running numpy.","archived":false,"fork":false,"pushed_at":"2024-02-25T19:50:49.000Z","size":311,"stargazers_count":352,"open_issues_count":12,"forks_count":38,"subscribers_count":22,"default_branch":"master","last_synced_at":"2024-06-05T11:53:57.962Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"http://tfdeploy.readthedocs.io","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/riga.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-03-07T13:08:21.000Z","updated_at":"2024-06-21T16:50:46.360Z","dependencies_parsed_at":"2024-06-21T16:50:44.315Z","dependency_job_id":"19674221-cfec-4822-954b-9d5909fc40db","html_url":"https://github.com/riga/tfdeploy","commit_stats":{"total_commits":166,"total_committers":6,"mean_commits":"27.666666666666668","dds":0.0662650602409639,"last_synced_commit":"22aea652fe12f081be43414e0f1f76c7d9aaf53c"},"previous_names":[],"tags_count":17,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/riga%2Ftfdeploy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/riga%2Ftfdeploy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/riga%2Ftfdeploy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/riga%2Ftfdeploy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/riga","download_url":"https://codeload.github.com/riga/tfdeploy/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247157283,"owners_count":20893220,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T18:00:55.392Z","updated_at":"2025-04-04T10:08:48.656Z","avatar_url":"https://github.com/riga.png","language":"Python","readme":"\u003cimg src=\"https://raw.githubusercontent.com/riga/tfdeploy/master/logo.png\" alt=\"tfdeploy logo\" width=\"250\"/\u003e\n\n[![Build Status](https://travis-ci.org/riga/tfdeploy.svg?branch=master)](https://travis-ci.org/riga/tfdeploy) [![Documentation Status](https://readthedocs.org/projects/tfdeploy/badge/?version=latest)](http://tfdeploy.readthedocs.org/en/latest/?badge=latest) [![Package Status](https://badge.fury.io/py/tfdeploy.svg)](https://badge.fury.io/py/tfdeploy)\n\nDeploy [tensorflow](https://www.tensorflow.org) graphs for *fast* evaluation and export to *tensorflow-less* environments running [NumPy](http://www.numpy.org).\n\n\u003e [!NOTE]  \n\u003e This project started as a personal playground to get an in-depth understanding of TensorFlow's operations and kernels.\n\u003e Up to a certain version, the NumPy based operations in tfdeploy provided full feature parity, but it is obvious that such a project cannot keep up with the vast development speed driven by TensorFlow devs and the open-source community.\n\u003e\n\u003e Therefore, tfdeploy is **no longer actively maintained**.\n\u003e However, the code base remains active as an easy-to-read reference implementation for most of the kernels that constitute the heart of todays ML landscape.\n\n\n##### Evaluation usage\n\n```python\nimport tfdeploy as td\nimport numpy as np\n\nmodel = td.Model(\"/path/to/model.pkl\")\ninp, outp = model.get(\"input\", \"output\")\n\nbatch = np.random.rand(10000, 784)\nresult = outp.eval({inp: batch})\n```\n\n\n##### Installation and dependencies\n\nVia [pip](https://pypi.python.org/pypi/tfdeploy)\n\n```bash\npip install tfdeploy\n```\n\nor by simply copying the file into your project.\n\nNumPy ≥ 1.10 should be installed on your system. [SciPy](http://www.scipy.org/) is optional. See [optimization](#optimization) for more info on optional packages.\n\nBy design, TensorFlow is required when creating a model.\n\n\n### Content\n\n- [Why?](#why)\n- [How?](#how)\n    - [Convert your graph](#convert-your-graph)\n    - [Load the model and evaluate](#load-the-model-and-evaluate)\n    - [Write your own operation](#write-your-own-operation)\n- [Ensembles](#ensembles)\n- [Optimization](#optimization)\n- [Performance](#performance)\n- [Contributing](#contributing)\n- [Development](#development)\n- [Authors](#authors)\n- [License](#license)\n\n## Why?\n\nWorking with TensorFlow is awesome. Model definition and training is simple yet powerful, and the range of built-in features is just striking.\n\nModel deployment in environments that are not able to run TensorFlow, however, things can be difficult (**note** that tfdeploy was developed before TensorFlow Lite was a thing).\n\nTo boil it down, tfdeploy\n\n- is lightweight. A single file with \u003c 150 lines of core code. Just copy it to your project.\n- [faster](#performance) than using TensorFlow's `Tensor.eval`.\n- **does not need TensorFlow** during evaluation.\n- only depends on NumPy.\n- can load one or more models from a single file.\n- does not support GPUs (maybe [gnumpy](http://www.cs.toronto.edu/~tijmen/gnumpy.html) is worth a try here).\n\n\n## How?\n\nThe central class is `tfdeploy.Model`. The following two examples demonstrate how a model can be created from a TensorFlow graph, saved to and loaded from disk, and eventually evaluated.\n\n##### Convert your graph\n\n```python\nimport tensorflow as tf\nimport tfdeploy as td\n\n# setup tfdeploy (only when creating models)\ntd.setup(tf)\n\n# build your graph\nsess = tf.Session()\n\n# use names for input and output layers\nx = tf.placeholder(\"float\", shape=[None, 784], name=\"input\")\nW = tf.Variable(tf.truncated_normal([784, 100], stddev=0.05))\nb = tf.Variable(tf.zeros([100]))\ny = tf.nn.softmax(tf.matmul(x, W) + b, name=\"output\")\n\nsess.run(tf.global_variables_initializer())\n\n# ... training ...\n\n# create a tfdeploy model and save it to disk\nmodel = td.Model()\nmodel.add(y, sess) # y and all its ops and related tensors are added recursively\nmodel.save(\"model.pkl\")\n```\n\n##### Load the model and evaluate\n\n```python\nimport numpy as np\nimport tfdeploy as td\n\nmodel = td.Model(\"model.pkl\")\n\n# shorthand to x and y\nx, y = model.get(\"input\", \"output\")\n\n# evaluate\nbatch = np.random.rand(10000, 784)\nresult = y.eval({x: batch})\n```\n\n##### Write your own `Operation`\n\ntfdeploy supports most of the `Operation`'s [implemented in tensorflow](https://www.tensorflow.org/versions/master/api_docs/python/math_ops.html). However, if you miss one (in that case, submit a PR or an issue ;) ) or if you're using custom ops, you might want to extend tfdeploy by defining a new class op that inherits from `tfdeploy.Operation`:\n\n```python\nimport tensorflow as tf\nimport tfdeploy as td\nimport numpy as np\n\n# setup tfdeploy (only when creating models)\ntd.setup(tf)\n\n# ... write you model here ...\n\n# let's assume your final tensor \"y\" relies on an op of type \"InvertedSoftmax\"\n# before creating the td.Model, you should add that op to tfdeploy\n\nclass InvertedSoftmax(td.Operation):\n    @staticmethod\n    def func(a):\n        e = np.exp(-a)\n        # ops should return a tuple\n        return np.divide(e, np.sum(e, axis=-1, keepdims=True)),\n\n# this is equivalent to\n# @td.Operation.factory\n# def InvertedSoftmax(a):\n#     e = np.exp(-a)\n#     return np.divide(e, np.sum(e, axis=-1, keepdims=True)),\n\n# now we're good to go\nmodel = td.Model()\nmodel.add(y, sess)\nmodel.save(\"model.pkl\")\n```\n\nWhen writing new ops, three things are important:\n\n- Try to avoid loops, prefer NumPy vectorization.\n- Return a tuple.\n- Don't change incoming tensors/arrays in-place, always work on and return copies.\n\n\n## Ensembles\n\ntfdeploy provides a helper class to evaluate an ensemble of models: `Ensemble`. It can load multiple models, evaluate them and combine their output values using different methods.\n\n```python\n# create the ensemble\nensemble = td.Ensemble([\"model1.pkl\", \"model2.pkl\", ...], method=td.METHOD_MEAN)\n\n# get input and output tensors (which actually are TensorEnsemble instances)\ninput, output = ensemble.get(\"input\", \"output\")\n\n# evaluate the ensemble just like a normal model\nbatch = ...\nvalue = output.eval({input: batch})\n```\n\nThe return value of `get()` is a `TensorEnsemble` istance. It is basically a wrapper around multiple tensors and should be used as keys in the `feed_dict` of the `eval()` call.\n\nYou can choose between `METHOD_MEAN` (the default), `METHOD_MAX` and `METHOD_MIN`. If you want to use a custom ensembling method, use `METHOD_CUSTOM` and overwrite the static `func_custom()` method of the `TensorEnsemble` instance.\n\n\n## Optimization\n\nMost ops are written using pure numpy. However, multiple implementations of the same op are allowed that may use additional third-party Python packages providing even faster functionality for some situations.\n\nFor example, NumPy does not provide a vectorized *lgamma* function. Thus, the standard `tfdeploy.Lgamma` op uses `math.lgamma` that was previously vectorized using `numpy.vectorize`. For these situations, additional implementations of the same op are possible (the *lgamma* example is quite academic, but this definitely makes sense for more sophisticated ops like pooling). We can simply tell the op to use its SciPy implementation instead:\n\n```python\ntd.Lgamma.use_impl(td.IMPL_SCIPY)\n```\n\nCurrently, allowed implementation types are NumPy (`IMPL_NUMPY`, the default) and SciPy (`IMPL_SCIPY`).\n\n\n##### Adding additional implementations\n\nAdditional implementations can be added by setting the `impl` attribute of the op factory or by using the `add_impl` decorator of existing operations. The first registered implementation will be the default one.\n\n```python\n# create the default lgamma op with numpy implementation\nlgamma_vec = np.vectorize(math.lgamma)\n\n@td.Operation.factory\n# equivalent to\n# @td.Operation.factory(impl=td.IMPL_NUMPY)\ndef Lgamma(a):\n    return lgamma_vec(a),\n\n# add a scipy-based implementation\n@Lgamma.add_impl(td.IMPL_SCIPY)\ndef Lgamma(a):\n    return sp.special.gammaln(a),\n```\n\n\n##### Auto-optimization\n\nIf SciPy is available on your system, it is reasonable to use all ops in their SciPy implementation (if it exists, of course). This should be configured before you create any model from TensorFlow objects using the second argument of the `setup` function:\n\n```python\ntd.setup(tf, td.IMPL_SCIPY)\n```\n\nOps that do not implement `IMPL_SCIPY` stick with the NumPy version (`IMPL_NUMPY`).\n\n\n## Performance\n\ntfdeploy is lightweight (1 file, \u003c 150 lines of core code) and fast. Internal evaluation calls have only very few overhead and tensor operations use NumPy vectorization. The actual performance depends on the ops in your graph. While most of the TensorFlow ops have a numpy equivalent or can be constructed from NumPy functions, a few ops require additional Python-based loops (e.g. `BatchMatMul`). But in many cases (and for small to medium graphs) it's potentially faster than using TensorFlow's `Tensor.eval`.\n\nThis is a comparison for a basic graph where all ops are vectorized (basically `Add`, `MatMul` and `Softmax`):\n\n```bash\n\u003e ipython -i tests/perf/simple.py\n\nIn [1]: %timeit -n 100 test_tf()\n100 loops, best of 3: 109 ms per loop\n\nIn [2]: %timeit -n 100 test_td()\n100 loops, best of 3: 60.5 ms per loop\n```\n\n## Contributing\n\nIf you want to contribute with new ops and features, I'm happy to receive pull requests. Just make sure to add a new test case to `tests/core.py` or `tests/ops.py` and run them via:\n\n```bash\n\u003e python -m unittest tests\n```\n\n\n##### Test grid\n\nIn general, tests should be run for different environments:\n\n|     Variation      |  Values |\n| ------------------ | ------- |\n| tensorflow version | `1.0.1` |\n| python version     | 2, 3    |\n| `TD_TEST_SCIPY`    | 0, 1    |\n| `TD_TEST_GPU`      | 0, 1    |\n\n\n##### Docker\n\nFor testing purposes, it is convenient to use docker. Fortunately, the official [tensorflow images](https://hub.docker.com/r/tensorflow/tensorflow/) contain all we need:\n\n```bash\ngit clone https://github.com/riga/tfdeploy.git\ncd tfdeploy\n\ndocker run --rm -v `pwd`:/root/tfdeploy -w /root/tfdeploy -e \"TD_TEST_SCIPY=1\" tensorflow/tensorflow:1.0.1 python -m unittest tests\n```\n\n\n## Development\n\n- Source hosted at [GitHub](https://github.com/riga/tfdeploy)\n- Report issues, questions, feature requests on [GitHub Issues](https://github.com/riga/tfdeploy/issues)\n\n\n## Authors\n\n- [Marcel R.](https://github.com/riga)\n","funding_links":[],"categories":["The Data Science Toolbox","Python","Deep Learning","模型序列化和转换","其他_机器学习与深度学习","Tensor Flow"],"sub_categories":["Deep Learning Packages","TensorFlow","Automated Machine Learning"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Friga%2Ftfdeploy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Friga%2Ftfdeploy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Friga%2Ftfdeploy/lists"}