Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/vkuznet/tfaas
TensorFlow as a Service, a general purpose framework to serve TF models.
https://github.com/vkuznet/tfaas
deeplearning go inference machine-learning prediction python tensorflow tensorflow-models tf-model tfaas-server
Last synced: 8 days ago
JSON representation
TensorFlow as a Service, a general purpose framework to serve TF models.
- Host: GitHub
- URL: https://github.com/vkuznet/tfaas
- Owner: vkuznet
- License: mit
- Created: 2017-11-01T14:30:29.000Z (about 7 years ago)
- Default Branch: master
- Last Pushed: 2024-03-13T22:10:28.000Z (8 months ago)
- Last Synced: 2024-11-12T17:03:23.455Z (8 days ago)
- Topics: deeplearning, go, inference, machine-learning, prediction, python, tensorflow, tensorflow-models, tf-model, tfaas-server
- Language: JavaScript
- Homepage:
- Size: 11.5 MB
- Stars: 54
- Watchers: 10
- Forks: 6
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- Funding: .github/FUNDING.yml
- License: LICENSE
Awesome Lists containing this project
README
### TensorFlow as a Service (TFaaS)
[](https://travis-ci.org/vkuznet/TFaaS)
[](https://goreportcard.com/report/github.com/vkuznet/TFaaS)
[](https://github.com/vkuznet/LICENSE)
[](https://zenodo.org/badge/latestdoi/109141847)
[](https://twitter.com/intent/tweet?text=TensorFlow%20as%20a%20service%20&url=https://github.com/vkuznet/TFaaS&hashtags=tensorflow,go,python)
A general purpose framework (written in Go) to serve TensorFlow models.
It provides reach and flexible set of APIs to efficiently access your
favorite TF models via HTTP interface. The TFaaS supports JSON and ProtoBuffer
data-formats.
The following set of APIs is provided:
- `/upload` to push your favorite TF model to TFaaS server either for Form or
as tar-ball bundle, see examples below
- `/delete` to delete your TF model from TFaaS server
- `/models` to view existing TF models on TFaaS server
- `/predict/json` to serve TF model predictions in JSON data-format
- `/predict/proto` to serve TF model predictions in ProtoBuffer data-format
- `/predict/image` to serve TF model predictions forimages in JPG/PNG formats
### From deployment to production
#### ➀ install docker image (TFaaS port is 8083)
```
docker run --rm -h `hostname -f` -p 8083:8083 -i -t veknet/tfaas
```
#### ➁ upload your TF model to TFaaS server
```
# example of image based model upload
curl -X POST http://localhost:8083/upload
-F 'name=ImageModel' -F 'params=@/path/params.json'
-F 'model=@/path/tf_model.pb' -F 'labels=@/path/labels.txt'
# example of TF pb file upload
curl -s -X POST http:/localhost:8083/upload \
-F 'name=vk' -F 'params=@/path/params.json' \
-F 'model=@/path/model.pb' -F 'labels=@/path/labels.txt'
# example of bundle upload produce with Keras TF
# here is our saved model area
ls model
assets saved_model.pb variables
# we can create tarball and upload it to TFaaS via bundle end-point
tar cfz model.tar.gz model
curl -X POST -H "Content-Encoding: gzip" \
-H "content-type: application/octet-stream" \
--data-binary @/path/models.tar.gz http://localhost:8083/upload
```
#### ➂ get your predictions
```
# obtain predictions from your ImageModel
curl https://localhost:8083/image -F 'image=@/path/file.png' -F 'model=ImageModel'
# obtain predictions from your TF based model
cat input.json
{"keys": [...], "values": [...], "model":"model"}
# call to get predictions from /json end-point using input.json
curl -s -X POST -H "Content-type: application/json" \
-d@/path/input.json http://localhost:8083/json
```
Fore more information please visit [curl client](https://github.com/vkuznet/TFaaS/blob/master/doc/curl_client.md) page.
### TFaaS interface
Clients communicate with TFaaS via HTTP protocol. See examples for
[Curl](https://github.com/vkuznet/TFaaS/blob/master/doc/curl_client.md),
[Python](https://github.com/vkuznet/TFaaS/blob/master/doc/python_client.md)
and
[C++](https://github.com/vkuznet/TFaaS/blob/master/doc/cpp_client.md)
clients.
### TFaaS benchmarks
Benchmark results on CentOS, 24 cores, 32GB of RAM serving DL NN with
42x128x128x128x64x64x1x1 architecture (JSON and ProtoBuffer formats show similar performance):
- 400 req/sec for 100 concurrent clients, 1000 requests in total
- 480 req/sec for 200 concurrent clients, 5000 requests in total
For more information please visit
[bencmarks](https://github.com/vkuznet/TFaaS/blob/master/doc/Benchmarks.md)
page.
### More information
- [Install instructions](https://github.com/vkuznet/TFaaS/blob/master/doc/INSTALL.md) to build TFaaS from source code
- [End-to-end example for MNIST dataset](https://github.com/vkuznet/TFaaS/blob/master/doc/MNIST.md)
- [End-to-end example of serving TF model in Go-server](https://github.com/vkuznet/TFaaS/blob/master/doc/workflow.md)
- [Demo](https://github.com/vkuznet/TFaaS/blob/master/doc/DEMO.md)