https://github.com/chagmgang/how_to_deploy_tensorflow
https://github.com/chagmgang/how_to_deploy_tensorflow
docker flask seq2seq tensorflow tensorflow-serving tf-serving
Last synced: 23 days ago
JSON representation
- Host: GitHub
- URL: https://github.com/chagmgang/how_to_deploy_tensorflow
- Owner: chagmgang
- Created: 2018-09-23T20:49:18.000Z (over 7 years ago)
- Default Branch: master
- Last Pushed: 2018-10-07T04:48:11.000Z (over 7 years ago)
- Last Synced: 2025-04-22T12:37:40.663Z (about 1 year ago)
- Topics: docker, flask, seq2seq, tensorflow, tensorflow-serving, tf-serving
- Language: Python
- Size: 86.7 MB
- Stars: 4
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# how_to_deploy_tensorflow
## Make toy tensorflow model
```
python toy_model.py
```
## Fetch the variables you need from your stored model(.ckpt)
```
python serve.py
```
## Download a Docker image with TensorFlow serving already compile on it
```
docker run -it -p : — name -v : epigramai/model-server:light-universal-1.7 — port= — model_name= — model_base_path=
```
for example
```
docker run -it -p 9000:9000 --name tf-serve -v $(pwd)/serve/:/serve/ epigramai/model-server:light-universal-1.7 --port=9000 --model_name=test --model_base_path=/serve/test
```
or
```
docker run -it -p 9000:9000 --name tf-serve -v $(pwd)/serve/:/serve/ chagmgang/tensorflow_deploy:1.7 --port=9000 --model_name=test --model_base_path=/serve/test
```
## Create a client in order to send the gRPC request
Install library python for make client and end-point
```
pip install git+https://github.com/chagmgang/tfserving-python-predict-client.git
```
or
```
pip install git+https://github.com/epigramai/tfserving-python-predict-client.git
```
## Test the response of docker
```
python client.py
```
## Make a API endpoint via Flask
```
import numpy as np
from predict_client.prod_client import ProdClient
from flask import Flask
from flask import request
from flask import jsonify
HOST = 'localhost:9000'
MODEL_NAME = 'test'
MODEL_VERSION = 1
app = Flask(__name__)
client = ProdClient(HOST, MODEL_NAME, MODEL_VERSION)
def convert_data(raw_data):
return np.array(raw_data, dtype=np.float32)
def get_prediction_from_model(data):
req_data = [{'in_tensor_name': 'inputs', 'in_tensor_dtype': 'DT_FLOAT', 'data': data}]
prediction = client.predict(req_data, request_timeout=10)
return prediction
@app.route("/prediction", methods=['POST'])
def get_prediction():
req_data = request.get_json()
raw_data = req_data['data']
data = convert_data(raw_data)
prediction = get_prediction_from_model(data)
# ndarray cannot be converted to JSON
return jsonify({ 'predictions': prediction['outputs'].tolist() })
if __name__ == '__main__':
app.run(host='localhost',port=3000)
```
or
```
python api_endpoint.py
```
## test via postman
## without docker(only can be done in ubuntu)
```
echo "deb [arch=amd64] http://storage.googleapis.com/tensorflow-serving-apt stable tensorflow-model-server tensorflow-model-server-universal" | sudo tee /etc/apt/sources.list.d/tensorflow-serving.list
curl https://storage.googleapis.com/tensorflow-serving-apt/tensorflow-serving.release.pub.gpg | sudo apt-key add -
```
```
sudo apt-get update && sudo apt-get install tensorflow-model-server
```
```
tensorflow_model_server --port=port_number --model_name=model_name --model_base_path=absolute/path/to/model
```