{"id":15159407,"url":"https://github.com/yu-iskw/tensorflow-serving-example","last_synced_at":"2025-09-30T09:30:39.342Z","repository":{"id":145587351,"uuid":"129815045","full_name":"yu-iskw/tensorflow-serving-example","owner":"yu-iskw","description":"Examples to server tensorflow models with tensorflow serving","archived":true,"fork":false,"pushed_at":"2019-01-29T18:04:01.000Z","size":32,"stargazers_count":98,"open_issues_count":3,"forks_count":47,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-01-20T01:34:21.228Z","etag":null,"topics":["machine-learning","tensorflow","tensorflow-estimators","tensorflow-examples","tensorflow-serving"],"latest_commit_sha":null,"homepage":"https://medium.com/@yuu.ishikawa/serving-pre-modeled-and-custom-tensorflow-estimator-with-tensorflow-serving-12833b4be421","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/yu-iskw.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-04-16T22:47:40.000Z","updated_at":"2025-01-15T01:13:48.000Z","dependencies_parsed_at":null,"dependency_job_id":"9c84b42c-5be0-43cb-a798-40581fa79d8c","html_url":"https://github.com/yu-iskw/tensorflow-serving-example","commit_stats":{"total_commits":11,"total_committers":3,"mean_commits":"3.6666666666666665","dds":"0.18181818181818177","last_synced_commit":"68dc20c9626e479a1c925433477a7dbe85b8162f"},"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/yu-iskw/tensorflow-serving-example","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yu-iskw%2Ftensorflow-serving-example","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yu-iskw%2Ftensorflow-serving-example/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yu-iskw%2Ftensorflow-serving-example/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yu-iskw%2Ftensorflow-serving-example/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/yu-iskw","download_url":"https://codeload.github.com/yu-iskw/tensorflow-serving-example/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yu-iskw%2Ftensorflow-serving-example/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":277665420,"owners_count":25856477,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-30T02:00:09.208Z","response_time":75,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["machine-learning","tensorflow","tensorflow-estimators","tensorflow-examples","tensorflow-serving"],"created_at":"2024-09-26T21:21:19.851Z","updated_at":"2025-09-30T09:30:39.334Z","avatar_url":"https://github.com/yu-iskw.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Serving Tensorflow Estimator with Tensorflow Serving Examples\n\nIn this repository, I would like to show examples to server tensorflow estimator with tensorflow serving.\nThe goals of this repository are to understand:\n1. how to create tensorflow estimators to serve with tensorflow serving,\n2. how to server a model with tensorflow, and\n3. how to implement a gRPC client for tensorflow serving.\n\nAt the time when I am writing this, there are less documentation about serving custom tensorflow estimator.\nThat's why I made this for myself.\nI hope it would help someone with learning something similar.\n\n## Requirements\n- Python 2.7\n- Docker\n- Anaconda\n\n## How to run the examples\n\n### 1. You clone the repository.\n```\ngit clone git@github.com:yu-iskw/tensorflow-serving-example.git\n```\n\n### 2. You create an anaconda environment.\nAt the moment when I am creating this, versions of main components are the following.\n- tensorflow: 1.12.0\n- tensorflow-serving-api: 1.12.0\n- tensorflow-serving: 1.12.0\n```\nconda env create -f environment.yml\n```\n\nOnce you create the environment, you need to activate this using the following command:\n\n```\nsource activate tensorflow-serving-example\n```\n\n### 3. You train a model.\nYou train a model with `python/train/mnist_custom_estimator.py`.\nIt is used for training a model to the MNIST task.\n```\npython python/train/mnist_custom_estimator.py \\\n    --steps 100 \\\n    --saved_dir ./models/ \\\n    --model_dir /tmp/mnist_custom_estimator\n```\n\nWhere `--steps` is the number of steps to train a model, `--saved_dir` is a path to save the model for tensorflow serving  and `--model_dir` is a path to save traditional checkpoints, meta graph and so on.\n\nWhen you finish the trained model, the saved model exists under `./models` directory like below.\nHere, `1548714304` is the unix timestamp when the model was saved.\nThis is different from what you get.\nIt depends on when you run the python code.\n\nWe can server `saved_model.pb` with tensorflow serving.\n```\n./models\n└── 1548714304\n    ├── saved_model.pb\n        └── variables\n                ├── variables.data-00000-of-00001\n                        └── variables.index\n```\n\n### 4. You build a docker image for tensorflow serving.\nAs you can see `./Dockerfile`, it just installs the pre-built tensorflow serving package.\nThis is because building it takes a little long time to compile tensorflow serving.\nIf you want to make a docker image from the tensorflow serving source, [those docker files](https://github.com/tensorflow/serving/tree/master/tensorflow_serving/tools/docker) would help.\n```\ndocker build --rm -f Dockerfile -t tensorflow-serving-example:0.6 .\n```\n\nAt the time when I am writing this, there is something wrong with `tensorflow-serving-universal`.\nIf you are interested in the issue, please track [Package recently broken on ubuntu 16\\.04 ? · Issue \\#819 · tensorflow/serving](https://github.com/tensorflow/serving/issues/819).\n\n\n### 5. You run the docker container with the trained model.\nBefore running a docker container, you must prepare for the served model.\n```\n# Prepare for the served model.\nmkdir -p ./models_for_serving/mnist/1\ncp -R ./models/mnist_custom_estimator/pb/1548714304/* ./models_for_serving/mnist/1\n\n# As a result of copying the files, the directory should be like following.\nmodels_for_serving/\n└── mnist\n    └── 1\n        ├── saved_model.pb\n        └── variables\n            ├── variables.data-00000-of-00001\n            └── variables.index\n```\nAs you probably know, tenwoflow can handle multiple versions of served models.\n`1` at the tail of `./models_for_serving/mnist/1/` means the served model version.\n\nWe have prepared for the served model.\nNow let's move on to running a docker container to server the model.\nThe model serving supports not only gRPC API, but HTTP/REST API.\nThe port 8500 is used for gRPC.\nMeanwhile, the port 8501 is used for RESTful API.\n```\n# Run a docker container.\ndocker run --rm -it -v /Users/yuishikawa/local/src/github/tensorflow-serving-example/models_for_serving:/models \\\n    -e MODEL_NAME=mnist \\\n    -e MODEL_PATH=/models/mnist \\\n    -p 8500:8500  \\\n    -p 8501:8501  \\\n    --name tensorflow-serving-example \\\n    tensorflow-serving-example:0.6\n```\n\nWhere `MODEL_NAME` is an environment variable to identify the served model when a gRPC client requests,\n`MODEL_PATH` is an environment variable to identify the path to the saved model.\nBesides, we share `./models_for_serving` between docker host and guest.\n\n### 6. You create a gRPC client for tensorflow serving.\n`./python/grpc_mnist_client.py` is an example of gRPC client for tensorflow serving.\n```\nTENSORFLOW_SERVER_HOST=\"...\"\npython python/grpc_mnist_client.py \\\n  --image ./data/0.png \\\n  --model mnist \\\n  --host $TENSORFLOW_SERVER_HOST\n```\nYou can pass one with `--image` option.\nI already prepared for some sample MNIST images in `./data/`.\nIf you use `docker-machine`, you can get the host with `docker-machine ip`.\n`--model mnist` is defined, when running a docker container with `-e MODEL_NAME='mnist'`.\n\n## Appendix: Serving premodeled tensorflow estimator\n`./python/train/mnist_premodeled_estimator.py` is an example to save a trained model with a premodeled tensorflow estimator.\nOne of the differences from a custom tensorflow estimator is the model spec signature name.\nWhen saving a model with a custom tensorflow estimator, the signature name is `serving_default` by default.\nOn the other hand, when saving a model with a pre-modeled tensorflow estimator, the signature name is `predict` by default.\n```\nTENSORFLOW_SERVER_HOST=\"...\"\npython python/grpc_mnist_client.py \\\n  --image ./data/0.png \\\n  --model mnist \\\n  --host $TENSORFLOW_SERVER_HOST \\\n  --signature_name predict\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyu-iskw%2Ftensorflow-serving-example","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyu-iskw%2Ftensorflow-serving-example","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyu-iskw%2Ftensorflow-serving-example/lists"}