{"id":15159301,"url":"https://github.com/sahil210695/tensorflow-model-deploy","last_synced_at":"2026-01-21T12:05:49.671Z","repository":{"id":195788108,"uuid":"168469617","full_name":"sahil210695/tensorflow-model-deploy","owner":"sahil210695","description":"Example how to deploy your TensorFlow model in production using the Tensorflow-serving","archived":false,"fork":false,"pushed_at":"2019-01-31T08:47:38.000Z","size":259,"stargazers_count":1,"open_issues_count":0,"forks_count":2,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-04-07T16:32:46.205Z","etag":null,"topics":["deployment","docker","docker-image","linear-regression","machine-learning","numpy","prediction","python","savedmodel","tensorflow","tensorflow-examples","tensorflow-experiments","tensorflow-graph","tensorflow-model","tensorflow-models","tensorflow-serving","tensorflow-tutorials"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sahil210695.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-01-31T05:42:30.000Z","updated_at":"2022-01-04T05:34:35.000Z","dependencies_parsed_at":null,"dependency_job_id":"c75d4d42-b01b-40f9-a936-b05207808862","html_url":"https://github.com/sahil210695/tensorflow-model-deploy","commit_stats":null,"previous_names":["sahil210695/tensorflow-model-deploy"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/sahil210695/tensorflow-model-deploy","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sahil210695%2Ftensorflow-model-deploy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sahil210695%2Ftensorflow-model-deploy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sahil210695%2Ftensorflow-model-deploy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sahil210695%2Ftensorflow-model-deploy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sahil210695","download_url":"https://codeload.github.com/sahil210695/tensorflow-model-deploy/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sahil210695%2Ftensorflow-model-deploy/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28632781,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-21T04:47:28.174Z","status":"ssl_error","status_checked_at":"2026-01-21T04:47:22.943Z","response_time":86,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deployment","docker","docker-image","linear-regression","machine-learning","numpy","prediction","python","savedmodel","tensorflow","tensorflow-examples","tensorflow-experiments","tensorflow-graph","tensorflow-model","tensorflow-models","tensorflow-serving","tensorflow-tutorials"],"created_at":"2024-09-26T21:03:23.155Z","updated_at":"2026-01-21T12:05:49.656Z","avatar_url":"https://github.com/sahil210695.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# TensorFlow Serving for model deployment in production\n\nWe have build a model and now we want that model to be accessible from web. There are many ways to do that but we are going to do that using **Tensorflow Serving**.\n\n### Steps\n1. Train a dumb model and save that model [Train](train.py)\n2. Restore saved model to check everything is working [Restore](restore.py)\n3. Export saved model to format that can be used by tensorflow-serving [Export](export.py)\n4. Load exported model to check everything is working [Load exported](load_exported.py)\n5. Deploy exported model in tensorflow-serving using docker image\n\n#### Train a dumb model and save that model\nWe will generate random datasaet and create a very basic model that will learn relationship \nbetween our randomly generated dataset\n\n_Relation between our dataset_\n![relation_dataset](images/X%20and%20y%20plot.png)\n\n_Plot for Loss history_\n![loss_history](images/epcoh%20loss%20plot.png)\n\nwhen training gets completed it will save model in [Save Directory](saved_model) using `tf.train.Saver()` and it will also save summary in [Summary](summary) to visualize the graph in tensorboard.\n\n_Tensorflow graph_\n![tensorflow_graph](images/tensorboard%20graph.png)\n\n#### Restore saved model to check everything is working\nHere will load the saved model and make prediction using that by some input\n\n#### Export saved model to format that can be used by tensorflow-serving\nHere comes the interesting part where will take the saved model and export it to [SavedModel](https://www.tensorflow.org/api_docs/python/tf/saved_model) format that tensorflow-serving will use to serve. To do that first we need load meta graph file and fetch tensors from graph which are required for predictions using their names then build tensor info from them that will be used to create signature definition that will be passed to the SavedModelBuilder instance. We can finally build model signature that identifies what serving is going to expect from the client.\n\nNow let's take a look at the export directory.\n```\n$ ls serve/linear/\n1/\n```\nA sub-directory will be created for exporting each version of the model.\n```\n$ ls serve/linear/1/\nsaved_model.pb  variables/\n```\nWe'll use the command line utility saved_model_cli to look at the MetaGraphDefs (the models) and SignatureDefs (the methods you can call) in our SavedModel.\n```\n$ saved_model_cli show --dir serve/linear/1/ --all\n\nMetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:\n\nsignature_def['serving_default']:\n  The given SavedModel SignatureDef contains the following input(s):\n    inputs['x'] tensor_info:\n        dtype: DT_FLOAT\n        shape: (1)\n        name: data_pipeline/IteratorGetNext:0\n  The given SavedModel SignatureDef contains the following output(s):\n    outputs['y'] tensor_info:\n        dtype: DT_FLOAT\n        shape: (1)\n        name: prediction/add:0\n  Method name is: tensorflow/serving/predict\n```\n\n#### Load exported model to check everything is working\nHere will load the exported **SavedModel** and make prediction using that by some random input\n\n#### Deploy exported model in tensorflow-serving using docker image\nOne of the easiest ways to get started using TensorFlow Serving is with Docker.\n\nGeneral installation instructions are on the Docker site, but we give some quick links here:\n- [Docker for macOS](https://docs.docker.com/docker-for-mac/install/)\n- [Docker for Windows](https://docs.docker.com/docker-for-windows/install/) for Windows 10 Pro or later\n- [Docker Toolbox](https://docs.docker.com/toolbox/) for much older versions of macOS, or versions of Windows before Windows 10 Pro\n\nLets serve our model\n1. you can pull the latest TensorFlow Serving docker image by running \u003cbr\u003e\n  `docker pull tensorflow/serving`\n\n2. Run docker image that will serve your model \u003cbr\u003e\n  `sudo docker run --name tf_serving -p 8501:8501 --mount type=bind,source=$(pwd)/linear/,target=/models/linear -e MODEL_NAME=linear -t tensorflow/serving`\n\n  _if all goes well you'll see logs like_ 😎\n  ```\n  2019-01-31 07:55:30.133911: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:259] SavedModel load for tags { serve }; Status: success. Took 58911 microseconds.\n  2019-01-31 07:55:30.133937: I tensorflow_serving/servables/tensorflow/saved_model_warmup.cc:83] No warmup data file found at /models/linear/1/assets.extra/tf_serving_warmup_requests\n  2019-01-31 07:55:30.134095: I tensorflow_serving/core/loader_harness.cc:86] Successfully loaded servable version {name: linear version: 1}\n  2019-01-31 07:55:30.137262: I tensorflow_serving/model_servers/server.cc:286] Running gRPC ModelServer at 0.0.0.0:8500 ...\n  [warn] getaddrinfo: address family for nodename not supported\n  2019-01-31 07:55:30.143359: I tensorflow_serving/model_servers/server.cc:302] Exporting HTTP/REST API at:localhost:8501 ...\n  ```\n\n3. In addition to gRPC APIs TensorFlow ModelServer also supports RESTful APIs.\n- Model status API: It returns the status of a model in the ModelServer \u003cbr\u003e\n  `GET http://host:port/v1/models/${MODEL_NAME}[/versions/${MODEL_VERSION}]`\n  \n  Go to your browser type `http://localhost:8501/v1/models/linear/versions/1` and you'll get status \u003cbr\u003e\n  ```\n  {\n     \"model_version_status\":[\n        {\n           \"version\":\"1\",\n           \"state\":\"AVAILABLE\",\n           \"status\":{\n              \"error_code\":\"OK\",\n              \"error_message\":\"\"\n           }\n        }\n     ]\n  }\n  ```\n  \n- Model Metadata API: It returns the metadata of a model in the ModelServer \u003cbr\u003e\n  `GET http://host:port/v1/models/${MODEL_NAME}[/versions/${MODEL_VERSION}]/metadata`\n  \n  Go to your browser type `http://localhost:8501/v1/models/linear/versions/1/metadata` and you'll get metadata \u003cbr\u003e\n  ```\n  {\n     \"model_spec\":{\n        \"name\":\"linear\",\n        \"signature_name\":\"\",\n        \"version\":\"1\"\n     },\n     \"metadata\":{\n        \"signature_def\":{\n           \"signature_def\":{\n              \"serving_default\":{\n                 \"inputs\":{\n                    \"x\":{\n                       \"dtype\":\"DT_FLOAT\",\n                       \"tensor_shape\":{\n                          \"dim\":[\n                             {\n                                \"size\":\"1\",\n                                \"name\":\"\"\n                             }\n                          ],\n                          \"unknown_rank\":false\n                       },\n                       \"name\":\"data_pipeline/IteratorGetNext:0\"\n                    }\n                 },\n                 \"outputs\":{\n                    \"y\":{\n                       \"dtype\":\"DT_FLOAT\",\n                       \"tensor_shape\":{\n                          \"dim\":[\n                             {\n                                \"size\":\"1\",\n                                \"name\":\"\"\n                             }\n                          ],\n                          \"unknown_rank\":false\n                       },\n                       \"name\":\"prediction/add:0\"\n                    }\n                 },\n                 \"method_name\":\"tensorflow/serving/predict\"\n              }\n           }\n        }\n     }\n  }\n  ```\n  \n- Predict API: It returns the predictions from you model based on the inputs that you'll send \u003cbr\u003e\n  `POST http://host:port/v1/models/${MODEL_NAME}[/versions/${MODEL_VERSION}]:predict`\n  \n  Let's make a curl request and check we are getting predictions or not \u003cbr\u003e\n  `curl -d '{\"signature_name\":\"serving_default\", \"instances\":[10, 20, 30]}' -X POST http://localhost:8501/v1/models/linear:predict` \u003cbr\u003e \u003cbr\u003e\n  **output**\n  ```\n  {\n     \"predictions\":[\n        38.4256,\n        68.9495,\n        99.4733\n     ]\n  }\n  ```\n\nNow you can use anything to make **POST** request to the model server and even use Flask, Django or any other framework of you choice to integrate this **REST API** and make application on top it 😎👍✌️\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsahil210695%2Ftensorflow-model-deploy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsahil210695%2Ftensorflow-model-deploy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsahil210695%2Ftensorflow-model-deploy/lists"}