{"id":14483177,"url":"https://github.com/fpaupier/tensorflow-serving_sidecar","last_synced_at":"2025-08-30T03:33:38.458Z","repository":{"id":53430998,"uuid":"167009011","full_name":"fpaupier/tensorflow-serving_sidecar","owner":"fpaupier","description":"Serve machine learning models using tensorflow serving ","archived":true,"fork":false,"pushed_at":"2021-03-30T17:44:38.000Z","size":119670,"stargazers_count":45,"open_issues_count":3,"forks_count":21,"subscribers_count":1,"default_branch":"master","last_synced_at":"2024-09-04T00:05:02.978Z","etag":null,"topics":["coco","docker","object-detection","object-detection-pipelines","python","tensorflow","tensorflow-serving","tensorflow-tutorials"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fpaupier.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-01-22T14:34:19.000Z","updated_at":"2024-02-29T04:52:05.000Z","dependencies_parsed_at":"2022-09-03T17:31:43.172Z","dependency_job_id":null,"html_url":"https://github.com/fpaupier/tensorflow-serving_sidecar","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fpaupier%2Ftensorflow-serving_sidecar","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fpaupier%2Ftensorflow-serving_sidecar/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fpaupier%2Ftensorflow-serving_sidecar/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fpaupier%2Ftensorflow-serving_sidecar/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fpaupier","download_url":"https://codeload.github.com/fpaupier/tensorflow-serving_sidecar/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":231438919,"owners_count":18376823,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["coco","docker","object-detection","object-detection-pipelines","python","tensorflow","tensorflow-serving","tensorflow-tutorials"],"created_at":"2024-09-03T00:01:34.580Z","updated_at":"2024-12-27T04:31:00.592Z","avatar_url":"https://github.com/fpaupier.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"_Note: This repo is now archived and won't see further development_\n\n# How to serve your machine learning model with `tensorflow-serving`\n\n\nThis repo offers detailed _How-to_ about serving your tensorflow models with `tensorflow-serving`.\nIt can be followed as a step-by-step tutorial to serve your ``tensorflow`` model or other models with some adjustments. \n\n### Scope of the project\nI do not cover the parts of training or exporting models here.\nI focus solely on serving a model for inference.\n![Pipe overview](/assets/pipe_overview.png)\n\n\n### What's in the box?\n\nTwo tutorials: \n\n1. A basic tutorial to get you quickly serve an object detection model with `tensorflow-serving`. \nExpect to get such an awesome object-detection up and running in less than 10 mins.\n![horse labelled with faster rcnn resnet](assets/out_image1.jpeg) \n\n2. An advanced tutorial to deploy your tensorflow server docker image on Google Cloud Platform.\nUnleash the power of GCP to build a scalable machine learning server running in a ``kubernetes`` cluster.\n\n*The proposed object detection model is here to get you started quickly, feel free to use yours for more fun!*\n\nThe reader is expected to be familiar with tensorflow, knowing how to export a model for inference will be helpful (`tensorflow-server` works with `savedModel`).\nKnowing docker or kubernetes are not pre-requisites since the commands used are simple and explained when needed.\n\nThose tutorials are highly inspired by [tensorflow-serving official documentation](https://www.tensorflow.org/serving/docker)\nwith some tips and more detail on the installation process. \n\n## 0. Install the project\n\nThe installation process is thoroughly described in the [docs/setup.md](docs/setup.md). It covers everything you need to do\nprior being able to serve a model with tensorflow-serving.\n\n## 1. Serve your first model and perform inference with tensorflow-serving\n\n_Play Time!_\n\nTo make sure the installation went smoothly, get your first inference result from the object detection model tensorflow serving.\nFollow the first tutorial to serve an object detection model on your machine [docs/tf_server_local.md](docs/tf_server_local.md).\n\n## 2. Deploy your model on the cloud for high availability\n\nNow that we made sure our inference server works great on local, let's deploy it on the cloud at production scale.\nLet's dive in the [docs/tf_server_k8s.md](docs/tf_server_k8s.md).\n\n\n----------\n\n### Resources \n\nAdditional information you may find useful\n\n\nGoogle proposes a managed solution - _Google Cloud ML Engine_ - to serve your `saved_models.pb` models. \nI do not focus on it since [google documentation](https://cloud.google.com/ml-engine/docs/tensorflow/deploying-models) is\na more comprehensive source of information.\n\n**Pros**\n- Deploy new models easily\n\n**Cons**\n- Less flexibility\n- As of today, you are limited in size for your `savedModel.pb` file to `250 MB`. (That may change) \n\n----------\n\n### Credits\n\nThe `object_detection` directory comes from the\n[tensorflow-model](https://github.com/tensorflow/models) repository. \nIt offers useful `utils` functions to tag the image returned from the model.\n\nFeel free to investigate the models on the `tensorflow-model` repo since they are well documented and often comes with useful tutorials.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffpaupier%2Ftensorflow-serving_sidecar","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffpaupier%2Ftensorflow-serving_sidecar","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffpaupier%2Ftensorflow-serving_sidecar/lists"}