{"id":19753426,"url":"https://github.com/popfido/tensorflow-client-builder","last_synced_at":"2026-04-12T06:31:46.753Z","repository":{"id":74966439,"uuid":"247651582","full_name":"popfido/tensorflow-client-builder","owner":"popfido","description":"An example/introductory repository to generate tensorflow gRPC client code in java, C++, C#, Javascript, or Python","archived":false,"fork":false,"pushed_at":"2023-07-05T20:50:33.000Z","size":150,"stargazers_count":0,"open_issues_count":1,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-01-10T21:49:27.973Z","etag":null,"topics":["grpc","java","tensorflow","tensorflow-serving"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/popfido.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-03-16T08:43:39.000Z","updated_at":"2020-03-23T09:19:28.000Z","dependencies_parsed_at":"2025-01-10T21:49:16.941Z","dependency_job_id":"68bd65a7-d3b9-4352-b809-b50ec101d401","html_url":"https://github.com/popfido/tensorflow-client-builder","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/popfido%2Ftensorflow-client-builder","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/popfido%2Ftensorflow-client-builder/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/popfido%2Ftensorflow-client-builder/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/popfido%2Ftensorflow-client-builder/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/popfido","download_url":"https://codeload.github.com/popfido/tensorflow-client-builder/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241084604,"owners_count":19907145,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["grpc","java","tensorflow","tensorflow-serving"],"created_at":"2024-11-12T02:52:45.078Z","updated_at":"2025-12-31T01:12:44.345Z","avatar_url":"https://github.com/popfido.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Introduction\n\n[TensorFlow](https://github.com/tensorflow/tensorflow) is a popular Machine Learning toolkit, which includes [TF Serving](https://github.com/tensorflow/serving) which can serve the saved ML models via a Docker image that exposes [RESTful](https://github.com/tensorflow/serving/blob/master/tensorflow_serving/g3doc/api_rest.md) and [gRPC API](https://github.com/tensorflow/serving/tree/master/tensorflow_serving/apis).\n\nHere is a [introduction of gRPC](https://grpc.io/docs/guides/index.html). The TF Serving's gRPC APIs are defined inside protobuf files and provide slightly more functionalities than the RESTful API. With these .proto files, you can generate the necessary client source code for various languages, and integrate the model serving function into your own application.\n\nThis repository is aiming at giving a step by step introduction on how to generate tensorflow server client code on different language platform (mainly for Java).\n\n# Build your Tensorflow Java client\n\n## Preparation\n\n### Checkout this repository fron github\n\nBefore doing stuffs below to generate your tensorflow client, clone this repository from github to use it\n\n```bash\n$ git clone --recursive https://github.com/popfido/tensorflow-client-builder.git\n$ cd tensorflow-client-builder\n$ export SRC=$(pwd)\n```\n\nFollowing Steps will help you understand how to generate target platform codes. But if you don't care about details, just execute:\n\n```bash\n$ cd $SRC\n$ sh build.sh\n```\n\nThen target java serving codes will be generated under path `tensorflow-server-client/src/main/java` with version 1.15.0.\n\nIf you wanna generate codes of other release version, execute `sh build.sh -h` for detailed options.\n\n**Note**: the original tensorflow repository is quite large, thus time consuming of `--recursive` might be slower than your expectation due to network speed in different area. \n\n## Step 1. Get TensorFlow protobuf files\n\n**Note**: Step 1 can be skipped since this repository has contained a version of tensorflow protobuf files with release 1.15.0.\n\n### Check out the tensorflow projects somewhere\n\n```bash\n# Change 1.15.0 to any target release you wanna generate\n$ export TARGET_RELEASE=1.15.0\n$ cd $SRC/serving\n$ git checkout tags/$TARGET_RELEASE\n\n$ cd $SRC/tensorflow\n$ git checkout tags/v$TARGET_RELEASE\n```\n\n### Gather all the .proto files and organize them into a new Java project.\n\nThe libraries we checked out contain many files, but we only need part of .proto files in order to compile our gRPC Java client. \nThus let's make a project to host the source .proto files and future .java files.\n\n```bash\n$ export PROJECT_ROOT=$SRC/tensorflow-server-client\n$ rm -rf $PROJECT_ROOT/src/main/proto\n$ mkdir -p $PROJECT_ROOT/src/main/proto\n```\n\nOur end goal is to get all the .proto files required directly or indirectly by `tensorflow_serving/apis/*_service.proto` files. \nHowever, there are no tools that can start with a few .proto files and trace through the import statements and list all other .proto files required. \nSo figuring out what files are needed is done by trying to compile the resulting Java classes till no 'no class def found' complaints. \nAlternatively one could simply include all .proto files from tensorflow_serving/ and tensorflow/, but it will result in much bigger Java package.\n\nLet's try to pick out only .proto files and put `.proto` files into the new project's directories respectively under `src/main/proto` using the rsync commands, \nwhile still keep the directory structure, which is assumed by the import statements in these .proto files. \n\n```bash\n$ rsync -arv  --prune-empty-dirs --include=\"*/\" --include='*.proto' --exclude='*' $SRC/serving/tensorflow_serving  $PROJECT_ROOT/src/main/proto/\n$ rsync -arv  --prune-empty-dirs --include=\"*/\" --include=\"tensorflow/core/lib/core/*.proto\" --include='tensorflow/core/framework/*.proto' --include=\"tensorflow/core/example/*.proto\" --include=\"tensorflow/core/protobuf/*.proto\" --include=\"tensorflow/stream_executor/*.proto\" --exclude='*' $SRC/tensorflow/tensorflow  $PROJECT_ROOT/src/main/proto/\n```\n\n**Note**: The .proto files in these directories can change between releases, new files can be added, and file content can also change. So it is possible that the above 5 directories will contain .proto files that require other .proto files from directories outside. This repository is currently only tested under Tensorflow 1.15.0. So in case of that situation comes, you shall expand the .proto files to include those .proto files needed. But future test under other releases will be done.\n\n## Step 2. Generate the Java files\n\nNow we have a project with only .proto files under `src/main/proto/`. Let's compile them into Java source files.\n\n### Build with maven\n\nBuild can be automated by using maven, the key dependencies declared in pom file are:\n\n```xml\n    \u003cproperties\u003e\n        \u003cgrpc.version\u003e1.28.0\u003c/grpc.version\u003e\n        \u003cprotobuf.version\u003e3.8.0\u003c/protobuf.version\u003e\n    \u003c/properties\u003e\n    \n    \u003cdependencies\u003e\n        \u003c!-- gRPC protobuf client --\u003e\n        \u003cdependency\u003e\n            \u003cgroupId\u003eio.grpc\u003c/groupId\u003e\n            \u003cartifactId\u003egrpc-protobuf\u003c/artifactId\u003e\n            \u003cversion\u003e${grpc.version}\u003c/version\u003e\n        \u003c/dependency\u003e\n        \u003cdependency\u003e\n            \u003cgroupId\u003eio.grpc\u003c/groupId\u003e\n            \u003cartifactId\u003egrpc-stub\u003c/artifactId\u003e\n            \u003cversion\u003e${grpc.version}\u003c/version\u003e\n        \u003c/dependency\u003e\n        \u003cdependency\u003e\n            \u003cgroupId\u003eio.grpc\u003c/groupId\u003e\n            \u003cartifactId\u003egrpc-netty-shaded\u003c/artifactId\u003e\n            \u003cversion\u003e${grpc.version}\u003c/version\u003e\n        \u003c/dependency\u003e\n    \u003c/dependencies\u003e\n```\n\nAdditionally, use the `protobuf-maven-plugin` which will compile .proto files to .java files. It will also generate extra `*Grpc.java` service stub files for each `*_service.proto` files:\n\n```xml\n    \u003cbuild\u003e\n        \u003cextensions\u003e\n            \u003cextension\u003e\n                \u003cgroupId\u003ekr.motd.maven\u003c/groupId\u003e\n                \u003cartifactId\u003eos-maven-plugin\u003c/artifactId\u003e\n                \u003cversion\u003e1.6.2\u003c/version\u003e\n            \u003c/extension\u003e\n        \u003c/extensions\u003e\n        \u003cplugins\u003e\n            \u003cplugin\u003e\n                \u003cgroupId\u003eorg.xolstice.maven.plugins\u003c/groupId\u003e\n                \u003cartifactId\u003eprotobuf-maven-plugin\u003c/artifactId\u003e\n                \u003cversion\u003e0.6.1\u003c/version\u003e\n                \u003cexecutions\u003e\n                    \u003cexecution\u003e\n                        \u003cgoals\u003e\n                            \u003cgoal\u003ecompile\u003c/goal\u003e\n                            \u003cgoal\u003ecompile-custom\u003c/goal\u003e\n                        \u003c/goals\u003e\n                    \u003c/execution\u003e\n                \u003c/executions\u003e\n                \u003cconfiguration\u003e\n                    \u003ccheckStaleness\u003etrue\u003c/checkStaleness\u003e\n                    \u003cprotocArtifact\u003ecom.google.protobuf:protoc:${protobuf.version}:exe:${os.detected.classifier}\u003c/protocArtifact\u003e\n                    \u003cpluginId\u003egrpc-java\u003c/pluginId\u003e\n                    \u003cpluginArtifact\u003eio.grpc:protoc-gen-grpc-java:${grpc.version}:exe:${os.detected.classifier}\u003c/pluginArtifact\u003e\n                    \u003coutputDirectory\u003e${basedir}/src/main/java\u003c/outputDirectory\u003e\n                    \u003cclearOutputDirectory\u003etrue\u003c/clearOutputDirectory\u003e\n                \u003c/configuration\u003e\n            \u003c/plugin\u003e\n        \u003c/plugins\u003e\n    \u003c/build\u003e\n```\n\nNoted that the output directory has been set as `{project_base_dir}/src/main/java`. Thus after executing list of goals above, you shall find the compiled `.java` file inside `src/main/java`.\n\nHere is the [documentation of this plugin](https://www.xolstice.org/protobuf-maven-plugin/), including the list of goals available. You can see it can compile the .proto files to **Java**, **C++**, **C#**, **Javascript**, or **Python**.\n\nNotes:\n\n- The `compile-custom` goal in the above pom will generate the `*Grpc.java` files, which are essential for the Java client, so keep it in your goal list.\n- The plugin includes pre-compiled `protoc` executable for Linux, and is compiled using glibc, so it may not run correctly in Linux systems without glibc, e.g. alpine linux. So don't use a build server based on alpine Linux. See [more details](https://github.com/xolstice/protobuf-maven-plugin/issues/23#issuecomment-266098369).\n- The `os-maven-plugin` extension is used to provide `${os.detected.classifier}`, in order to pull the correct executable for the build server/OS. [Eclipse IDE may require some special handling](https://github.com/trustin/os-maven-plugin#issues-with-eclipse-m2e-or-other-ides).\n\n# Sample Java client code to make gRPC calls\n\n```java\n    String host = \"localhost\";\n    int port = 8501;\n    // the model's name. \n    String modelName = \"cool_model\";\n    // model's version\n    long modelVersion = 123456789;\n    // assume this model takes input of free text, and make some sentiment prediction.\n    String modelInput = \"some text input to make prediction with\";\n    \n    // create a channel\n    ManagedChannel channel = ManagedChannelBuilder.forAddress(host, port).usePlaintext().build();\n    PredictionServiceGrpc.PredictionServiceBlockingStub stub = PredictionServiceGrpc.newBlockingStub(channel);\n    \n    // create a modelspec\n    Model.ModelSpec.Builder modelSpecBuilder = Model.ModelSpec.newBuilder();\n    modelSpecBuilder.setName(modelName);\n    modelSpecBuilder.setVersion(Int64Value.of(modelVersion));\n    modelSpecBuilder.setSignatureName(\"serving_default\");\n\n    Predict.PredictRequest.Builder builder = Predict.PredictRequest.newBuilder();\n    builder.setModelSpec(modelSpecBuilder);\n    \n    // create the TensorProto and request\n    TensorProto.Builder tensorProtoBuilder = TensorProto.newBuilder();\n    tensorProtoBuilder.setDtype(DataType.DT_STRING);\n    TensorShapeProto.Builder tensorShapeBuilder = TensorShapeProto.newBuilder();\n    tensorShapeBuilder.addDim(TensorShapeProto.Dim.newBuilder().setSize(1));\n    tensorProtoBuilder.setTensorShape(tensorShapeBuilder.build());\n    tensorProtoBuilder.addStringVal(ByteString.copyFromUtf8(modelInput));\n    TensorProto tp = tensorProtoBuilder.build();\n\n    builder.putInputs(\"inputs\", tp);\n    \n    Predict.PredictRequest request = builder.build();\n    Predict.PredictResponse response = stub.predict(request);\n```\n\nAdditional engineering considerations:\n\n- Creating a channel is an expensive operation, thus a connected channel should be cached.\n- protobuf classes are dumb data holders, used for serialization and communication. You should build separate application specific object models that wraps around these protobuf classes, to provide additional behavior. Don't extend the protobuf classes for this purpose. See [Protobuf Java Tutorial](https://developers.google.com/protocol-buffers/docs/javatutorial).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpopfido%2Ftensorflow-client-builder","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpopfido%2Ftensorflow-client-builder","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpopfido%2Ftensorflow-client-builder/lists"}