{"id":23201271,"url":"https://github.com/adessose/kafka-streams-tensorflow","last_synced_at":"2026-05-11T03:02:57.241Z","repository":{"id":41041435,"uuid":"173326077","full_name":"adessoSE/kafka-streams-tensorflow","owner":"adessoSE","description":"Integrates Kafka Streams and Tensorflow","archived":false,"fork":false,"pushed_at":"2023-07-05T20:43:21.000Z","size":1043,"stargazers_count":0,"open_issues_count":3,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-03T00:41:30.253Z","etag":null,"topics":["grpc","java","kafka","kafka-streams","protobuf","tensorflow","tensorflow-serving"],"latest_commit_sha":null,"homepage":null,"language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/adessoSE.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-03-01T15:36:57.000Z","updated_at":"2019-06-22T11:48:52.000Z","dependencies_parsed_at":"2023-01-19T05:46:09.309Z","dependency_job_id":null,"html_url":"https://github.com/adessoSE/kafka-streams-tensorflow","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adessoSE%2Fkafka-streams-tensorflow","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adessoSE%2Fkafka-streams-tensorflow/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adessoSE%2Fkafka-streams-tensorflow/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adessoSE%2Fkafka-streams-tensorflow/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/adessoSE","download_url":"https://codeload.github.com/adessoSE/kafka-streams-tensorflow/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247314957,"owners_count":20918941,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["grpc","java","kafka","kafka-streams","protobuf","tensorflow","tensorflow-serving"],"created_at":"2024-12-18T15:14:43.975Z","updated_at":"2026-05-11T03:02:57.153Z","avatar_url":"https://github.com/adessoSE.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Kafka Streams Tensorflow integration\n\n## Introduction\n\nIntegrates Kafka Streams and Tensorflow asynchronously using gRPC\n\nThis project is based on\nhttps://github.com/kaiwaehner/kafka-streams-machine-learning-example\n\n## Use Cases\n\n### Fraud\n\nIn order to do fraud detection, this application will read from a Kafka topic using Kafka Streams, send each message to Tensorflow and write the resulting message (including the classification) to another Kafka topic.\n\n### Credit Scoring\nTo be implemented\n\n## Pseudocode (Fraud)\n\n```\n1) For each message in INPUT_TOPIC:\n    - Send to Tensorflow model\n    - Add callback to sent message\n\n2) For each sent message:\n    (onSuccess)\n        -Send message to output Kafka topic asynchronously, without waiting for acknowledgement.\n\n3) For each message sent to Kafka Topic\n    (Acknowledgement: Success/failure)\n        - Show status returned by Kafka acknowledgement\n````\n\n## Building and Running\n\n### Building this application\n\nBecause this is a Maven application, the following should suffice:\n```bash\ngit clone https://github.com/adessoAG/kafka-streams-tensorflow.git\nmvn clean package\n``` \n\n### Running this application\n\nAs described in [Confluent - Running Streams Application](https://docs.confluent.io/current/streams/developer-guide/running-app.html) this application can be executed as follows:\n```bash\njava -cp kafka-streams-tensorflow-1.0-SNAPSHOT-jar-with-dependencies.jar de.adesso.kafkaml.FraudModelServing test.fraud.properties\n```\nThis application also supports overwriting parameters from the configuration file via additional command line parameters:\n```bash\njava -cp kafka-streams-tensorflow-1.0-SNAPSHOT-jar-with-dependencies.jar de.adesso.kafkaml.FraudModelServing test.fraud.properties tf.model.version=2.0\n```\n \n\n### Building Tensorflow .proto files\n\nThe Tensorflow files were built using the following commands:\n\n```\n\nInstall ProtoC:\nsudo yum install autoconf automake libtool curl make g++ unzip libatomic\nhttps://gist.github.com/sofyanhadia/37787e5ed098c97919b8c593f0ec44d8\n\n\nwget tensorflow and tensorflow_serving from git releases\n \ncd /.../tensorflow/tensorflow/core/framework\n\nwget http://central.maven.org/maven2/io/grpc/protoc-gen-grpc-java/1.19.0/protoc-gen-grpc-java-1.19.0-linux-x86_64.exe\n\nKeep trying to build using:\nprotoc -I=. --plugin=protoc-gen-grpc-java=protoc-gen-grpc-java --grpc-java_out=java-pb-files --java_out=java-pb-files *.proto\n\nIf there is an error with something missing, copy all .proto-files from that directory here and remove/replace their paths using the following commands\nfind ./*.proto -type f -exec sed -i 's+tensorflow_serving/config/++g' {} +\n\n# Alternatively:\nDirectory strucutre:\ntest/tensorflow/core\ntest/tensorflow_serving/framework/\n\ncd test\n\nfor x in $(find '*.proto')\ndo\nprotoc -I=. --plugin=protoc-gen-grpc-java=protoc-gen-grpc-java-1.19.0-linux-x86_64.exe \\\n--grpc-java_out=java-pb-files --java_out=java-pb-files $x\ndone\n\n```\n## References\n\n### Framework Documentation\nhttps://grpc.io/docs/tutorials/basic/java.html\n\n### Code\nhttps://github.com/grpc/grpc-java/blob/master/benchmarks/src/main/java/io/grpc/benchmarks/qps/AsyncClient.java\n\nhttps://github.com/tensorflow/serving/blob/master/tensorflow_serving/apis/prediction_service.proto\n\nhttps://github.com/tobegit3hub/deep_image_model/blob/master/java_predict_client/src/main/java/com/tobe/InceptionPredictClient.java\n\nhttps://github.com/fzschornack/tensorflow-java-client/blob/master/src/main/java/tensorflow/serving/PredictionServiceGrpc.java\n\n \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fadessose%2Fkafka-streams-tensorflow","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fadessose%2Fkafka-streams-tensorflow","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fadessose%2Fkafka-streams-tensorflow/lists"}