{"id":27248836,"url":"https://github.com/bakdata/kserve-client","last_synced_at":"2025-04-10T23:48:15.911Z","repository":{"id":38458126,"uuid":"456412373","full_name":"bakdata/kserve-client","owner":"bakdata","description":"A Java client for KServe inference services","archived":false,"fork":false,"pushed_at":"2025-04-02T07:31:54.000Z","size":224,"stargazers_count":8,"open_issues_count":2,"forks_count":1,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-04-10T23:48:10.999Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bakdata.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-02-07T08:08:36.000Z","updated_at":"2025-04-02T07:31:57.000Z","dependencies_parsed_at":"2025-01-28T13:34:12.179Z","dependency_job_id":"e771e476-9509-4ec0-870e-4cd013406911","html_url":"https://github.com/bakdata/kserve-client","commit_stats":null,"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bakdata%2Fkserve-client","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bakdata%2Fkserve-client/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bakdata%2Fkserve-client/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bakdata%2Fkserve-client/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bakdata","download_url":"https://codeload.github.com/bakdata/kserve-client/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248317727,"owners_count":21083528,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-04-10T23:48:15.311Z","updated_at":"2025-04-10T23:48:15.901Z","avatar_url":"https://github.com/bakdata.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Build Status](https://dev.azure.com/bakdata/public/_apis/build/status/bakdata.kserve-client?repoName=bakdata%2Fkserve-client\u0026branchName=main)](https://dev.azure.com/bakdata/public/_build/latest?definitionId=32\u0026repoName=bakdata%2Fkserve-client\u0026branchName=main)\n[![Quality Gate Status](https://sonarcloud.io/api/project_badges/measure?project=com.bakdata.kserve%3Akserve-client\u0026metric=alert_status)](https://sonarcloud.io/summary/new_code?id=com.bakdata.kserve%3Akserve-client)\n[![Coverage](https://sonarcloud.io/api/project_badges/measure?project=com.bakdata.kserve%3Akserve-client\u0026metric=coverage)](https://sonarcloud.io/summary/new_code?id=com.bakdata.kserve%3Akserve-client)\n[![Maven](https://img.shields.io/maven-central/v/com.bakdata.kserve/kserve-client.svg)](https://search.maven.org/search?q=g:com.bakdata.kserve%20AND%20a:kserve-client\u0026core=gav)\n\n\n# kserve-client\n\nA Java client for calling KServe inference services which implement one of [the predict v1 or v2 protocols](https://kserve.github.io/website/modelserving/v1beta1/serving_runtime/). \n\nIt let's you easily configure the endpoint of the inference service which should be called. \nThe data shape of both the request and response can be modeled using Java classes. \nThe library includes a retry mechanism to automatically retry requests to the inference service in case it's scaled to zero upon the first request.\n\nYou can find a [blog post on medium](https://medium.com/bakdata/scalable-machine-learning-with-kafka-streams-and-kserve-85308858d867) where the kserve-client is used in the demo application.\n\n## Getting Started\n\nYou can add kserve-client via Maven Central.\n\n#### Gradle\n```gradle\ncompile group: 'com.bakdata.kserve', name: 'kserve-client', version: '1.0.1'\n```\n\n#### Maven\n```xml\n\u003cdependency\u003e\n    \u003cgroupId\u003ecom.bakdata.kserve\u003c/groupId\u003e\n    \u003cartifactId\u003ekserve-client\u003c/artifactId\u003e\n    \u003cversion\u003e1.0.1\u003c/version\u003e\n\u003c/dependency\u003e\n```\n\n\nFor other build tools or versions, refer to the [latest version in MvnRepository](https://mvnrepository.com/artifact/com.bakdata.kserve/kserve-client/latest).\n\n### Usage\n\nThis usage example is extracted from a [blog post on medium](https://medium.com/bakdata/scalable-machine-learning-with-kafka-streams-and-kserve-85308858d867) where the kserve-client is used. In the inference service, we use an [Argos Translate](https://github.com/argosopentech/argos-translate) model to obtain a translation for an input text.\n\nA KServe inference service supporting the [protocol version v2](https://kserve.github.io/website/modelserving/inference_api) is expected to run on `localhost:8080` with model `argos-translator-en-es` so that the endpoint `localhost:8080/v2/models/argos-translator-en-es/infer` can be used for requests.\nThis inference service knows how to deal with the fields defined in `TextToTranslate.java` and `Translation.java` for the request input and output data, respectively.\n\nA usage example compatible with protocol version v1 can be constructed analogously using the `KServeClientFactoryV1` class.\n\n\\\n`TextToTranslate.java`\n```java\n@Data\n@AllArgsConstructor\n@NoArgsConstructor\n@Builder\npublic class TextToTranslate {\n    private String textToTranslate;\n}\n```\n\n\\\n`Translation.java`\n```java\n@Data\n@AllArgsConstructor\n@NoArgsConstructor\n@Builder\npublic class Translation {\n    private String originalText;\n    private String translatedText;\n}\n```\n\n\\\n`TranslatorResponse.java`\n```java\npublic class TranslatorResponse extends InferenceResponse\u003cTranslation\u003e {\n}\n```\n\n\\\n`KServeRequester.java`\n```java\npublic class KServeRequester\u003cI, O\u003e {\n    private final KServeClient\u003cI\u003e kServeClient;\n\n    public KServeRequester() {\n        this.kServeClient = (KServeClient\u003cI\u003e) new KServeClientFactoryV2().getKServeClient(\n                \"localhost:8080\",\n                \"argos-translator-en-es\",\n                Duration.ofSeconds(2),\n                false\n        );\n    }\n\n    protected Optional\u003cO\u003e requestInferenceService(final I jsonObject) {\n        try {\n            return (Optional\u003cO\u003e) this.kServeClient.makeInferenceRequest(\n                    jsonObject,\n                    TranslatorResponse.class,\n                    \"\");\n        } catch (final IOException e) {\n            throw new IllegalArgumentException(\n                    \"Error occurred when sending the inference request or receiving the response\", e);\n        }\n    }\n}\n```\n\n\\\n`App.java`\n```java\npublic final class App {\n    private static Translation getTranslation(final TextToTranslate input) {\n        return new KServeRequester\u003cInferenceRequest\u003cTextToTranslate\u003e, TranslatorResponse\u003e()\n                .requestInferenceService(InferenceRequest.\u003cTextToTranslate\u003ebuilder()\n                        .inputs(List.of(\n                                RequestInput.\u003cTextToTranslate\u003ebuilder()\n                                        .name(\"Translation\")\n                                        .datatype(\"BYTES\")\n                                        .shape(List.of(1))\n                                        .datatype(\"BYTES\")\n                                        .parameters(Parameters.builder()\n                                                .contentType(\"str\")\n                                                .build())\n                                        .data(input)\n                                        .build()\n                        ))\n                        .build())\n                .map(InferenceResponse::getOutputs)\n                .stream()\n                .flatMap(Collection::stream)\n                .map(ResponseOutput::getData)\n                .findFirst()\n                .orElseThrow();\n    }\n\n    public static void main(final String[] args) {\n        final Translation translation = getTranslation(TextToTranslate.builder().textToTranslate(\"Hello World\").build());\n        System.out.println(translation.getTranslatedText());\n        // Hola Mundo\n    }\n}\n```\n\n## Development\n\nIf you want to contribute to this project, you can simply clone the repository and build it via Gradle.\nAll dependencies should be included in the Gradle files, there are no external prerequisites.\n\n```bash\n\u003e git clone git@github.com:bakdata/kserve-client.git\n\u003e cd kserve-client \u0026\u0026 ./gradlew build\n```\n\nPlease note, that we have [code styles](https://github.com/bakdata/bakdata-code-styles) for Java.\nThey are basically the Google style guide, with some small modifications.\n\n## Contributing\n\nWe are happy if you want to contribute to this project.\nIf you find any bugs or have suggestions for improvements, please open an issue.\nWe are also happy to accept your PRs.\nJust open an issue beforehand and let us know what you want to do and why.\n\n## License\nThis project is licensed under the MIT license.\nHave a look at the [LICENSE](https://github.com/bakdata/kserve-client/blob/main/LICENSE) for more details.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbakdata%2Fkserve-client","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbakdata%2Fkserve-client","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbakdata%2Fkserve-client/lists"}