{"id":15340872,"url":"https://github.com/tadayosi/torchserve-client-java","last_synced_at":"2026-03-03T14:36:00.981Z","repository":{"id":256577124,"uuid":"855811354","full_name":"tadayosi/torchserve-client-java","owner":"tadayosi","description":"A Java client library for PyTorch TorchServe","archived":false,"fork":false,"pushed_at":"2025-03-02T02:21:34.000Z","size":4648,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-28T11:42:38.405Z","etag":null,"topics":["ai","client","java","pytorch","torchserve"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tadayosi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-09-11T13:52:27.000Z","updated_at":"2025-03-02T02:21:31.000Z","dependencies_parsed_at":"2024-12-09T09:19:40.148Z","dependency_job_id":"35c7e349-cff5-4644-bebc-9bcf1bc7fcbc","html_url":"https://github.com/tadayosi/torchserve-client-java","commit_stats":{"total_commits":42,"total_committers":2,"mean_commits":21.0,"dds":0.09523809523809523,"last_synced_commit":"73b60f88f9ab1a96c9f12f2bf8cee00110d0c594"},"previous_names":["tadayosi/torchserve-client-java"],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tadayosi%2Ftorchserve-client-java","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tadayosi%2Ftorchserve-client-java/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tadayosi%2Ftorchserve-client-java/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tadayosi%2Ftorchserve-client-java/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tadayosi","download_url":"https://codeload.github.com/tadayosi/torchserve-client-java/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248980637,"owners_count":21193135,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","client","java","pytorch","torchserve"],"created_at":"2024-10-01T10:36:26.149Z","updated_at":"2026-03-03T14:36:00.976Z","avatar_url":"https://github.com/tadayosi.png","language":"Java","readme":"⚠️ **TorchServe is now in limited maintenance. This project is not maintained anymore as well.** ⚠️\n\n# TorchServe Client for Java\n\n[![Maven Central](https://maven-badges.herokuapp.com/maven-central/io.github.tadayosi.torchserve/torchserve-client/badge.svg?style=flat)](https://repo1.maven.org/maven2/io/github/tadayosi/torchserve/torchserve-client/)\n[![Test](https://github.com/tadayosi/torchserve-client-java/actions/workflows/test.yml/badge.svg)](https://github.com/tadayosi/torchserve-client-java/actions/workflows/test.yml)\n\nTorchServe Client for Java (TSC4J) is a Java client library for [TorchServe](https://pytorch.org/serve/index.html). It supports the following [TorchServe REST API](https://pytorch.org/serve/rest_api.html):\n\n- [Inference API](https://pytorch.org/serve/inference_api.html)\n- [Management API](https://pytorch.org/serve/management_api.html)\n- [Metrics API](https://pytorch.org/serve/metrics_api.html)\n\n## Requirements\n\n- Java 17+\n\n## Install\n\nAdd the dependency to your `pom.xml`:\n\n```xml\n\u003cdependency\u003e\n    \u003cgroupId\u003eio.github.tadayosi.torchserve\u003c/groupId\u003e\n    \u003cartifactId\u003etorchserve-client\u003c/artifactId\u003e\n    \u003cversion\u003e0.4.1\u003c/version\u003e\n\u003c/dependency\u003e\n```\n\n## Usage\n\n### Inference\n\n- Prediction:\n\n  ```java\n  TorchServeClient client = TorchServeClient.newInstance();\n\n  byte[] image = Files.readAllBytes(Path.of(\"0.png\"));\n  Object result = client.inference().predictions(\"mnist_v2\", image);\n  System.out.println(result);\n  // =\u003e 0\n  ```\n\n- With the inference API endpoint other than \u003chttp://localhost:8080\u003e:\n\n  ```java\n  TorchServeClient client = TorchServeClient.builder()\n      .inferenceAddress(\"http://localhost:12345\")\n      .build();\n  ```\n\n- With token authorization:\n\n  ```java\n  TorchServeClient client = TorchServeClient.builder()\n      .inferenceKey(\"\u003cinference-key\u003e\")\n      .build();\n  ```\n\n### Management\n\n- Register a model:\n\n  ```java\n  TorchServeClient client = TorchServeClient.newInstance();\n\n  Response response = client.management().registerModel(\n    \"https://torchserve.pytorch.org/mar_files/mnist_v2.mar\",\n    RegisterModelOptions.empty());\n  System.out.println(response.getStatus());\n  // =\u003e \"Model \"mnist_v2\" Version: 2.0 registered with 0 initial workers. Use scale workers API to add workers for the model.\"\n  ```\n\n- Scale workers for a model:\n\n  ```java\n  TorchServeClient client = TorchServeClient.newInstance();\n\n  Response response = client.management().setAutoScale(\n    \"mnist_v2\",\n    SetAutoScaleOptions.builder()\n      .minWorker(1)\n      .maxWorker(2)\n      .build());\n  System.out.println(response.getStatus());\n  // =\u003e \"Processing worker updates...\"\n  ```\n\n- Describe a model:\n\n  ```java\n  TorchServeClient client = TorchServeClient.newInstance();\n\n  List\u003cModelDetail\u003e model = client.management().describeModel(\"mnist_v2\");\n  System.out.println(model.get(0));\n  // =\u003e\n  // ModelDetail {\n  //     modelName: mnist_v2\n  //     modelVersion: 2.0\n  // ...\n  ```\n\n- Unregister a model:\n\n  ```java\n  TorchServeClient client = TorchServeClient.newInstance();\n\n  Response response = client.management().unregisterModel(\n    \"mnist_v2\",\n    UnregisterModelOptions.empty());\n  System.out.println(response.getStatus());\n  // =\u003e \"Model \"mnist_v2\" unregistered\"\n  ```\n\n- List models:\n\n  ```java\n  TorchServeClient client = TorchServeClient.newInstance();\n\n  ModelList models = client.management().listModels(10, null);\n  System.out.println(models);\n  // =\u003e\n  // ModelList {\n  //     nextPageToken: null\n  //     models: [Model {\n  //     modelName: mnist_v2\n  //     modelUrl: https://torchserve.pytorch.org/mar_files/mnist_v2.mar\n  // },\n  // ...\n  ```\n\n- Set default version for a model:\n\n  ```java\n  TorchServeClient client = TorchServeClient.newInstance();\n\n  Response response = client.management().setDefault(\"mnist_v2\", \"2.0\");\n  System.out.println(response.getStatus());\n  // =\u003e \"Default version successfully updated for model \"mnist_v2\" to \"2.0\"\"\n  ```\n\n- With the management API endpoint other than \u003chttp://localhost:8081\u003e:\n\n  ```java\n  TorchServeClient client = TorchServeClient.builder()\n      .managementAddress(\"http://localhost:12345\")\n      .build();\n  ```\n\n- With token authorization:\n\n  ```java\n  TorchServeClient client = TorchServeClient.builder()\n      .managementKey(\"\u003cmanagement-key\u003e\")\n      .build();\n  ```\n\n### Metrics\n\n- Get metrics in Prometheus format:\n\n  ```java\n  TorchServeClient client = TorchServeClient.newInstance();\n\n  String metrics = client.metrics().metrics();\n  System.out.println(metrics);\n  // =\u003e\n  // # HELP MemoryUsed Torchserve prometheus gauge metric with unit: Megabytes\n  // # TYPE MemoryUsed gauge\n  // MemoryUsed{Level=\"Host\",Hostname=\"3a9b51d41fbf\",} 2075.09765625\n  // ...\n  ```\n\n- With the metrics API endpoint other than \u003chttp://localhost:8082\u003e:\n\n  ```java\n  TorchServeClient client = TorchServeClient.builder()\n      .metricsAddress(\"http://localhost:12345\")\n      .build();\n  ```\n\n## Configuration\n\n### tsc4j.properties\n\n```properties\ninference.key = \u003cinference-key\u003e\ninference.address = http://localhost:8080\n# inference.address takes precedence over inference.port if it's defined\ninference.port = 8080\n\nmanagement.key = \u003cmanagement-key\u003e\nmanagement.address = http://localhost:8081\n# management.address takes precedence over management.port if it's defined\nmanagement.port = 8081\n\nmetrics.address = http://localhost:8082\n# metrics.address takes precedence over metrics.port if it's defined\nmetrics.port = 8082\n```\n\n### System properties\n\nYou can configure the TSC4J properties via system properties with prefix `tsc4j.`.\n\nFor instance, you can configure `inference.address` with the `tsc4j.inference.address` system property.\n\n### Environment variables\n\nYou can also configure the TSC4J properties via environment variables with prefix `TSC4J_`.\n\nFor instance, you can configure `inference.address` with the `TSC4J_INFERENCE_ADDRESS` environment variable.\n\n## Examples\n\nSee [examples](./examples/).\n\n## Build\n\n```console\nmvn clean install\n```\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftadayosi%2Ftorchserve-client-java","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftadayosi%2Ftorchserve-client-java","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftadayosi%2Ftorchserve-client-java/lists"}