{"id":41586278,"url":"https://github.com/wiremind/pythie-serving","last_synced_at":"2026-01-24T09:06:30.701Z","repository":{"id":37097524,"uuid":"221730819","full_name":"wiremind/pythie-serving","owner":"wiremind","description":"Framework to serve other ML model with tensorflow serving protobuff API","archived":false,"fork":false,"pushed_at":"2025-05-06T12:38:21.000Z","size":1901,"stargazers_count":11,"open_issues_count":2,"forks_count":4,"subscribers_count":9,"default_branch":"master","last_synced_at":"2025-05-06T13:24:06.006Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/wiremind.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2019-11-14T15:37:05.000Z","updated_at":"2025-05-06T12:37:56.000Z","dependencies_parsed_at":"2025-05-06T13:22:24.115Z","dependency_job_id":"5ff18d75-4145-4391-907b-800d9fb1f6f6","html_url":"https://github.com/wiremind/pythie-serving","commit_stats":null,"previous_names":[],"tags_count":38,"template":false,"template_full_name":null,"purl":"pkg:github/wiremind/pythie-serving","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wiremind%2Fpythie-serving","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wiremind%2Fpythie-serving/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wiremind%2Fpythie-serving/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wiremind%2Fpythie-serving/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/wiremind","download_url":"https://codeload.github.com/wiremind/pythie-serving/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wiremind%2Fpythie-serving/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28722048,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-24T08:27:05.734Z","status":"ssl_error","status_checked_at":"2026-01-24T08:27:01.197Z","response_time":89,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-01-24T09:06:30.634Z","updated_at":"2026-01-24T09:06:30.693Z","avatar_url":"https://github.com/wiremind.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"## Goal of this library\nTensorflow's official client implementations depend on the tensorflow package as of this writing.\nFor instance, if you want to run their example MNIST client, you'd need to have tensorflow installed even just\nto construct the prediction request predict_pb2.PredictRequest, and to call\ntheir tf.contrib.util.make_tensor_proto function. So you can't easily use their work to make a lightweight client API\nthat is totally free of tensorflow.\nTensorflow being a huge library, the estimated RAM overhead of having tensorflow installed is around 150MB which makes\nthe client heavy.\nAnother goal of this project is to expose other models then tensorflow with a GRPC API following the one defined for tensorflow serving (https://github.com/tensorflow/serving).\nFor instance, it allows to serve a XGBoost model with the same API as the one used to request a tensorflow serving server\n\n## How to update this lib with a new version of tensorflow_serving\nWe need to generate the GRPC client code ourselves from the .proto definition in\nhttps://github.com/tensorflow/serving/tree/master/tensorflow_serving/apis\n\nTo do this,\n\nrun \"sh generate_pbs.sh X.Y.Z\" (you should have an already created pew virtualenv using python3.11 called 'grpc-build'),\nthis will generate a working python package in ./tensorflow_proto from the tag version X.Y.Z of tensorflow repo\nyou can then copy/paste this package inside src/pythie-serving/ and import it as you need.\n\nThis will also generate the necessary stubs for type-checking\n\n\u003e For Mac users: replace `sed` command in `generate_pbs.sh` by GNU sed (in order to prevent the creation of back files): `brew install gnu-sed` and replace `sed` by `gsed` in `generate_pbs.sh`.\n\n## Available model architectures\nThe following models can be served by pythie-serving:\n* LightGBM: https://lightgbm.readthedocs.io/en/latest/\n* XGBoost: https://xgboost.readthedocs.io/en/stable/\n* scikit-learn: https://scikit-learn.org/stable/\n* treelite: https://treelite.readthedocs.io/en/latest/\n* csv table\n\n## How to run\n\n### Python\n\n1. Define a `models.config` file (see: https://www.tensorflow.org/tfx/serving/serving_config):\n   ```\n    model_config_list {\n        config {\n            name: \u003cmodel_name\u003e,\n            base_path: \u003cmy_base_path\u003e,\n            model_platform: \u003cmodel_platform\u003e\n        }\n    }\n    ```\n2. Run `python pythie-serving/src/pythie_serving/run.py \u003cmodel_config_file_path\u003e`.\\\n   The following options are available:\n   * `--worker-count`: Number of concurrent threads for the GRPC server.\n   * `--max-concurrent-rpcs`: The maximum number of concurrent RPCs this server.\n   * `--port`: Port number to listen to.\n\n#### Environment variables\n* `OPENTELEMETRY_COLLECTOR_HOST`: OpenTelemetry Collector receiver endpoint. If not defined OpenTelemetry will not be activated. See https://opentelemetry.io/docs/what-is-opentelemetry for more details.\nFor a treelite served model:\n* `TREELITE_NTHREAD`: Number of threads to use to compute predictions\n* `TREELINTE_BIND_THREADS`: Set to `0` to deactivate thread pinning. See https://treelite.readthedocs.io/en/latest/treelite-runtime-api.html\nFor an LGBM served model:\n* `LGBM_NTHREAD`: Number of threads to use to compute predictions\n* `GRPC_SERVER_TIMEOUT`: Timeout at which server will stop and the pod will shut down in seconds.\n\n### Docker\n\nThe project is published on GitHub Container Registry: https://github.com/wiremind/pythie-serving/pkgs/container/pythie-serving\n\n## Development\n\n### Add a new architecture type\nTo add a new architecture type, you need to implement a `prediction_service_pb2_grpc.PredictionServiceServicer`.\nTo facilitate this, pythie-serving implements an abstract `AbstractPythieServingPredictionServiceServicer` that already implements the necessary `Predict`\nmethod, taking a `PredictRequest` as input and outputing a `PredictResponse`.\nThis class leaves 2 abstract methods to be implemented:\n* `_create_model_specs` to read a `ModelConfig` from the list and instantiate the model and necessary variables\n* `_predict` to make a prediction using the model (as python class) on a numpy array\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwiremind%2Fpythie-serving","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwiremind%2Fpythie-serving","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwiremind%2Fpythie-serving/lists"}