https://github.com/kozistr/catboost-server-rs
CatBoost server in Rust + gRPC
https://github.com/kozistr/catboost-server-rs
catboost grpc machine-learning rust server serving
Last synced: 3 months ago
JSON representation
CatBoost server in Rust + gRPC
- Host: GitHub
- URL: https://github.com/kozistr/catboost-server-rs
- Owner: kozistr
- License: mit
- Created: 2022-07-26T09:06:18.000Z (about 3 years ago)
- Default Branch: main
- Last Pushed: 2022-09-09T15:16:21.000Z (about 3 years ago)
- Last Synced: 2023-03-01T07:22:39.310Z (over 2 years ago)
- Topics: catboost, grpc, machine-learning, rust, server, serving
- Language: Rust
- Homepage:
- Size: 513 KB
- Stars: 2
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# catboost-server-rs
CatBoost server in Rust + gRPC
## Model
Simple CatBoost model
* 2 numeric (float) features
* 3 categorical features
## Run
### gRPC Server
```shell
$ cargo run --release --bin cb-server
```
### gRPC Client
```shell
# binary http://host:port batch_size iterations timeout
$ cargo run --release --bin cb-client http://127.0.0.1:50051 32 100000 10
```
## Performance
* timeout : 10ms
* latency : ms / req
* warm up with 10 times
| batch size | requests | mean | p95 | p99 | p99.9 | max | timeouts |
| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| 1 | 100k | 0.131 ms | 0.165 ms | 0.233 ms | 0.324 ms | 0.485 ms | 0 |
| 4 | 100k | 0.179 ms | 0.245 ms | 0.337 ms | 0.497 ms | 0.912 ms | 0 |
| 32 | 100k | 0.217 ms | 0.291 ms | 0.408 ms | 0.533 ms | 0.803 ms | 0 |
| 64 | 100k | 0.299 ms | 0.409 ms | 0.537 ms | 0.786 ms | 4.672 ms | 0 |
| 128 | 100k | 0.461 ms | 0.630 ms | 0.782 ms | 0.990 ms | 1.696 ms | 0 |
| 256 | 100k | 0.920 ms | 1.232 ms | 1.500 ms | 1.896 ms | 2.865 ms | 0 |
## Reference
* https://github.com/leosmerling/catboost_inference/tree/main/rust/grpc-server