https://github.com/teiacare/teiacareinferenceclient
TeiaCareInferenceClient is a C++ inference client library that implements KServe protocol.
https://github.com/teiacare/teiacareinferenceclient
cpp cross-platform deep-learning grpc inference kserve machine-learning
Last synced: 6 months ago
JSON representation
TeiaCareInferenceClient is a C++ inference client library that implements KServe protocol.
- Host: GitHub
- URL: https://github.com/teiacare/teiacareinferenceclient
- Owner: TeiaCare
- License: apache-2.0
- Created: 2024-06-19T06:55:43.000Z (almost 2 years ago)
- Default Branch: develop
- Last Pushed: 2024-10-02T10:10:33.000Z (over 1 year ago)
- Last Synced: 2025-03-26T12:46:39.561Z (about 1 year ago)
- Topics: cpp, cross-platform, deep-learning, grpc, inference, kserve, machine-learning
- Language: C
- Homepage: https://teiacare.github.io/TeiaCareInferenceClient/
- Size: 453 KB
- Stars: 2
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG
- Contributing: CONTRIBUTING.md
- License: LICENSE
Awesome Lists containing this project
README
# TeiaCareInferenceClient
Welcome to TeiaCareInferenceClient!
*TeiaCareInferenceClient* is a C++ inference client library that implements KServe protocol.
[](https://teiacare.github.io/TeiaCareInferenceClient/md_README.html)
[](https://app.codacy.com/gh/TeiaCare/TeiaCareInferenceClient/dashboard?utm_source=gh&utm_medium=referral&utm_content=&utm_campaign=Badge_grade)
[](https://app.codacy.com/gh/TeiaCare/TeiaCareInferenceClient/dashboard?utm_source=gh&utm_medium=referral&utm_content=&utm_campaign=Badge_coverage)
## Getting Started
This project uses git submodules so it is required to clone it using the *--recursive* flag in order to retrive the required submodules.
```bash
git clone https://github.com/TeiaCare/TeiaCareInferenceClient.git --recursive
```
### Create Development Environment
In order to setup a development environment it is sufficient to run the script *scripts/env/setup.* depending on your operating system.
```bash
# Linux/MacOS
scripts/env/setup.sh
# Windows
scripts\env\setup.bat
```
### Start Development Environment
In order to start the development environment it is sufficient to activate the Python Virtual Environment just created the step above.
```bash
# Linux/MacOS
source .venv/bin/activate
# Windows
.venv\Scripts\activate.bat
```
### Setup Build Environment (Windows Only)
When building from command line on Windows it is necessary to activate the Visual Studio Developer Command Prompt.
Depending on the version of Visual Studio compiler and on its install location it is required to run *vcvars64.bat* script the set the development environment properly.
*Note*: using Visual Studio IDE or the CMake extension for VSCode this step is already managed in the background, so no action is required.
Examples:
```bash
# Visual Studio 2022 - Build Tools
"C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\Build\vcvars64.bat"
# Visual Studio 2019 - Enterprise
"C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Auxiliary\Build\vcvars64.bat"
```
### Dependencies Setup
This script must be executed in order to setup the 3rd party dependencies using conan packages.
```bash
conan remote add teiacare https://artifactory.app.teiacare.com/artifactory/api/conan/teiacare --insert 0 --force
conan user -p -r teiacare
python scripts/conan/setup.py
```
### Configure, Build and Install
This script configures, builds and installs the library.
```bash
python scripts/cmake.py
```
## Examples
```bash
# Build all the examples
python scripts/cmake.py --examples --warnings
# Run all the examples
python scripts/tools/run_examples.py install/examples
```
Examples are installed in $PWD/install/examples.
## Unit Tests and Code Coverage
```bash
# Build Unit Tests with Code Coverage enabled (if supported)
python scripts/cmake.py --coverage --warnings
# Run Unit Tests
python scripts/tools/run_unit_tests.py
# Run Code Covergae
python scripts/tools/run_coverage.py
```
Note that code coverage is not available on Windows.
Unit tests results are available in $PWD/results/unit_tests.
Coverage results are available in $PWD/results/coverage.
## Sanitizers
### Address Sanitizer
```bash
# Build Unit Tests with Address Sanitizer enabled (if supported)
python scripts/cmake.py --address_sanitizer --unit_tests
# Run Unit Tests with Address Sanitizer
python scripts/tools/run_sanitizer.py --address_sanitizer install/unit_tests/teiacare_inference_client_unit_tests
```
Note that Address Sanitizer is supported only on Linux.
### Thread Sanitizer
```bash
# Build Unit Tests with Thread Sanitizer enabled (if supported)
python scripts/cmake.py --thread_sanitizer --unit_tests
# Run Unit Tests with Thread Sanitizer
python scripts/tools/run_sanitizer.py --thread_sanitizer install/unit_tests/teiacare_inference_client_unit_tests
```
Note that Thread Sanitizer is supported only on Linux.
## Benchmarks
```bash
# Build Benkmarks
python scripts/cmake.py --benchmarks --warnings
# Run Benchmarks
python scripts/tools/run_benchmarks.py
```
Benchmarks are installed in $PWD/install/benchmarks.
## Code Formatting
- [clang-format](https://clang.llvm.org/docs/ClangFormat.html)
*clang-format* can be installed via *pip* using the provided *scripts/requirements.txt*
```bash
python scripts/tools/run_clang_format.py -r -i inference_client
```
## Code Analysis
- [clang-tidy](https://clang.llvm.org/extra/clang-tidy/)
*clang-tidy* can be installed via *pip* using the provided *scripts/requirements.txt*
```bash
python scripts/tools/run_clang_tidy.py -header-filter=.* inference_client
```
- [cppcheck](http://cppcheck.net/)
First install and setup *cppcheck* from your OS package manager.
```bash
# Linux
sudo apt install cppcheck
# Windows
winget install cppcheck
```
Then run CppCheck using the provided python script:
```bash
python scripts/tools/run_cppcheck.py
```
## Generate Documentation
First install and setup *Doxygen* from your OS package manager.
```bash
# Linux
apt-get install doxygen graphviz
# Windows
winget install doxygen
```
Then run Doxygen using the provided python script:
```bash
python scripts/tools/run_doxygen.py
```
Documentation is now installed in $PWD/docs.
## Conan Package
### Local Install
Create, test and install local package.
Notes:
1) The install directory path must be a valid Conan cache (i.e. ".conan" folder) located in the current directory.
So, in order to install the package in a desired repository folder, it is required to run this script from the repository folder directly.
2) The Conan package tests are automatically run during package creation.
The directory test_package contains a test project that is built to validate the proper package creation.
```bash
# Create the Conan package locally
python scripts/conan/create.py
# Build and install the test package executable
python test_package/build.py
# Run the test package executable
$PWD/install/test_package/teiacare_inference_client_test_package
```
### Artifactory Upload
In order to upload a Conan package to TeiaCare Artifactory server it is required to setup you local Conan client once with the following commands:
```bash
# Add TeiaCare Artifactory remote to local Conan client
conan remote add teiacare $(artifactory.url)/teiacare
# Authenticate with Artifactory credentials
conan user $(artifactory.username) -p $(artifactory.password) -r teiacare
```
Now it is possible to create and upload a Conan package with the following commands:
```bash
# Create the Conan package locally
python scripts/conan/create.py
# Upload the package to Artifactory on the teicare remote
python scripts/conan/upload.py teiacare teiacare_inference_client
```
## Contributing
In order to contribute to TeiaCareInferenceClient, please follow our [contribution guidelines](./CONTRIBUTING).
[](./CONTRIBUTING)
## License
This project is licensed under the [Apache License, Version 2.0](./LICENSE).
Copyright © 2024 [TeiaCare](https://teiacare.com/)
[](./LICENSE)