{"id":19733972,"url":"https://github.com/scaleoutsystems/tee-serve","last_synced_at":"2026-03-19T11:20:47.360Z","repository":{"id":104982597,"uuid":"431061048","full_name":"scaleoutsystems/tee-serve","owner":"scaleoutsystems","description":"Lightweight model serving in Trusted Execution Environments.","archived":false,"fork":false,"pushed_at":"2022-01-12T13:37:19.000Z","size":75,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-01-10T18:23:31.542Z","etag":null,"topics":["ai","mlops","modelserving","tee"],"latest_commit_sha":null,"homepage":"https://medium.com/p/f17f26a65646","language":"Dockerfile","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/scaleoutsystems.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-11-23T10:43:24.000Z","updated_at":"2022-12-28T22:03:45.000Z","dependencies_parsed_at":null,"dependency_job_id":"f438574d-4727-4c43-abe0-5e6374b7d585","html_url":"https://github.com/scaleoutsystems/tee-serve","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scaleoutsystems%2Ftee-serve","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scaleoutsystems%2Ftee-serve/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scaleoutsystems%2Ftee-serve/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scaleoutsystems%2Ftee-serve/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/scaleoutsystems","download_url":"https://codeload.github.com/scaleoutsystems/tee-serve/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241059015,"owners_count":19902296,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","mlops","modelserving","tee"],"created_at":"2024-11-12T00:34:49.636Z","updated_at":"2026-03-03T07:36:19.950Z","avatar_url":"https://github.com/scaleoutsystems.png","language":"Dockerfile","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Lightweight model serving in TEE\nLightweight model serving in [Intel SGX TEE](https://www.intel.com/content/www/us/en/developer/tools/software-guard-extensions/overview.html) using [Gramine](https://grapheneproject.io) and [TensorFlow lite](https://www.tensorflow.org/lite) C++ API. For more details please refer to the companion blog post: https://medium.com/p/f17f26a65646.\n\n## Table of contents\n- [Lightweight model serving in TEE](#lightweight-model-serving-in-tee)\n  - [Table of contents](#table-of-contents)\n  - [Quickstart](#quickstart)\n  - [Building the binaries](#building-the-binaries)\n  - [Running in direct mode (without SGX)](#running-in-direct-mode-without-sgx)\n  - [Running in SGX](#running-in-sgx)\n  - [Getting access to SGX](#getting-access-to-sgx)\n\n## Quickstart\n\nThe only prerequisite to run this proof of concept is https://docker.io. Once you have installed docker you can clone this repository, locate into it and launch the environment as if follows.\n\n```console\n$ bin/launch.sh\n```\n\nThis is going to build and start a container with all of the required dependencies to build and run the codebase.\n\n## Building the binaries\n\nTo build the `server` and the `client` binaries you can run:\n\n```console\n$ bin/build.sh\n```\n\nThis command is going to build the binaries, sign the `server` and generate other necessary Gramine files.\n\n## Running in direct mode (without SGX)\nThe `gramine-direct` command can be used for testing purposed (or if a SGX machine is not available). To start the server in this manner you can run:\n\n```console\n$ gramine-direct server resources/plain/model.tflite\n```\n\n\u003e **Note** `server` is the server executable and `resources/plain/model.tflite` is a TensorFlow lite model.\n\nTo test the server you can open a new console and run e.g. `./client 0.5`.\n\n## Running in SGX\nThe `gramine-sgx` command can be used run the server in the SGX enclave as it follows.\n\n```console\n$ gramine-sgx server resources/model.tflite\n```\n\n\u003e **Note** `server` is the server executable and `resources/model.tflite` is an encrypted TensorFlow lite model.\n\nTo test the server you can open a new console and run e.g. `./client 0.5`.\n\n## Getting access to SGX\nAzure offers Intel SGX instances that you can get on demand: https://azure.microsoft.com/en-us/solutions/confidential-compute.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscaleoutsystems%2Ftee-serve","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fscaleoutsystems%2Ftee-serve","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscaleoutsystems%2Ftee-serve/lists"}