{"id":15640375,"url":"https://github.com/robmarkcole/tensorflow-lite-rest-server","last_synced_at":"2025-07-09T14:34:30.518Z","repository":{"id":38397367,"uuid":"214781883","full_name":"robmarkcole/tensorflow-lite-rest-server","owner":"robmarkcole","description":"Expose tensorflow-lite models via a rest API using FastAPI","archived":false,"fork":false,"pushed_at":"2024-02-06T07:39:31.000Z","size":15600,"stargazers_count":81,"open_issues_count":17,"forks_count":21,"subscribers_count":13,"default_branch":"master","last_synced_at":"2025-06-29T12:42:09.004Z","etag":null,"topics":["face-detection","fastapi","home-assistant","image-processing","machine-learning","object-detection","python","rest-api","tensorflow","tensorflow-lite","tflite"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/robmarkcole.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":"robmarkcole"}},"created_at":"2019-10-13T07:55:19.000Z","updated_at":"2025-01-12T16:16:47.000Z","dependencies_parsed_at":"2024-02-06T08:42:41.550Z","dependency_job_id":"0992b1fa-b802-4b68-ba1d-ba935b9c412d","html_url":"https://github.com/robmarkcole/tensorflow-lite-rest-server","commit_stats":null,"previous_names":[],"tags_count":11,"template":false,"template_full_name":null,"purl":"pkg:github/robmarkcole/tensorflow-lite-rest-server","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/robmarkcole%2Ftensorflow-lite-rest-server","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/robmarkcole%2Ftensorflow-lite-rest-server/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/robmarkcole%2Ftensorflow-lite-rest-server/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/robmarkcole%2Ftensorflow-lite-rest-server/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/robmarkcole","download_url":"https://codeload.github.com/robmarkcole/tensorflow-lite-rest-server/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/robmarkcole%2Ftensorflow-lite-rest-server/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":264476513,"owners_count":23614514,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["face-detection","fastapi","home-assistant","image-processing","machine-learning","object-detection","python","rest-api","tensorflow","tensorflow-lite","tflite"],"created_at":"2024-10-03T11:35:13.832Z","updated_at":"2025-07-09T14:34:30.482Z","avatar_url":"https://github.com/robmarkcole.png","language":"Jupyter Notebook","funding_links":["https://github.com/sponsors/robmarkcole"],"categories":[],"sub_categories":[],"readme":"# tensorflow-lite-rest-server\nExpose tensorflow-lite models via a rest API. Currently object, face \u0026 scene detection is supported. Can be hosted on any of the common platforms including RPi, linux desktop, Mac and Windows. A service can be used to have the server run automatically on an RPi.\n\n## Setup\nIn this process we create a virtual environment (venv), then install tensorflow-lite [as per these instructions](https://www.tensorflow.org/lite/guide/python) which is platform specific, and finally install the remaining requirements. **Note** on an RPi (only) it is necessary to system wide install pip3, numpy, pillow.\n\nAll instructions for mac:\n```\npython3 -m venv venv\nsource venv/bin/activate\npip3 install https://dl.google.com/coral/python/tflite_runtime-2.1.0.post1-cp37-cp37m-macosx_10_14_x86_64.whl\npip3 install -r requirements.txt\n```\n\n## Models\nFor convenience a couple of models are included in this repo and used by default. A description of each model is included in its directory. Additional models are available [here](https://github.com/google-coral/edgetpu/tree/master/test_data).\n\nIf you want to create custom models, there is the easy way, and the longer but more flexible way. The easy way is to use [teachablemachine](https://teachablemachine.withgoogle.com/train/image), which I have done in this repo for the dogs-vs-cats model. The teachablemachine service is limited to image classification but is very straightforward to use. The longer way allows you to use any neural network architecture to produce a tensorflow model, which you then convert to am optimized tflite model. An example of this approach is described in [this article](https://towardsdatascience.com/inferences-from-a-tf-lite-model-transfer-learning-on-a-pre-trained-model-e16e7c5f0ee6), or jump straight [to the code](https://github.com/arshren/TFLite/blob/master/Transfer%20Learning%20with%20TFLite-Copy1.ipynb).\n\n## Usage\nStart the tflite-server on port 5000 :\n```\n(venv) $ uvicorn tflite-server:app --reload --port 5000 --host 0.0.0.0\n```\n\nOr via Docker:\n\n```\n# Build\ndocker build -t tensorflow-lite-rest-server .\n# Run\ndocker run -p 5000:5000 tensorflow-lite-rest-server:latest\n```\n\nYou can check that the tflite-server is running by visiting `http://ip:5000/` from any machine, where `ip` is the ip address of the host (`localhost` if querying from the same machine). The docs can be viewed at `http://localhost:5000/docs`\n\nPost an image to detecting objects via cURL:\n```\ncurl -X POST \"http://localhost:5000/v1/vision/detection\" -H  \"accept: application/json\" -H  \"Content-Type: multipart/form-data\" -F \"image=@tests/people_car.jpg;type=image/jpeg\"\n```\nWhich should return:\n```\n{\n  \"predictions\": [\n    {\n      \"confidence\": 0.93359375,\n      \"label\": \"car\",\n      \"x_max\": 619,\n      \"x_min\": 302,\n      \"y_max\": 348,\n      \"y_min\": 120\n    },\n    {\n      \"confidence\": 0.7890625,\n      \"label\": \"person\",\n      \"x_max\": 363,\n      \"x_min\": 275,\n      \"y_max\": 323,\n      \"y_min\": 126\n    },\n.\n.\n.\n'success': True}\n```\nAn example request using the python requests package is in `tests/live-test.py`\n\n## Additional models\n\nIf you would like to serve additional models to the 3 that are shipped out-of-the-box with this project, you can do it adding an `additional` folder to the `models` one.\n\nYou can then ask for predictions to the additonal models using this  for `detection`:\n\n```\ncurl -X POST \"http://localhost:5000/v1/detection/{model_name}\" -H  \"accept: application/json\" -H  \"Content-Type: multipart/form-data\" -F \"image=........;type=image/jpeg\"\n```\n\nreplacing `{model_name}` with the folder name where are store the `model.tflite` and optionally the `labels.txt` files.\n\nIf you would like instead ask for a prediction to a `classification` model, the `curl` request template is:\n```\ncurl -X POST \"http://localhost:5000/v1/classification/{model_name}\" -H  \"accept: application/json\" -H  \"Content-Type: multipart/form-data\" -F \"image=........;type=image/jpeg\"\n```\n\n\n## Add tflite-server as a service\nYou can run tflite-server as a [service](https://www.raspberrypi.org/documentation/linux/usage/systemd.md), which means tflite-server will automatically start on RPi boot, and can be easily started \u0026 stopped. Create the service file in the appropriate location on the RPi using: ```sudo nano /etc/systemd/system/tflite-server.service```\n\nEntering the following (adapted for your `tflite-server.py` file location and args):\n```\n[Unit]\nDescription=Rest API exposing tensorflow lite models\nAfter=network.target\n\n[Service]\nExecStart=/home/pi/.local/bin/uvicorn tflite-server:app --reload --port 5000 --host 0.0.0.0\nWorkingDirectory=/home/pi/github/tensorflow-lite-rest-server\nStandardOutput=inherit\nStandardError=inherit\nRestart=always\nUser=pi\n\n[Install]\nWantedBy=multi-user.target\n```\n\nOnce this file has been created you can to start the service using:\n```sudo systemctl start tflite-server.service```\n\nView the status and logs with:\n```sudo systemctl status tflite-server.service```\n\nStop the service with:\n```sudo systemctl stop tflite-server.service```\n\nRestart the service with:\n```sudo systemctl restart tflite-server.service```\n\nYou can have the service auto-start on rpi boot by using:\n```sudo systemctl enable tflite-server.service```\n\nYou can disable auto-start using:\n```sudo systemctl disable tflite-server.service```\n\n## Deepstack, Home Assistant \u0026 UI\nThis API can be used as a drop in replacement for [deepstack object detection](https://github.com/robmarkcole/HASS-Deepstack-object) and [deepstack face detection](https://github.com/robmarkcole/HASS-Deepstack-face) (configuring `detect_only: True`) in Home Assistant. I also created a UI for viewing the predictions of the object detection model [here](https://github.com/robmarkcole/deepstack-ui).\n\n## FastAPI vs Flask\nThe master branch is using FastAPI whilst I am also maintaining a Flask branch. FastAPI offers a few nice features like less boilerplate code and auto generated docs. FastAPI is also widely considered to be faster than Flask.\n\n| Platform                             | Speed | Number of predictions |\n| :----------------------------------- | :---: | --------------------: |\n| Mac Pro with tflite-server (fastapi) |  2.2  |                   178 |\n| Mac Pro with tflite-server (flask)   |  2.1  |                   176 |\n| RPi4 with tflite-server (fastapi)    |  7.6  |                   176 |\n| RPi4 with tflite-server (flask)      |  8.9  |                   159 |\n\n## Development\nI am developing on a mac/pi4 using VScode, with black formatting \u0026 line length 120.\n\n* First time only, create venv: `python3.7 -m venv venv`\n* Activate venv: `source venv/bin/activate`\n* Install the dev requirements: `pip3 install -r requirements.txt` \u0026 `pip3 install -r requirements-dev.txt`\n* Sort requirements: `venv/bin/isort tflite-server.py`\n* Black format: `venv/bin/black tflite-server.py`","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frobmarkcole%2Ftensorflow-lite-rest-server","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frobmarkcole%2Ftensorflow-lite-rest-server","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frobmarkcole%2Ftensorflow-lite-rest-server/lists"}