{"id":13583963,"url":"https://github.com/robmarkcole/coral-pi-rest-server","last_synced_at":"2025-04-15T11:07:44.024Z","repository":{"id":43093111,"uuid":"176674493","full_name":"robmarkcole/coral-pi-rest-server","owner":"robmarkcole","description":"Perform inferencing of tensorflow-lite models on an RPi with acceleration from Coral USB stick","archived":false,"fork":false,"pushed_at":"2023-10-26T13:25:17.000Z","size":25167,"stargazers_count":69,"open_issues_count":27,"forks_count":20,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-04-15T11:07:20.994Z","etag":null,"topics":["deep-learning","object-detection","python","raspberry-pi","ssd","tensorflow","tensorflow-lite"],"latest_commit_sha":null,"homepage":"https://coral.ai/products/accelerator","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/robmarkcole.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null},"funding":{"github":"robmarkcole"}},"created_at":"2019-03-20T07:03:46.000Z","updated_at":"2025-01-21T19:04:36.000Z","dependencies_parsed_at":"2024-01-24T13:10:03.868Z","dependency_job_id":null,"html_url":"https://github.com/robmarkcole/coral-pi-rest-server","commit_stats":null,"previous_names":[],"tags_count":13,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/robmarkcole%2Fcoral-pi-rest-server","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/robmarkcole%2Fcoral-pi-rest-server/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/robmarkcole%2Fcoral-pi-rest-server/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/robmarkcole%2Fcoral-pi-rest-server/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/robmarkcole","download_url":"https://codeload.github.com/robmarkcole/coral-pi-rest-server/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249058370,"owners_count":21205910,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","object-detection","python","raspberry-pi","ssd","tensorflow","tensorflow-lite"],"created_at":"2024-08-01T15:03:55.704Z","updated_at":"2025-04-15T11:07:43.999Z","avatar_url":"https://github.com/robmarkcole.png","language":"Jupyter Notebook","funding_links":["https://github.com/sponsors/robmarkcole"],"categories":["Jupyter Notebook"],"sub_categories":[],"readme":"## coral-pi-rest-server\nPerform inference using tensorflow-lite deep learning models with hardware acceleration provided by a Coral usb accelerator running on a raspberry pi or linux/mac. The models are exposed via a REST API allowing inference over a network. To run the app with default model:\n```\n$ python3 coral-app.py --models-directory models\n```\nThen use curl to query:\n```\ncurl -X POST -F image=@images/test-image3.jpg -F min_confidence=.5 'http://localhost:5000/v1/vision/detection'\n\n{'predictions': [{'confidence': 0.953125,\n   'label': 'person',\n   'x_max': 918,\n   'x_min': 838,\n   'y_max': 407,\n   'y_min': 135},\n  {'confidence': 0.58203125,\n   'label': 'person',\n   'x_max': 350,\n   'x_min': 226,\n   'y_max': 374,\n   'y_min': 143}]\n 'success': True}\n```\n\nTo see the help run:\n```\n$ python3 coral-app.py -h\n```\n\nSee the [coral-app-usage.ipynb](https://github.com/robmarkcole/coral-pi-rest-server/blob/master/coral-app-usage.ipynb) Jupyter notebook for usage from python.\n\n**Warning**: it has been quite frustrating working with this hardware, with many tedious setup problems, hardware timeouts. Therefore I created a fork of this project that does not require a Coral at https://github.com/robmarkcole/tensorflow-lite-rest-server On an RPI 4 inference times are very fast even without a coral.\n\n## Models\nIf you have installed the raspberry pi disk images from edgetpu-platforms then you already have all the models in `home/pi/all_models`. If you are using a mac/linux desktop you can download the models [from here](https://github.com/google-coral/edgetpu/tree/master/test_data). It is [also possible to train your own models](https://coral.withgoogle.com/tutorials/edgetpu-models-intro/) -\u003e try using Google Colaboratory as the free environment for training or -\u003e https://cloud-annotations.github.io/training/object-detection/cli/index.html\n\n## Pi setup on existing pi\nFollow the instructions on -\u003e https://coral.ai/docs/accelerator/get-started/#1-install-the-edge-tpu-runtime Note I had headaches with this approach, better to use the disk image.\n\n## Pi setup with disk image\nInstall one of the disk images from [edgetpu-platforms](https://github.com/google-coral/edgetpu-platforms). In the `/home/pi` directory `git clone` this repository. You wil now have a file structure like (pi3 \u0026pi4 only, pi-zero [differs](https://github.com/google-coral/edgetpu-platforms/issues/13)):\n```\n$ ls\n\nall_models  coral-pi-rest-server  edgetpu_api  examples-camera  project-posenet  project-teachable  simple-demo\n```\n\nUse the `cd` command to enter `~/simple-demo` and test you installation:\n```\npi@pi:~/simple-demo $ python3 classify_image.py --model /home/pi/all_models/mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite --label  /home/pi/all_models/inat_bird_labels.txt --image  parrot.jpg\n\nINFO: Initialized TensorFlow Lite runtime.\n---------------------------\nAra macao (Scarlet Macaw)\nScore :  0.76171875\n```\n\nAssuming you saw the result above, use the `cd` command to enter `~/coral-pi-rest-server` and (system wide, no virtual environment) install the required dependencies:\n```\n~/coral-pi-rest-server $ pip3 install -r requirements.txt\n```\nNow run the app:\n```\n~/coral-pi-rest-server $ python3 coral-app.py\n```\n### Service\nYou can run the app as a [service](https://www.raspberrypi.org/documentation/linux/usage/systemd.md). Edit `coral.service` file to fit your needs and copy into `/etc/systemd/system` as root using `sudo cp coral.service /etc/systemd/system/coral.service`. Once this has been copied, you can attempt to start the service using `sudo systemctl start coral.service`. See the status and logs with `sudo systemctl status coral.service`. Stop the service with `sudo systemctl stop coral.service`. You can have it start automatically on reboot by using `sudo systemctl enable coral.service`.\n\n### Pi 3\nI am running a pi3 with the raspi camera below. FYI the camera is mounted on a [pan-tilt stage](https://shop.pimoroni.com/products/pan-tilt-hat).\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"https://github.com/robmarkcole/coral-pi-rest-server/blob/master/images/my_setup.png\" width=\"500\"\u003e\n\u003c/p\u003e\n\nUsing the pi3 (which has only usb2) processing a single request from another machine via this server takes ~ 300 to 500 ms. Therefore this is certainly fast enough to process images at 1 FPS which in my experience is suitable for tasks such as counting people in a room.\n\n### Pi 4\nThe pi4 has USB3 so we would expect better speeds. However I get inference time consistently ~ 500 ms when querying from another machine. Therefore it appears the coral library does not yet take advantage of the USB3.\n\n## Pi-zero\nI found that I get inference time in the range 2.5 to 5s when querying from another machine, so significantly slower than the pi3/pi4.\n\n## Power\nI recommend using a power supply which can supply 3 Amps, [I use this one](https://www.amazon.co.uk/gp/product/B017YW2CKM/ref=ppx_yo_dt_b_asin_title_o00_s00?ie=UTF8\u0026psc=1). Note that the official RPI supply delivers only 2.5 Amp and I found this was causing an issue where the stick would 'go to sleep' after a day of continual use.\n\n## Mac\nTo use the coral with a mac follow the official install instructions [here](https://coral.ai/docs/accelerator/get-started/#on-mac)\n\n## No coral?\nI created a fork of this project that does not require a Coral at https://github.com/robmarkcole/tensorflow-lite-rest-server\n\n## Deepstack \u0026 Home Assistant\nThe data returned by the app is as close as possible in format to that returned by Deepstack object detection endpoint, allowing you to use this app as the backend for [HASS-Deepstack-object](https://github.com/robmarkcole/HASS-Deepstack-object). There is also support for running as an addon via [github.com/grinco/HASS-coral-rest-api](https://github.com/grinco/HASS-coral-rest-api)\n\n## Troubleshooting\n-----------------------------\nQ: I get the error: `HandleQueuedBulkIn transfer in failed. Not found: USB transfer error 5 [LibUsbDataInCallback]`\nA: I reflashed the SD card and tried again with success\n\nQ: I get error `ValueError: Failed to load delegate from libedgetpu.1.dylib`\nA: libedgetpu is not in the expected location\n\n## References\n* https://github.com/google-coral\n* [Using the official pi camera with Coral](https://github.com/nickoala/edgetpu-on-pi)\n* https://github.com/snmsung716/Following-camera-with-Coral-Accelerator-on-Raspberry-Pi\n* https://github.com/lemariva/raspbian-EdgeTPU\n* [Pyimagesearch article on Coral](https://www.pyimagesearch.com/2019/04/22/getting-started-with-google-corals-tpu-usb-accelerator/)\n* [Hands on with coral + docker](https://lemariva.com/blog/2019/04/edge-tpu-coral-usb-accelerator-dockerized)\n* [Official Raspicam example](https://github.com/google-coral/examples-camera/blob/master/raspicam/classify_capture.py)\n\n## Credit\nI forked the code in this excellent article -\u003e https://blog.keras.io/building-a-simple-keras-deep-learning-rest-api.html [code](https://github.com/jrosebr1/simple-keras-rest-api)\n\n## Development Mac\nNote: important to use USB-C to USB-C cable. Had some connection issues initially documented in [this issue](https://github.com/google-coral/pycoral/issues/35)\n\n* `python3 -m venv venv`\n* `source venv/bin/activate`\n* `pip install -r requirements.txt`\n* `python3 -m pip install --index-url https://google-coral.github.io/py-repo/ --extra-index-url=https://pypi.python.org/simple pycoral`\n* `python3 coral-app.py`\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frobmarkcole%2Fcoral-pi-rest-server","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frobmarkcole%2Fcoral-pi-rest-server","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frobmarkcole%2Fcoral-pi-rest-server/lists"}