{"id":23557051,"url":"https://github.com/verisilicon/tflite-vx-delegate","last_synced_at":"2025-04-09T09:09:44.241Z","repository":{"id":37102698,"uuid":"345848444","full_name":"VeriSilicon/tflite-vx-delegate","owner":"VeriSilicon","description":"Tensorflow Lite external delegate based on TIM-VX","archived":false,"fork":false,"pushed_at":"2025-01-07T10:05:27.000Z","size":291,"stargazers_count":47,"open_issues_count":10,"forks_count":24,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-04-02T07:11:18.627Z","etag":null,"topics":["deep-learning","tensorflow","tflite","tflite-delegate"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/VeriSilicon.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-03-09T01:40:17.000Z","updated_at":"2025-03-11T02:28:54.000Z","dependencies_parsed_at":"2024-01-04T10:38:55.472Z","dependency_job_id":"af62acb8-6860-420e-b556-a584fd0b7684","html_url":"https://github.com/VeriSilicon/tflite-vx-delegate","commit_stats":{"total_commits":173,"total_committers":13,"mean_commits":"13.307692307692308","dds":0.6589595375722543,"last_synced_commit":"7c582a4d507cc7bc4508969533a94ad9bebf81b1"},"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VeriSilicon%2Ftflite-vx-delegate","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VeriSilicon%2Ftflite-vx-delegate/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VeriSilicon%2Ftflite-vx-delegate/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VeriSilicon%2Ftflite-vx-delegate/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/VeriSilicon","download_url":"https://codeload.github.com/VeriSilicon/tflite-vx-delegate/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248008630,"owners_count":21032556,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","tensorflow","tflite","tflite-delegate"],"created_at":"2024-12-26T14:19:08.740Z","updated_at":"2025-04-09T09:09:44.221Z","avatar_url":"https://github.com/VeriSilicon.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# TfLite-vx-delegate\nTfLite-vx-delegate constructed with TIM-VX as an openvx delegate for tensorflow lite. Before vx-delegate, you may have nnapi-linux version from VeriSilicon, we suggest you move to this new delegate because:\n\n    1. without nnapi, it's flexible to enable more AI operators.\n    2. vx-delegate is opensourced, and will promised compatible with latest tensorflow release(currently v2.14.0).\n# Use tflite-vx-delegate\n\n## Prepare source code\n```sh\nmkdir wksp \u0026\u0026 cd wksp\n# tim-vx is optional, it will be downloaded by CMake automatically for none-cross build\n# if you want to do cross build with cmake, you have to build tim-vx firstly\ngit clone https://github.com/VeriSilicon/TIM-VX.git tim-vx\ngit clone https://github.com/VeriSilicon/tflite-vx-delegate.git\n# tensorflow is optional, it will be downloaded automatically if not present\ngit clone https://github.com/tensorflow/tensorflow.git\n```\n# Build from source with cmake\n\n```sh\n# default built for x86-64 simulator\ncd tflite-vx-delegate\nmkdir build \u0026\u0026 cd build\ncmake ..\nmake vx_delegate -j12\n\n# benchmark_model\nmake benchmark_model -j12\n# label_image\nmake label_image -j12\n```\nIf you would like to build with your own vivante driver sdk and tim-vx build, you need do cross-build as\n```sh\ncd tim-vx\nmkdir build \u0026\u0026 cd build\ncmake .. -DCMAKE_TOOLCHAIN_FILE=\u003ctoolchain.cmake\u003e -DEXTERNAL_VIV_SDK=\u003csdk_root\u003e\n# we can also build from a specific ovxlib instead of use default one by set\n# TIM_VX_USE_EXTERNAL_OVXLIB=ON\n# OVXLIB_INC=\u003cdirectory_to_ovxlib_include\u003e\n# OVXLIB_LIB=\u003cfull_path_to_libovxlib.so\u003e\n```\n\nIf you would like to build using local version of tensorflow, you can use `FETCHCONTENT_SOURCE_DIR_TENSORFLOW` cmake variable. Point this variable to your tensorflow tree. For additional details on this variable please see the [official cmake documentation](https://cmake.org/cmake/help/latest/module/FetchContent.html#command:fetchcontent_populate)\n\n``` sh\ncmake -DFETCHCONTENT_SOURCE_DIR_TENSORFLOW=/my/copy/of/tensorflow \\\n    -DOTHER_CMAKE_DEFINES...\\\n    ..\n```\nAfter cmake execution completes, build and run as usual. Beware that cmake process will apply a patch to your tensorflow tree. The patch is requred to enable the external delegate support and the NBG support.\n\n## Enable external delegate support in benchmark_model/label_image\n\nFor tensorflow v2.8.0, addtional patch `pwd`/patches/0001-TensorFlow-V280-Enable-External-Delegate.patch requred to enable enable external delegate in benchmark_model/label_image. For higher versions of TensorFlow, the benchmark_model has automatically enabled the external delegate mechanism, but it is still necessary to apply patch `pwd`/patches/label_image_support.patch to enable the external delegate in label_image.\nIf tensorflow source code downloaded by cmake, you can find it in \u003cbuild_output_dir\u003e/_deps/tensorflow-src\n\nThe patch get merged into Tensorflow master branch, no patch required for master branch.\n\n## benchmark_model/label_image compatible with Tflite+NBG\nWith our Acuity Toolkit, you can generate tflite file with compiled NBG(**N**etwork **B**inary **G**raph) as a custom operator. To support this special format, you should build benchmark_model/label_image from our delegate repo and not use the offical one.\n\n## Run\n```sh\n# For default x86 build, you can find prebuilt sdk from tim-vx\n# export VSIMULATOR_CONFIG=\u003cyour_target_npu_id\u003e for x86-simulator\nexport VIVANTE_SDK_DIR=\u003cdirect_to_sdk_root\u003e\n# Please copy libtim-vx.so to drivers/ directory\nexport LD_LIBRARY_PATH=${VIVANTE_SDK_DIR}/drivers:$LD_LIBRARY_PATH # the \"drivers\" maybe named as lib\n./benchmark_model --external_delegate_path=\u003cpath_to_libvx_delegate.so\u003e --graph=\u003ctflite_model.tflite\u003e\n# If you would like to use cache mode which save and load binary graph in local disk\n./benchmark_model --external_delegate_path=\u003cpath_to_libvx_delegate.so\u003e \\\n                  --external_delegate_options='allowed_cache_mode:true;cache_file_path:\u003ccache_file\u003e' \\\n                  --graph=\u003ctflite_model.tflite\u003e\n```\n\n## Test\nIntroduced unit test with tensorflow keras api and convert it to tflite with quantized or none-quantized model,\nGolden generated from CPU implementation of tflite\n[Details for run test](./test/python/README.md)\n\n[Model verification script](./test/python/run_model.py) to compare NPU result with CPU result\n\n# Examples\nexamples/python/label_image.py\nmodified based on [offical label_image](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/examples/python/label_image.py)\n\n    1. build tensorflow-lite runtime python package follow by [offical build instruction](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/tools/pip_package#readme)\n    2. Added \"-e\" option to provide external provider, [Offical Label Image Instruction](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/examples/python/README.md)\n\nexamples/minimal\nmodified based on [offical minimal](https://cs.opensource.google/tensorflow/tensorflow/+/master:tensorflow/lite/examples/minimal/)\n\n```sh\nminimal \u003cpath_to_libvx_delegate.so\u003e \u003ctflite_model.tflite\u003e\n# If you would like to use cache mode which save and load binary graph in local disk\nminimal \u003cpath_to_libvx_delegate.so\u003e \u003ctflite_model.tflite\u003e use_cache_mode \u003ccache_file\u003e\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fverisilicon%2Ftflite-vx-delegate","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fverisilicon%2Ftflite-vx-delegate","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fverisilicon%2Ftflite-vx-delegate/lists"}