{"id":13628688,"url":"https://github.com/merrymercy/tvm-mali","last_synced_at":"2025-04-17T04:32:17.818Z","repository":{"id":114457833,"uuid":"117677477","full_name":"merrymercy/tvm-mali","owner":"merrymercy","description":"Optimizing Mobile Deep Learning on ARM GPU with TVM","archived":true,"fork":false,"pushed_at":"2018-10-15T08:59:20.000Z","size":345,"stargazers_count":181,"open_issues_count":2,"forks_count":27,"subscribers_count":12,"default_branch":"master","last_synced_at":"2025-04-16T12:14:23.990Z","etag":null,"topics":["arm","deep-learning","mali","opencl","tvm"],"latest_commit_sha":null,"homepage":"http://tvmlang.org/2018/01/16/opt-mali-gpu.html","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/merrymercy.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2018-01-16T11:53:13.000Z","updated_at":"2025-03-23T16:05:43.000Z","dependencies_parsed_at":"2023-06-08T06:00:48.972Z","dependency_job_id":null,"html_url":"https://github.com/merrymercy/tvm-mali","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/merrymercy%2Ftvm-mali","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/merrymercy%2Ftvm-mali/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/merrymercy%2Ftvm-mali/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/merrymercy%2Ftvm-mali/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/merrymercy","download_url":"https://codeload.github.com/merrymercy/tvm-mali/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249315981,"owners_count":21249868,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["arm","deep-learning","mali","opencl","tvm"],"created_at":"2024-08-01T22:00:55.784Z","updated_at":"2025-04-17T04:32:14.380Z","avatar_url":"https://github.com/merrymercy.png","language":"C","funding_links":[],"categories":["\u003ca name=\"Tools\"\u003e\u003c/a\u003e13. Tools","\u003ca name=\"Tools\"\u003e\u003c/a\u003e9. Tools"],"sub_categories":["13.7 Deployment"],"readme":"# Note: The data and scripts here are all stale. Please go to https://github.com/dmlc/tvm/wiki/Benchmark#mobile-gpu For the latest results.\n\n\u003cp\u003e\u003cbr\u003e\u003c/p\u003e\n\u003cp\u003e\u003cbr\u003e\u003c/p\u003e\n\u003cp\u003e\u003cbr\u003e\u003c/p\u003e\n\u003cp\u003e\u003cbr\u003e\u003c/p\u003e\n\u003cp\u003e\u003cbr\u003e\u003c/p\u003e\n\u003cp\u003e\u003cbr\u003e\u003c/p\u003e\n\u003cp\u003e\u003cbr\u003e\u003c/p\u003e\n\n# Benchmarking Deep Neural Networks on ARM CPU/GPU\n\nThis repo is the supporting material for [Optimizing Mobile Deep Learning on ARM GPU with TVM](http://tvmlang.org/2018/01/16/opt-mali-gpu.html)\n\n## Inference Speed on ImageNet\nTested on \n```\nFirefly-RK3399 4G, CPU: dual-core Cortex-A72 + quad-core Cortex-A53, GPU: Mali-T860MP4\nArm Compute Library: v17.12,  MXNet: v1.0.1,  Openblas: v0.2.18\n```\n\n![result](results.png)\n\n \n## Set Test Environment\n```\nsudo /etc/init.d/lightdm stop\nsudo -i\necho performance \u003e /sys/class/misc/mali0/device/devfreq/ff9a0000.gpu/governor\n```\nThis can make the environment more stable.\n\n**Note**: You need more than 2.5GB of memory to run the following test.\nOtherwise, you must skip the test of vgg16 by replacing `--model all` with `--model resnet18` or `--model mobilenet`\nin the commond.\n\n## Run Test for TVM/NNVM\nIn TVM, we use [RPC](http://nnvm.tvmlang.org/tutorials/deploy_model_on_mali_gpu.html) to do test,\nso you should build TVM runtime and start a RPC server on your device.\n```\npython -m tvm.exec.rpc_server --host 0.0.0.0 --port=9090\n```\n\nThen in your host machine, run the test commond\n``` bash\npython mali_imagenet_bench.py --target-host TARGET_HOST --host HOST --port PORT --model all\n```\nReplace the `TARGET_HOST`, `HOST` and `PORT` with the corresponding values in your environment.\n\nFor example, on my Firefly-RK3399, the commond is \n``` bash\npython mali_imagenet_bench.py --target-host 'llvm -target=aarch64-linux-gnu -mattr=+neon' --host 10.42.0.96 --port 9090 --model all\n```\n\n## Run Test for MXNet + Openblas\nThis test is executed locally on your device. So you need install the mxnet with openblas on your device first.\n\n``` bash\npython mxnet_test.py --model all\n```\n\n## Run Test for Arm Compute Library\nBuild ACL by cross-compile on host system.\n``` bash\nscons Werror=1 neon=1 opencl=1 examples=1 benchmark_tests=1 os=linux arch=arm64-v8a embed_kernels=1 -j$(nproc)\n```\n\ncopy acl\\_test.cc to the root directoy of ACL and build the acl\\_test by\n``` bash\naarch64-linux-gnu-g++ acl_test.cc build/utils/*.o -O2 -std=c++11\\\n    -I. -Iinclude -Lbuild -Lbuild/opencl-1.2-stubs/\\\n     -larm_compute -larm_compute_graph -larm_compute_core -lOpenCL -o acl_test\n```\n\ncopy the binary file acl\\_test to your device and run\n```\n./acl_test all\ncat result-acl.txt\n```\nresults are recored in `result-acl.txt`\n\n**Note** Some testcases (e.g. resnet) are missing because Arm Compute Library currently (v17.12) does not \nsupport skip connection in its graph runtime. Also some testcases are too slow so that be skipped.\n\n## Result\nPaste the outputs on my board here.\n\n### TVM/NNVM\n```\n============================================================\nmodel: vgg16, dtype: float32\nwarm up..\ntest..\ncost per image: 1.2926s\n============================================================\nmodel: vgg16, dtype: float16\nwarm up..\ntest..\ncost per image: 0.6896s\n============================================================\nmodel: resnet18, dtype: float32\nwarm up..\ntest..\ncost per image: 0.2041s\n============================================================\nmodel: resnet18, dtype: float16\nwarm up..\ntest..\ncost per image: 0.1183s\n============================================================\nmodel: mobilenet, dtype: float32\nwarm up..\ntest..\ncost per image: 0.0767s\n============================================================\nmodel: mobilenet, dtype: float16\nwarm up..\ntest..\ncost per image: 0.0479s\n```\n\n### MXNet + Openblas\n```\n============================================================\nmodel: vgg16, dtype: float32\nwarm up...\ntest..\ncost per image: 3.0250s\n============================================================\nmodel: resnet18, dtype: float32\nwarm up...\ntest..\ncost per image: 0.3977s\n============================================================\nmodel: mobilenet, dtype: float32\nwarm up...\ntest..\ncost per image: 0.2914s\n```\n\n### ACL\n```\nbackend: cl    model: vgg16      conv_method: gemm     dtype: float32   cost: 1.64456\nbackend: cl    model: vgg16      conv_method: gemm     dtype: float16   cost: 0.969372\nbackend: cl    model: vgg16      conv_method: direct   dtype: float32   cost: 3.90031\nbackend: cl    model: vgg16      conv_method: direct   dtype: float16   cost: 1.61179\nbackend: cl    model: mobilenet  conv_method: gemm     dtype: float32   cost: 0.170934\nbackend: cl    model: mobilenet  conv_method: direct   dtype: float32   cost: 0.173883\nbackend: neon  model: vgg16      conv_method: gemm     dtype: float32   cost: 4.10269\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmerrymercy%2Ftvm-mali","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmerrymercy%2Ftvm-mali","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmerrymercy%2Ftvm-mali/lists"}