https://github.com/tum-ei-eda/utvm_staticrt_codegen

This project contains a code generator that produces static C NN inference deployment code targeting tiny micro-controllers (TinyML) as replacement for other µTVM runtimes. This tools generates a runtime, which statically executes the compiled model. This reduces the overhead in terms of code size and execution time compared to having a dynamic on-device runtime.
https://github.com/tum-ei-eda/utvm_staticrt_codegen

tvm

Last synced: about 1 month ago
JSON representation

This project contains a code generator that produces static C NN inference deployment code targeting tiny micro-controllers (TinyML) as replacement for other µTVM runtimes. This tools generates a runtime, which statically executes the compiled model. This reduces the overhead in terms of code size and execution time compared to having a dynamic on-device runtime.

• Host: GitHub
• URL: https://github.com/tum-ei-eda/utvm_staticrt_codegen
• Owner: tum-ei-eda
• License: apache-2.0
• Created: 2021-01-27T08:20:45.000Z (over 4 years ago)
• Default Branch: master
• Last Pushed: 2021-09-22T09:50:54.000Z (about 4 years ago)
• Last Synced: 2025-09-01T16:58:15.576Z (about 1 month ago)
• Topics: tvm
• Language: C
• Homepage:
• Size: 643 KB
• Stars: 30
• Watchers: 3
• Forks: 9
• Open Issues: 1
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# µTVM StaticRT CodeGen

This project contains a code generator that produces static C NN inference deployment code targeting tiny micro-controllers (TinyML) as replacement for other µTVM runtimes. This tools generates a runtime, which statically executes the compiled model.  This reduces the overhead in terms of code size and execution time compared to having a dynamic on-device runtime.

![Tool flow](doc/flow.png)

## Requirements

- Install required system packages

- Install required python packages

- Clone TVM

- Apply patches in `tvm_patches/`

- Build TVM

For detailed commands see [CI Config](.github/workflows/ci.yml).

## Building

The generator is a conventional CMake project.

    mkdir build && cd build

    cmake ..

    cmake --build .

## Usage

- Point python to the TVM installation:

      export PYTHONPATH=../tvm/python:${PYTHONPATH}

- Generate a `graph.json`, `params.bin` and `kernels.c` file for your model with µTVM. A complete example is shown in [this µTVM example script](examples/utvm_gen_graph_and_params.py).

- Inspect `kernels.c` and determine the required workspace size by looking at the `TVMBackendAllocWorkspace` calls and picking the largest used byte size. If there are no calls, the size should be zero.

- Execute the µTVM StaticRT CodeGen:

      ./utvm_staticrt_codegen graph.json params.bin staticrt.c $WORKSPACE_SIZE

- Now the `kernels.c` and `staticrt.c` can be compiled together with some calling code into a complete application. An example is given in `examples/target_src/`.

For detailed commands see [example full flow script](examples/run_flow.sh).