{"id":19293640,"url":"https://github.com/utensor/utensor_cgen","last_synced_at":"2025-05-05T18:44:50.890Z","repository":{"id":27402084,"uuid":"112056490","full_name":"uTensor/utensor_cgen","owner":"uTensor","description":"C++ code generator for uTensor https://utensor-cgen.readthedocs.io/en/latest/","archived":false,"fork":false,"pushed_at":"2023-11-25T12:55:56.000Z","size":19696,"stargazers_count":51,"open_issues_count":27,"forks_count":38,"subscribers_count":9,"default_branch":"master","last_synced_at":"2025-04-27T13:01:52.598Z","etag":null,"topics":["deep-learning","edge-computing","embedded","iot","microcontroller","python","utensor"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/uTensor.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null},"funding":{"github":null,"patreon":null,"open_collective":"utensorai","ko_fi":null,"tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"otechie":null,"custom":null}},"created_at":"2017-11-26T04:48:08.000Z","updated_at":"2025-04-21T21:07:30.000Z","dependencies_parsed_at":"2023-02-18T21:01:13.376Z","dependency_job_id":"c045d148-c4d7-4f50-bfc5-46a4f3de21f6","html_url":"https://github.com/uTensor/utensor_cgen","commit_stats":{"total_commits":663,"total_committers":13,"mean_commits":51.0,"dds":"0.16742081447963797","last_synced_commit":"eccd6859028d0b6a350dced25ea72ff02faaf9ad"},"previous_names":[],"tags_count":32,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uTensor%2Futensor_cgen","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uTensor%2Futensor_cgen/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uTensor%2Futensor_cgen/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uTensor%2Futensor_cgen/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/uTensor","download_url":"https://codeload.github.com/uTensor/utensor_cgen/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252555606,"owners_count":21767196,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","edge-computing","embedded","iot","microcontroller","python","utensor"],"created_at":"2024-11-09T22:35:35.222Z","updated_at":"2025-05-05T18:44:50.874Z","avatar_url":"https://github.com/uTensor.png","language":"Python","funding_links":["https://opencollective.com/utensorai"],"categories":[],"sub_categories":[],"readme":"readthedocs: https://utensor-cgen.readthedocs.io/en/latest/\n\n.. readme_begin\n\n.. _readme:\n\n.. _install:\n\nInstallation (Python 2 \u0026 3)\n===========================\n\nFor Users\n---------\n\n-  with ``setup.py``\n\n.. code:: console\n\n    $ python setup.py install\n\n-  with pip_\n\n.. code:: console\n\n    $ pip install utensor_cgen\n\n.. _install_dev:\n\nFor Developers\n--------------\n\n-  with pipenv_\n\n.. code:: console\n\n    # install `utensor_cgen` (develop mode)\n    $ PIPENV_VENV_IN_PROJECT=1 pipenv install -d\n\n    # spawn a subshell and activate virtualenv\n    $ pipenv shell\n\n    # get help message of `utensor-cli`\n    $ utensor-cli -h\n\nTroubleshooting with pipenv_\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\n- If you have troubles with installation using pipenv_, try\n\n  .. code:: console\n\n    $ PIPENV_VENV_IN_PROJECT=1 pipenv install -d --skip-lock\n- there is known issue of pip_ and pipenv_, plz refer to this\n  `issue \u003chttps://github.com/pypa/pipenv/issues/2924\u003e`_ for detail\n\n  -  short answer: downgrade to ``pip==18.0`` may help :)\n\n- Tensorflow_ requires ``setuptools\u003c=39.1.0`` (the latest is ``40.4.3``\n  by the time this README is writen)\n\n  - plz downgrade to ``setuptools==39.1.0``\n  - my recommendation is to use ``virtualenv``\n\nOverall Architecture\n====================\n\n\\ |utensor-cli-components|\n\nBasic Usage\n===========\n\nModel File Inspection\n---------------------\n\n.. code-block:: console\n\n  $ utensor-cli show \u003cmodel.pb\u003e\n\nShow all nodes and detailed information of given pb file or\na :class:`.uTensorGraph` pickle file\n\nRun ``utensor-cli show --help`` for detailed information.\n\nConvert Model File to C/C++ Code\n--------------------------------\n\n**IMPORTANT**: ``pb`` file is deprecated in favor of Tensorflow 2.x, please refer to `End-to-End Training with Keras`_ for detail\n\n.. code-block:: console\n\n  $ utensor-cli convert \u003cmodel.pb\u003e \\\n    --output-nodes=\u003cnode name\u003e[,\u003cnode name\u003e,...] \\\n    [--config=config.toml]\n\nConvert given pb file into cpp/hpp files.\n\nNote that ``--output-nodes`` is required options. It's the names of\nnodes you want to output, seperated by comma for multiple values.\n\nIn graph theory terminology, they are ``leaf`` nodes of your graph.\n\nUse ``--config`` to pass a configuration file to the cli, you can use ``generate-config`` command to generate one (see below).\n\nexample\n~~~~~~~\n\n.. code-block:: console\n\n  $ utensor-cli convert simple_model.pb --output-nodes=pred,logits\n\nRun ``utensor-cli convert --help`` for detailed information.\n\nConfiguration\n-------------\n\n``utensor-cli`` use ``toml`` as configuration format.\n\nYou can generate configuration file of given target as following:\n\n.. code-block:: console\n\n  $ utensor-cli generate-config --target \u003ctarget name\u003e [-o filename.toml]\n\nThis command will generate a ``toml`` file listing all configurable values with its defaults.\n\nYou can modify the value and pass the file to cli with ``--config`` flag.\n\nexample\n~~~~~~~\n\n.. code-block:: console\n\n  # generate config file\n  $ utensor-cli generate-config --target utensor -o myconfig.toml\n\n  # after editting myconfig.toml\n  $ utensor-cli convert mymodel.pb --config=myconfig.toml --output-nodes=output,...\n\nUse :mod:`utensor_cgen` as Library\n==================================\n\n.. subgraph-match-begine\n\nSubgraph Isomorphic Matcher\n---------------------------\n\nWith :class:`.uTensorGraphMatcher`, performing isomorphic subgraph matching\nalong with replacing or manipulating the matched subgraph(s) takes just a\nfew line of code:\n\n.. code-block:: python\n\n  from utensor_cgen.matcher import uTensorGraphMatcher\n\n  # `pattrn_ugraph` is the pattern to match with\n  pattrn_ugraph = ...\n  matcher = uTensorGraphMatcher(pattrn_ugraph)\n\n  # a larget graph to perform subgraph match\n  subject_ugraph = ...\n\n  # matches is a list of `uTensorGraphMatch` objects\n  matches = matcher.match_all(subject_ugraph)\n  if matches:\n    # do stuff with the matches\n\nUse Case: Node Fusion\n~~~~~~~~~~~~~~~~~~~~~\n\nNote: we'll use **operation**/**node**/**layer** interchangeably in the\ndocumentation\n\n-  It's commonly seen pattern in convolution neural network (``CNN``),\n   ``conv -\u003e relu -\u003e pooling``. That is, a 2D convolution followed by a\n   relu layer and then a pooling down sampling layer.\n-  With our :class:`.uTensorGraphMatcher`, you can locate such pattern in your\n   ``CNN`` model and fuse/replace matched nodes into one optimized\n   :class:`.QuantizedFusedConv2DMaxpool` node.\n\n  -  Left: original graph\n  -  Middle: matched convolution layer\n  -  Right: replace the matched layer with specialized\n     ``QuantizedFusedConv2DMaxpool`` node\n\n\\ |conv-pool-fuse|\n\n\nUse Case: Dropout Layer Removal\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\n-  Though ``dropout`` is an effective technique to improve training\n   performance of your model, it's not necessary during inference\n   phrase.\n-  In the mainstream frameworks such as `Tensorflow`_ or `PyTorch`_,\n   an ``dropout`` layer is typically implemented with other elementary\n   operations/nodes. As a result, finding and removing those nodes for\n   inference optimization (both in model size and prediciton time) is\n   not trivial and error prone.\n-  With our :class:`.uTensorGraphMatcher`, you can find and remove the dropout\n   nodes as illustrated in the following picture.\n\n   -  Left: original graph with dropout Layers\n   -  Middle: matched dropout layers\n   -  Right: graph with dropout layers removed\n\n\\ |cnn-dropout|\n\nWe use mainly `Tensorflow`_ for declaring the pattern graph for matcher now.\n\nHigh-level graph builder is on its way, see `Future Works \u003c#future-works\u003e`_ for detail.\n\n.. subgraph-match-end\n\n.. offline-tensor-alloc-start\n\nOffline Tensor Memory Allocation\n--------------------------------\n\nConsidering following simple multi layers perceptron (`simple_mnist.pb`_):\n\n\\ |mlp-alloc-graph|\n\nOnce enabled the optimization transformer, ``tensor_alloc``, an offline tensor memory allocation planner,\n``utensor-cli`` will generate ``uTensor`` runtime codes that use following optimized allocation plan:\n\n\\ |mlp-alloc|\n\n- y-axis: tensor names ordered by topological sorting\n- x-axis: these are the memory span occupied by each tensor, that is, the memory address offset and\nthe size of the tensor\n\n.. offline-tensor-alloc-end\n\nTutorials\n=========\n\n-  `End-to-End Training with Keras`_\n-  `Extending uTensor Backend by Adding Custom Operators \u003chttps://github.com/uTensor/utensor_cgen/tree/master/tutorials/add_custom_operators\u003e`_\n-  `Wrighting Plugins: Component Registration \u003chttps://github.com/uTensor/utensor_cgen/tree/master/tutorials/component_registration\u003e`_\n\nHow to Serve Your Model on uTenosr\n==================================\n\nKeras_ (Recommended)\n--------------------\n\nPlease refer to `End-to-End Training with Keras`_ for detail\n\n\nTensorFlow_\n-----------\n\n1. Freeze your `tensorflow.Graph`\n\n  - please refer to this `issue track \u003chttps://github.com/tensorflow/tensorflow/issues/27614\u003e`_ for detail\n  - especially this `comment \u003chttps://github.com/tensorflow/tensorflow/issues/27614#issuecomment-571889676\u003e`_ by Robin2091\n\n2. Follow instructions in :ref:`install` section to install :mod:`utensor_cgen`\n\n  - then `utensor-cli` should be available in your console\n\n3. Inspect your pb file to find the output node\n\n  .. code-block:: console\n\n    # verbose mode\n    $ utensor-cli show graph.pb\n\n    # or oneline mode\n    $ utensor-cli show graph.pb --oneline\n\n4. convert the protobuf file to C/C++ source code with `utensor-cli`\n\n  - supose the output node is ``pred`` in **graph.pb**\n\n  .. code-block:: console\n\n    $ utensor-cli convert --output-nodes=pred graph.pb\n\n5. Compile your application code with generated C/C++ and weights files\n\n  - You should find your model C/C++ and weights files in directories\n    **models** and **constants** respectively\n\n\\ |convert-example|\n\nTesting\n=======\n\n1. follow the steps in :ref:`install_dev` section\n2. run tests as following\n\n  .. code-block:: console\n\n    # run with `make`\n    $ make tests\n\n    # run with `pipenv`\n    $ pipenv run pytest -m 'not slow_test and not deprecated' tests\n\n.. design philosophy\n..     `12 Factor CLI App \u003chttps://medium.com/@jdxcode/12-factor-cli-apps-dd3c227a0e46?fbclid=IwAR1Gfq0D7oh3b-mXaIMV3RwYu39TAPrPXfz5sBKC4Rz1t-cckvC8WjBVl_w\u003e`_\n\n\nFuture Works\n============\n\n1.  High-level graph builder api for building :class:`.uTensorGraph`.\n\n    - Currently ``utensor_cgen`` uses ``TensorFlow`` api for building IR graph, ``uTensorGraph``.\n    - With high-level graph builder, users can build their ``uTensorGraph`` easily and do not need\n      to take care of the integrity of the graph.\n      The builder will take care of it automatically.\n\n.. _pip: https://pip.pypa.io/en/stable/\n.. _pipenv: https://github.com/pypa/pipenv\n.. _Tensorflow: https://www.tensorflow.org\n.. _Keras: https://keras.io/\n.. _End-to-End Training with Keras: https://github.com/uTensor/utensor_cgen/tree/master/tutorials/end2end_training\n.. _PyTorch: https://pytorch.org/\n.. _uTensor: https://github.com/uTensor/uTensor\n.. _simple_mnist.pb: https://github.com/uTensor/utensor_cgen/blob/develop/tests/deep_mlp/simple_mnist.pb\n\n2. Automaic-Update TFLite fbs file\n\n   - `schema files \u003chttps://github.com/tensorflow/tensorflow/tree/cf28969fb2ea72c9738cd3acdddc0b69fd7dff5e/tensorflow/lite/schema\u003e`_\n\n.. readme_end\n\n.. |cnn-dropout| image:: doc/source/_images/cnn_dropout.png\n    :alt: cnn-dropout\n.. |conv-pool-fuse| image:: doc/source/_images/conv_pool_fuse.png\n    :alt: conv-pool-fuse\n.. |convert-example| image:: doc/source/_images/convert_example.png\n    :alt: convert-example\n.. |mlp-alloc| image:: doc/source/_images/mlp_alloc.png\n    :alt: mlp-alloc\n.. |mlp-alloc-graph| image:: doc/source/_images/mlp_alloc_graph.png\n    :alt: mlp-alloc-graph\n.. |utensor-cli-components| image:: doc/source/_images/utensor-cli-components.drawio.svg\n    :alt: utensor-cli-components\n\n.. TODOs\n.. =====\n\n.. 1. (done?) core code generator implementation\n\n..    -  We need some refactoring, PRs are welcomed!\n\n.. 2. type alias in C/C++\n\n..    -  ex: use ``uint8_t`` or ``unsigned char``?\n..    -  a lot more about this....\n\n.. 3. Relation among snippets/containers\n\n..    -  shared template variables? (headers, shared placeholders...etc)\n\n.. 4. Better configuration schema\n\n..    -  json\n..    -  yaml\n..    -  or ?\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Futensor%2Futensor_cgen","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Futensor%2Futensor_cgen","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Futensor%2Futensor_cgen/lists"}