{"id":20280563,"url":"https://github.com/kiritigowda/openvx-model-compiler","last_synced_at":"2025-04-11T06:37:08.254Z","repository":{"id":81714225,"uuid":"273781849","full_name":"kiritigowda/OpenVX-Model-Compiler","owner":"kiritigowda","description":"Tools to convert Caffe / NNEF / ONNX pre-trained neural net models to an OpenVX Graph","archived":false,"fork":false,"pushed_at":"2021-09-16T20:46:56.000Z","size":1126,"stargazers_count":8,"open_issues_count":0,"forks_count":3,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-03-25T04:32:45.433Z","etag":null,"topics":["amd-nnir","caffe","mivisionx","mivisionx-inference-analyzer","nnef","nnef-model","nnir-model","onnx","onnx-models","openvx","openvx-graph","openvx-runtime"],"latest_commit_sha":null,"homepage":"http://kiritigowda.com/OpenVX-Model-Compiler/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kiritigowda.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-06-20T20:46:38.000Z","updated_at":"2025-01-03T04:41:27.000Z","dependencies_parsed_at":null,"dependency_job_id":"72f13be8-c7e6-4498-be7b-0f67ef10f7b4","html_url":"https://github.com/kiritigowda/OpenVX-Model-Compiler","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kiritigowda%2FOpenVX-Model-Compiler","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kiritigowda%2FOpenVX-Model-Compiler/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kiritigowda%2FOpenVX-Model-Compiler/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kiritigowda%2FOpenVX-Model-Compiler/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kiritigowda","download_url":"https://codeload.github.com/kiritigowda/OpenVX-Model-Compiler/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248358533,"owners_count":21090400,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["amd-nnir","caffe","mivisionx","mivisionx-inference-analyzer","nnef","nnef-model","nnir-model","onnx","onnx-models","openvx","openvx-graph","openvx-runtime"],"created_at":"2024-11-14T13:35:57.501Z","updated_at":"2025-04-11T06:37:08.245Z","avatar_url":"https://github.com/kiritigowda.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![MIT licensed](https://img.shields.io/badge/license-MIT-blue.svg)](https://opensource.org/licenses/MIT)\n\n\u003cp align=\"center\"\u003e\u003cimg width=\"60%\" src=\"https://raw.githubusercontent.com/GPUOpen-ProfessionalCompute-Libraries/MIVisionX/master/docs/images/OpenVX_logo.png\" /\u003e\u003c/p\u003e\n\n# OpenVX Neural Net Model Compiler \u0026 Optimizer\n\n\u003cp align=\"center\"\u003e\u003cimg width=\"80%\" src=\"data/images/modelCompilerWorkflow.png\" /\u003e\u003c/p\u003e\n\nOpenVX Neural Net Model Compiler \u0026 Optimizer converts pre-trained neural network models to OpenVX runtime code for optimized inference.\n\n* [OpenVX RunTime](#openvx-runtime)\n* [Pre-requisites](#pre-requisites)\n* [Model Compiler \u0026 Optimizer Usage](#model-compiler--optimizer-usage)\n* [Examples for OpenVX C code generation](#examples-for-openvx-c-code-generation)\n* [Models \u0026 Operators currently supported](#models--operators-currently-supported)\n* [Contributing to Model Compiler](#contributing-to-model-compiler)\n\n\nPre-trained models in [ONNX](https://onnx.ai/), [NNEF](https://www.khronos.org/nnef), \u0026 [Caffe](http://caffe.berkeleyvision.org/) formats are supported by the model compiler \u0026 optimizer. The model compiler first converts the pre-trained models to AMD Neural Net Intermediate Representation (NNIR), once the model has been translated into AMD NNIR (AMD's internal open format), the Optimizer goes through the NNIR and applies various optimizations which would allow the model to be deployed on to target hardware most efficiently. Finally, AMD NNIR is converted into OpenVX C code, which could be compiled and deployed on any targeted AMD hardware.\n\n\u003cp align=\"center\"\u003e\u003cimg width=\"100%\" src=\"data/images/frameworks.png\" /\u003e\u003c/p\u003e\n\n## OpenVX RunTime\n\nOpenVX RunTime allows hundreds of different [OpenVX](https://www.khronos.org/registry/OpenVX/specs/1.0/html/da/db6/group__group__vision__functions.html) and [OpenCV interop](../amd_openvx_extensions/amd_opencv#list-of-opencv-interop-kernels) vision functions to be directly added into the OpenVX C code generated by the model compiler \u0026 optimizer for preprocessing the input to the neural network model and post-processing the model results, hence allowing users to create an end to end solution to be deployed on any targeted AMD hardware.\n\n\u003cp align=\"center\"\u003e\u003cimg width=\"100%\" src=\"data/images/runtime.png\" /\u003e\u003c/p\u003e\n\n## Pre-requisites\n\n* Ubuntu `16.04`/`18.04` or CentOS `7.5`/`7.6`\n* [OpenVX](https://github.com/KhronosGroup/OpenVX-sample-impl) - Install any conformat OpenVX Implementation  \n\n#### ONNX\n* numpy\n* onnx\n\n````\n% pip install onnx numpy\n````\n**Note:** ONNX Models are available at [ONNX Model Zoo](https://github.com/onnx/models)\n\n#### NNEF\n* numpy\n* [nnef-parser](https://github.com/KhronosGroup/NNEF-Tools) - Build the nnef python module\n\t\n````\n% pip install numpy\n````\n**Note:** NNEF Models are available at [NNEF Model Zoo](https://github.com/KhronosGroup/NNEF-Tools/tree/master/models#nnef-model-zoo)\n\n## Model Compiler \u0026 Optimizer Usage\n\n### Step 1 - Convert Pre-trained model to AMD NNIR\n\n#### Caffe\n\nTo convert a pre-trained caffemodel into AMD NNIR model:\n\n```\n% python caffe_to_nnir.py \u003cnet.caffeModel\u003e \u003cnnirOutputFolder\u003e --input-dims \u003cn,c,h,w\u003e [--verbose \u003c0|1\u003e]\n```\n\n#### ONNX\n\nTo convert an ONNX model into AMD NNIR model:\n\n```\n% python onnx_to_nnir.py \u003cmodel.onnx\u003e \u003cnnirModelFolder\u003e [OPTIONS]\n\nOPTIONS:\n\t--input_dims n,c,h,w\n```\n#### NNEF\n\nTo convert a NNEF model into AMD NNIR model:\n\n```\n% python nnef_to_nnir.py \u003cnnefInputFolder\u003e \u003cnnirOutputFolder\u003e\n```\n**Note:** If you want to create NNEF models from pre-trained caffe or tensorflow models, use [NNEF Converter](https://github.com/KhronosGroup/NNEF-Tools) or try NNEF models at [NNEF Model Zoo](https://github.com/KhronosGroup/NNEF-Tools/tree/master/models#nnef-model-zoo)\n\n### Step 2 - Apply Optimizations\n\nTo update batch size in AMD NNIR model:\n````\n% python nnir_update.py --batch-size \u003cN\u003e \u003cnnirModelFolder\u003e \u003cnnirModelFolderN\u003e\n````\n\nTo fuse operations in AMD NNIR model (like batch normalization into convolution):\n````\n% python nnir_update.py --fuse-ops \u003c1\u003e \u003cnnirModelFolderN\u003e \u003cnnirModelFolderFused\u003e\n````\n\nTo quantize the model to float 16\n````\n% python nnir_update.py --convert-fp16 \u003c1\u003e \u003cnnirModelFolderN\u003e \u003cnnirModelFolderFused\u003e\n````\n\nTo workaround groups using slice and concat operations in AMD NNIR model:\n````\n% python nnir_update.py --slice-groups \u003c1\u003e \u003cnnirModelFolderFused\u003e \u003cnnirModelFolderSliced\u003e\n````\n### Step 3 - Convert AMD NNIR to OpenVX C code\n\nTo convert an AMD NNIR model into OpenVX C code:\n````\n% python nnir_to_openvx.py --help\n\nUsage: python nnir_to_openvx.py [OPTIONS] \u003cnnirInputFolder\u003e \u003coutputFolder\u003e\n\n  OPTIONS:\n    --argmax UINT8                    -- argmax at the end with 8-bit output\n    --argmax UINT16                   -- argmax at the end with 16-bit output\n    --argmax \u003cfileNamePrefix\u003ergb.txt  -- argmax at the end with RGB color mapping using LUT\n    --argmax \u003cfileNamePrefix\u003ergba.txt -- argmax at the end with RGBA color mapping using LUT\n    --help                            -- show this help message\n\n  LUT File Format (RGB): 8-bit R G B values one per each label in text format\n    R0 G0 B0\n    R1 G1 B1\n    ...\n\n  LUT File Format (RGBA): 8-bit R G B A values one per each label in text format\n    R0 G0 B0 A0\n    R1 G1 B1 A1\n    ...\n````\n## Sample workflow for Model Compiler\n\n### Trained Caffe Model conversion to AMD NNIR to OpenVX Graph\n\n1. Convert net.caffemodel into NNIR model using the following command\n   ````\n\t    % python caffe_to_nnir.py \u003cnet.caffeModel\u003e \u003cnnirOutputFolder\u003e --input-dims n,c,h,w [--verbose 0|1]\n   ````\n2. Compile NNIR model into OpenVX C code with CMakelists.txt for compiling and building inference library\n   ````\n\t    % python nnir_to_openvx.py \u003cnnirModelFolder\u003e \u003cnnirModelOutputFolder\u003e\n   ````\n3. cmake and make the project inside the nnirModelOutputFolder\n   ````\n\t    % cd nnirModelOutputFolder\n\t    % cmake .\n\t    % make\n   ````\n4. Run anntest application for testing the inference with input and output tensor\n   ````\n\t    % ./anntest weights.bin\n   ````\n5. The shared C library (libannmodule.so) can be used in any customer application\n\n## Examples for OpenVX C code generation\n\n* Generate OpenVX and test code that can be used dump and compare raw tensor data:\n\n````\n% python nnir_to_openvx.py nnirInputFolderFused openvxCodeFolder\n% mkdir openvxCodeFolder/build\n% cd openvxCodeFolder/build\n% cmake ..\n% make\n% ./anntest\n````\n**Note:**\n```\nUsage: anntest \u003cweights.bin\u003e [\u003cinput-data-file(s)\u003e [\u003coutput-data-file(s)\u003e]]\u003c--add ADD\u003e \u003c--multiply MULTIPLY\u003e]\n\n   \u003cinput-data-file\u003e: is filename to initialize tensor\n     .jpg or .png: decode and initialize for 3 channel tensors\n         (use %04d in fileName to when batch-size \u003e 1: batch index starts from 0)\n     other: initialize tensor with raw data from the file\n\n   \u003coutput-data-file\u003e[,\u003creference-for-compare\u003e,\u003cmaxErrorLimit\u003e,\u003crmsErrorLimit\u003e]:\n     \u003creferece-to-compare\u003e is raw tensor data for comparision\n     \u003cmaxErrorLimit\u003e is max absolute error allowed\n     \u003crmsErrorLimit\u003e is max RMS error allowed\n     \u003coutput-data-file\u003e is filename for saving output tensor data\n       '-' to ignore\n       other: save raw tensor into the file\n       \n   \u003cadd\u003e: input preprocessing factor [optional - default:[0,0,0]]\n   \n   \u003cmultiply\u003e: input preprocessing factor [optional - default:[1,1,1]]\n\n% ./anntest ../weights.bin input.f32 output.f32,reference.f32,1e-6,1e-9 --add -2.1,-2.07,-1.8 --multiply 0.017,0.017,0.017\n...\n```\n\n* Generate OpenVX and test code with argmax that can be used dump and compare 16-bit argmax output tensor:\n````\n% python nnir_to_openvx.py --argmax UINT16 nnirInputFolderFused openvxCodeFolder\n% mkdir openvxCodeFolder/build\n% cd openvxCodeFolder/build\n% cmake ..\n% make\n% ./anntest\n````\n**Note:**\n```\nUsage: anntest \u003cweights.bin\u003e [\u003cinput-data-file(s)\u003e [\u003coutput-data-file(s)\u003e]]]\n\n   \u003cinput-data-file\u003e: is filename to initialize tensor\n     .jpg or .png: decode and initialize for 3 channel tensors\n         (use %04d in fileName to when batch-size \u003e 1: batch index starts from 0)\n     other: initialize tensor with raw data from the file\n\n   \u003coutput-data-file\u003e[,\u003creference-for-compare\u003e,\u003cpercentErrorLimit\u003e]:\n     \u003creferece-to-compare\u003e is raw tensor data of argmax output for comparision\n     \u003cpercentMismatchLimit\u003e is max mismatch (percentage) allowed\n     \u003coutput-data-file\u003e is filename for saving output tensor data\n       '-' to ignore\n       other: save raw tensor into the file\n\n% ./anntest ../weights.bin input-%04d.png output.u16,reference.u16,0.01\n...\n```\n\n* Generate OpenVX and test code with argmax and LUT that is designed for semantic segmentation use cases. You can dump output in raw format or PNGs and additionally compare with reference data in raw format.\n````\n% python nnir_to_openvx.py --argmax lut-rgb.txt nnirInputFolderFused openvxCodeFolder\n% mkdir openvxCodeFolder/build\n% cd openvxCodeFolder/build\n% cmake ..\n% make\n% ./anntest\n````\n**Note:**\n```\nUsage: anntest \u003cweights.bin\u003e [\u003cinput-data-file(s)\u003e [\u003coutput-data-file(s)\u003e]]]\n\n   \u003cinput-data-file\u003e: is filename to initialize tensor\n     .jpg or .png: decode and initialize for 3 channel tensors\n         (use %04d in fileName to when batch-size \u003e 1: batch index starts from 0)\n     other: initialize tensor with raw data from the file\n\n   \u003coutput-data-file\u003e[,\u003creference-for-compare\u003e,\u003cpercentErrorLimit\u003e]:\n     \u003creferece-to-compare\u003e is raw tensor data of LUT output for comparision\n     \u003cpercentMismatchLimit\u003e is max mismatch (percentage) allowed\n     \u003coutput-data-file\u003e is filename for saving output tensor data\n       .png: save LUT output as PNG file(s)\n         (use %04d in fileName when batch-size \u003e 1: batch index starts from 0)\n       '-' to ignore\n       other: save raw tensor into the file\n\n% ./anntest ../weights.bin input-%04d.png output.rgb,reference.rgb,0.01\n...\n% ./anntest ../weights.bin input-%04d.png output-%04d.png,reference.rgb,0.01\n...\n```\n\nTest code with preprocessing add / multiply values to normalize the input tensor. Some models(e.g. Inception v4) require input tensor to be normalized. You can pass the preprocessing values using --add \u0026 --multiply option.\n\n````\n% ./anntest ../weights.bin input.f32 output.f32 --add -2.1,-2.07,-1.8 --multiply 0.017,0.017,0.017\n...\n````\n\n## Models \u0026 Operators currently supported\n###  Models\n\n\u003cp align=\"center\"\u003e\u003cimg width=\"60%\" src=\"data/images/modelTrainedFrameWorks.png\" /\u003e\u003c/p\u003e\n\n|Networks|Caffe|ONNX|NNEF|\n|--------|-----|----|----|\n|AlexNet||\u0026#9745;|\u0026#9745;|\n|Caffenet||\u0026#9745;||\n|DenseNet||\u0026#9745;||\t\t\t\t\t\t\n|Googlenet|\u0026#9745;|\u0026#9745;|\u0026#9745;|\t\t\n|Inception-V1||\u0026#9745;|\u0026#9745;|\t\t\t\n|Inception-V2||\u0026#9745;|\u0026#9745;|\t\t\t\n|Inception-V3||||\t\t\t\n|Inception-V4|\u0026#9745;|||\t\t\t\n|MNIST|\u0026#9745;||\u0026#9745;|\t\t\n|Mobilenet||\u0026#9745;|\u0026#9745;|\t\t\n|MobilenetV2|||\u0026#9745;|\n|ResNet-18|||\u0026#9745;|\t\t\t\n|ResNet-34|||\u0026#9745;|\t\t\t\n|ResNet-50|\u0026#9745;|\u0026#9745;|\u0026#9745;|\t\t\t\n|ResNet-101|\u0026#9745;||\u0026#9745;|\t\t\t\n|ResNet-152|\u0026#9745;||\u0026#9745;|\t\t\t\n|ResNetV2-18|||\u0026#9745;|\t\t\t\n|ResNetV2-34|||\u0026#9745;|\t\t\t\n|ResNetV2-50|||\u0026#9745;|\t\t\t\n|ResNetV2-101|||\u0026#9745;|\t\t\t\n|Squeezenet||\u0026#9745;|\u0026#9745;|\t\t\t\n|Tiny-Yolo-V2|\u0026#9745;|||\t\t\t\n|VGGNet-16|\u0026#9745;||\u0026#9745;|\t\t\t\n|VGGNet-19|\u0026#9745;|\u0026#9745;|\u0026#9745;|\t\t\t\n|Yolo-V3|\u0026#9745;|||\t\t\t\n|ZFNet||\u0026#9745;||\n\n**Note:**\n* Currently supporting [ONNX models](https://github.com/onnx/models) with `release 1.1` and `release 1.3` tags\n\n### Operators\n\n\u003cp align=\"center\"\u003e\u003cimg width=\"60%\" src=\"data/images/modelCompilerFrameWorks.png\" /\u003e\u003c/p\u003e\n\n|Layers|Caffe|ONNX|NNEF|\n|-------|----|----|----|\n|Add||\u0026#9745;|\u0026#9745;|\n|Argmax||\u0026#9745;|\u0026#9745;|\n|AveragePool||\u0026#9745;|\u0026#9745;|\n|BatchNormalization|\u0026#9745;|\u0026#9745;|\u0026#9745;|\n|Cast||\u0026#9745;||\n|Clamp|||\u0026#9745;|\n|Clip||\u0026#9745;||\n|Concat|\u0026#9745;|\u0026#9745;|\u0026#9745;|\n|Constant||\u0026#9745;||\n|Conv|\u0026#9745;|\u0026#9745;|\u0026#9745;|\n|ConvTranspose|\u0026#9745;|\u0026#9745;|\u0026#9745;|\n|Copy||\u0026#9745;|\u0026#9745;|\n|Crop|\u0026#9745;|||\n|CropAndResize||||\n|Deconv|\u0026#9745;|\u0026#9745;|\u0026#9745;|\n|DetectionOutput|\u0026#9745;|||\n|Div||\u0026#9745;|\u0026#9745;|\n|Dropout||||\n|Eltwise|\u0026#9745;|||\n|Exp||\u0026#9745;|\u0026#9745;|\n|Flatten|\u0026#9745;|||\n|GEMM|\u0026#9745;|\u0026#9745;|\u0026#9745;|\n|GlobalAveragePool||\u0026#9745;|\u0026#9745;|\n|InnerProduct|\u0026#9745;|||\n|Interp|\u0026#9745;|||\n|LeakyRelu||\u0026#9745;|\u0026#9745;|\n|Linear|||\u0026#9745;|\n|Log||\u0026#9745;|\u0026#9745;|\n|LRN|\u0026#9745;|\u0026#9745;|\u0026#9745;|\n|Matmul||\u0026#9745;|\u0026#9745;|\n|Max||\u0026#9745;|\u0026#9745;|\n|MaxPool||\u0026#9745;|\u0026#9745;|\n|MeanReduce|||\u0026#9745;|\n|Min||\u0026#9745;|\u0026#9745;|\n|Mul||\u0026#9745;|\u0026#9745;|\n|MulAdd||||\n|Permute|\u0026#9745;||\u0026#9745;|\n|PriorBox|\u0026#9745;|||\n|Relu|\u0026#9745;|\u0026#9745;|\u0026#9745;|\n|Reshape|\u0026#9745;|\u0026#9745;|\u0026#9745;|\n|Shape||\u0026#9745;||\n|Sigmoid||\u0026#9745;|\u0026#9745;|\n|Slice|||\u0026#9745;|\n|Split|\u0026#9745;|||\n|Softmax|\u0026#9745;|\u0026#9745;|\u0026#9745;|\n|SoftmaxWithLoss|\u0026#9745;|||\n|Squeeze||\u0026#9745;|\u0026#9745;|\n|Sub||\u0026#9745;|\u0026#9745;|\n|Sum||\u0026#9745;||\n|Transpose||\u0026#9745;|\u0026#9745;|\n|Unsqueeze||\u0026#9745;|\u0026#9745;|\n|Upsample|\u0026#9745;||\u0026#9745;|\n\n## Contribution\n\nThe OpenVX Model Compiler is contributed by [AMD](https://www.amd.com/en) from their [MIVisionX](https://gpuopen-professionalcompute-libraries.github.io/MIVisionX/) Toolkit.\n\n## Contributing to OpenVX Model Compiler\n\nWe welcome contributions to Model Compiler to extend the functionalities and add support to more layers and models. When contributing to this repository, please first discuss the changes you wish to make via issues and then submit a pull request.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkiritigowda%2Fopenvx-model-compiler","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkiritigowda%2Fopenvx-model-compiler","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkiritigowda%2Fopenvx-model-compiler/lists"}