{"id":20685439,"url":"https://github.com/yas-sim/pyopenvino","last_synced_at":"2025-04-22T13:40:47.712Z","repository":{"id":185235207,"uuid":"464719268","full_name":"yas-sim/pyopenvino","owner":"yas-sim","description":"Experimental Python implementation of OpenVINO Inference Engine (very slow, limited functionality). All codes are written in Python. Easy to read and modify.","archived":false,"fork":false,"pushed_at":"2022-03-11T14:33:29.000Z","size":66087,"stargazers_count":7,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-29T15:34:51.997Z","etag":null,"topics":["ai","convolution","convolutional-neural-network","deep-learning","experimental","inference","inference-engine","inference-library","mnist","mnist-classification","object-detection","openvino","python","reference-implementation","ssd-mobilenet"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/yas-sim.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2022-03-01T02:41:12.000Z","updated_at":"2022-10-31T17:36:37.000Z","dependencies_parsed_at":"2023-08-01T07:09:28.391Z","dependency_job_id":null,"html_url":"https://github.com/yas-sim/pyopenvino","commit_stats":null,"previous_names":["yas-sim/pyopenvino"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yas-sim%2Fpyopenvino","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yas-sim%2Fpyopenvino/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yas-sim%2Fpyopenvino/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yas-sim%2Fpyopenvino/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/yas-sim","download_url":"https://codeload.github.com/yas-sim/pyopenvino/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250250245,"owners_count":21399597,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","convolution","convolutional-neural-network","deep-learning","experimental","inference","inference-engine","inference-library","mnist","mnist-classification","object-detection","openvino","python","reference-implementation","ssd-mobilenet"],"created_at":"2024-11-16T22:27:22.099Z","updated_at":"2025-04-22T13:40:47.668Z","avatar_url":"https://github.com/yas-sim.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# PyOpenVINO - An Experimental Python Implementation of OpenVINO Inference Engine (minimum-set)\n----------------------------\n\n## Description\nThe PyOpenVINO is a spin-off product from my deep learning algorithm study work. This project is aiming at neither practical performance nor rich functionalities.\nPyOpenVINO can load an OpenVINO IR model (.xml/.bin) and run it.\nThe implementation is quite straightforward and naive. No Optimization technique is used. Thus, the code is easy to read and modify.\nSupported API is quite limited, but it mimics OpenVINO IE Python API. So, you can easily read and modify the sample code too.  \n- Developed as a spin-off from my deep learning study work.  \n- Very slow and limited functionality. Not a general DL inference engine.\n- Naive and straightforward code: (I hope) This is a good reference for learning deep-learning technology.  \n- Extensible ops: Ops are implemented as plugins. You can easily add your ops as needed.  \n- MNIST CNN, Googlenet-v1, and ssd-mobilenet-v1 are working.  \n\n### Tested DL Models  \n- MNIST (Conv+FC+SoftMax)\n- Googlenet-v1\n- SSD-mobilenet-v1\n![resources/ssd_mobilenet_v1.png](resources/ssd_mobilenet_v1.png)\n\n------------------------\n\n## How to run\n\nSteps 1 and 2 are optional since the converted MNIST IR model is provided.  \n\n0. Install required Python packages (`opencv-python`, `numpy`, `networkx`)\n```sh\npython -m pip install --upgrade pip setuptools\npython -m pip install -r requirements.txt\n```\n\n1. (Optional) Train a model and generate a '`saved_model`' with TensorFlow  \n```sh\npython mnist-tf-training.py\n```\nThe trained model data will be created under `./mnist-savedmodel` directory.\n\n2. (Optional) Convert TF saved_model into OpenVINO IR model  \nPrerequisite: You need to have OpenVINO installed (Model Optimizer is required).  \n```sh\nconvert-model.bat\n```\nConverted IR model (.xml/.bin) will be generated in `./models` directory.  \n\n3. Run pyOpenVINO sample program\n```sh\npython test_pyopenvino.py\n```\nYou'll see the output like this.  \n```sh\npyopenvino\u003epython test_pyopenvino.py\ninputs: [{'name': 'conv2d_input', 'type': 'Parameter', 'version': 'opset1', 'data': {'element_type': 'f32', 'shape': (1, 1, 28, 28)}, 'output': {0: {'precision': 'FP32', 'dims': (1, 1, 28, 28)}}}]\noutputs: [{'name': 'Func/StatefulPartitionedCall/output/_11:0', 'type': 'Result', 'version': 'opset1', 'input': {0: {'precision': 'FP32', 'dims': (1, 10)}}}]\n# node_name, time (sec)\nconv2d_input Parameter, 0.0\nconv2d_input/scale_copy Const, 0.0\nStatefulPartitionedCall/sequential/conv2d/Conv2D Convolution, 0.11315417289733887\nStatefulPartitionedCall/sequential/conv2d/BiasAdd/ReadVariableOp Const, 0.0\nStatefulPartitionedCall/sequential/conv2d/BiasAdd/Add Add, 0.0\nStatefulPartitionedCall/sequential/conv2d/Relu ReLU, 0.0010142326354980469\nStatefulPartitionedCall/sequential/max_pooling2d/MaxPool MaxPool, 0.020931482315063477\n          :\nStatefulPartitionedCall/sequential/dense_1/BiasAdd/Add Add, 0.0\nStatefulPartitionedCall/sequential/dense_1/Softmax SoftMax, 0.0009992122650146484\nFunc/StatefulPartitionedCall/output/_11:0 Result, 0.0\n@TOTAL_TIME, 0.21120882034301758\n0.21120882034301758 sec/inf\nRaw result: {'Func/StatefulPartitionedCall/output/_11:0': array([[7.8985136e-07, 2.0382247e-08, 9.9999917e-01, 1.0367385e-10,\n        1.0184062e-10, 1.6024957e-12, 2.0729640e-10, 1.6014919e-08,\n        6.5354638e-10, 9.5946295e-14]], dtype=float32)}\nResult: [2 0 1 7 8 6 3 4 5 9]\n```\n\n4. Run MNIST `Draw-and-Inter` demo  \nThis demo program recognizes a number drawn by the user in real-time. You can draw a number on the screen by pointing device such as a mouse and the demo tells you the recognition result.  \n```sh\npython draw-and-infer.py\n```\n#### How to Operate  \n- Left click to draw points.  \n- Right click to clear the canvas.  \nThis demo program is using 'special' kernels for performance.  \n![draw-and-infer](resources/draw_and_infer.png)\n----------------------------------\n## A Littile Description of the Implementation  \n\n### IR model internal representation\nThis inference engine uses `networkx.DiGraph` as the internal representation of the IR model.\nIR model will be translated into `node`s and `edge`s.  \nThe nodes represent the `ops`, and it holds the attributes of the ops (e.g., strides, dilations, etc.).  \nThe edges represent the connection between the nodes. The edges hold the port number for both ends.  \nThe intermediate output from the nodes (feature maps) will be stored in the `data` attributes in the `output` port of the node (`G.nodes[node_id_num]['output'][port_num]['data'] = feat_map`)  \n\n### An example of the contents (attributes) of a node  \n```sh\nnode id= 14\n name : StatefulPartitionedCall/sequential/target_conv_layer/Conv2D\n type : Convolution\n version : opset1\n data :\n     auto_pad : valid\n     dilations : 1, 1\n     pads_begin : 0, 0\n     pads_end : 0, 0\n     strides : 1, 1\n input :\n     0 :\n         precision : FP32\n         dims : (1, 64, 5, 5)\n     1 :\n         precision : FP32\n         dims : (64, 64, 3, 3)\n output :\n     2 :\n         precision : FP32\n         dims : (1, 64, 3, 3)\n```\n\n### An example of the contents of an edge  \nformat = (from-layer, from-port, to-layer, to-port)\n```sh\nedge_id= (0, 2)\n   {'connection': (0, 0, 2, 0)}\n```\n\n### Ops plugins\nOperators are implemented as plugins. You can develop an Op in Python and place the file in the `op_plugins` directory. The inference_engine of pyOpenVINO will search the Python source files in the `op_plugins` directory at the start time and register them as the Ops plugin.  \nThe file name of the Ops plugin will be treated as the Op name, so it must match the `layer type` attribute field in the IR XML file.  \nThe inference engine will call the `compute()` function of the plugin to perform the calculation.  The `compute()` function is the only API between the inference engine and the plugin. The inference engine will collect the required input data and pass it to the `compute()` function. The input data is in the form of Python `dict`. (`{port_num:data[, port_num:data[, ...]]}`)  \nThe op needs to calculate the result from the input data and return it as a Python `dict`. (`{port_num:result[, port_num:result[, ...]]}`)  \n\n### Kernel implementation: NumPy version and Naive version  \nNot all, but some Ops have dual kernel implementation, a naive implementation (easy to read), and a NumPy version implementation (a bit faster).  \nThe NumPy version might be x10+ faster than the naive version.  \nThe kernel type can be specified with `Executable_Network.kernel_type` attribute. You can specify eitgher one of `'naive'` (default) or `'numpy'`. Please refer to the sample program `test_pyopenvino.py` for the details.  \n\n### Special Thanks\n\nThe fastest 'special' convolution kernel is taken from '[deep-learning-from-scratch](https://github.com/oreilly-japan/deep-learning-from-scratch)' project.\n\n\nEND\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyas-sim%2Fpyopenvino","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyas-sim%2Fpyopenvino","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyas-sim%2Fpyopenvino/lists"}