{"id":27266342,"url":"https://github.com/supersecurehuman/segmentation_oneapi","last_synced_at":"2025-04-11T08:47:16.663Z","repository":{"id":188274859,"uuid":"664939200","full_name":"SuperSecureHuman/Segmentation_OneAPI","owner":"SuperSecureHuman","description":"Segementation with oneAPI","archived":false,"fork":false,"pushed_at":"2023-09-03T04:43:47.000Z","size":11143,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2023-09-04T08:44:27.389Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/SuperSecureHuman.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-07-11T05:00:50.000Z","updated_at":"2023-08-21T13:07:55.000Z","dependencies_parsed_at":"2023-08-14T17:38:18.315Z","dependency_job_id":null,"html_url":"https://github.com/SuperSecureHuman/Segmentation_OneAPI","commit_stats":null,"previous_names":["supersecurehuman/segmentation_oneapi"],"tags_count":1,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SuperSecureHuman%2FSegmentation_OneAPI","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SuperSecureHuman%2FSegmentation_OneAPI/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SuperSecureHuman%2FSegmentation_OneAPI/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SuperSecureHuman%2FSegmentation_OneAPI/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/SuperSecureHuman","download_url":"https://codeload.github.com/SuperSecureHuman/Segmentation_OneAPI/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248362957,"owners_count":21091240,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-04-11T08:47:13.597Z","updated_at":"2025-04-11T08:47:16.654Z","avatar_url":"https://github.com/SuperSecureHuman.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Segmentation with OpenVINO: Enhancing Semantic Segmentation\n\n## Table of Contents\n\n- [Introduction](#introduction)\n- [Demo](#demo)\n- [Model Weights](#model-weights)\n- [Dataset](#dataset)\n- [Methods](#methods)\n- [Inference Performance](#inference-performance)\n- [Integration of OneAPI](#integration-of-oneapi)\n- [Other Integration Possiblities](#other-integration-possiblites)\n- [Further Works](#further-works)\n- [License](#license)\n\n## Introduction\n\nSemantic segmentation, a cornerstone of computer vision, involves understanding images at a pixel level. This repository embarks on a journey from training a robust semantic segmentation model on Oxford's Pets dataset to optimizing it for real-time inference using OpenVINO. The goal is to maintain exceptional accuracy while achieving lightning-fast performance.\n\n## Demo\n\nA short video demonstration showcases the efficiency and accuracy achieved through the integration of OpenVINO.\n\nRun `demo.py`. Make sure to have model weights in the path `./model/model_int8.xml`\n\n\n\n\u003chttps://github.com/SuperSecureHuman/Segmentation_OneAPI/assets/88489071/efe54327-e7c7-4e03-bf90-008340190f96\u003e\n\n\n\n## Model Weights\n\nModel weights can be found in the Releases page. In order to run the final demo, you need only the INT8 files.\n\n## Dataset\n\nI used the [Oxford's Pets dataset](https://www.robots.ox.ac.uk/~vgg/data/pets/) for our training and evaluation. This dataset offers a rich collection of diverse pet images along with meticulous pixel-wise annotations, providing the foundation for seamless semantic segmentation tasks.\n\n## Project Stack\n\n1. Pytorch\n2. OpenCV\n3. Pytorch Segmentation Models\n4. NNCF (Neural Network Compression Framework for enhanced OpenVINO™ inference)\n5. OpenVINO\n6. Intel Extention for Pytorch\n\n## Methods\n\nThe evolution of our model is outlined through these pivotal methods:\n\n1. **Torch**: Training of the model. And using torch for initial baseline to start with.\n2. **TorchScript**: Transitioning to TorchScript enables dynamic Just-In-Time (JIT) compilation, enhancing computational efficiency.\n3. **Intel PyTorch Extension (IPEX) - BFFloat**: IPEX with BFFloat optimization leverages oneDNN. IPEX applies graph fusion, which is accelerated by oneDNN.\n4. **Testing Quantization**: Quantization with Callibration data was used then to compress the model to INT8 without compermising on accuracy.\n5. **OpenVINO Integration**: The optimized model seamlessly converts to OpenVINO's Intermediate Representation (IR) format, delivering fast inference speed.\n\n## Inference Performance\n\nThe leap in inference performance is astonishing:\n\n- Initial PyTorch CPU inference speed: ~16 frames per second (fps).\n- Inference speed with OpenVINO CPU (INT8 precision): ~100 fps.\n\nNotably, this performance enhancement is achieved without any compromise on segmentation accuracy.\n\n## Integration of OneAPI\n\nOptimization is at the heart of this project, facilitated by OneAPI tools:\n\n1. **Export to ONNX**: Model Optimizer (MO) exports the model to ONNX format for compatibility with OpenVINO.\n2. **NNCF Quantization**: Neural Network Compression Framework (NNCF) enables INT8 quantization and exporting.\n3. **Intel OpenVINO Inference Engine**: The Intel OpenVINO Inference Engine drives rapid execution during inference.\n\n### Other Integration Possiblites\n\n1. Can use other Intel Hardware (Intel Xeon Processors, Habana Gaudi Instances, Intel Server GPUs) via DevCloud for training by leveraging IPEX and Torch XPU interface. Since I had a GPU locally, I preffered using it.\n2. Host an async inference server, and use the model for real-time inference with data from other devices. Couldn't do because of lack of hardware presently with me.\n\n## Further Works\n\n1. Use the model for real-time inference with data from other devices (Make use of inference server within OpenVINO).\n2. Extend this to use Intel NCS on edge Devices\n\n## License\n\nThis project operates under the [MIT License](LICENSE), granting you the freedom to manipulate, adjust, and share the code while adhering to the original license terms.\n\n---\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsupersecurehuman%2Fsegmentation_oneapi","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsupersecurehuman%2Fsegmentation_oneapi","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsupersecurehuman%2Fsegmentation_oneapi/lists"}