{"id":13574492,"url":"https://github.com/oneapi-src/visual-process-discovery","last_synced_at":"2025-04-04T15:31:09.900Z","repository":{"id":152282194,"uuid":"610975620","full_name":"oneapi-src/visual-process-discovery","owner":"oneapi-src","description":"AI Starter Kit for Visual Process Discovery using Intel® Extension for Pytorch","archived":true,"fork":false,"pushed_at":"2024-02-01T23:56:34.000Z","size":114,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-11-05T09:44:40.876Z","etag":null,"topics":["deep-learning","pytorch"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/oneapi-src.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-03-07T21:21:59.000Z","updated_at":"2024-08-24T11:18:45.000Z","dependencies_parsed_at":"2024-11-05T09:44:26.571Z","dependency_job_id":null,"html_url":"https://github.com/oneapi-src/visual-process-discovery","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oneapi-src%2Fvisual-process-discovery","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oneapi-src%2Fvisual-process-discovery/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oneapi-src%2Fvisual-process-discovery/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oneapi-src%2Fvisual-process-discovery/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/oneapi-src","download_url":"https://codeload.github.com/oneapi-src/visual-process-discovery/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247202678,"owners_count":20900827,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","pytorch"],"created_at":"2024-08-01T15:00:52.069Z","updated_at":"2025-04-04T15:31:04.891Z","avatar_url":"https://github.com/oneapi-src.png","language":"Python","readme":"PROJECT NOT UNDER ACTIVE MANAGEMENT\n\nThis project will no longer be maintained by Intel.\n\nIntel has ceased development and contributions including, but not limited to, maintenance, bug fixes, new releases, or updates, to this project.  \n\nIntel no longer accepts patches to this project.\n\nIf you have an ongoing need to use this project, are interested in independently developing it, or would like to maintain patches for the open source software community, please create your own fork of this project.  \n\nContact: webadmin@linux.intel.com\n# Applications of Visual Process Discovery Using PyTorch\r\n## Introduction\r\nProcess discovery has been crucial for businesses from traditional times to understand the existing business process flows at every level and identify the gaps and bottlenecks to improve productivity. With the growing digitization of businesses, visual process discovery tools and techniques have found broad applications across multiple industries.\r\n\r\n## Table of Contents \r\n - [Purpose](#purpose)\r\n - [Reference Solution](#reference-solution)\r\n - [Reference Implementation](#reference-implementation)\r\n - [Intel® Optimized Implementation](#optimizing-the-end-to-end-solution-with-intel%C2%AE-oneapi-components)\r\n - [Performance Observations](#performance-observations)\r\n\r\n## Purpose\r\n\r\nAs more and more enterprises transform their operations to digital mode, it becomes critical to get detailed granular-level insights on how their users/stakeholders interact with enterprises to understand and provide better experiences to the users. They depend on expensive external subject matter experts or consultants to do the process investigation. Also, due to insufficient documentation, the as-is process is frequently not well defined, and organizations mostly fail to offer the necessary inputs to identify important concerns. Relying solely on human expertise to understand the business processes would not result in optimal process changes.\r\n\r\nVisual process discovery (VPD) captures real-time interactions between users and workflows; maps and analyses the workflows and provides objective data-driven insights to enhance processes; and identifies processes that can be automated using Robotic Process Automation (RPA).\r\n\r\nFor example, in the financial services sector, VPD can help enhance its customer experience by providing personalized solutions knowing their needs as they navigate through the company's websites; even help improve and automate loan processing, suspicious activity report generation, customer onboarding and account opening/closure, Know Your Customer (KYC) and other repetitive processes.\r\n\r\nDeep Learning algorithms can anticipate UI components from screenshots taken from the website and analyze sophisticated features quickly and efficiently. Generating a near-real-time report of recognized UI components through RPA on a low-power edge device is critical.\r\n\r\n## Reference Solution  \r\nThe reference solution aims to detect the UI elements (Buttons, Links, Texts, Images, Headings, Fields, Labels, IFrames) from the website screenshots using a PyTorch-based Deep Learning model.\r\n\r\nThis reference kit uses an advanced PyTorch-based pre-trained Faster R-CNN RESNET-50 model to perform transfer learning on a Roboflow Website Screenshots dataset. This model is an object detection algorithm that enables the Convolutional Neural Network (CNN) itself to learn the Region Proposals and is further trained to detect UI elements in the input website screenshot. A promising technique to detect UI components from website screenshots is the Faster R-CNN RESNET-50 object detection model. Any visual process automation might use this trained model to replace the UI element detection module.\r\n\r\nEdge-deployed AI model offers real-time analysis of UI element detection, conducting computation at extremely low latency, boosting the whole visual process discovery. Thus, by quantizing/compressing the model (from floating point to integer model), while maintaining a similar level of accuracy as the floating point model, efficient utilization of underlying resources can be demonstrated when deployed on edge devices with low processing and memory capabilities.\r\n\r\nSince GPUs are typically the choice for Deep Learning and AI processing to achieve a higher Frames Per Second (FPS) rate, to offer a more cost-effective option leveraging a CPU, the quantization technique can be used, leveraging Intel® Analytics toolkit, to achieve a high FPS by performing vectorized operations on CPUs itself.\r\n## Key Implementation Details\r\n\r\n- Highlighted the performance gain using Intel® oneAPI packages especially, Intel® Extension for PyTorch* v1.13.0 over the stock version of PyTorch v1.13.0.\r\n- Used the pre-trained Faster R-CNN RESNET-50 model to detect UI elements in the input website screenshots. Captured training time and hyperparameter-tuning time, inference time and model accuracy for multiple runs on both stock version and Intel® oneAPI version. Compared the averages of these runs.\r\n- Quantized model using Intel® Neural Compressor that has shown high-performance vectorized operations on Intel® platforms.\r\n\r\n## Reference Implementation\r\n### Use Case End-To-End flow\r\n![Use_case_flow](assets/stock_e2e.png)\r\n### Expected Input-Output\r\n\r\n\r\n|              **Input**               |   **Output**   |\r\n|:------------------------------------:|-----|\r\n| Website Screenshot |   UI Elements (Buttons, Links, Texts, Images, Headings, Fields, Labels, IFrames)  |\r\n\r\n\r\n### Reference Sources\r\n\r\n*DataSet*: https://public.roboflow.com/object-detection/website-screenshots/1 (236 MB dataset for this use case)\u003cbr\u003e\r\n*Case Study \u0026 Repo*: https://github.com/harshatejas/pytorch_custom_object_detection \r\n\r\n\u003e ***Please see this data set's applicable license for terms and conditions. Intel®Corporation does not own the rights to this data set and does not confer any rights to it.***\r\n\r\n### Repository clone and Anaconda installation\r\n\r\n```\r\ngit clone https://github.com/oneapi-src/visual-process-discovery\r\ncd visual-process-discovery\r\n```\r\n\r\n\u003e **Note**: If you are beginning to explore the reference kits on client machines such as a windows laptop, go to the [Running on Windows](#running-on-windows) section to ensure you are all set and come back here\r\n\r\n\u003e **Note**: The performance measurements were captured on Xeon based processors. The instructions will work on WSL, however some portions of the reference kits may run slower on a client machine, so utilize the flags supported to modify the epochs/batch size to run the training or inference faster. Additionally performance claims reported may not be seen on a windows based client machine.\r\n\r\n\u003e In this reference kit implementation already provides the necessary conda environment configurations to set up the software requirements. To utilize these environment scripts, first, install Anaconda/Miniconda by following the instructions at the following link\u003cbr\u003e[Anaconda installation](https://docs.anaconda.com/anaconda/install/linux/)\r\n\r\n### Usage and Instructions\r\n\r\nBelow are the steps to reproduce the benchmarking results given in this repository\r\n1. Creating the execution environment\r\n2. Dataset preparation\r\n3. Training Faster R-CNN RESNET-50 model\r\n4. Hyperparameter Tuning \u0026 Evaluation\r\n5. Model Inference\r\n6. Quantize trained models using Intel® Neural Compressor and benchmarking\r\n7. Observations\r\n\r\n\r\n### Software Requirements\r\n| **Package**              | **Stock Python**                   \r\n|:-------------------------| :---                               \r\n| Python                   | python==3.9.12                       \r\n| PyTorch                  | pytorch==1.13.0                 \r\n\r\n### Environment\r\nBelow are the developer environment used for this module on Azure. All the observations captured are based on this environment setup.\r\n\r\n\r\n| **Size** | **CPU Cores** | **Memory** | **Intel® CPU Family** |\r\n|----------|:-------------:|:----------:|:---------------------:|\r\n|    NA    |       8       |    32GB    |        ICELAKE        |\r\n\r\n### Solution setup\r\nThe below file is used to create an environment as follows:\r\n\r\n\r\n|        **YAML file**         | **Environment Name** |           **Configuration**            |\r\n|:----------------------------:|----------------------|:--------------------------------------:|\r\n| `env/stock/stock-visual.yml` |     `stock-visual`      | Python=3.9.12 with stock PyTorch 1.13.0 |\r\n\r\n### Dataset\r\nThe Roboflow Website Screenshots dataset is composed of screenshots from over 1.2K of the world's top websites. Annotated screenshots are very useful in Robotic Process Automation. \r\n\r\n| **Use case**                   | Visual Process Discovery\r\n|:-------------------------------| :---\r\n| **Objects of interest**         | Buttons, Links, Texts, Images, Headings, Fields, Labels, IFrames \r\n| **Size**                       | Total 1206 Labelled Images\u003cbr\u003e\r\n| **Source**                     | https://public.roboflow.com/object-detection/website-screenshots/1\r\n\r\n\r\n### Training\r\nTo identify Region Proposals, Faster R-CNN employs Selective Search on the layers of the Convolutional Neural Network itself. Faster R-CNN employs an object detection approach that allows the Convolutional Neural Network to learn Region Proposals and objects of interest from the input web screenshots. PyTorch v1.13.0 is used to perfrom transfer learning using Faster R-CNN RESNET-50 object detection architecture on a dataset that has been downloaded and preprocessed with OpenCV and NumPy.\r\n\r\n\r\n| **Input Image Dimension** | 1024x768 (preprocessed data)\r\n| :--- | :---\r\n| **Output Model format** | PyTorch\r\n  **Output**  | UI Elements (Buttons, Links, Texts, Images, Headings, Fields, Labels, IFrames)\r\n\r\n### Inference\r\nPerformed inferencing on the trained model using Stock PyTorch v1.13.0.\r\n\r\n#### 1. Environment Creation\r\n**Setting up the environment for Stock PyTorch**\u003cbr\u003eFollow the below conda installation commands to set up the Stock PyTorch environment for the model training and prediction. \r\n```sh\r\nconda env create -f env/stock/stock-visual.yml\r\n```\r\n\r\n*Activate stock conda environment*\r\n\r\nUse the following command to activate the environment that was created:\r\n\r\n```sh\r\nconda activate stock-visual\r\n```\r\n\r\n#### 2. Data preparation\r\n\u003e The Roboflow Website Screenshots Dataset is downloaded and extracted in a folder before running the training python module. Elements are annotated in Pascal VOC format.\r\n\r\n\u003e **Note**: For model building 19 images from the dataset were used for testing, 198 images for training, and 49 images for validation. Details of the image names can be found in the text files named \"train.txt,\" \"test.txt,\" and \"valid.txt\" inside the \"data\" folder. \r\n\r\n```sh\r\ncd data\r\n\r\ncurl -L \"https://public.roboflow.com/ds/nEKEk2xTVl?key=5RsNd1Ak4t\" \u003e roboflow.zip\r\nunzip roboflow.zip \r\n\r\nrm roboflow.zip\r\ncd ../\r\n```\r\n\r\nThe folder structure Looks as below after extraction of the dataset.\r\n```\r\n- data\r\n    - train\r\n    - valid\r\n    - test\r\n```\r\n\r\n\u003e **Note**: For step by step description for the above instructions to download the dataset, refer to the data.txt file inside the data folder.\r\n\r\n\u003e **Now the data folder contains the below structure** \r\n\u003cbr\u003edata=\"data/{train/valid/test}\"\r\n\r\n#### 3. Training Faster R-CNN RESNET-50 model\r\nRun the training module as given below to start training using the active environment. \r\n\r\nThis module takes option to run the training.\r\n```\r\nusage: training.py [-h] [-d DATA_PATH] [-c CHECKPOINT_PATH] [-o OUTPUT_MODEL_PATH] [-hy HYPERPARAMETER_TUNING]\r\n                   [-i INTEL] [-ep NUMBER_OF_EPOCHS]\r\n\r\noptional arguments:\r\n  -h, --help            show this help message and exit\r\n  -d DATA_PATH, --data_path DATA_PATH\r\n                        dataset path which consists of train and valid folders\r\n  -c CHECKPOINT_PATH, --checkpoint_path CHECKPOINT_PATH\r\n                        use absolute path to load the model for tuning, default is None\r\n  -o OUTPUT_MODEL_PATH, --output_model_path OUTPUT_MODEL_PATH\r\n                        output file name without extension to save the model\r\n  -hy HYPERPARAMETER_TUNING, --hyperparameter_tuning HYPERPARAMETER_TUNING\r\n                        use 1 for hyperparameter tuning , default is 0\r\n  -i INTEL, --intel INTEL\r\n                        use 1 for enabling intel pytorch optimizations, default is 0\r\n  -ep NUMBER_OF_EPOCHS, --number_of_epochs NUMBER_OF_EPOCHS\r\n                        Number of epochs used to train the model , default is 10\r\n```\r\n**Command to run training**\r\n```sh\r\npython src/training.py -d data -o models/stock/default -ep 10\r\n```\r\n\r\n**Expected Output**\u003cbr\u003e\r\n\u003eThe output trained model will be saved in `models/stock/default` as `vpd_model`. Training time in seconds would be generated at the end of the training module for 10 epochs.\r\n\r\n#### 4. Hyperparameter Tuning \u0026 Evaluation\r\n\r\n**Command to run Hyperparameter tuning**\r\n```sh\r\npython src/training.py -d data -c models/stock/default/vpd_model -o models/stock/tuning -hy 1\r\n```\r\n\u003e The output tuned model would be saved in `models/stock/tuning` as `vpd_best_model`.\r\n\r\n**Hyperparameters used here are as below**\r\n\r\nDataset used with a 70:20 split for training and validation.\r\n- Batch Size = 8 \r\n- Epochs = 2\r\n- Learning Rates      : [0.0001, 0.005]\r\n- Optimizers          : [\"Adam\", \"SGD\"]\r\n\r\n\u003e We recommend using the model trained using a regular training module as the starting point for Hyperparameter tuning. So that one can identify the best possible combinations to get better accuracy.\u003cbr\u003e\r\n\r\n\u003eNote: The test mAP of the model could be improved further by (1) Adding more data samples (2) Training for more epochs (3) Extending hyperparameter tuning for varied learning rates, optimizers etc.\r\n\r\n*Evaluating the model on Test Dataset*\r\n\r\nRun the evaluation module to find out the mAP.\r\n```\r\nusage: evaluation.py [-h] [-d DATASET] [-i INTEL] [-c CHECKPOINT_PATH]\r\n\r\noptional arguments:\r\n  -h, --help            show this help message and exit\r\n  -d DATASET, --dataset DATASET\r\n                        dataset path which consists of test folder\r\n  -i INTEL, --intel INTEL\r\n                        use 1 for enabling intel pytorch optimizations, default is 0\r\n  -c CHECKPOINT_PATH, --checkpoint_path CHECKPOINT_PATH\r\n                        use 1 for enabling intel pytorch optimizations, default is 0\r\n```\r\n\r\n**Command to run evaluation**\r\n\r\n```sh\r\npython src/evaluation.py -d data -c models/stock/tuning/vpd_best_model\r\n```\r\n\r\n#### 5. Inference\r\n*Running inference using Test dataset*\r\n\r\n```\r\nusage: inference.py [-h] [-d DATASET] [-i INTEL] [-c CHECKPOINT_PATH] [-b TEST_BATCH_SIZE]\r\n\r\noptional arguments:\r\n  -h, --help            show this help message and exit\r\n  -d DATASET, --dataset DATASET\r\n                        dataset path which consists of test folder\r\n  -i INTEL, --intel INTEL\r\n                        use 1 for enabling intel pytorch optimizations, default is 0\r\n  -c CHECKPOINT_PATH, --checkpoint_path CHECKPOINT_PATH\r\n                        use 1 for enabling intel pytorch optimizations, default is 0\r\n  -b TEST_BATCH_SIZE, --test_batch_size TEST_BATCH_SIZE\r\n                        use different batch sizes, default is 1\r\n```\r\n**Command to run inference**\r\n\r\n```sh\r\npython src/inference.py -d data -c models/stock/tuning/vpd_best_model -b 1 \r\n```\r\n\u003e Above inference script can be run using different batch sizes. Use `-b` to test with different batch sizes (e.g. `-b 10`) \u003cbr\u003e\r\n\r\n## Optimizing the End To End solution with Intel® oneAPI components\r\n### Use Case End-To-End flow\r\n![Use_case_flow](assets/intel_e2e.png)\r\n\r\n### Optimized software requirements\r\n\r\n| **Package**               | **Intel® Python**                   \r\n|:--------------------------| :---                               \r\n| Python                    | python==3.9.12\r\n| Intel® Extension for PyTorch*                   | intel-extension-for-pytorch==1.13.0\r\n| Intel® Neural Compressor | neural-compressor==2.0\r\n\r\n**YAML file**                                 | **Environment Name** |            **Configuration**             |\r\n| :---: |:--------------------:|:----------------------------------------:|\r\n`env/intel/intel-visual.yml`             |     `intel-visual`      | Python=3.9.12 with Intel® Extension for PyTorch* v1.13.0 |\r\n\r\n\r\n### 1. Environment Creation\r\n\r\n**Before setting up the Intel environment please deactivate the existing Stock environment by executing the below**\r\n```sh\r\nconda deactivate \r\n``` \r\n\r\n**Setting up the environment for Intel® Extension for PyTorch**\u003cbr\u003eFollow the below conda installation commands to set up the Intel® Extension for PyTorch* environment for the model training and prediction.\r\n```sh\r\nconda env create -f env/intel/intel-visual.yml\r\n```\r\n\r\n**Activate intel conda environment**\r\n\u003cbr\u003eUse the following command to activate the environment that was created:\r\n```sh\r\nconda activate intel-visual\r\n```\r\n\r\n#### 2. Training Faster R-CNN RESNET-50 model\r\nRun the training module as given below to start training using the active environment. \r\n\r\nThis module takes the option to run the training.\r\n\r\n**Command to run training**\r\n\r\n```sh\r\npython src/training.py -d data -o models/intel/default -ep 10 -i 1\r\n```\r\n\r\n\u003e The output trained model would be saved in `models/intel/default` as `vpd_model`. Training time in seconds would be generated at the end of the training module for 10 epochs.\r\n\r\n#### 3. Hyperparameter Tuning \u0026 Evaluation\r\n\r\n**Command to run Hyperparameter tuning**\r\n```sh\r\npython src/training.py -d data -c models/intel/default/vpd_model -o models/intel/tuning -hy 1 -i 1\r\n```\r\n\u003e The output tuned model would be saved in `models/intel/tuning` as `vpd_best_model`.\r\n\r\n**Hyperparameters used here are as below**\r\n\r\nDataset remains the same with a 70:20 split for training and validation.\r\n- Batch Size = 8 \r\n- Epochs = 2\r\n- Learning Rates      : [0.0001, 0.005]\r\n- Optimizers          : [\"Adam\", \"SGD\"]\r\n\r\n\r\n\u003e We recommend using the model trained using a regular training module as the starting point for Hyperparameter tuning. So that one can identify the best possible combinations to get better accuracy.\u003cbr\u003e\r\n\r\n\u003eNote: The test mAP of the model could be improved further by (1) Adding more data samples (2) Training for more epochs (3) Extending hyperparameter tuning for varied learning rates, optimizers etc.\r\n\r\n*Evaluating the model on Test Dataset*\r\n\r\nRun the evaluation module to find out the mAP.\r\n\r\n**Command to run evaluation**\r\n```sh\r\npython src/evaluation.py -d data -i 1 -c models/intel/tuning/vpd_best_model \r\n```\r\n\r\n#### 4. Inference\r\n*Running inference using test dataset and PyTorch*\r\n\r\n**Command to run inference**\r\n\r\n```sh\r\npython src/inference.py -d data -i 1 -c models/intel/tuning/vpd_best_model -b 1\r\n```\r\n\u003e Above inference script can be run using different batch sizes. Use `-b` to test with different batch sizes (e.g. `-b 10`) \u003cbr\u003e\r\n\r\n\r\n#### 5. Quantize trained models using Intel® Neural Compressor\r\nIntel® Neural Compressor is used to quantize the FP32 Model to the INT8 Model. An optimized model is used here for evaluating and timing analysis.\r\nIntel® Neural Compressor supports many optimization methods. In this case, we used the post-training accuracy aware quantization method to quantize the FP32 model.\r\n\r\n*Step-1: Conversion of FP32 Model to INT8 Model also called the Quantization*\r\n\r\n```\r\nusage: run_quantization.py [-h] [-o OUTPATH] [-d DATAPATH] [-c CHECKPOINT_PATH]\r\n\r\noptional arguments:\r\n  -h, --help            show this help message and exit\r\n  -o OUTPATH, --outpath OUTPATH\r\n                        absolute path to save quantized model. By default it will be saved in \"./inc_compressed_model/output\" folder\r\n  -d DATAPATH, --datapath DATAPATH\r\n                        Path to test dataset folder\r\n  -c CHECKPOINT_PATH, --checkpoint_path CHECKPOINT_PATH\r\n                        path to load the quantization model\r\n```\r\n\r\n**Command to run the neural_compressor_conversion**\r\n\r\n**Note**: Activate Intel® Environment before running\r\n```sh\r\npython src/intel_neural_compressor/run_quantization.py -o models/inc_compressed_model/output -d data -c models/intel/tuning/vpd_best_model\r\n```\r\n\u003e Quantized model will be saved by default in `models/inc_compressed_model/output` folder \r\n\r\n\r\n*Step-2: Inferencing using quantized Model*\r\n```\r\nusage: run_quant_inference.py [-h] [-d DATA_PATH] [-b BATCH_SIZE] [-qw QUANT_WEIGHTS] [-eval EVAL_MODE]\r\n\r\noptional arguments:\r\n  -h, --help            show this help message and exit\r\n  -d DATA_PATH, --data_path DATA_PATH\r\n                        path to the test data\r\n  -b BATCH_SIZE, --batch_size BATCH_SIZE\r\n                        batch size for the dataloader....default is 1\r\n  -qw QUANT_WEIGHTS, --quant_weights QUANT_WEIGHTS\r\n                        Quantization Model Weights folder containing \".pt\" format model\r\n  -eval EVAL_MODE, --eval_mode EVAL_MODE\r\n                        Enable evaluation mode to evaluate quantized model...default is False                   \r\n```\r\n*Command to run inference for realtime `(batchsize =1)`*\r\n```sh\r\npython src/intel_neural_compressor/run_quant_inference.py -d data -b 1 -qw models/inc_compressed_model/output\r\n\r\n```\r\n\u003e Use `-b` to test with different batch size (e.g. `-b 10`)\r\n\r\n*Command to run evaluation with quantized model*\r\n```sh\r\npython src/intel_neural_compressor/run_quant_inference.py -d data -b 1 -qw models/inc_compressed_model/output -eval True\r\n\r\n```\r\n\r\n```sh\r\nconda deactivate \r\n``` \r\n\r\n\r\n## Performance Observations\r\n\r\nThis section covers the training time and inference time comparison between Stock PyTorch v1.13.0 and Intel® Extension for PyTorch* v1.13.0.\r\n\r\n### Training and Hyperparameter tuning  benchmarking results: Stock PyTorch v1.13.0 vs Intel® Extension for PyTorch* v1.13.0\r\n\r\n![image](assets/training.png)\r\n\r\n\r\n\r\n\u003cbr\u003e**Key Takeaways**\u003cbr\u003e\r\nIntel® Extension for PyTorch* v1.13.0 offers speed up of training time up to 1.56x during regular model training and up to 1.42x speed up during hyperparameter tuned training, as compared to the Stock PyTorch v1.13.0.\r\n\r\n### Inference benchmarking results: Stock PyTorch v1.13.0 FP32 Model vs Intel® Extension for PyTorch* v1.13.0 FP32 Model vs Intel® Neural Compressor INT8 Model.\r\n\r\n![image](assets/inference.png)\r\n\r\n\r\n\u003cbr\u003e**Key Takeaways**\u003cbr\u003e\r\n\r\n-  Realtime prediction time with Intel® Extension for PyTorch* v1.13.0 FP32 Model shows up to 1.24x against Stock PyTorch v1.13.0 FP32 Model\r\n-  Batch prediction time speedup with Intel® Extension for PyTorch* v1.13.0 FP32 Model shows up to 1.34x against Stock PyTorch v1.13.0 FP32 Model\r\n-  Intel® Neural Compressor quantization offers real-time prediction time speedup of up to  2.54x against Stock PyTorch v1.13.0  FP32 model\r\n-  Intel® Neural Compressor quantization offers batch prediction time speedup of up to 2.07x against the Stock PyTorch v1.13.0 FP32 model.\r\n-  Accuracy drop of only 0.001% is observed post-quantization of the FP32 model in both phases, which is around 57% for 10 epochs of training.\r\n-  Model Footprint is reduced from 158 MB to 40 MB (~75% Compressed)\r\n\r\n## Conclusion\r\nTo build a webpage UI elements detection model for visual process discovery using the Deep Learning approach, machine learning engineers will need to train models with a large dataset and run inference more frequently. The accelerated training will allow them to train their models more frequently to achieve better accuracy. With faster inferencing speeds, they will be able to implement predictions in real time for faster process automation.\r\n\r\nThis reference kit implementation provides a performance-optimized guide for building an optimal UI element detection model that can be used on website screenshots to test efficient and successful process automation.\r\n\r\n### Notices \u0026 Disclaimers\r\nPerformance varies by use, configuration, and other factors. Learn more on the [Performance Index site](https://edc.intel.com/content/www/us/en/products/performance/benchmarks/overview/). \r\nPerformance results are based on testing as of dates shown in configurations and may not reflect all publicly available updates.  See backup for configuration details.  No product or component can be absolutely secure. \r\nYour costs and results may vary. \r\nIntel technologies may require enabled hardware, software, or service activation.\r\n© Intel Corporation.  Intel, the Intel logo, and other Intel marks are trademarks of Intel Corporation or its subsidiaries.  Other names and brands may be claimed as the property of others.  \r\n\r\nTo the extent that any public or non-Intel datasets or models are referenced by or accessed using tools or code on this site those datasets or models are provided by the third party indicated as the content source. Intel does not create the content and does not warrant its accuracy or quality. By accessing the public content, or using materials trained on or with such content, you agree to the terms associated with that content and that your use complies with the applicable license.\r\n \r\nIntel expressly disclaims the accuracy, adequacy, or completeness of any such public content, and is not liable for any errors, omissions, or defects in the content, or for any reliance on the content. Intel is not liable for any liability or damages relating to your use of public content.\r\n\r\n\r\n## Appendix\r\n\r\n### **Running on Windows**\r\n\r\nThe reference kits commands are linux based, in order to run this on Windows, goto Start and open WSL and follow the same steps as running on a linux machine starting from git clone instructions. If WSL is not installed you can [install WSL](https://learn.microsoft.com/en-us/windows/wsl/install).\r\n\r\n\u003e **Note** If WSL is installed and not opening, goto Start ---\u003e Turn Windows feature on or off and make sure Windows Subsystem for Linux is checked. Restart the system after enabling it for the changes to reflect.\r\n\r\n### **Experiment Setup**\r\n- Testing performed on: March 2023\r\n- Testing performed by: Intel Corporation\r\n- Configuration Details: Azure Standard_D8_V5 (Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz), 1 Socket, 4 Cores per Socket, 2 Threads per Core, Turbo:On, Total Memory: 32 GB, OS: Ubuntu 20.04, Kernel: Linux 5.13.0-1031-azure , Software: Intel® Extension for PyTorch* v1.13.0, Intel® Neural Compressor v2.0 \r\n\r\n| Platform                          | Ubuntu 20.04\r\n| :---                              | :---\r\n| Hardware                          | Azure Standard_D8_V5 (Icelake)\r\n| Software                          | Intel® Extension for PyTorch*, Intel® Neural Compressor.\r\n| What you will learn               | Advantage of using components in Intel® oneAPI AI Analytics Toolkit over the stock version for the computer vision-based model build and inferencing.\r\n\r\n### Known Issues\r\n\r\n1. Environment Creation Issue: Could not build wheels for pycocotools\r\n\r\n    **Issue:**\r\n      When creating an intel environment using `intel-visual` \r\n      ```\r\n      error: command 'gcc' failed: No such file or directory\r\n      ERROR: Could not build wheels for pycocotools\r\n      ```\r\n\r\n    **Solution:**\r\n\r\n    Install gcc.  For Ubuntu, this will be: \r\n\r\n      ```bash\r\n      apt install gcc\r\n      ```\r\n\r\n2. ImportError: libGL.so.1: cannot open shared object file: No such file or directory\r\n   \r\n    **Issue:**\r\n      While running the script \"python src/run_quantization.py\":,\r\n      ```\r\n      ImportError: libGL.so.1: cannot open shared object file: No such file or directory\r\n      or\r\n      libgthread-2.0.so.0: cannot open shared object file: No such file or directory\r\n      ```\r\n\r\n    **Solution:**\r\n\r\n      Install the libgl11-mesa-glx and libglib2.0-0 libraries. For Ubuntu this will be:\r\n\r\n      ```bash\r\n     sudo apt install libgl1-mesa-glx\r\n     sudo apt install libglib2.0-0\r\n      ```\r\n","funding_links":[],"categories":["Table of Contents"],"sub_categories":["AI - Frameworks and Toolkits"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foneapi-src%2Fvisual-process-discovery","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Foneapi-src%2Fvisual-process-discovery","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foneapi-src%2Fvisual-process-discovery/lists"}