{"id":13574357,"url":"https://github.com/oneapi-src/engineering-design-optimization","last_synced_at":"2025-04-04T14:32:28.102Z","repository":{"id":152282018,"uuid":"610975765","full_name":"oneapi-src/engineering-design-optimization","owner":"oneapi-src","description":"AI Starter Kit for Engineering Design Optimization using Intel® Extension for Pytorch","archived":true,"fork":false,"pushed_at":"2024-02-01T23:56:44.000Z","size":248,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-11-05T09:44:28.083Z","etag":null,"topics":["deep-learning","pytorch"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/oneapi-src.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-03-07T21:22:29.000Z","updated_at":"2024-10-26T19:21:38.000Z","dependencies_parsed_at":null,"dependency_job_id":"c094f98c-1ce1-4ca9-a5fc-51e6901aabb7","html_url":"https://github.com/oneapi-src/engineering-design-optimization","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oneapi-src%2Fengineering-design-optimization","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oneapi-src%2Fengineering-design-optimization/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oneapi-src%2Fengineering-design-optimization/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oneapi-src%2Fengineering-design-optimization/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/oneapi-src","download_url":"https://codeload.github.com/oneapi-src/engineering-design-optimization/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247194358,"owners_count":20899472,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","pytorch"],"created_at":"2024-08-01T15:00:50.785Z","updated_at":"2025-04-04T14:32:27.317Z","avatar_url":"https://github.com/oneapi-src.png","language":"Python","readme":"PROJECT NOT UNDER ACTIVE MANAGEMENT\n\nThis project will no longer be maintained by Intel.\n\nIntel has ceased development and contributions including, but not limited to, maintenance, bug fixes, new releases, or updates, to this project.  \n\nIntel no longer accepts patches to this project.\n\nIf you have an ongoing need to use this project, are interested in independently developing it, or would like to maintain patches for the open source software community, please create your own fork of this project.  \n\nContact: webadmin@linux.intel.com\n# **Engineering Design Optimizations**\r\n## **Table of Contents**\r\n - [Purpose](#Purpose)\r\n - [Reference Solution](#proposed-solution)\r\n - [Reference Implementation](#Reference-Implementation)\r\n - [Intel® Optimized Implementation](#optimizing-the-E2E-solution-with-Intel®-oneAPI)\r\n - [Performance Observations](#performance-observations)\r\n\r\n## **Purpose**\r\nIn the past, products were mainly valued by customers for their utility. Gradually, aesthetics, novelty and slick designs have also become important factors among other factors that customers look for when purchasing products. Designers and engineers in the manufacturing industry often struggle with introducing novelty as it is a time-consuming process that includes iterating through multiple designs, comparing designs, ensuring that utility and quality of products are not compromised on account of design changes, etc. Depending on the product or tool being designed, it could take days, months or even years to come up with the right design for a product that meets the usefulness, novelty, and quality criteria for products.  \r\n\r\nCustomers have become extremely demanding over the years and expect new products to be introduced often. Companies also run the risk of losing market share to competitors if they are unable to keep up with customer demands.  \r\n\r\nGenerative design is the next frontier in CAD design for engineers working in virtually all manufacturing industries. It harnesses the power of artificial intelligence (AI) to develop new high-performance design iterations that help solve complex challenges, reduce component weights and manufacturing costs, scale customization, optimize performance and reduce product development cycle time.  \r\n\r\nThis reference kit demonstrates how Generative Adversarial Networks can be used to optimize and improve novelty in product design.\r\n\r\n## **Reference Solution**\r\nConventional engineering design methods involve exploring designs and ideas interactively and even sometimes manually which add time and complexity to the design exploration. In fact, designers should spend vital time examining various design alternatives till they determine helpful solutions, so GAN data-driven methods such as the one described in this reference kit can enable designers to quickly synthesize new designs by reducing the complexity of design exploration.\r\nThis reference kit is a data-driven reference solution for design synthesis and proposes a method for guiding existing GAN models to synthesize novel designs. A state-of-the-art [StyleGAN2](https://github.com/lucidrains/stylegan2-pytorch) model is used to generate realistic designs. Other models (WideResNet, ResUNet) are then applied to automatically detect and locate the novel features in the generated designs, to then rewrite the original StyleGAN2 model using the detected most novel features through the concept of [rewriting GANs](https://rewriting.csail.mit.edu/).\\\r\n\u003cbr\u003eUsing a bicycle designs dataset to re-train the model, the StyleGAN2  model can create new bicycle designs with unique frames and handles, and generalize rare novelties to a broad set of designs, completely automatic and without requiring human intervention. As a post-processing step, a custom algorithm is applied on generated novel design images to suggest the most compact and novel designs.\r\n### *Key Implementation Details*\r\nThe method implemented in this reference kit leverages multiple models and machine learning algorithms to detect the novel features and applies these features to rewrite the original GAN model. Details of the components implemented are as follows:\r\n1. StyleGAN2 model architecture to synthesize design images\r\n2. Novelty Detection and localization\\\r\n   a. WideResNet model architecture to extract features from the GAN generated images and training dataset\\\r\n   b. Find the top novel features and localize them in the image\\\r\n   c. ResUNet based segmentation model architecture to predict the part of the bike for a new novel design image\r\n3. Rewrite the original StyleGAN2 model using detected most novel features.\r\n\r\nAs Generative Adversarial Networks (GANs) models are often slow to generate images, the overall process of generating the novel and optimized design is relatively slow. This reference kit also demonstrates how Intel® AI Analytics Toolkit can help in minimize the time taken by GAN and other deep learning models used in generating the most optimized and novel design and in accelerating the overall process. This will help productionize the suggested method to generate the novel designs on scale.\r\n\r\nThree benchmarkings are performed in this reference kit and are as follows:\r\n1. Feature Extraction Inference time - Stock vs Intel® Extension for PyTorch\r\n2. Segmentation Inference time- Stock vs Intel® Extension for PyTorch vs Intel® Neural Compressor\r\n3. E2E pipeline Inference time - Stock vs Intel® Extension for PyTorch vs Intel® Neural Compressor\r\n\r\n\u003e Note: Additional info/inputs required from Intel on Quantization of models (Currently quantization done for models in E2E pipeline)\r\n\r\n## **Reference Implementation**\r\nThe reference implementation of different deep learning models in this reference kit is based on PyTorch v1.13.0.\r\n\u003cbr\u003eThe StyleGAN2 model used in this implementation to generate bike design images is pre-trained on the Biked Dataset . Then, the novelty feature detection and localization method, which relies on features extracted from generated images using a pre-trained Wide Residual Newtorks50 (WideResNet50) architecture and a pre-trained ResUNet based segmentation model architecture used to segment the different parts of the bike in the generated bike design images. Finally, the segmented feature mask is used to modify the StyleGAN2 model using the model rewriting approach so that it generates the most novel and optimized bike designs.\r\n### **Use Case E2E Flow**\r\n\r\n![Use_case_flow](assets/e2e_flow.png)\r\n\r\n### **Experiment Details**\r\n\r\n### **Expected Input-Output**\r\n\r\n**Input**                                         | **Output** | \r\n| :---: | :---: |\r\n| Random Noise Data (400x512 size vector to generate 10 design images)|      Novel and Optimized Bike Design Image (512x512x3 size image)|\r\n\r\n### **Reference Sources**\r\n**Case Study Reference:**\r\nhttps://github.com/mfrashad/creativegan\r\n\r\n### ***Software Requirements***\r\n\r\n1. Python v3.9.15\r\n2. PyTorch v1.13.0\r\n3. Intel® Extension for PyTorch v1.13.0\r\n4. Intel® Neural Compressor v2.0\r\n\r\n### ***Solution Setup Prerequisite***\r\n#### ***Anaconda Installation***\r\nNote that this reference kit implementation already provides the necessary scripts to setup the software requirements. \r\nTo utilize these environment scripts, first install Anaconda/Miniconda by following the instructions at the following link\r\n\r\nhttps://docs.conda.io/projects/conda/en/latest/user-guide/install/index.html\r\nor\r\n[Anaconda installation](https://docs.anaconda.com/anaconda/install/linux/)\\\r\n**Note: If the conda is installed using pip, it will not work for this purpose. So please install conda from the above link only.**\r\n\r\n### ***Reference Solution Setup***\r\n### ***Repository Clone:***\r\nClone the repository to the desired location on your computer using the command given below:\r\n```sh\r\ngit clone https://github.com/oneapi-src/engineering-design-optimization\r\ncd engineering-design-optimization\r\n```\r\n\u003e Note: It is assumed that the present working directory is the root directory of this code repository. You can verify the present working directory location using the following command.\r\n```sh\r\npwd\r\n```\r\n```\r\noutput: \r\n\u003cAbsolute path to cloned directory\u003e/engineering-design-optimization\r\n```\r\n### ***Apply Solution Patch***\r\n\u003e The original repo has been restructured and modified to add some post-processing steps and to enable it for Intel optimizations to use this reference kit. A patch has been provided of these changes for users to apply on top of original repo.\r\nSteps to apply the patch are as follows:\r\n```\r\ncd src\r\ngit clone https://github.com/mfrashad/creativegan.git\r\ncd creativegan\r\ngit reset --hard 9b1abc2afdac2d9e168422613a457158f5952ad0\r\ngit apply --reject --whitespace=fix  ../creativegan_patch.patch\r\ncd ..\r\nmv run_benchmark_feature_extraction.py creativegan/run_benchmark_feature_extraction.py\r\nmv run_benchmark_resunet.py creativegan/run_benchmark_resunet.py\r\ncd ..\r\n```\r\n\u003e Note: It is assumed that the present working directory is the root directory of this code repository. You can verify the present working directory location using the following command. \r\n### ***Stock Environment Setup***\r\nFollow the below conda installation commands to setup the Stock environment along with the necessary packages for this model training and prediction.\r\n```sh\r\nconda env create --file env/stock/eng-design-stock.yml\r\n```\r\nThis command utilizes the dependencies found in the `env/stock/eng-design-stock.yml` file to create an environment as follows:\r\n\r\n**YAML file**                       | **Environment Name**         |  **Configuration** |\r\n| :---: | :---: | :---: |\r\n| `env/stock/eng-design-stock.yml`             | `eng-design-stock` | Python v3.9.15, PyTorch v1.13.0\r\n\r\nUse the following command to activate the environment that was created:\r\n*Activate stock conda environment*\r\n```sh\r\nconda activate eng-design-stock\r\n```\r\n###  **Dataset and Pretrained Models**\r\n\r\n**Dataset**                     |  **Dataset Reference**                      |**Pretrained Models**                                                                                |**Pretrained Models Reference** |\r\n| :---:                         | :---:                                       | :---:                                                                                               | :---: |\r\n| Bicycle design images : 4512   | https://github.com/Lyleregenwetter/BIKED   |StyleGAN2 model pretrained on Biked dataset, Segmentation model pretrained on Bike Part dataset      |https://github.com/mfrashad/creativegan\r\n\r\n#### ***Biked Dataset***\r\nIn this reference solution, Biked dataset has been used which is a diverse assortment of 4512 manually-designed bicycle models that span all common bicycle styles and feature unique models from remote corners of the bicycle design space.\r\n\r\n#### ***Pretrained Models***\r\nIn this reference solution, pretrained models have been used for StyleGAN2 image generation model and ResUNet Segmentation model.\r\n\r\nA python script \"src/download.sh\" has been provided in the source code to download both dataset and pretrained models.\\\r\nRun the following command to download the dataset and pretrained models.\r\n```\r\nsh src/download.sh\r\n```\r\nAfter running the above command, \"datasets\" and \"models\" folders will be created in the root directory of the code repository and dataset and pretrained models will be downloaded and extracted in the respective folders.\r\n###  **Benchmarking**\r\n\r\n#### ***Novel Feature Extraction***\r\nThis module uses pretrained WideResNet model to extract features from training images.\r\n```\r\nusage: python src/creativegan/run_benchmark_feature_extraction.py [--data_path][--intel][--novelty_infer_batch_size][--use_quantized_models][--wideresnet_quantized_model_path]\r\n```\r\n\r\nArguments:\u003cbr\u003e\r\n```\r\n  --data_path                         Path to dataset, the folder should directly contain the images\r\n  --intel                             Give 1 for enabling intel PyTorch optimizations, default is 0  \r\n  --novelty_infer_batch_size          Batch size to extract the features from design images using WideResNet model\r\n  --use_quantized_models              Use INC Quantized model in pipeline\r\n  --wideresnet_quantized_model_path   WideResNet quantized model path\r\n\r\n```\r\n\u003cb\u003eExample:\u003c/b\u003e\r\n```sh\r\npython src/creativegan/run_benchmark_feature_extraction.py \\\r\n --data_path ./datasets/biked_subset \\\r\n --intel 0 \\\r\n --novelty_infer_batch_size 32 \\\r\n --use_quantized_models 0\r\n```\r\n\r\n#### ***Novel feature extraction by Parts using ResUNet Segmentation Model***\r\nThis module uses pretrained ResUNet model to segment novel feature masks from extracted features.\r\n```\r\nusage: run_benchmark_resunet.py [-h] -d DATA_PATH [-i INTEL] [-b BATCH_SIZE] [-uqm] -m SEG_MODEL_PATH [-qm QUANT_SEG_MODEL_PATH] [-cls SEG_TOTAL_CLASS]\r\n```\r\n\r\nArguments:\u003cbr\u003e\r\n```\r\n-d DATA_PATH, --data_path DATA_PATH Path to dataset, the folder should directly contain the images\r\n-i INTEL, --intel INTEL To apply Intel IPEX optimization\r\n-b BATCH_SIZE, --batch_size BATCH_SIZE Batch size to be used by the model for inference\r\n-uqm, --use_quantized_models Use INC Quantized model in pipeline\r\n-m SEG_MODEL_PATH, --seg_model_path SEG_MODEL_PATH Path to segmentation model\r\n-qm QUANT_SEG_MODEL_PATH, --quant_seg_model_path QUANT_SEG_MODEL_PATH Path to quantized segmentation model\r\n-cls SEG_TOTAL_CLASS, --seg_total_class SEG_TOTAL_CLASS Total class/channel in segmentation model  \r\n```\r\n\u003cb\u003eExample:\u003c/b\u003e\r\n```sh\r\npython src/creativegan/run_benchmark_resunet.py \\\r\n --data_path ./datasets/biked_subset \\\r\n --intel 0 \\\r\n --batch_size 32 \\\r\n --seg_model_path ./models/segmentation_bike.pt \\\r\n --seg_total_class 7\r\n```\r\n\r\n#### ***End-to-End Novel Design Generation Pipeline***\r\nThe end-to-end pipeline script given below is used to generate most novel, unique and optimized bike design suggestions. In this pipeline, pretrained StyleGAN2 model has been used to generate unique design suggestions, pretrained WideResNet model has been used to extract novel features in the suggested designs and pretrained ResUNet model has been used to segment the novel feature mask in the design images. The segmented feature masks have been used to modify the StyleGAN2 model using model rewriting approach so that it generates most novel and optimized bike designs.\r\n\r\n```\r\nusage: sh src/experiment.sh\r\n```\r\nThe following parameters can be configured in the above script as per the current configuration and need.\r\n```\r\n--model_path                            Path to pretrained StyleGAN2 model\r\n--seg_model_path                        Path to pretrained Segmentation model\r\n--data_path                             Path to dataset, the folder should directly contain the images\r\n--copy_id                               Random seed to create a noise to input to the pipeline\r\n--gan_infer_batch_size                  Batch size to generate design samples using StyleGAN2 model (should be less than 14 as per the given machine configuration)\r\n--segment_infer_batch_size              Batch size to segment the design images using UNET model\r\n--novelty_infer_batch_size              Batch size to extract the features from design images using WideResNet model (between 32-512 as per the given machine configuration)\r\n--intel                                 Give 1 for enabling intel PyTorch optimizations, default is 0\r\n```\r\nOutput of this pipeline is the most optimized bike design image, which will be saved as \"best_design_image.jpg\" in current directory.\r\n\r\n\u003cb\u003eExample:\u003c/b\u003e\r\n```sh\r\nsh src/experiment.sh\r\n```\r\n\r\n**Sample output**\r\n\r\n![image](assets/sample_output.png)\r\n\r\n## **Optimizing the E2E solution with Intel Optimizations for Engineering Design Optimizations**\r\nIn order to derive the most insightful and beneficial actions to take, the data generated through various feature sets and algorithms needs to be analysed, thus requiring frequent re-runs of the algorithms under many different parameter sets. To utilize all the hardware resources efficiently, software \r\noptimizations cannot be ignored.   \r\n \r\nThis reference kit solution extends to demonstrate the advantages of using the Intel® AI Analytics Toolkit on the task of building a pipeline for generating optimized engineering designs.\r\n\r\n### Use Case E2E flow\r\n![image](assets/e2e_optimized.png)\r\n\r\n### **Optimized Solution Setup**\r\n\r\n#### ***Intel Environment Setup***\r\nFollow the below conda installation commands to setup the Intel environment along with the necessary packages.\r\n\u003eNote: It is assumed that the present working directory is the root directory of this code repository\r\n\r\n```shell\r\nconda env create --file env/intel/eng-design-intel.yml\r\n```\r\nThis command utilizes the dependencies found in the `env/intel/eng-design-intel.yml` file to create an environment as follows:\r\n\r\n**YAML file**                                 | **Environment Name** |  **Configuration** |\r\n| :---: | :---: | :---: |\r\n| `env/intel/eng-design-intel.yml`             | `eng-design-intel` | Python v3.9.15 with Intel® Extension for PyTorch v1.13.0\r\n\r\nIntel environment will be used for the workload implementation to arrive at first level solution.\r\n\r\nUse the following command to activate the environment that was created:\r\n```shell\r\nconda activate eng-design-intel\r\n```\r\n#### ***Optimized Software Components***\r\nIntel® Extension for PyTorch (version 1.13.0) framework has been optimized using oneAPI Deep Neural Network Library (oneDNN) primitives, a popular performance \r\nlibrary for deep learning applications. It provides accelerated implementations of numerous popular DL algorithms that optimize performance on \r\nIntel® hardware only requiring simple lines of modifications to existing code.\r\n\r\n**Intel® Extension for PyTorch (IPEX)** extends PyTorch with Optimizations for extra performance boost on Intel hardware.\r\nThe below changes have been done to the stock PyTorch training code base to utilize the Intel® Extension for PyTorch* performance.\r\nOne can enable the intel flag to incorporate below Intel PyTorch optimizations.\r\n```\r\nimport intel_extension_for_pytorch  as ipex\r\n\r\nmodel=ipex.optimize(model)\r\n```\r\n###  **Benchmarking**\r\n\r\n#### ***Intel Optimized Novel Feature Extraction***\r\nThis module uses pretrained WideResNet model to extract features from training images.\r\n```\r\nusage: python src/creativegan/run_benchmark_feature_extraction.py [--data_path][--intel][--novelty_infer_batch_size][--use_quantized_models][--wideresnet_quantized_model_path]\r\n```\r\n\r\nArguments:\u003cbr\u003e\r\n```\r\n  --data_path                         Path to dataset, the folder should directly contain the images\r\n  --intel                             Give 1 for enabling intel PyTorch optimizations, default is 0  \r\n  --novelty_infer_batch_size          Batch size to extract the features from design images using WideResNet model\r\n  --use_quantized_models              Use INC Quantized model in pipeline\r\n  --wideresnet_quantized_model_path   WideResNet quantized model path\r\n\r\n```\r\n\u003cb\u003eExample:\u003c/b\u003e\r\n```sh\r\npython src/creativegan/run_benchmark_feature_extraction.py \\\r\n --data_path ./datasets/biked_subset \\\r\n --intel 1 \\\r\n --novelty_infer_batch_size 32 \\\r\n --use_quantized_models 0\r\n```\r\n\r\n#### ***Intel Optimized Novel feature extraction by Parts using ResUNet Segmentation Model***\r\nThis module uses pretrained ResUNet model to segment novel feature masks from extracted features.\r\n```\r\nusage: run_benchmark_resunet.py [-h] -d DATA_PATH [-i INTEL] [-b BATCH_SIZE] [-uqm] -m SEG_MODEL_PATH [-qm QUANT_SEG_MODEL_PATH] [-cls SEG_TOTAL_CLASS]\r\n```\r\n\r\nArguments:\u003cbr\u003e\r\n```\r\n-d DATA_PATH, --data_path DATA_PATH Path to dataset, the folder should directly contain the images\r\n-i INTEL, --intel INTEL To apply Intel IPEX optimization\r\n-b BATCH_SIZE, --batch_size BATCH_SIZE Batch size to be used by the model for inference\r\n-uqm, --use_quantized_models Use INC Quantized model in pipeline\r\n-m SEG_MODEL_PATH, --seg_model_path SEG_MODEL_PATH Path to segmentation model\r\n-qm QUANT_SEG_MODEL_PATH, --quant_seg_model_path QUANT_SEG_MODEL_PATH  Path to quantized segmentation model\r\n-cls SEG_TOTAL_CLASS, --seg_total_class SEG_TOTAL_CLASS Total class/channel in segmentation model  \r\n```\r\n\u003cb\u003eExample:\u003c/b\u003e\r\n```sh\r\npython src/creativegan/run_benchmark_resunet.py \\\r\n --data_path ./datasets/biked_subset \\\r\n --intel 1 \\\r\n --batch_size 32 \\\r\n --seg_model_path ./models/segmentation_bike.pt \\\r\n --seg_total_class 7\r\n```\r\n\r\n#### ***Intel Optimized End-to-End Novel Design Generation Pipeline***\r\nThe end-to-end pipeline script given below is used to generate most novel, unique and optimized bike design suggestions. In this pipeline, pretrained StyleGAN2 model has been used to generate unique design suggestions, pretrained WideResNet model has been used to extract novel features in the suggested designs and pretrained ResUNet model has been used to segment the novel feature mask in the design images. The segmented feature masks have been used to modify the StyleGAN2 model using model rewriting approach so that it generates the most novel and optimized bike designs.\r\n\r\n```\r\nusage: sh src/intel-experiment.sh\r\n```\r\nThe following parameters can be configured in the above script as per the current configuration and need.\r\n```\r\n--model_path                            Path to pretrained StyleGAN2 model\r\n--seg_model_path                        Path to pretrained Segmentation model\r\n--data_path                             Path to dataset, the folder should directly contain the images\r\n--copy_id                               Random seed to create a noise to input to the pipeline\r\n--gan_infer_batch_size                  Batch size to generate design samples using StyleGAN2 model (should be less than 14 as per the given machine configuration)\r\n--segment_infer_batch_size              Batch size to segment the design images using UNET model\r\n--novelty_infer_batch_size              Batch size to extract the features from design images using WideResNet model (between 32-512 as per the given machine configuration)\r\n--intel                                 Give 1 for enabling intel PyTorch optimizations, default is 0\r\n```\r\nOutput of this pipeline is the most optimized bike design image, which will be saved as \"best_design_image.jpg\" in current directory.\r\n\r\n\u003cb\u003eExample:\u003c/b\u003e\r\n```sh\r\nsh src/intel-experiment.sh\r\n```\r\n**Sample output**\r\n\r\n![image](assets/sample_output.png)\r\n\r\n### **Model Conversion using Intel® Neural Compressor**\r\nIntel® Neural Compressor is used to quantize the FP32 Model to the INT8 Model.\r\nIntel® Neural Compressor supports many optimization methods. Here, post-training optimization with `Default quantization mode` method has been used to \r\nquantize the FP32 model.\r\n\r\n##### **Quantization of ResUNet Model**\r\n\r\nThe below script is used to quantize the pretrained ResUNet FP32 segmentation model. The output of this script is the quantized ResUNet INT8 segmentation model.\r\n```\r\nusage: python src/quantize_resunet_inc.py [--model_path] [--out_path]\r\n```\r\n\r\nArguments:\r\n```\r\n  --model_path             Path to pretrained FP32 segmentation model       \r\n  --out_path               Directory path to save the quantized INT8 segmentation model\r\n```\r\n\r\n\u003cb\u003eExample\u003c/b\u003e\r\n```sh\r\npython src/quantize_resunet_inc.py --model_path './models/segmentation_bike.pt' --out_path './models/quantized_segmentation_model'\r\n```\r\nThe above script will quantize the FP32 segmentation model and save the quantized INT8 segmentation model in the \"./models/quantized_segmentation_model\" directory.\r\n\r\nTo use Quantized Model for inference, run the below command.\r\n```sh\r\npython src/creativegan/run_benchmark_resunet.py \\\r\n --data_path ./datasets/biked_subset \\\r\n --intel 1 \\\r\n --batch_size 32 \\\r\n --seg_model_path ./models/segmentation_bike.pt \\\r\n --seg_total_class 7 \\\r\n --use_quantized_models \\\r\n --quant_seg_model_path ./models/quantized_segmentation_model/best_model.pt\r\n```\r\n\r\n\r\n#### ***Intel Optimized Implementation of End-to-End Novel Design Generation using Intel® Neural Compressor Quantized Int8 Models***\r\nThe end-to-end pipeline script given below is used to generate most novel, unique and optimized bike design suggestions. In this pipeline, pretrained StyleGAN2 model has been used to generate unique design suggestions, pretrained WideResNet model has been used to extract novel features in the suggested designs and pretrained ResUNet model has been used to segment the novel feature mask in the design images. The segmented feature masks have been used to modify the StyleGAN2 model using model rewriting approach so that it generates most novel and optimized bike designs.\r\n\r\n```\r\nusage: sh src/inc-experiment.sh\r\n```\r\nThe following parameters can be configured in the above script as per the current configuration and need.\r\n```\r\n--model_path                            Path to pretrained StyleGAN2 model\r\n--seg_model_path                        Path to pretrained Segmentation model\r\n--data_path                             Path to dataset, the folder should directly contain the images\r\n--copy_id                               Random seed to create a noise to input to the pipeline\r\n--gan_infer_batch_size                  Batch size to generate design samples using StyleGAN2 model (should be less than 14 as per the given machine configuration)\r\n--segment_infer_batch_size              Batch size to segment the design images using UNET model\r\n--novelty_infer_batch_size              Batch size to extract the features from design images using WideResNet model (between 32-512 as per the given machine configuration)\r\n--intel                                 Give 1 for enabling intel PyTorch optimizations, default is 0\r\n--resunet_quantized_model_path          Path to quantized INT8 segmentation model\r\n```\r\nOutput of this pipeline is the most optimized bike design image, which will be saved as \"best_design_image.jpg\" in current directory.\r\n\r\n\u003cb\u003eExample:\u003c/b\u003e\r\n```sh\r\nsh src/inc-experiment.sh\r\n```\r\n\r\n## **Performance Observations**\r\nThis section covers the novelty detection (feature extraction) time comparison between Stock PyTorch version and Intel PyTorch Extension for WideResNet model. The results are used to calculate the performance gain achieved by using Intel One API packages over \r\nstock version of similar packages.\r\n\r\n![image](assets/feature_extraction_plot.png)\r\n\u003cbr\u003e**Takeaway**\u003cbr\u003eIntel® PyTorch Extension with Intel Optimizations offers batch inference time speed-up of \r\n1.38x using pretrained model compared to stock versions without Intel Optimizations.\r\n\r\nThis section covers the segmentation time comparison between Stock PyTorch version and Intel PyTorch Extension for ResUNet model. The results are used to calculate the performance gain achieved by using Intel One API packages over \r\nstock version of similar packages.\r\n\r\n![image](assets/segmentation_plot.png)\r\n\u003cbr\u003e**Takeaway**\u003cbr\u003eIntel® PyTorch Extension with Intel Optimizations offers batch inference time speed-up of \r\n1.36x using pretrained model compared to stock versions without Intel Optimizations.\r\n\u003cbr\u003eIntel® PyTorch Extension with Intel Optimizations and Intel® Neural Compressor offers batch inference time speed-up of \r\n2.93x using pretrained model compared to stock versions without Intel Optimizations. \r\n\r\nThis section covers the end-to-end novel image generation (image generation, image segmentation and feature extraction) time comparison between Stock PyTorch version and Intel PyTorch Extension for StyleGAN2, ResUNet and WideResNet model. The results are used to calculate the performance gain achieved by using Intel One API packages over \r\nstock version of similar packages.\r\n\r\n![image](assets/e2e_plot.png)\r\n\u003cbr\u003e**Takeaway**\u003cbr\u003eIntel® PyTorch Extension with Intel Optimizations and Intel® Neural Compressor collectively offers batch inference time speed-up of \r\n1.04x using pretrained models compared to stock versions without Intel Optimizations for end-to-end pipeline.\r\n\r\n#### **Conclusion**\r\nThis reference kit can be used by any ML engineer for creating optimized engineering designs and also accelerate their implementation using the Intel® oneAPI packages.\r\n\r\n### Notices \u0026 Disclaimers\r\nPerformance varies by use, configuration, and other factors. Learn more on the [Performance Index site](https://edc.intel.com/content/www/us/en/products/performance/benchmarks/overview/). \r\nPerformance results are based on testing as of dates shown in configurations and may not reflect all publicly available updates. See backup for configuration details.  No product or component can be absolutely secure. \r\nCosts and results may vary. \r\nIntel technologies may require enabled hardware, software or service activation. \u003cbr\u003e\r\n© Intel Corporation. Intel, the Intel logo, and other Intel marks are trademarks of Intel Corporation or its subsidiaries.  Other names and brands may be claimed as the property of others.  \r\n\r\n**Date Testing Performed**: March 2023 \r\n\r\n**Configuration Details and Workload Setup**: Azure D8v5 (Intel® Xeon® Platinum 8370C CPU @ 2.80GHz), 1 Socket, 4 Cores per Socket, 2 Threads per Core, Turbo: On, Total Memory: 32 GB, OS: Ubuntu 20.04, Kernel: Linux 5.15.0-1019-azure. Framework/Toolkit: PyTorch v1.13.0, Intel® Extension for PyTorch v1.13.0, Intel® Neural Compressor v2.0, Python 3.9.15. Dataset size: ~2k training RGB images of bike Model: StyleGAN2 for image generation, WideResNet for feature extraction and ResUNet for segmentation of novel features. \r\n\r\nTo the extent that any public or non-Intel datasets or models are referenced by or accessed using tools or code on this site those datasets or models are provided by the third party indicated as the content source. Intel does not create the content and does not warrant its accuracy or quality. By accessing the public content, or using materials trained on or with such content, you agree to the terms associated with that content and that your use complies with the applicable license.\r\n \r\nIntel expressly disclaims the accuracy, adequacy, or completeness of any such public content, and is not liable for any errors, omissions, or defects in the content, or for any reliance on the content. Intel is not liable for any liability or damages relating to your use of public content.\r\n\r\n**Testing performed by** Intel Corporation\r\n\r\n## Appendix\r\n### **Running on Windows**\r\nThe reference kits commands are linux based, in order to run this on Windows, goto Start and open WSL and follow the same steps as running on a linux machine starting from git clone instructions. If WSL is not installed you can [install WSL](https://learn.microsoft.com/en-us/windows/wsl/install).\r\n\u003e **Note** If WSL is installed and not opening, goto Start ---\u003e Turn Windows feature on or off and make sure Windows Subsystem for Linux is checked. Restart the system after enabling it for the changes to reflect.\r\n\r\n### **Experiment setup**\r\n\r\n| Platform                          | Ubuntu 20.04\r\n| :---                              | :---\r\n| Hardware                          | Azure Standard_D8_V5 (Icelake)\r\n| Software                          | Intel® Distribution for Python v3.9.15, Intel® Extension for PyTorch v1.13.0, Intel® Neural Compressor v2.0\r\n| What you will learn               | Advantage of using components in Intel® Extension for PyTorch over the stock versions\r\n\r\n\r\n### Known Issues\r\n\r\n1. Common prerequisites required to run python scripts in linux system.\r\n    Install gcc and curl.  For Ubuntu, this will be: \r\n\r\n      ```bash\r\n      apt-get update\r\n      apt-get install gcc\r\n      apt-get install git\r\n      ```\r\n\r\n2. ImportError: libGL.so.1: cannot open shared object file: No such file or directory\r\n   \r\n    **Issue:**\r\n      ```\r\n      ImportError: libGL.so.1: cannot open shared object file: No such file or directory\r\n      or\r\n      libgthread-2.0.so.0: cannot open shared object file: No such file or directory\r\n      ```\r\n\r\n    **Solution:**\r\n\r\n      Install the libgl1 and libglib2.0-0 libraries. For Ubuntu this will be:\r\n\r\n      ```bash\r\n     sudo apt install libgl1\r\n     sudo apt install libglib2.0-0\r\n      ```\r\n3. Optimization instructions for StyleGAN2 and quantization instructions for StyleGAN2 and WideResNet, as well as performance benchmarks, to come.\r\n","funding_links":[],"categories":["Table of Contents"],"sub_categories":["AI - Frameworks and Toolkits"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foneapi-src%2Fengineering-design-optimization","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Foneapi-src%2Fengineering-design-optimization","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foneapi-src%2Fengineering-design-optimization/lists"}