{"id":13574480,"url":"https://github.com/oneapi-src/traffic-camera-object-detection","last_synced_at":"2025-04-04T15:31:05.397Z","repository":{"id":66145935,"uuid":"592528673","full_name":"oneapi-src/traffic-camera-object-detection","owner":"oneapi-src","description":"AI Starter Kit for traffic camera object detection using Intel® Extension for Pytorch","archived":true,"fork":false,"pushed_at":"2024-05-08T23:57:55.000Z","size":395,"stargazers_count":9,"open_issues_count":0,"forks_count":4,"subscribers_count":5,"default_branch":"main","last_synced_at":"2024-11-05T09:44:40.324Z","etag":null,"topics":["ai-starter-kit","deep-learning","pytorch"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/oneapi-src.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-01-23T23:12:17.000Z","updated_at":"2024-09-26T02:46:56.000Z","dependencies_parsed_at":null,"dependency_job_id":"5323ee65-c392-4955-a4e9-6d81773422bb","html_url":"https://github.com/oneapi-src/traffic-camera-object-detection","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oneapi-src%2Ftraffic-camera-object-detection","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oneapi-src%2Ftraffic-camera-object-detection/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oneapi-src%2Ftraffic-camera-object-detection/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oneapi-src%2Ftraffic-camera-object-detection/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/oneapi-src","download_url":"https://codeload.github.com/oneapi-src/traffic-camera-object-detection/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247202655,"owners_count":20900825,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai-starter-kit","deep-learning","pytorch"],"created_at":"2024-08-01T15:00:51.990Z","updated_at":"2025-04-04T15:31:00.388Z","avatar_url":"https://github.com/oneapi-src.png","language":"Python","readme":"PROJECT NOT UNDER ACTIVE MANAGEMENT\n\nThis project will no longer be maintained by Intel.\n\nIntel has ceased development and contributions including, but not limited to, maintenance, bug fixes, new releases, or updates, to this project.  \n\nIntel no longer accepts patches to this project.\n\nIf you have an ongoing need to use this project, are interested in independently developing it, or would like to maintain patches for the open source software community, please create your own fork of this project.  \n\nContact: webadmin@linux.intel.com\n# Traffic Camera Object Detection\r\n\r\n## Introduction\r\n\r\nThis reference kit provides an end-to-end (E2E) Machine Learning (ML) workflow for traffic management using Pytorch*, [Intel® Neural Compressor](https://www.intel.com/content/www/us/en/developer/tools/oneapi/neural-compressor.html), [OpenVINO*](https://github.com/openvinotoolkit/openvino) and [Intel® Optimizations for PyTorch*](https://www.intel.com/content/www/us/en/developer/tools/oneapi/optimization-for-pytorch.html). Traffic management is an important issue plaguing well-established and rapidly growing cities. Bad traffic management and accidents impact performance at work, but mainly quality of life.\r\n\r\nCheck out more workflow examples in the [Developer Catalog](https://developer.intel.com/aireferenceimplementations).\r\n\r\n## Solution Technical Overview\r\n\r\nTraffic accidents are dangerous and often result in fatalities. The time taken to respond to accidents and send medical aid depends on multiple human factors. Often a lack of timely response impacts the likelihood of survival.\r\n\r\nIntelligent traffic management systems leveraging video surveillance, automated accident detection, and prediction solutions will go a long way in improving safety and traffic flow in cities. These solutions need to provide real-time insights and recommendations to be effective.\r\n\r\nLeveraging the power of edge computing, communication between devices with very low latency can be achieved. An example scenario includes traffic signals and vehicles exchanging information about pedestrians on crosswalks, communication between surveillance cameras and vehicles regarding proximity, and the possibility of an accident with very low latency; thus enabling preventive action in near real-time.\r\n\r\nDeep Learning (DL) algorithms can predict traffic accidents based on live traffic camera feeds. Computer vision tasks and complex feature analysis can be accomplished easily with high performance by leveraging DL algorithms.\r\n\r\nArtificial Intelligence (AI) based detection algorithms deployed at the edge enable real-time analytics of video feeds and detection of accidents and other issues, thus improving overall traffic management. AI-enabled traffic camera imaging aid helps address traffic management challenges by reducing congestion on road, improving the accuracy of pedestrian/vehicle identification, improving device-2-device communication, and helping reduce accidents.\r\n\r\nThis workflow implementation is a reference solution to the described use case that includes an optimized reference E2E architecture enabled with [Intel® Optimizations for PyTorch*](https://www.intel.com/content/www/us/en/developer/tools/oneapi/optimization-for-pytorch.html#gs.1sivqb) and [Intel® Neural Compressor](https://www.intel.com/content/www/us/en/developer/tools/oneapi/neural-compressor.html#gs.1sixws) available as part of [Intel® AI Tools](https://www.intel.com/content/www/us/en/developer/tools/oneapi/ai-analytics-toolkit.html#gs.1sgo4k) optimizations and [OpenVINO*](https://github.com/openvinotoolkit/openvino):\r\n\r\n* **Intel® Optimizations for PyTorch***\r\n\r\n  PyTorch* is an AI and machine learning framework popular for both research and production usage. This open source library is often used for deep learning applications whose compute-intensive training and inference test the limits of available hardware resources. Intel releases its newest optimizations and features in Intel® Optimizations for PyTorch* before upstreaming them into open source PyTorch.\r\n\r\n* **Intel® Neural Compressor**\r\n\r\n  [Intel® Neural Compressor](https://www.intel.com/content/www/us/en/developer/tools/oneapi/neural-compressor.html) performs model compression to reduce the model size and increase the speed of deep learning inference for deployment on CPUs or GPUs. This open source Python* library automates popular model compression technologies, such as quantization, pruning, and knowledge distillation across multiple deep learning frameworks.\r\n\r\n* **OpenVINO***\r\n\r\n  [OpenVINO*](https://github.com/openvinotoolkit/openvino) is an open-source toolkit for optimizing and deploying AI inference.\r\n\r\n  * Boost deep learning performance in computer vision, automatic speech recognition, natural language processing and other common tasks.\r\n  * Use models trained with popular frameworks like TensorFlow, PyTorch and more.\r\n  * Reduce resource demands and efficiently deploy on a range of Intel® platforms from edge to cloud.\r\n\r\n## Solution Technical Details\r\n\r\n### Dataset\r\n\r\nThe dataset to be used is the [Pascal VOC dataset](http://host.robots.ox.ac.uk/pascal/VOC/index.html). It will be downloaded automatically when running the `train.py` script and divided into folders for training and validation.\r\n\r\nThe training data consists of a set of images; each image has an annotation file giving a bounding box and object class label for each object in one of the twenty classes present in the image. Multiple objects from multiple classes may be present in the same image.\r\n\r\n\u003e***Note:*** Please see this dataset applicable license for terms and conditions. Intel®Corporation does not own the rights to this dataset and does not confer any rights to it.\r\n\r\nFolder structure looks as below after data downloaded:\r\n\r\n```text\r\ndata/\r\n  ├── images\r\n  │   ├── VOCdevkit\r\n  │   │   ├── VOC2007\r\n  │   │   │   ├── Annotations\r\n  │   │   │   ├── ImageSets\r\n  │   │   │   │   ├── Layout\r\n  │   │   │   │   ├── Main\r\n  │   │   │   │   └── Segmentation\r\n  │   │   │   ├── JPEGImages\r\n  │   │   │   ├── SegmentationClass\r\n  │   │   │   └── SegmentationObject\r\n  │   │   └── VOC2012\r\n  │   │       ├── Annotations\r\n  │   │       ├── ImageSets\r\n  │   │       │   ├── Action\r\n  │   │       │   ├── Layout\r\n  │   │       │   ├── Main\r\n  │   │       │   └── Segmentation\r\n  │   │       ├── JPEGImages\r\n  │   │       ├── SegmentationClass\r\n  │   │       └── SegmentationObject\r\n  │   ├── test2007\r\n  │   ├── train2007\r\n  │   ├── train2012\r\n  │   ├── val2007\r\n  │   └── val2012\r\n  └── labels\r\n      ├── test2007\r\n      ├── train2007\r\n      ├── train2012\r\n      ├── val2007\r\n      └── val2012\r\n```\r\n\r\n## Validated Hardware Details\r\n\r\n[Intel® oneAPI](https://www.intel.com/content/www/us/en/developer/tools/oneapi/overview.html) is used to achieve quick results even when the data for a model is huge. It provides the capability to reuse the code present in different languages so that the hardware utilization is optimized to provide these results.\r\n\r\n| Recommended Hardware                                            | Precision\r\n| ----------------------------------------------------------------|-----------\r\n| CPU: Intel® 2nd Gen Xeon® Platinum 8280 CPU @ 2.70GHz or higher | FP32, INT8, BF16\r\n| RAM: 187 GB                                                     |\r\n| Recommended Free Disk Space: 20 GB or more                      |\r\n\r\n\u003e***Note:*** BF16 can be enabled on Intel® Fourth Gen Xeon®, previous generations of Xeon® might not be compatible.\r\n\r\nCode was tested on Ubuntu\\* 22.04 LTS.\r\n\r\n\r\n## How it Works\r\n\r\nThe following diagram shows the traffic camera object detection E2E workflow:\r\n\r\n![Use_case_flow](assets/intel-pipeline.png)\r\n\r\nThe following is an example of expected input and output:\r\n\r\n**Input**                                 | **Output** |\r\n| :---: | :---: |\r\n| Traffic Camera Live Feed          |  Detected Objects(Vehicle/Pedestrians) and Alarm(High Risk/Low Risk) for possible accident scenarios\r\n| ![raw image](assets/sample_input.png)      |  ![output image](assets/sample_output.png)\r\n\r\n### Intelligent Traffic Management\r\n\r\nThis reference kit uses a general detection model capable of distinguishing objects that would be relevant to traffic cameras. It preprocesses a Pascal VOC dataset by combining it with COCO classes using OpenCV*. A transfer learning approach is performed using an advanced pre-trained real-time object detection YOLOv5 model, which is further trained to detect vehicles and pedestrians. The dataset is first preprocessed using OpenCV* and NumPy, and then NumPy based postprocessing is performed using Non-Maxima Suppression (NMS) and centroid-based distance calculations for possible collision detection, which could be used for example to warn vehicle drivers via device-2-device communication.\r\n\r\nSince GPUs are typically the choice for Deep Learning and AI processing to achieve a higher Frames Per Second (FPS) rate, to offer a more cost-effective option leveraging a CPU, we use the quantization technique, leveraging [Intel® AI Tools](https://www.intel.com/content/www/us/en/developer/tools/oneapi/ai-analytics-toolkit.html#gs.1sgmnd), to achieve high FPS by performing vectorized operations on CPUs itself.\r\n\r\nBy quantizing/compressing the model (from floating point to integer model), while maintaining a similar level of accuracy as the floating point model, we demonstrate efficient utilization of underlying resources when deployed on edge devices with low processing and memory capabilities.\r\nModel has been quantized using Intel® Neural Compressor and Intel® Distribution of OpenVINO* which has shown high-performance vectorized operations on Intel® platforms.\r\n\r\n### Get Started\r\n\r\nStart by defining an environment variable that will store the workspace path, these directories will be created in further steps and will be used for all the commands executed using absolute paths.\r\n\r\n[//]: # (capture: baremetal)\r\n```bash\r\nexport WORKSPACE=$PWD/traffic-camera-object-detection\r\nexport DATA_DIR=$WORKSPACE/data\r\nexport OUTPUT_DIR=$WORKSPACE/output\r\nexport CONFIG_DIR=$WORKSPACE/config\r\nexport YOLO_DIR=$WORKSPACE/src/yolov5\r\n```\r\n\r\n### Download the Workflow Repository\r\n\r\nCreate a working directory for the workflow and clone the [main repository](https://github.com/oneapi-src/traffic-camera-object-detection) into your working directory.\r\n\r\n[//]: # (capture: baremetal)\r\n```bash\r\nmkdir -p $WORKSPACE \u0026\u0026 cd $WORKSPACE\r\n```\r\n\r\n```bash\r\ngit clone https://github.com/oneapi-src/traffic-camera-object-detection.git $WORKSPACE\r\n```\r\n\r\n[//]: # (capture: baremetal)\r\n```bash\r\nmkdir -p $DATA_DIR $OUTPUT_DIR/models $OUTPUT_DIR/images $YOLO_DIR\r\n```\r\n\r\n### Set Up Conda\r\n\r\nTo learn more, please visit [install anaconda on Linux](https://docs.anaconda.com/free/anaconda/install/linux/).\r\n\r\n```bash\r\nwget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh\r\nbash Miniconda3-latest-Linux-x86_64.sh\r\nrm Miniconda3-latest-Linux-x86_64.sh\r\n```\r\n\r\n### Set Up Environment\r\n\r\nThe conda environment dependencies are kept in `$WORKSPACE/env/intel_env.yml`.\r\n\r\n| **Package** | **Version**\r\n| :---        | :---\r\n| python | 3.9\r\n| opencv | 4.8.0\r\n| intel-aikit-pytorch | 2024.0\r\n| neural-compressor | 2.3.1\r\n| torchvision | 0.15.2\r\n| seaborn | 0.13.0\r\n| tqdm | 4.66.1\r\n| tensorboard | 2.15.1\r\n| pillow | 10.0.1\r\n| ultralytics | 8.0.227\r\n| gitpython | 3.1.40\r\n| pip | 23.3.1\r\n| opencv-python-headless | 4.8.1.78\r\n| thop | 0.1.1-2209072238\r\n| ipython | 8.18.1\r\n| openvino-dev[pytorch,onnx] | 2023.2.0\r\n| onnx | 1.14.1\r\n\r\nTo install the conda environment follow the next instructions:\r\n\r\n```bash\r\n# If you have conda 23.10.0 or greater you can skip the following two lines\r\n# since libmamba is already set as the default solver.\r\nconda install -n base conda-libmamba-solver -y\r\nconda config --set solver libmamba\r\nconda env create -f $WORKSPACE/env/intel_env.yml\r\n```\r\n\r\nEnvironment setup is required only once. Make sure no conda environment exists with the same name since this step does not cleanup/overwrite the existing environment. During this setup a new conda environment will be created with the dependencies listed in the YAML file.\r\n\r\nOnce the appropriate environment is created, activate it using the conda command given below:\r\n\r\n```bash\r\nconda activate traffic_detection_intel\r\n```\r\n\r\n## Download the Datasets\r\n\r\nData will be downloaded automatically through the script while running `train.py`; users can change the data download path by setting the value of `$DATA_DIR`. In order to download the datasets `curl` must be installed.\r\n\r\n```bash\r\nsudo apt install curl -y\r\n```\r\n\r\n## Supported Runtime Environment\r\n\r\nYou can execute the reference pipeline using the following environments:\r\n\r\n* Bare Metal\r\n\r\n---\r\n\r\n### Run Using Bare Metal\r\n\r\n#### Set Up System Software\r\n\r\nOur examples use the `conda` package and environment on your local computer. If you don't already have `conda` installed or the `conda` environment created, go to [Set Up Conda*](#set-up-conda) or see the [Conda* Linux installation instructions](https://docs.conda.io/projects/conda/en/stable/user-guide/install/linux.html).\r\n\r\n### Run Workflow\r\n\r\nYOLOv5 is needed to run the workflow, download it by executing the following instructions:\r\n\r\n[//]: # (capture: baremetal)\r\n```bash\r\ngit clone https://github.com/ultralytics/yolov5.git $YOLO_DIR\r\ncd $YOLO_DIR\r\ngit reset --hard 63555c8e2230328585d09fdc50a6601822a70ded\r\n\r\n# Intel® Extension for PyTorch training patch\r\ngit apply --reject --whitespace=fix $CONFIG_DIR/training.patch\r\n\r\n# Copying required files to the cloned repo\r\ncp $CONFIG_DIR/data/VOC.yaml $YOLO_DIR/data/\r\ncp $CONFIG_DIR/deploy.yaml $YOLO_DIR\r\ncp $WORKSPACE/src/run* $YOLO_DIR\r\ncp -r $WORKSPACE/src/openvino $YOLO_DIR\r\n\r\n#Changes default data download path from `../data/VOC` to `$DATA_DIR`.\r\nsed -i \"s+../data/VOC+$DATA_DIR+g\" $YOLO_DIR/data/VOC.yaml\r\n```\r\n\r\n### Training\r\n\r\nTo start training run `train.py` python script inside `src/yolov5`. The script downloads the dataset, preprocesses it and runs the training routine. The trained model will be saved in `output/models/train/exp{}/weights` folder.\r\n\r\n```txt\r\nusage: train.py [-h] [--weights WEIGHTS] [--cfg CFG] [--data DATA] [--hyp HYP] [--epochs EPOCHS] [--batch-size BATCH_SIZE] [--imgsz IMGSZ]\r\n                [--rect] [--resume [RESUME]] [--nosave] [--noval] [--noautoanchor] [--noplots] [--evolve [EVOLVE]] [--bucket BUCKET]\r\n                [--cache [CACHE]] [--image-weights] [--device DEVICE] [--multi-scale] [--single-cls] [--optimizer {SGD,Adam,AdamW}]\r\n                [--sync-bn] [--workers WORKERS] [--project PROJECT] [--name NAME] [--exist-ok] [--quad] [--cos-lr]\r\n                [--label-smoothing LABEL_SMOOTHING] [--patience PATIENCE] [--freeze FREEZE [FREEZE ...]] [--save-period SAVE_PERIOD]\r\n                [--seed SEED] [--local_rank LOCAL_RANK] [--entity ENTITY] [--upload_dataset [UPLOAD_DATASET]]\r\n                [--bbox_interval BBOX_INTERVAL] [--artifact_alias ARTIFACT_ALIAS] [--bf16]\r\n\r\noptional arguments:\r\n  -h, --help            show this help message and exit\r\n  --weights WEIGHTS     initial weights path\r\n  --cfg CFG             model.yaml path\r\n  --data DATA           dataset.yaml path\r\n  --hyp HYP             hyperparameters path\r\n  --epochs EPOCHS       total training epochs\r\n  --batch-size BATCH_SIZE\r\n                        total batch size for all GPUs, -1 for autobatch\r\n  --imgsz IMGSZ, --img IMGSZ, --img-size IMGSZ\r\n                        train, val image size (pixels)\r\n  --rect                rectangular training\r\n  --resume [RESUME]     resume most recent training\r\n  --nosave              only save final checkpoint\r\n  --noval               only validate final epoch\r\n  --noautoanchor        disable AutoAnchor\r\n  --noplots             save no plot files\r\n  --evolve [EVOLVE]     evolve hyperparameters for x generations\r\n  --bucket BUCKET       gsutil bucket\r\n  --cache [CACHE]       image --cache ram/disk\r\n  --image-weights       use weighted image selection for training\r\n  --device DEVICE       cuda device, i.e. 0 or 0,1,2,3 or cpu\r\n  --multi-scale         vary img-size +/- 50%\r\n  --single-cls          train multi-class data as single-class\r\n  --optimizer {SGD,Adam,AdamW}\r\n                        optimizer\r\n  --sync-bn             use SyncBatchNorm, only available in DDP mode\r\n  --workers WORKERS     max dataloader workers (per RANK in DDP mode)\r\n  --project PROJECT     save to project/name\r\n  --name NAME           save to project/name\r\n  --exist-ok            existing project/name ok, do not increment\r\n  --quad                quad dataloader\r\n  --cos-lr              cosine LR scheduler\r\n  --label-smoothing LABEL_SMOOTHING\r\n                        Label smoothing epsilon\r\n  --patience PATIENCE   EarlyStopping patience (epochs without improvement)\r\n  --freeze FREEZE [FREEZE ...]\r\n                        Freeze layers: backbone=10, first3=0 1 2\r\n  --save-period SAVE_PERIOD\r\n                        Save checkpoint every x epochs (disabled if \u003c 1)\r\n  --seed SEED           Global training seed\r\n  --local_rank LOCAL_RANK\r\n                        Automatic DDP Multi-GPU argument, do not modify\r\n  --entity ENTITY       Entity\r\n  --upload_dataset [UPLOAD_DATASET]\r\n                        Upload data, \"val\" option\r\n  --bbox_interval BBOX_INTERVAL\r\n                        Set bounding-box image logging interval\r\n  --artifact_alias ARTIFACT_ALIAS\r\n                        Version of dataset artifact to use\r\n  --bf16                Enable only on Intel® Fourth Gen Xeon, BF16\r\n```\r\n\r\nAn example of how to use the above command is provided in the following code block. It will automatically download the dataset and `yolov5s.pt` model, preprocess the dataset, run the training script for 10 epochs using the values stored in `yolov5s.pt` for fine-tunning and `$YOLO_DIR/data/VOC.yaml` as configuration parameters. Output models will be stored at `$OUTPUT_DIR/models/train`.\r\n\r\n[//]: # (capture: baremetal)\r\n```bash\r\npython $YOLO_DIR/train.py --weights $OUTPUT_DIR/models/yolov5s.pt --data $YOLO_DIR/data/VOC.yaml --epochs 10 --project $OUTPUT_DIR/models/train\r\n```\r\n\r\n`train.py` script also includes a command line flag `--bf16` that enables bf16 mixed precision training (on CPUs that support it) along with the optimizations.\r\n\r\nThe training process for Intel® Optimizations for PyTorch* along with bf16 mixed precision training can be enabled using the `train.py` script as:\r\n\r\n```bash\r\npython $YOLO_DIR/train.py --weights $OUTPUT_DIR/models/yolov5s.pt --data $YOLO_DIR/data/VOC.yaml --epochs 10 --project $OUTPUT_DIR/models/train --bf16\r\n```\r\n\r\n\u003e***Note***: You can enable bf16 training by setting the bf16. Please note that this flag MUST be enabled only on Intel® Fourth Gen Xeon® Scalable processors codenamed Sapphire Rapids that has bf16 training support and optimizations to utilize AMX, the latest ISA introduced in this family of processors.\r\n\r\n### Inference\r\n\r\nTo run inference use the `run_inference.py` python script inside `src/yolov5`.\r\n\r\n```txt\r\nusage: run_inference.py [-h] [-c CONFIG] [-d DATA_YAML] [-b BATCHSIZE] [-w WEIGHTS] [-int8inc] [-qw QUANT_WEIGHTS] [-si]\r\n                        [-sip SAVE_IMAGE_PATH]\r\n\r\noptional arguments:\r\n  -h, --help            show this help message and exit\r\n  -c CONFIG, --config CONFIG\r\n                        Yaml file for quantizing model, default is \"./deploy.yaml\"\r\n  -d DATA_YAML, --data_yaml DATA_YAML\r\n                        Absolute path to the data yaml file containing configurations\r\n  -b BATCHSIZE, --batchsize BATCHSIZE\r\n                        batchsize for the dataloader....default is 1\r\n  -w WEIGHTS, --weights WEIGHTS\r\n                        Model Weights \".pt\" format\r\n  -int8inc              Run INC quantization when present\r\n  -qw QUANT_WEIGHTS, --quant_weights QUANT_WEIGHTS\r\n                        Quantization Model Weights folder containing \".pt\" format model\r\n  -si, --save_image     Save images in the specified save_image_path when present.\r\n  -sip SAVE_IMAGE_PATH, --save_image_path SAVE_IMAGE_PATH\r\n                        Path to save images after post processing/ detected results\r\n```\r\n\r\nAn example of how to use the inference script is provided in the following code block. The script can be run in an Intel® environment using different batch sizes `-b`:{1/8/16/32/64/128/256}. To set the output folder path to save the output images `--save_image` and `--save_image_path {path/to/folder}` must be specified.\r\n\r\n[//]: # (capture: baremetal)\r\n```bash\r\npython $YOLO_DIR/run_inference.py -c $CONFIG_DIR/deploy.yaml -d $YOLO_DIR/data/VOC.yaml -b 1 -w $OUTPUT_DIR/models/yolov5s.pt --save_image --save_image_path $OUTPUT_DIR/images/fp32\r\n```\r\n\r\n### Quantize trained models using Intel® Neural Compressor\r\n\r\nIntel® Neural Compressor is used to quantize the FP32 Model to the INT8 Model. Optimized model is used here for evaluating and timing analysis.\r\nIntel® Neural Compressor supports many optimization methods, for this case post-training quantization with the `Accuracy Aware Mode` method is used to quantize the FP32 model.\r\n\r\n#### Step-1: Conversion of FP32 Model to INT8 Model\r\n\r\n```txt\r\nusage: run_inc_quantization.py [-h] [-o OUTPATH] [-c CONFIG] [-d DATA_YAML] [-w WEIGHTS]\r\n\r\noptional arguments:\r\n  -h, --help            show this help message and exit\r\n  -o OUTPATH, --outpath OUTPATH\r\n                        absolute path to save quantized model. By default it will be saved in \"./inc_compressed_model/output\" folder\r\n  -c CONFIG, --config CONFIG\r\n                        Yaml file for quantizing model, default is \"./deploy.yaml\"\r\n  -d DATA_YAML, --data_yaml DATA_YAML\r\n                        Absolute path to the data yaml file containing configurations\r\n  -w WEIGHTS, --weights WEIGHTS\r\n                        Model Weights \".pt\" format\r\n```\r\n\r\nAn example of how to quantize trained models is provided in the following code block. Quantized model will be saved by default in `$OUTPUT_DIR/models/inc_compressed_model/` folder as `best_model.pt`\r\n\r\n[//]: # (capture: baremetal)\r\n```bash\r\npython $YOLO_DIR/run_inc_quantization.py -o $OUTPUT_DIR/models/inc_compressed_model/ -c $CONFIG_DIR/deploy.yaml -d $YOLO_DIR/data/VOC.yaml -w $OUTPUT_DIR/models/yolov5s.pt\r\n```\r\n\r\n#### Step-2: Inference using quantized Model\r\n\r\nAn example of how to run inference with quantize models is provided in the following code block.\r\n\r\n[//]: # (capture: baremetal)\r\n```bash\r\npython $YOLO_DIR/run_inference.py -c $CONFIG_DIR/deploy.yaml -d $YOLO_DIR/data/VOC.yaml -b 1 -w $OUTPUT_DIR/models/yolov5s.pt -int8inc -qw $OUTPUT_DIR/models/inc_compressed_model/best_model.pt --save_image --save_image_path $OUTPUT_DIR/images/int8\r\n```\r\n\r\n### Using Intel® Distribution of OpenVINO*\r\n\r\nWhen it comes to the deployment of this model on Edge devices, with less computing and memory resources, we further need to explore options for quantizing and compressing the model which brings out the same level of accuracy and efficient utilization of underlying computing resources. Intel® Distribution of OpenVINO* Toolkit facilitates the optimization of a deep learning model from a framework and deployment using an inference engine on such computing platforms based on Intel hardware accelerators. This section covers the steps to use this toolkit for model quantization and measure its performance.\r\n\r\n#### Model conversion to OpenVINO* Intermediate Representation (IR) conversion\r\n\r\nBelow script is used to convert FP32 model to ONNX model representation.\r\n\r\n```txt\r\nusage: convert_to_onnx.py [-h] [-o OUTPATH] [-w WEIGHTS] [-mname MODEL_NAME]\r\n\r\noptional arguments:\r\n  -h, --help            show this help message and exit\r\n  -o OUTPATH, --outpath OUTPATH\r\n                        absolute path to save converted model. By default it will be saved in \"./openvino/openvino_models/openvino_onnx\" folder\r\n  -w WEIGHTS, --weights WEIGHTS\r\n                        Model Weights in \".pt\" format\r\n  -mname MODEL_NAME, --model_name MODEL_NAME\r\n                        Name of the model to be created in \".onnx\" format, default \"TrafficOD\"\r\n```\r\n\r\nHere is an example of how to use `convert_to_onnx.py`:\r\n\r\n[//]: # (capture: baremetal)\r\n```bash\r\npython $YOLO_DIR/openvino/convert_to_onnx.py -o $OUTPUT_DIR/models/openvino_models/openvino_onnx -w $OUTPUT_DIR/models/yolov5s.pt\r\n```\r\n\r\nThe converted model will be saved to the `$OUTPUT_DIR/models/openvino_models/openvino_onnx` in .onnx format.\r\n\r\nBelow command is used to convert the onnx model to OpenVINO* IR model format. `mo` has more arguments for the user to use, but for this case only `--input_model` and `--output_dir` are needed. For more options check `--help`.\r\n\r\n```txt\r\nmo --input_model \u003connx model\u003e --output_dir \u003coutput dir path to save the IR model\u003e\r\n\r\narguments:\r\n\r\n  --input_model     onnx model\r\n  --output_dir      path of the folder to save the OpenVINO IR model format\r\n```\r\n\r\nThe above command will generate `bin` and `xml` files as output which can be used for OpenVINO* inference, with FP32 as default precision.\r\n\r\nHere is an example of how to use `mo`:\r\n\r\n[//]: # (capture: baremetal)\r\n```bash\r\nmo --input_model $OUTPUT_DIR/models/openvino_models/openvino_onnx/TrafficOD_Onnx_Model.onnx --output_dir $OUTPUT_DIR/models/openvino_models/openvino_ir\r\n```\r\n\r\n#### Model Performance with OpenVINO* Post-Training Optimization Tool\r\n\r\nPost-training Optimization Tool (POT) is designed to accelerate the inference of deep learning models by applying special methods without model retraining or fine-tuning. One such method is post-training quantization.\r\n\r\n`benchmark_app` has more arguments for the user to use, but for this case only `--modelpath` is needed. For more options check `--help`.\r\n\r\n```txt\r\nbenchmark_app -m \u003cpath of onnx model\u003e\r\n\r\nargument:\r\n\r\n  -m,--modelpath   path of model in onnx format\r\n```\r\n\r\nThe example below is used to run the benchmark tool for the ONNX model generated.\r\n\r\n[//]: # (capture: baremetal)\r\n```bash\r\nbenchmark_app -m $OUTPUT_DIR/models/openvino_models/openvino_onnx/TrafficOD_Onnx_Model.onnx -api async -niter 120 -nireq 1 -b 1 -nstreams 1 -nthreads 8 -hint none\r\n```\r\n\r\nBelow example is used to run the benchmark tool for the OpenVINO* IR model.\r\n\r\n[//]: # (capture: baremetal)\r\n```bash\r\nbenchmark_app -m $OUTPUT_DIR/models/openvino_models/openvino_ir/TrafficOD_Onnx_Model.xml -api async -niter 120 -nireq 1 -b 1 -nstreams 1 -nthreads 8 -hint none\r\n```\r\n\r\n#### Model Conversion Using OpenVINO* Quantization\r\n\r\n`openvino_quantization.py` is used to convert OpenVINO* IR model to OpenVINO* INT8 model representation.\r\n\r\n```txt\r\nusage: openvino_quantization.py [-h] [-m FPIR_MODELPATH] [-o OUTPATH] [-d DATA_YAML] [-b BATCHSIZE]\r\n\r\noptional arguments:\r\n  -h, --help            show this help message and exit\r\n  -m FPIR_MODELPATH, --FPIR_modelpath FPIR_MODELPATH\r\n                        FP32 IR Model absolute path without extension\r\n  -o OUTPATH, --outpath OUTPATH\r\n                        default output quantized model will be save in path specified by outpath\r\n  -d DATA_YAML, --data_yaml DATA_YAML\r\n                        Absolute path to the yaml file containing paths data/ download data\r\n  -b BATCHSIZE, --batchsize BATCHSIZE\r\n                        batch size used for loading the data\r\n```\r\n\r\nRun the following example to run OpenVINO* quantization.\r\n\r\n[//]: # (capture: baremetal)\r\n```bash\r\npython $YOLO_DIR/openvino/openvino_quantization.py -o $OUTPUT_DIR/models/openvino_models/openvino_quantized -d $YOLO_DIR/data/VOC.yaml -b 1 -m $OUTPUT_DIR/models/openvino_models/openvino_ir/\r\n```\r\n\r\nAfter running the above command, we can verify that `bin`, `xml` and `mapping` files (quantized model) got generated on `$OUTPUT_DIR/models/openvino_models/openvino_quantized` path.\r\n\r\n#### Model Performance Using Quantized (INT8) Model\r\n\r\nUse the example below to run the benchmark tool for the Quantized OpenVINO* IR model.\r\n\r\n[//]: # (capture: baremetal)\r\n```bash\r\nbenchmark_app -m $OUTPUT_DIR/models/openvino_models/openvino_quantized/torch_jit.xml -api async -niter 120 -nireq 1 -b 1 -nstreams 1 -nthreads 8 -hint none\r\n```\r\n\r\n### Clean Up Bare Metal\r\n\r\nBefore proceeding to the cleaning process, it is strongly recommended to make a backup of the data that the user wants to keep. To clean the previously downloaded and generated data, run the following commands:\r\n\r\n```bash\r\nconda activate base\r\nconda env remove -n traffic_detection_intel\r\n```\r\n\r\n```bash\r\nrm -rf $DATA_DIR $OUTPUT_DIR $YOLO_DIR\r\n```\r\n\r\nTo remove WORKSPACE:\r\n\r\n```bash\r\nrm -rf $WORKSPACE\r\n```\r\n\r\n## Expected Output\r\n\r\nA successful execution of `train.py` should return similar results as shown below:\r\n\r\n```txt\r\n      Epoch    GPU_mem   box_loss   obj_loss   cls_loss  Instances       Size\r\n        9/9         0G    0.03788    0.03211    0.01635         46        640: 100%|██████████| 1035/1035 [21:51\u003c00:00,  1.27s/it]\r\n                 Class     Images  Instances          P          R      mAP50   mAP50-95: 100%|██████████| 155/155 [02:50\u003c00:00,  1.10s/it]\r\n                   all       4952      12032      0.723       0.68       0.74      0.459\r\n\r\n10 epochs completed in 3.938 hours.\r\nOptimizer stripped from /workspace/traffic-detection/output/models/train/exp/weights/last.pt, 14.8MB\r\nOptimizer stripped from /workspace/traffic-detection/output/models/train/exp/weights/best.pt, 14.8MB\r\n\r\nValidating /workspace/traffic-detection/output/models/train/exp/weights/best.pt...\r\nFusing layers...\r\nModel summary: 157 layers, 7225885 parameters, 0 gradients, 16.4 GFLOPs\r\n                 Class     Images  Instances          P          R      mAP50   mAP50-95: 100%|██████████| 155/155 [02:33\u003c00:00,  1.01it/s]\r\n                   all       4952      12032      0.757       0.75      0.807      0.547\r\n                person       4952       4528      0.832      0.838      0.898      0.602\r\n               bicycle       4952        337      0.785      0.774      0.862      0.579\r\n                   car       4952       1201      0.827      0.883       0.92      0.667\r\n             motorbike       4952        325      0.862      0.786      0.881      0.557\r\n             aeroplane       4952        285      0.897      0.786      0.885      0.562\r\n                   bus       4952        213       0.78      0.878       0.92      0.724\r\n                 train       4952        282      0.832      0.841      0.889      0.599\r\n                  boat       4952        263      0.594      0.662      0.659      0.359\r\n                  bird       4952        459      0.686      0.743      0.772      0.495\r\n                   cat       4952        358      0.815      0.691      0.817      0.548\r\n                   dog       4952        489      0.783      0.685      0.785      0.526\r\n                 horse       4952        348      0.899      0.853      0.904      0.635\r\n                 sheep       4952        242      0.718      0.822      0.849      0.628\r\n                   cow       4952        244      0.736       0.77      0.828       0.59\r\n                bottle       4952        469      0.623      0.825       0.79       0.53\r\n                 chair       4952        756      0.632      0.589      0.628      0.412\r\n                  sofa       4952        239      0.689      0.653      0.717      0.535\r\n           pottedplant       4952        480      0.694      0.425      0.546      0.277\r\n           diningtable       4952        206      0.721      0.651      0.736       0.52\r\n             tvmonitor       4952        308      0.732      0.842       0.85      0.593\r\nResults saved to /workspace/traffic-detection/output/models/train/exp\r\n```\r\n\r\nA successful execution of `run_inference.py` should return similar results as shown below:\r\n\r\n```txt\r\nMean Average Precision for all images is  0.5856925404676505\r\nBatch Size used here is  1\r\nAverage Inference Time Taken --\u003e  0.016135698727872794 for images :: 1011\r\n```\r\n\r\nA successful execution of `run_inc_quantization.py` should return similar results as shown below:\r\n\r\n```txt\r\n2023-12-22 01:11:12 [INFO] Tune 6 result is: [Accuracy (int8|fp32): 0.5543|0.5541, Duration (seconds) (int8|fp32): 158.9356|255.7827], Best tune result is: [Accuracy: 0.5543, Duration (seconds): 158.9356]\r\n2023-12-22 01:11:12 [INFO] |***********************Tune Result Statistics**********************|\r\n2023-12-22 01:11:12 [INFO] +--------------------+-----------+---------------+------------------+\r\n2023-12-22 01:11:12 [INFO] |     Info Type      |  Baseline | Tune 6 result | Best tune result |\r\n2023-12-22 01:11:12 [INFO] +--------------------+-----------+---------------+------------------+\r\n2023-12-22 01:11:12 [INFO] |      Accuracy      |  0.5541   |    0.5543     |     0.5543       |\r\n2023-12-22 01:11:12 [INFO] | Duration (seconds) | 255.7827  |   158.9356    |    158.9356      |\r\n2023-12-22 01:11:12 [INFO] +--------------------+-----------+---------------+------------------+\r\n2023-12-22 01:11:12 [INFO] Save tuning history to /workspace/traffic-detection/src/yolov5/nc_workspace/2023-12-22_00-53-31/./history.snapshot.\r\n2023-12-22 01:11:12 [INFO] Specified timeout or max trials is reached! Found a quantized model which meet accuracy goal. Exit.\r\n2023-12-22 01:11:12 [INFO] Save deploy yaml to /workspace/traffic-detection/src/yolov5/nc_workspace/2023-12-22_00-53-31/deploy.yaml\r\n2023-12-22 01:11:12 [INFO] Save config file and weights of quantized model to /workspace/traffic-detection/output/models/inc_compressed_model.\r\n******************************\r\nSuccesfully Quantized model and saved at : /workspace/traffic-detection/output/models/inc_compressed_model/\r\n```\r\n\r\nA successful execution of `convert_to_onnx.py` should return similar results as shown below:\r\n\r\n```txt\r\n============== Diagnostic Run torch.onnx.export version 2.0.1+cpu ==============\r\nverbose: False, log level: Level.ERROR\r\n======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================\r\n```\r\n\r\nA successful execution of `mo` for this workflow should return similar results as shown below:\r\n\r\n```txt\r\n[ INFO ] Generated IR will be compressed to FP16. If you get lower accuracy, please consider disabling compression explicitly by adding argument --compress_to_fp16=False.\r\nFind more information about compression to FP16 at https://docs.openvino.ai/2023.0/openvino_docs_MO_DG_FP16_Compression.html\r\n[ INFO ] The model was converted to IR v11, the latest model format that corresponds to the source DL framework input/output format. While IR v11 is backwards compatible with OpenVINO* Inference Engine API v1.0, please use API v2.0 (as of 2022.1) to take advantage of the latest improvements in IR v11.\r\nFind more information about API v2.0 and IR v11 at https://docs.openvino.ai/2023.0/openvino_2_0_transition_guide.html\r\n[ SUCCESS ] Generated IR version 11 model.\r\n[ SUCCESS ] XML file: /workspace/traffic-detection/output/models/openvino_models/openvino_ir/TrafficOD_Onnx_Model.xml\r\n[ SUCCESS ] BIN file: /workspace/traffic-detection/output/models/openvino_models/openvino_ir/TrafficOD_Onnx_Model.bin\r\n```\r\n\r\nA successful execution of `benchmark_app` for this workflow should return similar results as shown below:\r\n\r\n```txt\r\n[Step 11/11] Dumping statistics report\r\n[ INFO ] Execution Devices:['CPU']\r\n[ INFO ] Count:            120 iterations\r\n[ INFO ] Duration:         2144.58 ms\r\n[ INFO ] Latency:\r\n[ INFO ]    Median:        17.50 ms\r\n[ INFO ]    Average:       17.72 ms\r\n[ INFO ]    Min:           17.06 ms\r\n[ INFO ]    Max:           29.55 ms\r\n[ INFO ] Throughput:   55.95 FPS\r\n```\r\n\r\nA successful execution of `openvino_quantization.py` should return similar results as shown below:\r\n\r\n```txt\r\n[2023-12-22 17:24:55][INFO] Step 1/9: Load the model\r\n[2023-12-22 17:24:57][INFO] Step 2/9: Initialize the data loader\r\nval: Scanning /workspace/traffic-detection/data/labels/test2007.cache... 4952 images, 0 backgrounds, 0 corrupt: 100%|██████████| 4952/4952 [00:00\u003c?, ?it/s]\r\n[2023-12-22 17:24:57][INFO] Step 3/9: Initialize the metric\r\n[2023-12-22 17:24:57][INFO] Step 4/9: Initialize the engine for metric calculation and statistics collection\r\n[2023-12-22 17:24:57][INFO] This will take time, please wait!\r\n[2023-12-22 17:24:57][INFO] Step 5/9: Create a pipeline of compression algorithms\r\n[2023-12-22 17:29:52][INFO] This will take time, please wait!\r\n[2023-12-22 17:29:52][INFO] Step 6/9: Execute the pipeline\r\n[2023-12-22 17:34:29][INFO] Step 7/9: Compress model weights quantized precision in\"                     \" order to reduce the size of final .bin file\r\n[2023-12-22 17:34:32][INFO] Step 8/9: Save the compressed model and get the path to the model\r\n[2023-12-22 17:34:36][INFO] The quantized model is stored in /workspace/traffic-detection/output/models/openvino_models/openvino_quantized/torch_jit.xml\r\n[2023-12-22 17:34:36][INFO] Step 9 (Optional): Evaluate the original and compressed model. Print the results\r\n[2023-12-22 17:37:11][INFO] MeanAP of the quantized model: 0.57575\r\n[2023-12-22 17:37:11][INFO] MeanAP of the original model:  0.58256\r\n```\r\n\r\n## Summary and Next Steps\r\n\r\nThe ML pipeline can be break down into the following main tasks:\r\n\r\n* Preprocessing (normalization and resizing) of VOC dataset using COCO classes.\r\n* Set yolov5 weights for fine-tuning by using `yolov5s.pt` pretrained model and run training using the preprocessed dataset.\r\n* Run yolov5 inference on a subset of images.\r\n* The YOLO results are post processed; by using NMS and centroid-based distance calculation of detected objects possible collision can be detected.\r\n* Provide a risk assessment as output.\r\n\r\nThis exercise for traffic camera object detection can be used as a reference implementation across similar use cases with Intel AI optimizations enabled to accelerate the E2E process.\r\n\r\n## Learn More\r\n\r\nFor more information about or to read about other relevant workflow examples, see these guides and software resources:\r\n\r\n* [Intel® AI Tools](https://www.intel.com/content/www/us/en/developer/tools/oneapi/ai-analytics-toolkit.html)\r\n* [Intel® Optimizations for PyTorch*](https://www.intel.com/content/www/us/en/developer/tools/oneapi/optimization-for-pytorch.html)\r\n* [Intel® Neural Compressor](https://www.intel.com/content/www/us/en/developer/tools/oneapi/neural-compressor.html)\r\n* [OpenVINO*](https://github.com/openvinotoolkit/openvino)\r\n\r\n## Support\r\n\r\nThe End-to-end Traffic Camera Object Detection team tracks both bugs and\r\nenhancement requests using [GitHub\r\nissues](https://github.com/oneapi-src/traffic-camera-object-detection/issues).\r\nBefore submitting a suggestion or bug report,\r\nsee if your issue has already been reported.\r\n\r\n## Appendix\r\n\r\n### Disclaimer\r\n\r\n© Intel Corporation.  Intel, the Intel logo, and other Intel marks are trademarks of Intel Corporation or its subsidiaries.  Other names and brands may be claimed as the property of others.\r\n\r\nTo the extent that any public or non-Intel datasets or models are referenced by or accessed using tools or code on this site those datasets or models are provided by the third party indicated as the content source. Intel does not create the content and does not warrant its accuracy or quality. By accessing the public content, or using materials trained on or with such content, you agree to the terms associated with that content and that your use complies with the applicable license.\r\n\r\nIntel expressly disclaims the accuracy, adequacy, or completeness of any such public content, and is not liable for any errors, omissions, or defects in the content, or for any reliance on the content. Intel is not liable for any liability or damages relating to your use of public content.\r\n\r\n\\*Names and brands that may be claimed as the property of others. [Trademarks](https://www.intel.com/content/www/us/en/legal/trademarks.html).\r\n","funding_links":[],"categories":["Table of Contents"],"sub_categories":["AI - Frameworks and Toolkits"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foneapi-src%2Ftraffic-camera-object-detection","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Foneapi-src%2Ftraffic-camera-object-detection","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foneapi-src%2Ftraffic-camera-object-detection/lists"}