{"id":13574212,"url":"https://github.com/SamSamhuns/yolov5_export_cpu","last_synced_at":"2025-04-04T14:32:08.845Z","repository":{"id":46768954,"uuid":"381263836","full_name":"SamSamhuns/yolov5_export_cpu","owner":"SamSamhuns","description":"Exporting YOLOv5 for CPU inference with ONNX and OpenVINO","archived":false,"fork":false,"pushed_at":"2024-08-07T06:31:14.000Z","size":5840,"stargazers_count":36,"open_issues_count":2,"forks_count":9,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-11-05T09:44:03.501Z","etag":null,"topics":["cpu-inference-api","object-detection","onnx-inference","openvino-inference-engine","yolov5"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/SamSamhuns.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-06-29T06:35:26.000Z","updated_at":"2024-10-08T10:25:39.000Z","dependencies_parsed_at":"2024-08-07T09:55:20.231Z","dependency_job_id":"a4076d17-2d28-42fd-ac23-11e22053a237","html_url":"https://github.com/SamSamhuns/yolov5_export_cpu","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SamSamhuns%2Fyolov5_export_cpu","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SamSamhuns%2Fyolov5_export_cpu/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SamSamhuns%2Fyolov5_export_cpu/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SamSamhuns%2Fyolov5_export_cpu/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/SamSamhuns","download_url":"https://codeload.github.com/SamSamhuns/yolov5_export_cpu/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247194223,"owners_count":20899448,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cpu-inference-api","object-detection","onnx-inference","openvino-inference-engine","yolov5"],"created_at":"2024-08-01T15:00:48.204Z","updated_at":"2025-04-04T14:32:03.827Z","avatar_url":"https://github.com/SamSamhuns.png","language":"Python","funding_links":[],"categories":["Table of Contents","Table of content"],"sub_categories":["AI - Computer Vision","AI Computer Vision"],"readme":"# YOLOv5 CPU Export and OpenVINO Inference\n\n[![Codacy Badge](https://app.codacy.com/project/badge/Grade/55c3d2e474f14e7b8cb6c611504457d9)](https://www.codacy.com/gh/SamSamhuns/yolov5_export_cpu/dashboard?utm_source=github.com\u0026amp;utm_medium=referral\u0026amp;utm_content=SamSamhuns/yolov5_export_cpu\u0026amp;utm_campaign=Badge_Grade)\n\nDocumentation on exporting YOLOv5 models for fast CPU inference using Intel's OpenVINO framework (Tested on commits up to June 6, 2022 in docker).\n\n## Google Colab Conversion\n\nConvert yolov5 model to IR format with Google Colab. [![Google Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1K8gnZEka47Gbcp1eJbBaSe3GxngJdvio?usp=sharing) (Recommended)\n\n## 1. Clone and set up the Official YOLOv5 GitHub repository\n\n\u003cdetails\u003e\n  \u003csummary\u003eSetup\u003c/summary\u003e\n\n  All package installations should be done in a virtualenv or conda env to prevent package conflict errors.\n\n-   Install required requirements for onnx and openvino Inference\n\n```bash\npip install --upgrade pip\npip install -r inf_requirements.txt\n```\n\n-   Clone and install requirements for yolov5 repository\n\n```bash\ngit clone https://github.com/ultralytics/yolov5                    # clone repo\ncd yolov5\npip install -r requirements.txt                                    # base requirements\n```\n\n\u003c/details\u003e\n\n## 2. Export a Trained YOLOv5 Model as ONNX\n\n\u003cdetails\u003e\n  \u003csummary\u003eExport\u003c/summary\u003e\n\nExport a pre-trained or custom trained YOLOv5 model to generate the respective ONNX, TorchScript and CoreML formats of the model. The pre-trained `yolov5s.pt` is the lightest and fastest model for CPU inference. Other slower but more accurate models include `yolov5m.pt, yolov5l.pt` and `yolov5x.pt`. All available model details at Ultralytics YOLOv5 [README](https://github.com/ultralytics/yolov5#pretrained-checkpoints).\n\nA custom training checkpoint i.e. `runs/exp/weights/best.pt` can be used for conversion as well.\n\n-   Export a pre-trained light yolov5s.pt model at 640x640 with batch size 1\n\n```bash\npython export.py --weights yolov5s.pt --include onnx --img 640 --batch 1\n```\n\n-   Export a custom checkpoint for dynamic input shape {BATCH_SIZE, 3, HEIGHT, WIDTH}. Note, for CPU inference mode, BATCH_SIZE must be set to 1. Install onnx-simplifier for simplifying onnx exports\n\n```bash\npip install onnx-simplifier==0.3.10                                \npython export.py --weights runs/exp/weights/best.pt --include onnx  --dynamic --simplify\n```\n\n-  Cd to `yolov5_export_cpu` dir and move the onnx model to `yolov5_export_cpu/models` directory\n\n```bash\nmv \u003cPATH_TO_ONNX_MODEL\u003e yolov5_export_cpu/models/\n```\n\n\u003c/details\u003e\n\n## 3. Test YOLOv5 ONNX model inference\n\n\u003cdetails\u003e\n  \u003csummary\u003eONNX inference\u003c/summary\u003e\n\n```bash\npython detect_onnx.py -m image -i \u003cIMG_FILE_PATH/IMG_DIR_PATH\u003e\npython detect_onnx.py -m video -i \u003cVID_PATH_FILE\u003e\n# python detect_onnx.py -h for more info\n```\n\nOptional: To convert the all frames in the `output` directory into a mp4 video using `ffmpeg`, use `ffmpeg -r 25 -start_number 00001 -i output/frame_onnx_%5d.jpg -vcodec libx264 -y -an onnx_result.mp4`\n\n\u003c/details\u003e\n\n## 4. Export ONNX to OpenVINO\n\n**Recommended Option A**\n\n### Option A. Use OpenVINO's python dev library\n\n\u003cdetails\u003e\n  \u003csummary\u003e A1. Install OpenVINO python dev library\u003c/summary\u003e\n\n  Instructions for setting OpenVINO available [here](https://docs.openvino.ai/latest/openvino_docs_install_guides_install_dev_tools.html)\n\n```bash\n# install required OpenVINO lib to convert ONNX to OpenVINO IR\npip install openvino-dev[onnx]\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003e A2. Export ONNX to OpenVINO IR\u003c/summary\u003e\n\nThis will create the OpenVINO Intermediate Model Representation (IR) model files (xml and bin) in the directory `models/yolov5_openvino`.\n\n**Important Note:** --input_shape must be provided and match the img shape used to export ONNX model. Batching might not supported for CPU inference\n\n```bash\n# export onnx to OpenVINO IR\nmo \\\n  --progress \\\n  --input_shape [1,3,640,640] \\\n  --input_model models/yolov5s.onnx \\\n  --output_dir models/yolov5_openvino\n```\n\n[Full OpenVINO export options](https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model_General.html)\n\n\u003c/details\u003e\n\n### Option B. Use OpenVINO Docker\n\n\u003cdetails\u003e\n  \u003csummary\u003eB1. Download Docker and OpenVINO Docker Image\u003c/summary\u003e\n\n[Install docker](https://docs.docker.com/get-docker/) in your system if not already installed.\n\nPass the docker run command below in a terminal which will automatically download the OpenVINO Docker Image and run it. The `models` directory containing the ONNX model must be in the current working directory.\n\n```bash\ndocker run -it --rm \\\n            -v $PWD/models:/home/openvino/models \\\n            openvino/ubuntu18_dev:latest \\\n            /bin/bash -c \"cd /home/openvino/; bash\"\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003eB2. Export ONNX model to an OpenVINO IR representation\u003c/summary\u003e\n\nThis will create the OpenVINO Intermediate Model Representation (IR) model files (xml and bin) in the directory `models/yolov5_openvino` which will be available in the host system outside the docker container.\n\n**Important Note:** --input_shape must be provided and match the img shape used to export ONNX model. Batching might not supported for CPU inference\n\n```bash\n# inside the OpenVINO docker container\nmo \\\n  --progress \\\n  --input_shape [1,3,640,640] \\\n  --input_model models/yolov5s.onnx \\\n  --output_dir models/yolov5_openvino\n# exit OpenVINO docker container\nexit  \n```\n\n[Full OpenVINO export options](https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model_General.html)\n\n\u003c/details\u003e\n\n## 5. Test YOLOv5 OpenVINO IR model CPU inference\n\n\u003cdetails\u003e\n  \u003csummary\u003eOpenVINO model inference\u003c/summary\u003e\n\n```bash\npython detect_openvino.py -m image -i \u003cIMG_FILE_PATH/IMG_DIR_PATH\u003e\npython detect_openvino.py -m video -i \u003cVID_PATH_FILE\u003e\n# python detect_openvino.py -h for more info\n```\n\nOptional: To convert the all frames in the `output` directory into a mp4 video using `ffmpeg`, use `ffmpeg -r 25 -start_number 00001 -i output/frame_openvino_%5d.jpg -vcodec libx264 -y -an openvino_result.mp4`\n\n\u003c/details\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FSamSamhuns%2Fyolov5_export_cpu","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FSamSamhuns%2Fyolov5_export_cpu","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FSamSamhuns%2Fyolov5_export_cpu/lists"}