{"id":13442717,"url":"https://github.com/ENOT-AutoDL/ONNX-Runtime-with-TensorRT-and-OpenVINO","last_synced_at":"2025-03-20T15:30:30.744Z","repository":{"id":38311446,"uuid":"389500829","full_name":"ENOT-AutoDL/ONNX-Runtime-with-TensorRT-and-OpenVINO","owner":"ENOT-AutoDL","description":"Docker scripts for building ONNX Runtime with TensorRT and OpenVINO in manylinux environment","archived":false,"fork":false,"pushed_at":"2023-05-11T04:47:28.000Z","size":121,"stargazers_count":22,"open_issues_count":0,"forks_count":1,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-03-10T22:45:59.804Z","etag":null,"topics":["cuda","nvidia","onnx","onnxruntime","openvino","tensorrt"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ENOT-AutoDL.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2021-07-26T03:49:04.000Z","updated_at":"2025-02-07T02:30:51.000Z","dependencies_parsed_at":"2024-01-18T14:41:17.306Z","dependency_job_id":"6bf775d9-8890-4432-b2fa-6387b496f57e","html_url":"https://github.com/ENOT-AutoDL/ONNX-Runtime-with-TensorRT-and-OpenVINO","commit_stats":null,"previous_names":[],"tags_count":13,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ENOT-AutoDL%2FONNX-Runtime-with-TensorRT-and-OpenVINO","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ENOT-AutoDL%2FONNX-Runtime-with-TensorRT-and-OpenVINO/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ENOT-AutoDL%2FONNX-Runtime-with-TensorRT-and-OpenVINO/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ENOT-AutoDL%2FONNX-Runtime-with-TensorRT-and-OpenVINO/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ENOT-AutoDL","download_url":"https://codeload.github.com/ENOT-AutoDL/ONNX-Runtime-with-TensorRT-and-OpenVINO/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244639841,"owners_count":20485939,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cuda","nvidia","onnx","onnxruntime","openvino","tensorrt"],"created_at":"2024-07-31T03:01:49.598Z","updated_at":"2025-03-20T15:30:30.488Z","avatar_url":"https://github.com/ENOT-AutoDL.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# ONNX Runtime with TensorRT and OpenVINO\n\nDocker scripts for building [ONNX Runtime](https://github.com/microsoft/onnxruntime) with [TensorRT](https://github.com/NVIDIA/TensorRT) and [OpenVINO](https://github.com/openvinotoolkit/openvino) in manylinux environment.\n\nSupports `x86_64` and `aarch64 (JetPack)` architectures.\n\n## Build requirements\n\n - [CUDA 11.6](https://developer.nvidia.com/cuda-downloads) (and CUDA 11.1 for tests)\n - [cuDNN 8.4](https://developer.nvidia.com/cudnn-download-survey)\n - [TensorRT 8.4](https://developer.nvidia.com/nvidia-tensorrt-download)\n\nPlace CUDA (`.run`), cuDNN (`tar.gz`) and TensorRT (`tar.gz`) files into `distrib` folder.\n\n## Building\n\nSimply type the following command in your terminal and press `Enter`:\n```\nbash docker-run.sh\n```\n\nWheels will be placed into `wheelhouse` folder.\n\n## Customization\n\n - To specify `Python` versions for which wheels will be built, edit `PYTHON_TARGETS` variable in `docker-run.sh`\n - To change number of parallel threads edit `THREADS_NUM` variable in `docker-run.sh`\n\n## Using\n\nWheels compiled for `x86_64` architecture depend on the following packages from NVIDIA repository:\n - `nvidia-cuda-runtime-cu116 (11.6)`\n - `nvidia-cublas-cu116 (11.9)`\n - `nvidia-cudnn-cu116 (8.4)`\n - `nvidia-cufft-cu116 (10.7)`\n - `nvidia-curand-cu116 (10.2)`\n - `tensorrt (8.5)`\n\nand `openvino (2022.2)` from standard PyPI repository.\\\nCompiled wheels do not explicitly depend on NVIDIA packages, you can install them by the following commands:\n```\npip install --extra-index-url https://pypi.ngc.nvidia.com \\\n    nvidia-cuda-runtime-cu116==11.6.55 \\\n    nvidia-cudnn-cu116==8.4.0.27 \\\n    nvidia-cufft-cu116==10.7.2.124 \\\n    nvidia-curand-cu116==10.2.9.124 \\\n    nvidia-cublas-cu116==11.9.2.110\n\npip install --no-deps --extra-index-url https://pypi.ngc.nvidia.com tensorrt==8.5.2.2\n\npip install openvino==2022.2 openvino-dev==2022.2\n```\n\nThe **recommended way** to install this ONNX Runtime package is to use our `install.sh` script,\nwhich installs ONNX Runtime with all dependencies automatically.\n\nInstall `GPU` version (with all NVIDIA dependencies):\n```\nwget -O - https://raw.githubusercontent.com/ENOT-AutoDL/ONNX-Runtime-with-TensorRT-and-OpenVINO/master/install.sh | bash\n```\nInstall `CPU`-only version (without NVIDIA packages, use this version if your target device has no `GPU`):\n```\nwget -O - https://raw.githubusercontent.com/ENOT-AutoDL/ONNX-Runtime-with-TensorRT-and-OpenVINO/master/install.sh | bash -s -- -t CPU\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FENOT-AutoDL%2FONNX-Runtime-with-TensorRT-and-OpenVINO","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FENOT-AutoDL%2FONNX-Runtime-with-TensorRT-and-OpenVINO","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FENOT-AutoDL%2FONNX-Runtime-with-TensorRT-and-OpenVINO/lists"}