{"id":28309848,"url":"https://github.com/chansoopark98/windows-tensorrt-python","last_synced_at":"2026-02-08T09:05:59.655Z","repository":{"id":173534983,"uuid":"650890179","full_name":"chansoopark98/Windows-TensorRT-Python","owner":"chansoopark98","description":"Repository on how to install and inference TensorRT Python on Windows","archived":false,"fork":false,"pushed_at":"2023-06-19T15:23:22.000Z","size":19,"stargazers_count":8,"open_issues_count":0,"forks_count":3,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-08-10T03:35:26.311Z","etag":null,"topics":["inference","onnx","pytorch","pytorch-lightning","tensorflow","tensorflow2","tensorrt"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/chansoopark98.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-06-08T03:15:26.000Z","updated_at":"2025-01-23T09:46:05.000Z","dependencies_parsed_at":null,"dependency_job_id":"900c12d4-7d21-455c-825f-2ed186aa3772","html_url":"https://github.com/chansoopark98/Windows-TensorRT-Python","commit_stats":null,"previous_names":["chansoopark98/windows-tensorrt-python"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/chansoopark98/Windows-TensorRT-Python","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chansoopark98%2FWindows-TensorRT-Python","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chansoopark98%2FWindows-TensorRT-Python/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chansoopark98%2FWindows-TensorRT-Python/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chansoopark98%2FWindows-TensorRT-Python/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/chansoopark98","download_url":"https://codeload.github.com/chansoopark98/Windows-TensorRT-Python/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chansoopark98%2FWindows-TensorRT-Python/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29225750,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-08T06:05:31.539Z","status":"ssl_error","status_checked_at":"2026-02-08T05:58:33.853Z","response_time":57,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["inference","onnx","pytorch","pytorch-lightning","tensorflow","tensorflow2","tensorrt"],"created_at":"2025-05-24T10:11:48.888Z","updated_at":"2026-02-08T09:05:59.650Z","avatar_url":"https://github.com/chansoopark98.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Hits](https://hits.seeyoufarm.com/api/count/incr/badge.svg?url=https%3A%2F%2Fgithub.com%2Fchansoopark98%2FWindows-TensorRT-Python\u0026count_bg=%2379C83D\u0026title_bg=%232980E5\u0026icon=python.svg\u0026icon_color=%23E7E7E7\u0026title=hits\u0026edge_flat=true)](https://hits.seeyoufarm.com)\n\n# Windows-TensorRT-Python\nRepository on how to install and infer TensorRT Python on Windows\n\nIncludes examples of converting Tensorflow and PyTorch models to TensorRT in the Windows environment and inferring the converted models.\n\n\n## 한국어 [README.md](https://github.com/chansoopark98/Windows-TensorRT-Python/blob/main/README_kr.md) 지원\n\n\u003cbr\u003e\n\u003chr\u003e\n\n# Table of Contents\n\n ## 1. [Install CUDA \u0026 CuDNN \u0026 TensorRT](#1-install-cuda--cudnn--tensorrt)\n ## 2. [Install TensorRT python](#2-install-tensorrt-python)\n ## 3. [Convert DL Models](#3-convert-dl-models)\n ## 4. [Inference](#4-inference)\n\n\u003cbr\u003e\n\u003chr\u003e\n\n## Dependency\n\n| Type | Name |\n| :-- | :-: |\n| **OS** | Windows 11 Pro (22H2 Version) |\n| **CPU** | Intel i7-12650H 2.30GHz |\n| **RAM** | 16GB |\n| **GPU** | Nvidia RTX 3050ti laptop |\n| **Tensorflow** | Tensorflow 2.9.1 |\n| **TensorRT** | TensorRT-8.2.5.1 |\n| **CUDA Toolkit** | CUDA Toolkit 11.4 |\n| **CuDNN** | CuDNN v8.4.1 (May 27th, 2022), for CUDA 11.x |\n\n\u003cbr\u003e\n\u003chr\u003e\n\u003cbr\u003e\n\n# 1. Install CUDA \u0026 CuDNN \u0026 TensorRT\n\n## 1.1 Installation CUDA\n\u003cbr\u003e\n\n- **Install CUDA Toolkit** :\n\n    https://developer.nvidia.com/cuda-11-4-4-download-archive\n\n- **Set windows environment variable** :\n\n    ![image](https://github.com/chansoopark98/Windows-TensorRT-Python/assets/60956651/cb362cd5-5a64-4579-9aa9-5756b4370fd8)\n\n\n\u003cbr\u003e\n\n## 1.2 Installation CuDNN\n\n- **CuDNN** : https://developer.nvidia.com/rdp/cudnn-archive#a-collapse841-\n\u003cbr\u003e\n\n- **Copy files** :\n\n    Move the installed CuDNN **'bin', 'include', 'lib'** folder into the CUDA folder of the installed version\n    \u003cbr\u003e\n\n    ![image](https://github.com/chansoopark98/Windows-TensorRT-Python/assets/60956651/c603a448-8fcf-4d0e-90cc-6939c0ad0fba)\n\n    \u003cbr\u003e\n\n    Verify CUDA installation after reboot\n\n        cmd -\u003e nvcc -V\n    \n    ![image](https://github.com/chansoopark98/Windows-TensorRT-Python/assets/60956651/86d47736-2976-4430-ab86-afc66b59210f)\n\n## 1.3 Installation TensorRT SDK\n\n- **TensorRT** : https://developer.nvidia.com/nvidia-tensorrt-8x-download (TensorRT 8.2 GA Update 4)\n\n\u003cbr\u003e\n\n- **Move directory** :\n\n    Move the installed TensorRT .zip file to C:\\ root directory\n    ```cmd\n    cd c:\\TensorRT-8.2.5.1\u003e\n    ```\n\n    \u003cbr\u003e\n\n- **Copy \u0026 Paste .dll, .lib files**\n\n    Command Prompt (cmd) -\u003e Run commands sequentially\n    ```cmd\n    copy c:\\TensorRT-8.2.5.1\\include \"c:\\Program Files\\NVIDIA GPU Computing     Toolkit\\CUDA\\v11.4\\include\"\n\n    robocopy c:\\TensorRT-8.2.5.1\\lib \"c:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v11.4\\lib\\x64\" *.lib\n\n    robocopy c:\\TensorRT-8.2.5.1\\lib \"c:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v11.4\\bin\" *.dll\n    ```\n\n\u003cbr\u003e\n\u003chr\u003e\n\u003cbr\u003e\n\n\n# 2.1 Install TensorRT python\n\n## 2.1 Create virtual enviroments\n\n- **Setting up a virtual environment**:\n    ```cmd\n    conda create -n tensorrt python=3.8\n    ```\n\n\u003cbr\u003e\n\n- **Install TensorRT Python**:\n    ```cmd\n    conda activate tensorrt\n\n    cd c:\\TensorRT-8.2.5.1\n\n    pip install python/tensorrt-8.2.5.1-cp38-none-win_amd64.whl (가상환경 버전에 따라 cp36, cp37, cp38, cp39 선택)\n\n    pip install uff/uff-0.6.9-py2.py3-none-any.whl\n\n    pip install graphsurgeon/graphsurgeon-0.4.5-py2.py3-none-any.whl\n\n    pip install onnx_graphsurgeon/onnx_graphsurgeon-0.3.12-py2.py3-none-any.whl\n    ```\n\n- **Check installation**:\n\n    ![image](https://github.com/chansoopark98/Windows-TensorRT-Python/assets/60956651/0e44b042-b5aa-492d-9e0b-53c0c62319a9)\n    \n\n- **Install pycuda**:\n    ```cmd\n    pip install pycuda\n    ```\n\n\u003cbr\u003e\n\u003chr\u003e\n\u003cbr\u003e\n\n# 3. Convert DL Models\n\nTensorRT supports various DL frameworks including Tensorflow, PyTorch, and ONNX.\n\nThis repository contains examples of converting TensorRT models via ONNX.\n\n## 3.1 Install ONNX\n\nFor ONNX installation, install with the virtual environment activated.\n\n```cmd\npip install onnx onnxruntime\n```\n\n\u003cbr\u003e\n\n## 3.2 Convert to ONNX\n\n- 3.2.1 Tensorflow to ONNX\n\n    - Save model :\n\n        It is based on the tensorflow saved model format for easy conversion from Tensorflow to ONNX.\n\n        Store tensorflow model objects in your training or inference code.\n\n        ```python\n        import tensorflow as tf\n        \"\"\" load your tensorflow model \"\"\"\n        model = load_model_func(*args)\n        tf.saved_model.save(model, your_save_path)\n        ```\n\n        **your_save_path** is the save path, and no extension is required.\n\n    - Install tf2onnx:\n        ```cmd\n        pip install -U tf2onnx\n        ```\n\n    - Model conversion:\n        ```cmd\n        python -m tf2onnx.convert --saved-model ./your_save_path/ --output model.onnx --opset 13\n        ```\n\n        \u003cbr\u003e\n\n        **Caution**\n\n        1. You need to adjust the **--opset** version according to your onnx version.\n        2. It can be converted to other forms other than the saved model format. (frozen_graph, checkpoint)\n        3. Details can be checked through **python -m tf2onnx.convert --help**.\n        \n\n\u003cbr\u003e\n\n- 3.2.2 PyTorch to ONNX\n\n    The PyTorch framework uses built-in functions to export ONNX models.\n\n    - Model conversion\n\n        ```python\n        import torch\n        model = load_your_model()\n        torch.onnx.export(model,               \n        x,                         \n        your_save_path + '.onnx',\n        export_params=True,\n        opset_version=13,\n        do_constant_folding=True,\n        input_names = ['input'],\n        output_names = ['output'],\n        dynamic_axes={'input' : {0 : 'batch_size'},\n                    'output' : {0 : 'batch_size'}})\n        ```\n\n\n        **Caution**\n\n        1. You need to adjust the **--opset** version according to your onnx version.\n        2. Input_names and output_names are different for each PyTorch model, so convert according to the layer name.\n\n\u003cbr\u003e\n\n- 3.3 ONNX to TensorRT\n\n    Convert ONNX models converted from Tensorflow/PyTorch to TensorRT engine.\n\n    Copy the converted .onnx file to the path below.\n    ```cmd\n    copy your_saved_onnx_file.onnx c:\\TensorRT-8.2.5.1\\bin\\\n    ```\n\n    \u003cbr\u003e\n    \n    Convert to tensorRT engine using trtexec.\n    ```cmd\n    .\\trtexec.exe --onnx=your_saved_onnx_file.onnx --saveEngine=model.trt\n    ```\n\n    \u003cbr\u003e\n\n    During conversion, additional optimization options can be set using the **--help** command.\n    ```cmd\n    .\\trtexec.exe --help\n    ```\n\n    \u003cbr\u003e\n\n    When the conversion is complete, the tensorRT engine file is created in the path below.\n    ```cmd\n    c:\\TensorRT-8.2.5.1\\bin\\model.trt\n    ```\n\n\u003cbr\u003e\n\u003chr\u003e\n\u003cbr\u003e\n    \n# 4. Inference\n\nYou can check the inference speed and output results of the TensorRT engine file.\n\n```cmd\npython tensorRT_inference_example.py --model=model.trt --b 1 --h 224 --w 224 -c 3\n```\n\nPyTorch model shape(B,C,H,W) enable --torch_mode.\n```cmd\npython tensorRT_inference_example.py --model=model.trt --b 1 --h 224 --w 224 -c 3 --torch_mode\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchansoopark98%2Fwindows-tensorrt-python","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fchansoopark98%2Fwindows-tensorrt-python","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchansoopark98%2Fwindows-tensorrt-python/lists"}