{"id":29093242,"url":"https://github.com/nvidia-ai-iot/tf_trt_models","last_synced_at":"2025-06-28T08:07:50.712Z","repository":{"id":54510007,"uuid":"138945144","full_name":"NVIDIA-AI-IOT/tf_trt_models","owner":"NVIDIA-AI-IOT","description":"TensorFlow models accelerated with NVIDIA TensorRT","archived":false,"fork":false,"pushed_at":"2021-02-14T17:15:34.000Z","size":1291,"stargazers_count":681,"open_issues_count":70,"forks_count":246,"subscribers_count":27,"default_branch":"master","last_synced_at":"2024-05-14T00:14:43.600Z","etag":null,"topics":["image-classification","inference","jetson","models","neural-network","nvidia","object-detection","optimize","realtime","tensorflow","tensorrt","train","tx1","tx2"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/NVIDIA-AI-IOT.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-06-28T00:00:30.000Z","updated_at":"2024-04-28T18:04:23.000Z","dependencies_parsed_at":"2022-08-13T18:10:43.384Z","dependency_job_id":null,"html_url":"https://github.com/NVIDIA-AI-IOT/tf_trt_models","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/NVIDIA-AI-IOT/tf_trt_models","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NVIDIA-AI-IOT%2Ftf_trt_models","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NVIDIA-AI-IOT%2Ftf_trt_models/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NVIDIA-AI-IOT%2Ftf_trt_models/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NVIDIA-AI-IOT%2Ftf_trt_models/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/NVIDIA-AI-IOT","download_url":"https://codeload.github.com/NVIDIA-AI-IOT/tf_trt_models/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NVIDIA-AI-IOT%2Ftf_trt_models/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262396520,"owners_count":23304447,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["image-classification","inference","jetson","models","neural-network","nvidia","object-detection","optimize","realtime","tensorflow","tensorrt","train","tx1","tx2"],"created_at":"2025-06-28T08:07:41.067Z","updated_at":"2025-06-28T08:07:50.706Z","avatar_url":"https://github.com/NVIDIA-AI-IOT.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"TensorFlow/TensorRT Models on Jetson\n====================================\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"data/landing_graphic.jpg\" alt=\"landing graphic\" height=\"300px\"/\u003e\n\u003c/p\u003e\n\nThis repository contains scripts and documentation to use TensorFlow image classification and object detection models on NVIDIA Jetson.  The models are sourced from the [TensorFlow models repository](https://github.com/tensorflow/models)\nand optimized using TensorRT.\n\n* [Setup](#setup)\n* [Image Classification](#ic)\n  * [Models](#ic_models)\n  * [Download pretrained model](#ic_download)\n  * [Build TensorRT / Jetson compatible graph](#ic_build)\n  * [Optimize with TensorRT](#ic_trt)\n  * [Jupyter Notebook Sample](#ic_notebook)\n  * [Train for custom task](#ic_train)\n* [Object Detection](#od)\n  * [Models](#od_models)\n  * [Download pretrained model](#od_download)\n  * [Build TensorRT / Jetson compatible graph](#od_build)\n  * [Optimize with TensorRT](#od_trt)\n  * [Jupyter Notebook Sample](#od_notebook)\n  * [Train for custom task](#od_train)\n\n\u003ca name=\"setup\"\u003e\u003c/a\u003e\nSetup\n-----\n\n1. Flash your Jetson TX2 with JetPack 3.2 (including TensorRT).\n2. Install miscellaneous dependencies on Jetson\n\n   ```\n   sudo apt-get install python-pip python-matplotlib python-pil\n   ```\n   \n3. Install TensorFlow 1.7+ (with TensorRT support).  Download the [pre-built pip wheel](https://devtalk.nvidia.com/default/topic/1031300/jetson-tx2/tensorflow-1-8-wheel-with-jetpack-3-2-/) and install using pip.\n\n    ```\n    pip install tensorflow-1.8.0-cp27-cp27mu-linux_aarch64.whl --user\n    ```\n    \n    or if you're using Python 3.\n    \n    ```\n    pip3 install tensorflow-1.8.0-cp35-cp35m-linux_aarch64.whl --user\n    ```\n\n    \n4. Clone this repository\n\n    ```\n    git clone --recursive https://github.com/NVIDIA-Jetson/tf_trt_models.git\n    cd tf_trt_models\n    ```\n\n5. Run the installation script\n\n    ```\n    ./install.sh\n    ```\n    \n    or if you want to specify python intepreter\n    \n    ```\n    ./install.sh python3\n    ```\n\n\u003ca name=\"ic\"\u003e\u003c/a\u003e\nImage Classification\n--------------------\n\n\n\u003cimg src=\"data/classification_graphic.jpg\" alt=\"classification\" height=\"300px\"/\u003e\n\n\n\u003ca name=\"ic_models\"\u003e\u003c/a\u003e\n### Models\n\n| Model | Input Size | TF-TRT TX2 | TF TX2 |\n|:------|:----------:|-----------:|-------:|\n| inception_v1 | 224x224 | 7.36ms | 22.9ms |\n| inception_v2 | 224x224 | 9.08ms | 31.8ms |\n| inception_v3 | 299x299 | 20.7ms | 74.3ms |\n| inception_v4 | 299x299 | 38.5ms | 129ms  |\n| inception_resnet_v2 | 299x299 |   | 158ms |\n| resnet_v1_50 | 224x224 | 12.5ms | 55.1ms |\n| resnet_v1_101 | 224x224 | 20.6ms | 91.0ms |\n| resnet_v1_152 | 224x224 | 28.9ms | 124ms |\n| resnet_v2_50 | 299x299 | 26.5ms | 73.4ms |\n| resnet_v2_101 | 299x299 | 46.9ms |    |\n| resnet_v2_152 | 299x299 | 69.0ms |    |\n| mobilenet_v1_0p25_128 | 128x128 | 3.72ms | 7.99ms |\n| mobilenet_v1_0p5_160 | 160x160 | 4.47ms | 8.69ms |\n| mobilenet_v1_1p0_224 | 224x224 | 11.1ms | 17.3ms |\n\n**TF** - Original TensorFlow graph (FP32)\n\n**TF-TRT** - TensorRT optimized graph (FP16)\n\nThe above benchmark timings were gathered after placing the Jetson TX2 in MAX-N\nmode.  To do this, run the following commands in a terminal:\n\n```\nsudo nvpmodel -m 0\nsudo ~/jetson_clocks.sh\n```\n\n\u003ca name=\"ic_download\"\u003e\u003c/a\u003e\n### Download pretrained model\n\nAs a convenience, we provide a script to download pretrained models sourced from the\nTensorFlow models repository.  \n\n```python\nfrom tf_trt_models.classification import download_classification_checkpoint\n\ncheckpoint_path = download_classification_checkpoint('inception_v2')\n```\nTo manually download the pretrained models, follow the links [here](https://github.com/tensorflow/models/tree/master/research/slim#Pretrained).\n\n\u003ca name=\"ic_build\"\u003e\u003c/a\u003e\n\n### Build TensorRT / Jetson compatible graph\n\n```python\nfrom tf_trt_models.classification import build_classification_graph\n\nfrozen_graph, input_names, output_names = build_classification_graph(\n    model='inception_v2',\n    checkpoint=checkpoint_path,\n    num_classes=1001\n)\n```\n\n### Optimize with TensorRT\n\n```python\nimport tensorflow.contrib.tensorrt as trt\n\ntrt_graph = trt.create_inference_graph(\n    input_graph_def=frozen_graph,\n    outputs=output_names,\n    max_batch_size=1,\n    max_workspace_size_bytes=1 \u003c\u003c 25,\n    precision_mode='FP16',\n    minimum_segment_size=50\n)\n```\n\n\u003ca name=\"ic_notebook\"\u003e\u003c/a\u003e\n### Jupyter Notebook Sample\n\nFor a comprehensive example of performing the above steps and executing on a real\nimage, see the [jupyter notebook sample](examples/classification/classification.ipynb).\n\n\u003ca name=\"ic_train\"\u003e\u003c/a\u003e\n### Train for custom task\n\nFollow the documentation from the [TensorFlow models repository](https://github.com/tensorflow/models/tree/master/research/slim).\nOnce you have obtained a checkpoint, proceed with building the graph and optimizing\nwith TensorRT as shown above.\n\n\u003ca name=\"od\"\u003e\u003c/a\u003e\nObject Detection \n----------------\n\n\u003cimg src=\"data/detection_graphic.jpg\" alt=\"detection\" height=\"300px\"/\u003e\n\n\u003ca name=\"od_models\"\u003e\u003c/a\u003e\n### Models\n\n| Model | Input Size | TF-TRT TX2 | TF TX2 |\n|:------|:----------:|-----------:|-------:|\n| ssd_mobilenet_v1_coco | 300x300 | 50.5ms | 72.9ms |\n| ssd_inception_v2_coco | 300x300 | 54.4ms | 132ms  |\n\n**TF** - Original TensorFlow graph (FP32)\n\n**TF-TRT** - TensorRT optimized graph (FP16)\n\nThe above benchmark timings were gathered after placing the Jetson TX2 in MAX-N\nmode.  To do this, run the following commands in a terminal:\n\n```\nsudo nvpmodel -m 0\nsudo ~/jetson_clocks.sh\n```\n\n\u003ca name=\"od_download\"\u003e\u003c/a\u003e\n### Download pretrained model\n\nAs a convenience, we provide a script to download pretrained model weights and config files sourced from the\nTensorFlow models repository.  \n\n```python\nfrom tf_trt_models.detection import download_detection_model\n\nconfig_path, checkpoint_path = download_detection_model('ssd_inception_v2_coco')\n```\nTo manually download the pretrained models, follow the links [here](https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md).\n\n\u003e **Important:** Some of the object detection configuration files have a very low non-maximum suppression score threshold (ie. 1e-8).\n\u003e This can cause unnecessarily large CPU post-processing load.  Depending on your application, it may be advisable to raise \n\u003e this value to something larger (like 0.3) for improved performance.  We do this for the above benchmark timings.  This can be done by modifying the configuration\n\u003e file directly before calling build_detection_graph.  The parameter can be found for example in this [line](https://github.com/tensorflow/models/blob/master/research/object_detection/samples/configs/ssd_mobilenet_v1_coco.config#L130).\n\n\u003ca name=\"od_build\"\u003e\u003c/a\u003e\n### Build TensorRT / Jetson compatible graph\n\n```python\nfrom tf_trt_models.detection import build_detection_graph\n\nfrozen_graph, input_names, output_names = build_detection_graph(\n    config=config_path,\n    checkpoint=checkpoint_path\n)\n```\n\n\u003ca name=\"od_trt\"\u003e\u003c/a\u003e\n### Optimize with TensorRT\n\n```python\nimport tensorflow.contrib.tensorrt as trt\n\ntrt_graph = trt.create_inference_graph(\n    input_graph_def=frozen_graph,\n    outputs=output_names,\n    max_batch_size=1,\n    max_workspace_size_bytes=1 \u003c\u003c 25,\n    precision_mode='FP16',\n    minimum_segment_size=50\n)\n```\n\n\u003ca name=\"od_notebook\"\u003e\u003c/a\u003e\n### Jupyter Notebook Sample\n\nFor a comprehensive example of performing the above steps and executing on a real\nimage, see the [jupyter notebook sample](examples/detection/detection.ipynb).\n\n\u003ca name=\"od_train\"\u003e\u003c/a\u003e\n### Train for custom task\n\nFollow the documentation from the [TensorFlow models repository](https://github.com/tensorflow/models/tree/master/research/object_detection).\nOnce you have obtained a checkpoint, proceed with building the graph and optimizing\nwith TensorRT as shown above.  Please note that all models are not tested so \nyou should use an object detection\nconfig file during training that resembles one of the ssd_mobilenet_v1_coco or\nssd_inception_v2_coco models.  Some config parameters may be modified, such as the number of\nclasses, image size, non-max supression parameters, but the performance may vary.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnvidia-ai-iot%2Ftf_trt_models","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnvidia-ai-iot%2Ftf_trt_models","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnvidia-ai-iot%2Ftf_trt_models/lists"}