{"id":23117722,"url":"https://github.com/justincdavis/trtutils","last_synced_at":"2026-03-12T17:10:43.881Z","repository":{"id":207969648,"uuid":"716334284","full_name":"justincdavis/trtutils","owner":"justincdavis","description":"Utilities for enabling easier high-level usage of TensorRT in Python","archived":false,"fork":false,"pushed_at":"2026-03-06T23:54:38.000Z","size":81965,"stargazers_count":19,"open_issues_count":16,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2026-03-07T01:40:06.760Z","etag":null,"topics":["cuda","dnn-inference","gpu-acceleration","image-classification","inference","jetson","nvidia","object-detection","python","tensorrt","tensorrt-inference"],"latest_commit_sha":null,"homepage":"https://trtutils.readthedocs.io/en/latest/index.html","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/justincdavis.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.rst","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2023-11-08T23:19:24.000Z","updated_at":"2026-03-05T21:02:34.000Z","dependencies_parsed_at":null,"dependency_job_id":"1a9e6e25-2cc3-4d95-9685-e8dc73edb0b2","html_url":"https://github.com/justincdavis/trtutils","commit_stats":null,"previous_names":["justincdavis/trtutils"],"tags_count":24,"template":false,"template_full_name":null,"purl":"pkg:github/justincdavis/trtutils","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/justincdavis%2Ftrtutils","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/justincdavis%2Ftrtutils/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/justincdavis%2Ftrtutils/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/justincdavis%2Ftrtutils/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/justincdavis","download_url":"https://codeload.github.com/justincdavis/trtutils/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/justincdavis%2Ftrtutils/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30434285,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-12T14:34:45.044Z","status":"ssl_error","status_checked_at":"2026-03-12T14:09:33.793Z","response_time":114,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cuda","dnn-inference","gpu-acceleration","image-classification","inference","jetson","nvidia","object-detection","python","tensorrt","tensorrt-inference"],"created_at":"2024-12-17T04:23:42.367Z","updated_at":"2026-03-12T17:10:43.872Z","avatar_url":"https://github.com/justincdavis.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# trtutils\n\n[![](https://img.shields.io/pypi/pyversions/trtutils.svg)](https://pypi.org/pypi/trtutils/)\n![PyPI](https://img.shields.io/pypi/v/trtutils.svg?style=plastic)\n[![CodeFactor](https://www.codefactor.io/repository/github/justincdavis/trtutils/badge)](https://www.codefactor.io/repository/github/justincdavis/trtutils)\n\n![Ty](https://github.com/justincdavis/trtutils/actions/workflows/ty.yaml/badge.svg?branch=main)\n![Ruff](https://github.com/justincdavis/trtutils/actions/workflows/ruff.yaml/badge.svg?branch=main)\n![PyPI Build](https://github.com/justincdavis/trtutils/actions/workflows/build-check.yaml/badge.svg?branch=main)\n\nA high-level Python interface for TensorRT inference, providing a simple and unified way to run arbitrary TensorRT engines. This library abstracts away the complexity of CUDA memory management, binding management, and engine execution, making it easy to perform inference with any TensorRT engine.\n\n## Features\n\n- Simple, high-level interface for TensorRT inference\n- Automatic CUDA memory management\n- Support for arbitrary TensorRT engines\n- Built-in preprocessing and postprocessing capabilities\n- Comprehensive type hints and documentation\n- Support for both basic engine execution and end-to-end model inference\n\n## Supported Models\n\nThe models listed in the [documentation](https://trtutils.readthedocs.io/en/latest/models.html) are officially supported for inference.\n\n## Quick Start\n\n### Basic Engine Usage\n\nThe `TRTEngine` class provides a simple interface for running any TensorRT engine:\n\n```python\nfrom trtutils import TRTEngine\n\n# Load your TensorRT engine\nengine = TRTEngine(\"path_to_engine\")\n\n# Get input specifications\nprint(engine.input_shapes)  # Expected input shapes\nprint(engine.input_dtypes)  # Expected input data types\n\n# Run inference\ninputs = read_your_data()\noutputs = engine.execute(inputs)\n```\n\n## Installation\n\n```bash\npip install trtutils\n```\n\nFor additional features, you can install optional dependencies:\n\n```bash\n# For JIT compiler\npip install \"trtutils[jit]\"\n\n# For development\npip install \"trtutils[dev]\"\n```\n\n## Documentation\n\nFor detailed documentation, including advanced usage, examples, and API reference, visit our [documentation site](https://trtutils.readthedocs.io/).\n\n## Examples\n\nCheck out our [examples directory](examples/) for more detailed usage examples, including:\n- Basic engine usage\n- End-to-end model inference\n- YOLO model implementation\n- Benchmarking utilities\n\n## Performance\n\n| Device            | YOLOv8m                                                                 | YOLOv8n                                                                 |\n|-------------------|-------------------------------------------------------------------------|-------------------------------------------------------------------------|\n| OrinAGX-64GB      | ![OrinAGX-64GB YOLOv8m](benchmark/plots/OrinAGX-64GB/yolov8m.png)       | ![OrinAGX-64GB YOLOv8n](benchmark/plots/OrinAGX-64GB/yolov8n.png)       |\n| OrinAGX-32GB      | ![OrinAGX-32GB YOLOv8m](benchmark/plots/OrinAGX-32GB/yolov8m.png)       | ![OrinAGX-32GB YOLOv8n](benchmark/plots/OrinAGX-32GB/yolov8n.png)       |\n| OrinNX-16GB       | ![OrinNX-16GB YOLOv8m](benchmark/plots/OrinNX-16GB/yolov8m.png)         | ![OrinNX-16GB YOLOv8n](benchmark/plots/OrinNX-16GB/yolov8n.png)        |\n| OrinNano-8GB      | ![OrinNano-8GB YOLOv8m](benchmark/plots/OrinNano-8GB/yolov8m.png)       | ![OrinNano-8GB YOLOv8n](benchmark/plots/OrinNano-8GB/yolov8n.png)       |\n| XavierNX-8GB      | ![XavierNX-8GB YOLOv8m](benchmark/plots/XavierNX-8GB/yolov8m.png)       | ![XavierNX-8GB YOLOv8n](benchmark/plots/XavierNX-8GB/yolov8n.png)       |\n| 3080Ti            | ![3080Ti YOLOv8m](benchmark/plots/3080Ti/yolov8m.png)                   | ![3080Ti YOLOv8n](benchmark/plots/3080Ti/yolov8n.png)                   |\n| TitanRTX          | ![TitanRTX YOLOv8m](benchmark/plots/TitanRTX/yolov8m.png)               | ![TitanRTX YOLOv8n](benchmark/plots/TitanRTX/yolov8n.png)               |\n\n## License\n\nThis project is licensed under the MIT License - see the LICENSE file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjustincdavis%2Ftrtutils","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjustincdavis%2Ftrtutils","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjustincdavis%2Ftrtutils/lists"}