Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/nvidia/tensorrt-llm
TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines.
https://github.com/nvidia/tensorrt-llm
Last synced: 3 days ago
JSON representation
TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines.
- Host: GitHub
- URL: https://github.com/nvidia/tensorrt-llm
- Owner: NVIDIA
- License: apache-2.0
- Created: 2023-08-16T17:14:27.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2024-12-11T17:30:54.000Z (22 days ago)
- Last Synced: 2024-12-14T00:28:36.444Z (20 days ago)
- Language: C++
- Homepage: https://nvidia.github.io/TensorRT-LLM
- Size: 443 MB
- Stars: 8,890
- Watchers: 95
- Forks: 1,024
- Open Issues: 357
-
Metadata Files:
- Readme: README.md
- License: LICENSE