{"id":20058651,"url":"https://github.com/roboflow/trt-demos","last_synced_at":"2025-07-12T19:33:12.428Z","repository":{"id":98131837,"uuid":"585272836","full_name":"roboflow/trt-demos","owner":"roboflow","description":"This is a repo for Roboflow TFT python examples.","archived":false,"fork":false,"pushed_at":"2023-10-09T21:21:54.000Z","size":8606,"stargazers_count":3,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-30T20:50:53.712Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/roboflow.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-01-04T18:52:34.000Z","updated_at":"2023-10-17T16:01:37.000Z","dependencies_parsed_at":null,"dependency_job_id":"59859b4a-74fd-4a1d-ab99-f8bf9f2ee8d0","html_url":"https://github.com/roboflow/trt-demos","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/roboflow%2Ftrt-demos","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/roboflow%2Ftrt-demos/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/roboflow%2Ftrt-demos/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/roboflow%2Ftrt-demos/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/roboflow","download_url":"https://codeload.github.com/roboflow/trt-demos/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252514982,"owners_count":21760481,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-13T13:02:46.906Z","updated_at":"2025-05-05T14:31:29.940Z","avatar_url":"https://github.com/roboflow.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Roboflow TRT Container Demos\n\nThis repository is for quickly accessing examples on how to use the Roboflow TRT Docker container. Our docker container is an enterprise feature, please refer to our [Enterprise Documentation](https://docs.roboflow.com/inference/enterprise/) for more information.\n\n## Install Roboflow\n\nTo install this package, please use `Python 3.6` or higher. We provide three different ways to install the Roboflow\npackage to use within your own projects.\n\nInstall from PyPi (Recommended):\n\n```bash\npip install roboflow\n```\n\n## Building the Roboflow Container Load Balancer\n\nTo build the load balancer docker container use the command below.\n\n```\ndocker build . -t lb\n```\n\nMake sure that the names of the services in the docker-compose.yaml file are correctly reflected in the .conf/roboflow-nginx.conf file.\n\n```\ndocker-compose up\n```\n\nYour Docker should now be spinning up multiple GPU containers that all share a volume and a port with the load balancer. This way the load balencer can manage the throughput of each container for optimal speed.\n\n# Configuring Testing Scripts\n\nBefore we start docker and run the scripts we need to configure them to your Roboflow account first. For this next step we will need your Roboflow API Key. If you don't have your API key, you can learn how to get it in our [REST API documentation.](https://docs.roboflow.com/rest-api#obtaining-your-api-key)\n\n## Configure TRT-API-MultiContainer.py Test Script\n\n```python\n# Setting up arrays for Roboflow model varaibles (Lines 15-18)\nversion_number_array = [version_num1, version_num2, version_num3, version_num4]\nproject_id_array = [\"model_id1/\", \"model_id2/\", \"model_id3/\", \"model_id4/\"]\nimages_folder_array = [\"folder_path1\", \"folder_path2\", \"folder_path3\", \"folder_path4\"]\napi_key_array = ['API1', 'API2', 'API3', 'API4'] # It is likely you will have the same API keys\n```\n\n\n## Quickstart - GPU Guide\n\nMake sure that you have Docker Desktop running or if you are using Linux you can start Docker Daemon using `sudo service docker start` while having Docker installed.\n\nThese examples will require two terminal windows. Open them now for your convenience.\n\n\n### Run inference with API (Recommended)\n\n```bash\n## Terminal 1 - Run docker ('sudo' linux only)\nsudo docker run --gpus all -p 9001:9001 roboflow/inference-server-trt:latest\n## Terminal 2\npython3 TRT-API.py\n```\n\n### Run inference with Python Package\n\n```bash\n## Terminal 1 - Run docker ('sudo' linux only)\nsudo docker run --gpus all -p 9001:9001 roboflow/inference-server-trt:latest\n## Terminal 2\npython3 TRT-Python.py\n```\n\n### Run inference on multiple containers with API\n\n```bash\n## Terminal 1 - Run docker ('sudo' linux only)\nsudo docker compose up\n## Terminal 2\npython3 TRT-API-MultiContainer.py\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Froboflow%2Ftrt-demos","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Froboflow%2Ftrt-demos","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Froboflow%2Ftrt-demos/lists"}