{"id":13441676,"url":"https://github.com/zhiqwang/yolort","last_synced_at":"2025-05-15T01:06:29.074Z","repository":{"id":37027499,"uuid":"287517711","full_name":"zhiqwang/yolort","owner":"zhiqwang","description":"yolort is a runtime stack for yolov5 on specialized accelerators such as tensorrt, libtorch, onnxruntime, tvm and ncnn.","archived":false,"fork":false,"pushed_at":"2025-03-31T18:24:09.000Z","size":24041,"stargazers_count":729,"open_issues_count":41,"forks_count":151,"subscribers_count":14,"default_branch":"main","last_synced_at":"2025-05-06T04:27:41.412Z","etag":null,"topics":["deployment","detection","graghsurgeon","inference","jit","libtorch","lightning","ncnn","nms","onnx","onnxruntime","pytorch","tensorrt","torchscript","trt","tvm","visualization","yolo","yolort","yolov5"],"latest_commit_sha":null,"homepage":"https://zhiqwang.com/yolort","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zhiqwang.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":".github/CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":".github/CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2020-08-14T11:31:00.000Z","updated_at":"2025-04-10T11:46:34.000Z","dependencies_parsed_at":"2024-01-14T12:17:59.169Z","dependency_job_id":"54787c76-bc20-4b25-8567-634100f95587","html_url":"https://github.com/zhiqwang/yolort","commit_stats":{"total_commits":433,"total_committers":37,"mean_commits":"11.702702702702704","dds":0.5334872979214781,"last_synced_commit":"846334816de2580f254ff9ad8d9f17599d0f319e"},"previous_names":["zhiqwang/yolort","zhiqwang/yolov5-rt-stack"],"tags_count":18,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhiqwang%2Fyolort","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhiqwang%2Fyolort/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhiqwang%2Fyolort/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhiqwang%2Fyolort/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zhiqwang","download_url":"https://codeload.github.com/zhiqwang/yolort/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253171272,"owners_count":21865297,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deployment","detection","graghsurgeon","inference","jit","libtorch","lightning","ncnn","nms","onnx","onnxruntime","pytorch","tensorrt","torchscript","trt","tvm","visualization","yolo","yolort","yolov5"],"created_at":"2024-07-31T03:01:36.809Z","updated_at":"2025-05-15T01:06:24.064Z","avatar_url":"https://github.com/zhiqwang.png","language":"Python","funding_links":[],"categories":["Python","Lighter and Deployment Frameworks","Applications"],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\n\u003cimg src=\"docs/source/_static/yolort_logo.png\" width=\"400px\"\u003e\n\n**YOLOv5 Runtime Stack**\n\n______________________________________________________________________\n\n[Documentation](https://zhiqwang.com/yolort/) •\n[Installation Instructions](https://zhiqwang.com/yolort/installation.html) •\n[Deployment](#-deployment) •\n[Contributing](.github/CONTRIBUTING.md) •\n[Reporting Issues](https://github.com/zhiqwang/yolort/issues/new?assignees=\u0026labels=\u0026template=bug-report.yml)\n\n______________________________________________________________________\n\n[![Python Version](https://img.shields.io/badge/Python-3.6--3.10-FFD43B?logo=python)](https://pypi.org/project/yolort/)\n[![PyPI version](https://img.shields.io/pypi/v/yolort?color=4D97FF\u0026logo=PyPI)](https://badge.fury.io/py/yolort)\n[![PyPI downloads](https://static.pepy.tech/personalized-badge/yolort?period=total\u0026units=international_system\u0026left_color=grey\u0026right_color=brightgreen\u0026left_text=pypi%20downloads)](https://pepy.tech/project/yolort)\n[![Github downloads](https://img.shields.io/github/downloads/zhiqwang/yolort/total?label=Model%20downloads\u0026logo=PyTorch\u0026color=FF6F00\u0026logoColor=EE4C2C)](https://github.com/zhiqwang/yolort/releases)\n[![Slack](https://img.shields.io/badge/Slack%20chat-4A154B?logo=slack\u0026logoColor=white)](https://join.slack.com/t/yolort/shared_invite/zt-mqwc7235-940aAh8IaKYeWclrJx10SA)\n[![PRs Welcome](https://img.shields.io/badge/PRs%20welcome-792EE5?logo=GitHub-Sponsors\u0026logoColor=#white)](.github/CONTRIBUTING.md)\n\n[![CI testing](https://github.com/zhiqwang/yolort/actions/workflows/ci-test.yml/badge.svg)](https://github.com/zhiqwang/yolort/actions/workflows/ci-test.yml)\n[![Build \u0026 deploy docs](https://github.com/zhiqwang/yolort/actions/workflows/gh-pages.yml/badge.svg)](https://github.com/zhiqwang/yolort/tree/gh-pages)\n[![pre-commit.ci status](https://results.pre-commit.ci/badge/github/zhiqwang/yolort/main.svg)](https://results.pre-commit.ci/latest/github/zhiqwang/yolort/main)\n[![codecov](https://codecov.io/gh/zhiqwang/yolort/branch/main/graph/badge.svg?token=1GX96EA72Y)](https://codecov.io/gh/zhiqwang/yolort)\n\n______________________________________________________________________\n\n\u003c/div\u003e\n\n## 🤗 Introduction\n\n**What it is.** Yet another implementation of Ultralytics's [YOLOv5](https://github.com/ultralytics/yolov5). yolort aims to make the training and inference of the object detection task integrate more seamlessly together. yolort now adopts the same model structure as the official YOLOv5. The significant difference is that we adopt the dynamic shape mechanism, and within this, we can embed both pre-processing (letterbox) and post-processing (nms) into the model graph, which simplifies the deployment strategy. In this sense, yolort makes it possible to deploy the object detection more easily and friendly on `LibTorch`, `ONNX Runtime`, `TVM`, `TensorRT` and so on.\n\n**About the code.** Follow the design principle of [detr](https://github.com/facebookresearch/detr):\n\n\u003e object detection should not be more difficult than classification, and should not require complex libraries for training and inference.\n\n`yolort` is very simple to implement and experiment with. Do you like the implementation of torchvision's faster-rcnn, retinanet or detr? Do you like yolov5? You'll love `yolort`!\n\n\u003ca href=\"notebooks/assets/zidane.jpg\"\u003e\u003cimg src=\"notebooks/assets/zidane.jpg\" alt=\"YOLO inference demo\" width=\"500\"/\u003e\u003c/a\u003e\n\n## 🆕 What's New\n\n- *Dec. 27, 2021*. Add `TensorRT` C++ interface example. Thanks to [Shiquan](https://github.com/ShiquanYu).\n- *Dec. 25, 2021*. Support exporting to `TensorRT`, and inferencing with `TensorRT` Python interface.\n- *Sep. 24, 2021*. Add `ONNX Runtime` C++ interface example. Thanks to [Fidan](https://github.com/itsnine).\n- *Feb. 5, 2021*. Add `TVM` compile and inference notebooks.\n- *Nov. 21, 2020*. Add graph visualization tools.\n- *Nov. 17, 2020*. Support exporting to `ONNX`, and inferencing with `ONNX Runtime` Python interface.\n- *Nov. 16, 2020*. Refactor YOLO modules and support *dynamic shape/batch* inference.\n- *Nov. 4, 2020*. Add `LibTorch` C++ inference example.\n- *Oct. 8, 2020*. Support exporting to `TorchScript` model.\n\n## 🛠️ Usage\n\nThere are no extra compiled components in `yolort` and package dependencies are minimal, so the code is very simple to use.\n\n### Installation and Inference Examples\n\n- Above all, follow the [official instructions](https://pytorch.org/get-started/locally/) to install PyTorch 1.8.0+ and torchvision 0.9.0+\n\n- Installation via pip\n\n  Simple installation from [PyPI](https://pypi.org/project/yolort/)\n\n  ```shell\n  pip install -U yolort\n  ```\n\n  Or from Source\n\n  ```shell\n  # clone yolort repository locally\n  git clone https://github.com/zhiqwang/yolort.git\n  cd yolort\n  # install in editable mode\n  pip install -e .\n  ```\n\n- Install pycocotools (for evaluation on COCO):\n\n  ```shell\n  pip install -U 'git+https://github.com/ppwwyyxx/cocoapi.git#subdirectory=PythonAPI'\n  ```\n\n- To read a source of image(s) and detect its objects 🔥\n\n  ```python\n  from yolort.models import yolov5s\n\n  # Load model\n  model = yolov5s(pretrained=True, score_thresh=0.45)\n  model.eval()\n\n  # Perform inference on an image file\n  predictions = model.predict(\"bus.jpg\")\n  # Perform inference on a list of image files\n  predictions = model.predict([\"bus.jpg\", \"zidane.jpg\"])\n  ```\n\n### Loading via `torch.hub`\n\nThe models are also available via torch hub, to load `yolov5s` with pretrained weights simply do:\n\n```python\nmodel = torch.hub.load(\"zhiqwang/yolort:main\", \"yolov5s\", pretrained=True)\n```\n\n### Loading checkpoint from official yolov5\n\nThe following is the interface for loading the checkpoint weights trained with `ultralytics/yolov5`. Please see our documents on what we [share](https://zhiqwang.com/yolort/notebooks/how-to-align-with-ultralytics-yolov5.html) and how we [differ](https://zhiqwang.com/yolort/notebooks/comparison-between-yolort-vs-yolov5.html) from yolov5 for more details.\n\n```python\nfrom yolort.models import YOLOv5\n\n# Download checkpoint from https://github.com/ultralytics/yolov5/releases/download/v6.0/yolov5s.pt\nckpt_path_from_ultralytics = \"yolov5s.pt\"\nmodel = YOLOv5.load_from_yolov5(ckpt_path_from_ultralytics, score_thresh=0.25)\n\nmodel.eval()\nimg_path = \"test/assets/bus.jpg\"\npredictions = model.predict(img_path)\n```\n\n## 🚀 Deployment\n\n### Inference on LibTorch backend\n\nWe provide a [tutorial](https://zhiqwang.com/yolort/notebooks/inference-pytorch-export-libtorch.html) to demonstrate how the model is converted into `torchscript`. And we provide a [C++ example](deployment/libtorch) of how to do inference with the serialized `torchscript` model.\n\n### Inference on ONNX Runtime backend\n\nWe provide a pipeline for deploying yolort with ONNX Runtime.\n\n```python\nfrom yolort.runtime import PredictorORT\n\n# Load the serialized ONNX model\nengine_path = \"yolov5n6.onnx\"\ny_runtime = PredictorORT(engine_path, device=\"cpu\")\n\n# Perform inference on an image file\npredictions = y_runtime.predict(\"bus.jpg\")\n```\n\nPlease check out this [tutorial](https://zhiqwang.com/yolort/notebooks/export-onnx-inference-onnxruntime.html) to use yolort's ONNX model conversion and ONNX Runtime inferencing. And you can use the [example](deployment/onnxruntime) for ONNX Runtime C++ interface.\n\n### Inference on TensorRT backend\n\nThe pipeline for TensorRT deployment is also very easy to use.\n\n```python\nimport torch\nfrom yolort.runtime import PredictorTRT\n\n# Load the serialized TensorRT engine\nengine_path = \"yolov5n6.engine\"\ndevice = torch.device(\"cuda\")\ny_runtime = PredictorTRT(engine_path, device=device)\n\n# Perform inference on an image file\npredictions = y_runtime.predict(\"bus.jpg\")\n```\n\nBesides, we provide a [tutorial](https://zhiqwang.com/yolort/notebooks/onnx-graphsurgeon-inference-tensorrt.html) detailing yolort's model conversion to TensorRT and the use of the Python interface. Please check this [example](deployment/tensorrt) if you want to use the C++ interface.\n\n## 🎨 Model Graph Visualization\n\nNow, `yolort` can draw the model graph directly, checkout our [tutorial](https://zhiqwang.com/yolort/notebooks/model-graph-visualization.html) to see how to use and visualize the model graph.\n\n\u003ca href=\"notebooks/assets/yolov5_graph_visualize.svg\"\u003e\u003cimg src=\"notebooks/assets/yolov5_graph_visualize.svg\" alt=\"YOLO model visualize\" width=\"500\"/\u003e\u003c/a\u003e\n\n## 👋 Contributing\n\nWe love your input! Please see our [Contributing Guide](.github/CONTRIBUTING.md) to get started and for how to help out. Thank you to all our contributors! If you like this project please consider ⭐ this repo, as it is the simplest way to support us.\n\n[![Contributors](https://contrib.rocks/image?repo=zhiqwang/yolort)](https://github.com/zhiqwang/yolort/graphs/contributors)\n\n## 📖 Citing yolort\n\nIf you use yolort in your publication, please cite it by using the following BibTeX entry.\n\n```bibtex\n@Misc{yolort2021,\n  author =       {Zhiqiang Wang and Song Lin and Shiquan Yu and Wei Zeng and Fidan Kharrasov},\n  title =        {YOLORT: A runtime stack for object detection on specialized accelerators},\n  howpublished = {\\url{https://github.com/zhiqwang/yolort}},\n  year =         {2021}\n}\n```\n\n## 🎓 Acknowledgement\n\n- The implementation of `yolov5` borrow the code from [ultralytics](https://github.com/ultralytics/yolov5).\n- This repo borrows the architecture design and part of the code from [torchvision](https://github.com/pytorch/vision).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzhiqwang%2Fyolort","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzhiqwang%2Fyolort","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzhiqwang%2Fyolort/lists"}