{"id":13437653,"url":"https://github.com/olibartfast/object-detection-inference","last_synced_at":"2025-03-19T17:33:20.741Z","repository":{"id":49819781,"uuid":"129530299","full_name":"olibartfast/object-detection-inference","owner":"olibartfast","description":"C++ object detection inference from video or image input source","archived":false,"fork":false,"pushed_at":"2024-04-13T15:37:00.000Z","size":39894,"stargazers_count":41,"open_issues_count":1,"forks_count":7,"subscribers_count":6,"default_branch":"master","last_synced_at":"2024-04-13T18:02:06.341Z","etag":null,"topics":["gstreamer","libtensorflow","libtorch","object-detection","onnxruntime","rt-detr","tensorflow","tensorrt","yolo"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/olibartfast.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null},"funding":{"custom":["https://www.justgiving.com/page/francesco-oliva-greatnorthrun2024-spinalresearch"]}},"created_at":"2018-04-14T15:36:41.000Z","updated_at":"2024-04-21T23:09:18.128Z","dependencies_parsed_at":"2023-02-03T00:32:03.656Z","dependency_job_id":"c6f09ff0-305b-4062-83b5-8b8e4253f1fa","html_url":"https://github.com/olibartfast/object-detection-inference","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/olibartfast%2Fobject-detection-inference","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/olibartfast%2Fobject-detection-inference/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/olibartfast%2Fobject-detection-inference/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/olibartfast%2Fobject-detection-inference/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/olibartfast","download_url":"https://codeload.github.com/olibartfast/object-detection-inference/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244474208,"owners_count":20458607,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["gstreamer","libtensorflow","libtorch","object-detection","onnxruntime","rt-detr","tensorflow","tensorrt","yolo"],"created_at":"2024-07-31T03:00:59.028Z","updated_at":"2025-03-19T17:33:20.736Z","avatar_url":"https://github.com/olibartfast.png","language":"C++","funding_links":["https://www.justgiving.com/page/francesco-oliva-greatnorthrun2024-spinalresearch"],"categories":["C++","Lighter and Deployment Frameworks"],"sub_categories":[],"readme":"# Object Detection Inference\n\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n[![C++20](https://img.shields.io/badge/C++-20-blue.svg)](https://isocpp.org/std/the-standard)\n\nC++ framework for [real-time object detection](https://paperswithcode.com/sota/real-time-object-detection-on-coco), supporting multiple deep learning backends and input sources. Run state-of-the-art object detection models on video streams, video files, or images with configurable hardware acceleration.\n\n\n## 🚀 Key Features\n\n- Multiple model support (YOLO series from YOLOv4 to YOLOv12, RT-DETR, D-FINE, DEIM)\n- Switchable inference backends (OpenCV DNN, ONNX Runtime, TensorRT, Libtorch, OpenVINO, Libtensorflow)\n- Real-time video processing with GStreamer integration\n- GPU acceleration support\n- Docker deployment ready\n- Benchmarking tools included\n\n## 🔧 Requirements\n\n### Core Dependencies\n- CMake (≥ 3.15)\n- C++17 compiler (GCC ≥ 8.0)\n- OpenCV (≥ 4.6)\n  ```bash\n  apt install libopencv-dev\n  ```\n- Google Logging (glog)\n  ```bash\n  apt install libgoogle-glog-dev\n  ```\n\n\n### Fetched Dependencies\nThe project automatically fetches and builds the following dependencies using CMake's FetchContent:\n\n#### [VideoCapture Library](https://github.com/olibartfast/videocapture) (Only for the App module, not the library)\n```cmake\nFetchContent_Declare(\n    VideoCapture\n    GIT_REPOSITORY https://github.com/olibartfast/videocapture\n    GIT_TAG main\n)\n```\n- Handles video input processing\n- Provides unified interface for various video sources\n- Optional GStreamer integration\n\n\n#### [Inference Engines Library](https://github.com/olibartfast/inference-engines)\n```cmake\nFetchContent_Declare(\n    InferenceEngines\n    GIT_REPOSITORY https://github.com/olibartfast/inference-engines\n    GIT_TAG main\n)\n```\n- Provides abstraction layer for multiple inference backends\n- Supported backends:\n  - OpenCV DNN Module \n  - ONNX Runtime (default)\n  - LibTorch\n  - TensorRT(10.0.7.23)\n  - OpenVINO\n  - LibTensorflow(2.13)\n \n⚠️ **Note**: **After the CMake configuration step, fetched dependencies are cloned into the ``build/_deps`` folder.**\n\n## 🏗 Building\n\n### Complete Build (Shared Library + Application)\n```bash\nmkdir build \u0026\u0026 cd build\ncmake -DDEFAULT_BACKEND=\u003cbackend\u003e -DBUILD_ONLY_LIB=OFF -DCMAKE_BUILD_TYPE=Release ..\ncmake --build .\n```\n\n#### Enabling GStreamer Support\n```bash\ncmake -DDEFAULT_BACKEND=\u003cbackend\u003e -DBUILD_ONLY_LIB=OFF -DUSE_GSTREAMER=ON -DCMAKE_BUILD_TYPE=Release ..\ncmake --build .\n```\n\n---\n\n### Library-Only Build\n```bash\nmkdir build \u0026\u0026 cd build\ncmake -DBUILD_ONLY_LIB=ON -DDEFAULT_BACKEND=\u003cbackend\u003e -DCMAKE_BUILD_TYPE=Release ..\ncmake --build .\n```\n\n---\n\n### Backend Options\nReplace `\u003cbackend\u003e` with one of the following options:  \n- **`OPENCV_DNN`**   \n- **`ONNX_RUNTIME`**  \n- **`LIBTORCH`**  \n- **`TENSORRT`**  \n- **`OPENVINO`**  \n- **`LIBTENSORFLOW`**  \n\n---\n\n### Notes  \n\n1. **Custom Backend Paths**  \n   If the required backend package is not installed system-wide, you can manually specify its path:  \n\n   - **Libtorch**  \n     Modify [`LibTorch.cmake`](https://github.com/olibartfast/inference-engines/blob/master/cmake/LibTorch.cmake) or pass the `Torch_DIR` argument.  \n\n   - **ONNX Runtime**  \n     Modify [`ONNXRuntime.cmake`](https://github.com/olibartfast/inference-engines/blob/master/cmake/ONNXRuntime.cmake) or pass the `ONNX_RUNTIME_DIR` and `ORT_VERSION` arguments.  \n\n   - **TensorRT**  \n     Modify [`TensorRT.cmake`](https://github.com/olibartfast/inference-engines/blob/master/cmake/TensorRT.cmake) or pass the `TENSORRT_DIR` and `TRT_VERSION` arguments.  \n\n   - ⚠️ **Important:**  \n     - These CMake files belong to the [`InferenceEngines`](https://github.com/olibartfast/inference-engines) project and are cloned into the `build/_deps` folder after the configuration step.  \n     - Ensure your backend version is set correctly in [cmake/AddCompileDefinitions.cmake](cmake/AddCompileDefinitions.cmake).  \n\n2. **Cleaning the Build Folder**  \n   When switching backends or changing configuration options, clean the `build` directory before reconfiguring and compiling.  \n\n   **Full Clean (Major Changes)**  \n   - **Command:**  \n     ```sh\n     rm -rf build \u0026\u0026 mkdir build\n     ```  \n\n   **Partial Clean (Minor Changes)**  \n   - **Command:**  \n     ```sh\n     rm build/CMakeCache.txt\n     ```  \n---\n### Test Builds\n```bash\n# App tests\ncmake -DENABLE_APP_TESTS=ON ..\n\n# Library tests\ncmake -DENABLE_DETECTORS_TESTS=ON ..\n```\n\n## 💻 App Usage\n\n### Command Line Options\n\n```bash\n./object-detection-inference \\\n  [--help | -h] \\\n  --type=\u003cmodel_type\u003e \\\n  --source=\u003cinput_source\u003e \\\n  --labels=\u003clabels_file\u003e \\\n  --weights=\u003cmodel_weights\u003e \\\n  [--min_confidence=\u003cthreshold\u003e] \\\n  [--batch|-b=\u003cbatch_size\u003e] \\\n  [--input_sizes|-is='\u003cinput_sizes\u003e'] \\\n  [--use-gpu] \\\n  [--warmup] \\\n  [--benchmark] \\\n  [--iterations=\u003cnumber\u003e]\n```\n\n#### Required Parameters\n\n- `--type=\u003cmodel_type\u003e`: Specifies the type of object detection model to use. Possible values include `yolov4`, `yolov5`, `yolov6`, `yolov7`, `yolov8`, `yolov9`, `yolov10`, `yolo11`, `yolov12`, `rtdetr`,`rtdetrv2`, `rtdetrul`, `dfine`, `deim`.\n\n- `--source=\u003cinput_source\u003e`: Defines the input source for the object detection. It can be:\n  - A live feed URL, e.g., `rtsp://cameraip:port/stream`\n  - A path to a video file, e.g., `path/to/video.format`\n  - A path to an image file, e.g., `path/to/image.format`\n\n- `--labels=\u003cpath/to/labels/file\u003e`: Specifies the path to the file containing the class labels. This file should list the labels used by the model, with each label on a new line.\n\n- `--weights=\u003cpath/to/model/weights\u003e`: Defines the path to the file containing the model weights.\n\n#### Optional Parameters\n\n- `[--min_confidence=\u003cconfidence_value\u003e]`: Sets the minimum confidence threshold for detections. Detections with a confidence score below this value will be discarded. The default value is `0.25`.\n\n- `[--batch | -b=\u003cbatch_size\u003e]`: Specifies the batch size for inference. Default value is `1`, inference with batch size bigger than 1 is not currently supported.\n\n- `[--input_sizes | -is=\u003cinput_sizes\u003e]`: Input sizes for each model input when models have dynamic axes or the backend can't retrieve input layer information (like the OpenCV DNN module). Format: `CHW;CHW;...`. For example:\n  - `'3,224,224'` for a single input\n  - `'3,224,224;3,224,224'` for two inputs\n  - `'3,640,640;2'` for RT-DETR/D-FINE/DEIM models\n\n- `[--use-gpu]`: Activates GPU support for inference. This can significantly speed up the inference process if a compatible GPU is available. Default is `false`.\n\n- `[--warmup]`: Enables GPU warmup. Warming up the GPU before performing actual inference can help achieve more consistent and optimized performance. This parameter is relevant only if the inference is being performed on an image source. Default is `false`.\n\n- `[--benchmark]`: Enables benchmarking mode. In this mode, the application will run multiple iterations of inference to measure and report the average inference time. This is useful for evaluating the performance of the model and the inference setup. This parameter is relevant only if the inference is being performed on an image source. Default is `false`.\n\n- `[--iterations=\u003cnumber\u003e]`: Specifies the number of iterations for benchmarking. The default value is `10`.\n\n### To check all available options:\n\n```bash\n./object-detection-inference --help\n```\n\n### Common Use Case Examples\n\n```bash\n# YOLOv8 Onnx Runtime image processing\n./object-detection-inference \\\n  --type=yolov8 \\\n  --source=image.png \\\n  --weights=models/yolov8s.onnx \\\n  --labels=data/coco.names\n\n# YOLOv8 TensorRT video processing\n./object-detection-inference \\\n  --type=yolov8 \\\n  --source=video.mp4 \\\n  --weights=models/yolov8s.engine \\\n  --labels=data/coco.names \\\n  --min_confidence=0.4\n\n# RTSP stream processing using RT-DETR Ultralytics implementation\n    --type=rtdetrul \\\n    --source=\"rtsp://camera:554/stream\" \\\n    --weights=models/rtdetr-l.onnx \\\n    --labels=data/coco.names \\\n    --use-gpu\n```\n\n*Check the [`.vscode folder`](.vscode/launch.json) for other examples.*\n\n## 🐳 Docker Deployment\n\n### Building Images\nInside the project, in the [Dockerfiles folder](docker), there will be a dockerfile for each inference backend (currently onnxruntime, libtorch, tensorrt, openvino)\n```bash\n# Build for specific backend\ndocker build --rm -t object-detection-inference:\u003cbackend_tag\u003e  \\\n    -f docker/Dockerfile.backend .\n```\n\n### Running Containers\nReplace the wildcards with your desired options and paths:\n```bash\ndocker run --rm \\\n    -v\u003cpath_host_data_folder\u003e:/app/data \\\n    -v\u003cpath_host_weights_folder\u003e:/weights \\\n    -v\u003cpath_host_labels_folder\u003e:/labels \\\n    object-detection-inference:\u003cbackend_tag\u003e \\\n    --type=\u003cmodel_type\u003e \\\n    --weights=\u003cweight_according_your_backend\u003e \\\n    --source=/app/data/\u003cimage_or_video\u003e \\\n    --labels=/labels/\u003clabels_file\u003e\n```\n\n\nFor GPU support, add `--gpus all` to the docker run command.\n\n\n## 🗺 Project Structure\n\n```\n.\n├── app/            # Main application\n├── detectors/      # Detection library\n├── cmake/          # CMake modules\n└── docker/         # Dockerfiles\n└── build/_deps/    # Fetched dependencies after CMake configuration\n```\n\n## 📚 Additional Resources\n\n- [Supported Models](docs/TablePage.md)\n- [Model Export Guide](docs/ExportInstructions.md)\n- Backend-specific export documentation:\n  - [YOLOv5](docs/yolov5-export.md)\n  - [YOLOv8](docs/yolov8-export.md)\n  - [YOLOv6](docs/yolov6-export.md)\n  - [YOLOv7](docs/yolov7-export.md)\n  - [YOLOv8](docs/yolov8-export.md)\n  - [YOLOv9](docs/yolov9-export.md)\n  - [YOLOv10](docs/yolov10-export.md)\n  - [YOLO11](docs/yolo11-export.md)\n  - [YOLOv12](docs/yolov12-export.md)\n  - [YOLO-NAS](docs/yolo-nas-export.md)\n  - [RT-DETR (lyuwenyu implementation)](docs/rtdetr-lyuwenyu-export.md)\n  - [RT-DETRV2](docs/rtdetrv2-lyuwenyu-export.md)\n  - [RT-DETR (Ultralytics implementation)](docs/rtdetr-ultralytics-export.md)\n  - [D-FINE](docs/d-fine-export.md)\n  - [DEIM](docs/deim-export.md)\n\n## ⚠️ Known Limitations\n- Windows builds not currently supported\n- Some model/backend combinations may require specific export configurations\n\n## 🙏 Acknowledgments\n- [OpenCV YOLO detection with DNN module](https://github.com/opencv/opencv/blob/4.x/samples/dnn/yolo_detector.cpp)\n- [TensorRTx](https://github.com/wang-xinyu/tensorrtx)\n- [RT-DETR Deploy](https://github.com/CVHub520/rtdetr-onnxruntime-deploy)\n\n ## References\n - https://paperswithcode.com/sota/real-time-object-detection-on-coco\n - https://leaderboard.roboflow.com/\n\n## 📫 Support\n\n- Open an [issue](https://github.com/olibartfast/object-detection-inference/issues) for bug reports or feature requests\n- Check existing issues for solutions to common problems\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Folibartfast%2Fobject-detection-inference","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Folibartfast%2Fobject-detection-inference","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Folibartfast%2Fobject-detection-inference/lists"}