{"id":21068526,"url":"https://github.com/gustavz/realtime_object_detection","last_synced_at":"2025-05-16T03:33:29.175Z","repository":{"id":163900705,"uuid":"115018602","full_name":"gustavz/realtime_object_detection","owner":"gustavz","description":"Plug and Play Real-Time Object Detection App with Tensorflow and OpenCV","archived":false,"fork":false,"pushed_at":"2019-10-04T11:43:00.000Z","size":122226,"stargazers_count":279,"open_issues_count":24,"forks_count":147,"subscribers_count":24,"default_branch":"master","last_synced_at":"2023-10-20T20:08:02.758Z","etag":null,"topics":["api","deep-learning","deep-neural-networks","google","object-detection","opencv","python","real-time","tensorflow"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/gustavz.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2017-12-21T15:11:35.000Z","updated_at":"2023-10-20T20:08:04.081Z","dependencies_parsed_at":null,"dependency_job_id":"ff8787a5-9b3d-4aa0-b27c-e56b43c244b9","html_url":"https://github.com/gustavz/realtime_object_detection","commit_stats":null,"previous_names":[],"tags_count":3,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gustavz%2Frealtime_object_detection","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gustavz%2Frealtime_object_detection/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gustavz%2Frealtime_object_detection/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gustavz%2Frealtime_object_detection/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/gustavz","download_url":"https://codeload.github.com/gustavz/realtime_object_detection/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225403835,"owners_count":17469041,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api","deep-learning","deep-neural-networks","google","object-detection","opencv","python","real-time","tensorflow"],"created_at":"2024-11-19T18:22:36.304Z","updated_at":"2024-11-19T18:22:36.847Z","avatar_url":"https://github.com/gustavz.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# realtime_object_detection\nRealtime Object Detection based on Tensorflow's [Object Detection API](https://github.com/tensorflow/models/tree/master/research/object_detection) and [DeepLab Project](https://github.com/tensorflow/models/tree/master/research/deeplab)\n\u003cimg src=\"test_images/rod.png\" width=\"75.0%\"\u003e\n\u003e Version 1: use branch [r1.0](https://github.com/GustavZ/realtime_object_detection/tree/r1.0) for the original repo that was focused on high performance inference of `ssd_mobilenet` \u003cbr /\u003e\n(*x10 Performance Increase on Nvidia Jetson TX2*)\n\n\u003e Version 2: use branch [Master](https://github.com/GustavZ/realtime_object_detection/tree/master) or to be additionally able to run and test Mask-Detection Models, KCF-Tracking and DeepLab Models (*merge of the repo [realtime_segmenation](https://github.com/GustavZ/realtime_segmenation)*)\n\n\u003e ROS Support: To use this Repo as ROS-Package including detection and segmentation ROS-Nodes use branch [ros](https://github.com/GustavZ/realtime_object_detection/tree/ros).\nAlternativley use the repo [objectdetection_ros](https://github.com/GustavZ/objectdetection_ros)\n\n## About the Project\nThe Idea was to create a scaleable realtime-capable object detection pipeline that runs on various systems. \u003cbr /\u003e\nPlug and play, ready to use without deep previous knowledge.\u003cbr /\u003e \u003cbr /\u003e\nThe project includes following work:\n- optional download of tensorflow pretrained models\n- do Inference with OpenCV, either through video input or on selected test_images. \u003cbr /\u003e\nsupported Models are all `research/object_detection` as well as `research/deeplab` models\n- enjoy this project's own `ssd_mobilenet` speed hack, which splits the model in a mutlithreaded cpu and gpu session. \u003cbr /\u003e\nResults in up to x10 performance increase depending on the running system \u003cbr /\u003e\n⇒ which makes it (one of) the fastest inference piplines out there\n- run statistic tests on sets of images and get statistical information like mean and median fps, std dev and much more\n- create `timeline` files measuring the exact time consumption of each operation in your model\n- inspect, summarize, quantize, transform and benchmark models with the provided `scripts/`\n- Use this Repo as ROS Package. the detection subscirbes a ROS Image topic and publishes the detection as ROS Node.\n\n\n## Inference:  \n- create a copy of `config.sample.yml` named `config.yml` and only change configurations inside this file \u003cbr /\u003e\nFor example: If you are not interested in visualization: set `VISUALIZE` to `False`, \u003cbr /\u003e\nor if you want to switch off the speed hack set `SPLIT_MODEL` to `False`, \u003cbr /\u003e\n- to be able to use KCF_Tracking inside `scripts/` run `bash build_kcf.sh` to build it and set `USE_TRACKER` to `True` to use it \u003cbr /\u003e\n(currently only works for pure object detection models without `SPLIT_MODEL`)\n- new class (Model,Config,Visualizer) structure. Simply create your own test file with:\n    ```\n    from rod.model import ObjectDetectionModel, DeepLabModel\n    from rod.config import Config\n\n    model_type = 'od'                                              #or 'dl'\n    input_type = 'video'                                           #or 'image'\n    config = Config(model_type)\n    model = ObjectDetectionModel(config).prepare_model(input_type) #or DeepLabModel\n    model.run()\n    ```\n- Alternativley run `python` + `objectdetection_video.py` or `objectdetection_image.py` or `deeplab_video.py` or `deeplab_image.py` or `allmodels_image.py`\n\n\n## Scripts:\nTo make use of the tools provided inside `scripts/` follow this guide: \u003cbr /\u003e\n- first change all paths and variables inside `config_tools.sh` to your needs / according to your system\n- When using **the first time** run: `source config_tools.sh` and in the same terminal **run only once** `source build_tools.sh` to build the tools. this will take a while. \u003cbr /\u003e\n- For all following uses first run: `source config_tools.sh`(due to the exported variables) and after that you are able to run the wanted scripts **always from the same terminal** with `source script.sh`.\n- All scripts log the terminal output to `test_results/`\n\n\n## Setup:\nUse the following setup for best and verified performance\n- Ubuntu 16.04\n- Python 2.7\n- Tensorflow 1.4\n([this repo](https://github.com/peterlee0127/tensorflow-nvJetson) provides pre-build tf wheel files for jetson tx2)\n- OpenCV 3.3.1\n\u003e Note: tensorflow v1.7.0 seems to have massive performance issues (try to use other versions)\n\n## Current max Performance on `ssd_mobilenet`:\n- Dell XPS 15 with i7 @ 2.80GHZ x8 and GeForce GTX 1050 4GB:  **100 FPS**\n- Nvidia Jetson Tx2 with Tegra 8GB:                           **30 FPS**\n\n\n\n## Related Work:\n- [objectdetection_ros](https://github.com/GustavZ/objectdetection_ros): This Repository as ROS Package ready to use\n- [test_models](https://github.com/GustavZ/test_models): A repo for models i am currently working on for benchmark tests\n- [deeptraining_hands](https://github.com/GustavZ/deeptraining_hands): A repo for setting up the [ego](http://vision.soic.indiana.edu/projects/egohands/)- and [oxford](http://www.robots.ox.ac.uk/~vgg/data/hands/) hands-datasets.\u003cbr /\u003e\nIt also contains several scripts to convert various annotation formats to be able to train Networks on different deep learning frameworks \u003cbr /\u003e\ncurrently supports `.xml`, `.mat`, `.csv`, `.record`, `.txt` annotations\n- [yolo_for_tf_od_api](https://github.com/GustavZ/yolo_for_tf_od_api): A repo to be able to include Yolo V2 in tf's object detection api\n- [realtime_segmenation](https://github.com/GustavZ/realtime_segmenation): This repo was merged into v2.0\n- [Mobile_Mask_RCNN](https://github.com/GustavZ/Mobile_Mask_RCNN): a Keras Model for training Mask R-CNN for mobile deployment\n- [tf_training](https://github.com/GustavZ/tf_training): Train Mobile Mask R-CNN Models on AWS Cloud\n- [tf_models](https://github.com/GustavZ/tf_models): My `tensorflow/models` fork which includes `yolov2` and `mask_rcnn_mobilenet_v1_coco`\n- [eetfm_automation](https://github.com/GustavZ/eetfm_automation): Export and Evaluation of TensorFlow Models Automation based on the Object Detection API\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgustavz%2Frealtime_object_detection","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgustavz%2Frealtime_object_detection","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgustavz%2Frealtime_object_detection/lists"}