{"id":18277457,"url":"https://github.com/oaid/tenginegst","last_synced_at":"2025-04-05T04:30:33.203Z","repository":{"id":64906135,"uuid":"417012300","full_name":"OAID/TengineGst","owner":"OAID","description":"TengineGst is a streaming media analytics framework, based on GStreamer multimedia framework, for creating varied complex media analytics pipelines. It ensures pipeline interoperability and provides optimized media, and inference operations using Tengine Toolkit Inference Engine backend, across varied architecture - CPU, iGPU and  VPU. ","archived":false,"fork":false,"pushed_at":"2021-10-27T06:45:48.000Z","size":38542,"stargazers_count":57,"open_issues_count":1,"forks_count":13,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-03-20T21:42:05.355Z","etag":null,"topics":["gstreamer","mosquitto","opencv","tengine"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/OAID.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-10-14T06:21:07.000Z","updated_at":"2025-01-17T16:07:15.000Z","dependencies_parsed_at":"2022-12-17T17:23:03.920Z","dependency_job_id":null,"html_url":"https://github.com/OAID/TengineGst","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OAID%2FTengineGst","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OAID%2FTengineGst/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OAID%2FTengineGst/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OAID%2FTengineGst/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/OAID","download_url":"https://codeload.github.com/OAID/TengineGst/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247289382,"owners_count":20914463,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["gstreamer","mosquitto","opencv","tengine"],"created_at":"2024-11-05T12:19:15.105Z","updated_at":"2025-04-05T04:30:32.197Z","avatar_url":"https://github.com/OAID.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# TengineGst\n\n## 简介\nTengineGst 是 OPEN AI LAB 基于 GStreamer 多媒体框架的分析推理框架，用于创建各类多媒体处理管道，可以方便的利用各种成熟的插件快速搭建起稳定的应用，并使用 Tengine 优化推理操作，更快更优更专注的开发核心 AI 业务。\n完整的解决方案利用了：\n- 用于管道管理的开源 GStreamer 框架;\n- GStreamer 用于输入和输出的插件，如媒体文件和来自摄像头或网络的实时流媒体;\n- GStreamer 各种成熟插件，例如编解码、图形处理等;\n- 从主流训练框架 Caffe、TensorFlow、ONNX、Darknet 等转换而来的 Tengine 深度学习模型 tmfile。\n\nTengineGst 中深度学习推理的插件：\n- 推理插件利用Tengine 使用深度学习模型进行高性能推理；\n- 推理结果的可视化，带有检测对象的边界框和标签，位于视频流之上；\n- 推理结果可以通过MQTT等标准协议推送出去。\n\n## 架构\n![架构](https://github.com/OAID/TengineGst/blob/main/docs/TengineGst.png)\n数据流\n![pipeline](https://github.com/OAID/TengineGst/blob/main/docs/TengineGst-Flow.png)\n\n## 插件包括\n- streammux：多路流合并成一路由一路算法处理多路；\n- streamdemux：一路推理的结果分离出相对应的各路，与 streammux 配合使用；\n- videoanalysis：主要的推理插件，提供了标准二次插件接口，可以动态加载推理业务，在大多数不想写插件的时候，只需要实现一个业务动态库，由此插件把推理业务交给推理业务库即可。配合类“inferservice”的业务库使用。如果特别熟悉 GStreamer 插件开发，可以自己写一个插件来直接进行推理业务；\n- mqtt：把推理结果泵向 mqtt broker 的功能；\n- postprocess：简单的把推理结果叠加到视频流的功能。\n\n## 业务插件\ninferservice：调用Tengine 推理框架，加载模型，推理结果，并把结果输出到分析插件按照类 inferservice 库的框架编译的库，也可以作为 videoanalysis 插件的业务库传入，修改 businessdll 属性为业务库地址，即可以支持不同算法业务。\n\n## 需要安装依赖\n```\nsudo apt update\nsudo apt install pkg-config\nsudo apt install pkgconf\nsudo apt install -y build-essential cmake\nsudo apt install gstreamer1.0-tools libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev gstreamer1.0-libav gstreamer1.0-plugins-bad gstreamer1.0-plugins-good\napt install libssl-dev\n```\n\n目前在x86/khadas 环境编译，其他设备只需替换不同 Tengine 推理库即可。\n\n### 编译方法\n#### 依赖模块进行编译\n##### Tengine\n```\n//参考：https://github.com/OAID/Tengine/tree/tengine-lite/doc/docs_zh/source_compile，编译不同 npu 版本，下面例子编译x86\ncd Tengine\nmkdir build \ncd build\ncmake ..\nmake\nmake install\ncp -r install/include/ /usr/local/\ncp install/lib/* /usr/local/lib/\n```\n\n##### zlog：\n```\nwget https://github.com/HardySimpson/zlog/archive/refs/tags/1.2.15.tar.gz\ntar zxvf 1.2.15.tar.gz\ncd zlog-1.2.15\nmake PREFIX=/usr/local\nsudo make PREFIX\n=/usr/local install\n```\n##### mosquitto\n```\ngit clone https://github.com/eclipse/mosquitto.git\ncd mosquitto \u0026\u0026 cd lib\nmake \u0026\u0026 make install\n```\n\n##### turbo-jpeg\n```\ngit clone https://github.com/libjpeg-turbo/libjpeg-turbo.git\ncd libjpeg-turbo\nmkdir build\ncmake ..\nmake \u0026\u0026 make install\n```\n#### opencv\n```\nwget https://github.com/opencv/opencv/archive/3.4.16.zip\nunzip 3.4.16.zip\ncd opencv-3.4.16\nmkdir build\ncd build\ncmake ..\nmake \u0026\u0026 make install\n```\n\n### 编译工程\ncmake 目录下 cross.cmake 文件有交叉编译开关。交叉编译环境需调整交叉编译路径。对应的 khadas 支持库已经提供，打开开关即可。\n\n具体编译方法：\n```\nmkdir build\ncd build\ncmake ..\nmake\n```\n\n### 示例\nkhadas 交叉编译，需要 env-run.tar.gz 解压出来，可以减少编译步骤，这时需要把子目录 run 里面的库拷贝到设备里面，模型文件参看源码里面指定的路径，拷贝到相应的目录。\n```\ntar zxvf env-run.tar.gz\n```\n\n编译完成，build 目录会产生 aarc64/Release/lib 目录，里面即是所有的插件。把里面的插件库拷贝到GStreamer 目录下。类似命令：\n```bash\n// khadas\ncp aarch64/Release/lib/libgst* /usr/lib/aarch64-linux-gnu/gstreamer-1.0/\n// x86\ncp aarch64/Release/lib/libgst* /usr/lib/x86_64-linux-gnu/gstreamer-1.0/\n```\n\n执行类似命令检查插件 `gst-inspect-1.0 mqtt` 。因为示例插件，需要把子目录 `run` 里面的库拷贝到设备的 `/usr/lib/aarch64-linux-gnu/` 目录下\n模型文件拷贝到：`/home/khadas/`（见插件 inferservice） 即可，这个发布的时候，可以随意指定路径。\n\n### 测试命令\n```\ngst-launch-1.0 rtspsrc location=\"rtsp://**\" ! rtph264depay ! capsfilter caps=\"video/x-h264\" ! h264parse ! avdec_h264 !  videoanalysis businessdll=/dir/libinferservice.so  ! postprocess ! mqtt username=** userpwd=** servip=** servport=1883 ! fakevideosink\n```\n\n当推理得到结果，就会把结果通过 `mqtt` 插件发送到 `mqtt broker`。 `mqtt` 测试工具可以用 `MQTTBox`，订阅主题 `detect_result` 可以查看推理结果。\n\n## 致谢\n- [Tengine](https://github.com/OAID/Tengine)\n- [GStreamer](https://gstreamer.freedesktop.org/src/)\n- [curl](https://github.com/curl/curl.git)\n- [zlog](https://github.com/lisongmin/zlog)\n- [mosquitto](https://github.com/eclipse/mosquitto)\n- [turbojpeg](https://github.com/libjpeg-turbo/libjpeg-turbo)\n## License\n- [Apache 2.0](https://github.com/OAID/Tengine/blob/tengine-lite/LICENSE)\n- LGPL\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foaid%2Ftenginegst","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Foaid%2Ftenginegst","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foaid%2Ftenginegst/lists"}