https://github.com/mrtj/pyds_tracker_meta
A Python wrapper written in C++11 for past frame tracking result metadata classes in Nvidia's DeepStream framework, providing functionality not available in the Python SDK.
https://github.com/mrtj/pyds_tracker_meta
deepstream nvidia object-tracking pybind11
Last synced: 5 months ago
JSON representation
A Python wrapper written in C++11 for past frame tracking result metadata classes in Nvidia's DeepStream framework, providing functionality not available in the Python SDK.
- Host: GitHub
- URL: https://github.com/mrtj/pyds_tracker_meta
- Owner: mrtj
- License: mit
- Created: 2020-06-05T14:53:05.000Z (over 5 years ago)
- Default Branch: master
- Last Pushed: 2020-09-25T12:51:29.000Z (over 5 years ago)
- Last Synced: 2025-04-14T11:54:50.819Z (9 months ago)
- Topics: deepstream, nvidia, object-tracking, pybind11
- Language: C++
- Homepage:
- Size: 28.3 KB
- Stars: 30
- Watchers: 2
- Forks: 4
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# pyds_tracker_meta
[pybind11](https://github.com/pybind/pybind11) wrapper to access Nvidia [DeepStream](https://developer.nvidia.com/deepstream-sdk) tracker meta info (`NvDsPastFrame...` classes) from Python.
> The [`master`](//github.com/mrtj/pyds_tracker_meta/tree/master) branch of this repository is compatible with DeepStream SDK 5.0. If you are using an earlier version of the SDK (including DeepStrem SDK 5.0 Developer Preview), check out the [`deepstream-4.0`](//github.com/mrtj/pyds_tracker_meta/tree/deepstream-4.0) branch.
## Notes for DeepStreamer 5.0
Starting from DeepStream SDK 5.0 the python bindings for `NvDsPastFrame...` classes are included in the official SDK. However some vital functions are missing:
- to cast a generic user metadata to `NvDsPastFrameObjBatch` object
- to correctly iterate through the elements of `list` fields of `NvDsPastFrameObjBatch`, `NvDsPastFrameObjStream` and `NvDsPastFrameObjList`. These fields are standard C arrays in the native SDK, so simply exposing them in python will give access in the best case only the first element of the array. The pointer arithmetic to iterate over the elements is also included in this library.
## Introduction
This library provides utility functions to access to the `NvDsPastFrameObjBatch` type user metadata of DeepStream data streams. This metadata is generated by the object tracker of [nvtracker](https://docs.nvidia.com/metropolis/deepstream/dev-guide/#page/DeepStream%20Plugins%20Development%20Guide/deepstream_plugin_details.3.02.html#) plugin. The C API of these structures can be found in [nvds_tracker_meta.h](https://docs.nvidia.com/metropolis/deepstream/dev-guide/DeepStream_Development_Guide/baggage/nvds__tracker__meta_8h.html) header file of DeepStream SDK. The instances of these metadata structure contain information about the object tracking results in past frames.
Some trackers do not report the state of the tracked object if there are matching detection results in the current frame. When in a later frame a detection result confirms the state of the tracked object, the past history of the tracking is reported retroactively in `NvDsPastFrame...` metadata. This library provides Python access to this metadata. For more information refer to the [nvtracker](https://docs.nvidia.com/metropolis/deepstream/dev-guide/#page/DeepStream%20Plugins%20Development%20Guide/deepstream_plugin_details.3.02.html#) plugin documentation.
## Installation
### Prerequisites
1. Install [pybind11](https://github.com/pybind/pybind11). The recommended way is to [build it from source](https://pybind11.readthedocs.io/en/stable/basics.html?highlight=install#compiling-the-test-cases). Alternatively you might try simply `pip3 install pybind11`.
2. You should have `gstreamer-1.0` and `gstreamer-video-1.0` packages installed in your system. If you are using DeepStream, you most likely installed these packages.
3. You will need also the standard `c++` compiler you usually find in Linux distributions. `c++11` standard is used.
### Compile the source
1. The source should be compiled on your target platform (Jetson or x86).
2. Set your DeepStream version and path in `build.sh`.
3. Launch `build.sh`
4. Install the compiled module with `python setup.py install` (use sudo or python3 if needed).
## Usage
`pyds_tracker_meta` is meant to be used together with the standard [Python bindings for DeepStream](https://github.com/NVIDIA-AI-IOT/deepstream_python_apps). Make sure you have `pyds` available.
Ensure you have set `enable-past-frame` property of the `gst-nvtracker` plugin to `1`. (See [nvtracker](https://docs.nvidia.com/metropolis/deepstream/dev-guide/#page/DeepStream%20Plugins%20Development%20Guide/deepstream_plugin_details.3.02.html#) plugin documentation.)
Most likely you will use this library from the buffer probe callbacks of a gstreamer plugin pad, when the object tracking results are available. The [deepstream-test2](https://github.com/NVIDIA-AI-IOT/deepstream_python_apps/tree/master/apps/deepstream-test2) python app shows you how to set up such a callback.
The example snippet provided bellow shows how to cast a user meta to a past frame object batch, and how to access all fields of the metadata. Add the following lines to the `osd_sink_pad_buffer_probe` method found int `deepstream-test2.py`, just after the [`batch_meta` was acquired](https://github.com/NVIDIA-AI-IOT/deepstream_python_apps/blob/2931f6b295b58aed15cb29074d13763c0f8d47be/apps/deepstream-test2/deepstream_test_2.py#L61):
```python
import pyds_tracker_meta
def osd_sink_pad_buffer_probe(pad, info, u_data):
# ... code to acquire batch_meta ...
user_meta_list = batch_meta.batch_user_meta_list
while user_meta_list is not None:
user_meta = pyds.NvDsUserMeta.cast(user_meta_list.data)
print('user_meta:', user_meta)
print('user_meta.user_meta_data:', user_meta.user_meta_data)
print('user_meta.base_meta:', user_meta.base_meta)
if user_meta.base_meta.meta_type != pyds.NvDsMetaType.NVDS_TRACKER_PAST_FRAME_META:
continue
pfob = pyds_tracker_meta.NvDsPastFrameObjBatch_cast(user_meta.user_meta_data)
print('past_frame_object_batch:', pfob)
print(' list:')
for pfos in pyds_tracker_meta.NvDsPastFrameObjBatch_list(pfob):
print(' past_frame_object_stream:', pfos)
print(' streamID:', pfos.streamID)
print(' surfaceStreamID:', pfos.surfaceStreamID)
print(' list:')
for pfol in pyds_tracker_meta.NvDsPastFrameObjStream_list(pfos):
print(' past_frame_object_list:', pfol)
print(' numObj:', pfol.numObj)
print(' uniqueId:', pfol.uniqueId)
print(' classId:', pfol.classId)
print(' objLabel:', pfol.objLabel)
print(' list:')
for pfo in pyds_tracker_meta.NvDsPastFrameObjList_list(pfol):
print(' past_frame_object:', pfo)
print(' frameNum:', pfo.frameNum)
print(' tBbox.left:', pfo.tBbox.left)
print(' tBbox.width:', pfo.tBbox.width)
print(' tBbox.top:', pfo.tBbox.top)
print(' tBbox.right:', pfo.tBbox.height)
print(' confidence:', pfo.confidence)
print(' age:', pfo.age)
try:
user_meta_list = user_meta_list.next
except StopIteration:
break
```
Written by [@jtolgyesi](http://twitter.com/jtolgyesi)
Powered by [Neosperience](https://www.neosperience.com)