{"id":21802489,"url":"https://github.com/qengineering/rtsp-with-opencv","last_synced_at":"2025-07-12T20:03:49.505Z","repository":{"id":112947984,"uuid":"578206475","full_name":"Qengineering/RTSP-with-OpenCV","owner":"Qengineering","description":"OpenCV Universal Video Interface with neglectable latency.","archived":false,"fork":false,"pushed_at":"2023-08-14T07:22:22.000Z","size":41,"stargazers_count":38,"open_issues_count":0,"forks_count":7,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-27T09:23:06.209Z","etag":null,"topics":["cpp","ffmpeg","ffmpeg-script","gstreamer","gstreamer-pipeline","jetson-nano","no-lag","opencv","raspberry-pi","rtsp","rtsp-client","rtsp-stream","tcp","tcp-client","udp","udp-client"],"latest_commit_sha":null,"homepage":"https://qengineering.eu/install-gstreamer-1.18-on-raspberry-pi-4.html","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Qengineering.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-12-14T14:04:20.000Z","updated_at":"2025-02-04T16:56:47.000Z","dependencies_parsed_at":null,"dependency_job_id":"572c0358-ebcd-4f52-88eb-400a46a05079","html_url":"https://github.com/Qengineering/RTSP-with-OpenCV","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Qengineering%2FRTSP-with-OpenCV","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Qengineering%2FRTSP-with-OpenCV/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Qengineering%2FRTSP-with-OpenCV/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Qengineering%2FRTSP-with-OpenCV/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Qengineering","download_url":"https://codeload.github.com/Qengineering/RTSP-with-OpenCV/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248764418,"owners_count":21158079,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cpp","ffmpeg","ffmpeg-script","gstreamer","gstreamer-pipeline","jetson-nano","no-lag","opencv","raspberry-pi","rtsp","rtsp-client","rtsp-stream","tcp","tcp-client","udp","udp-client"],"created_at":"2024-11-27T11:29:03.302Z","updated_at":"2025-04-13T18:41:56.243Z","avatar_url":"https://github.com/Qengineering.png","language":"C++","funding_links":["https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick\u0026hosted_button_id=CPZTM5BB3FCYL"],"categories":[],"sub_categories":[],"readme":"# RTSP - UDP - TCP streams in OpenCV (with neglectable latency)\nIt is a known issue with RTSP streams and time-consuming algorithms such as deep learning frameworks. You're getting out of sync if individual frames take longer than your stream's frame rate to process. There is an increasing delay between reality and the captured images.\u003cbr\u003e\u003cbr\u003e\nThere are two possible solutions to this problem.\u003cbr\u003e\n* You could continuously grabbing images in a [seperated thread](https://github.com/Qengineering/Multi-Thread-Camera-OpenCV).\u003cbr\u003e\n* Or determine the missed images and skipping them before grabbing a new frame. The latter solution is implemented here.\u003cbr\u003e\n\nThe code speaks for itself.\u003cbr\u003e\nYou can either use GStreamer or FFmpeg to open the stream.\u003cbr\u003e\nRTSP streams usually are compressed with H264. Therefore your decompression is sensitive for timing issues, like threads halted for some time.\n\n------------\n\n## Dependencies.\nTo run the application, you have to:\n- OpenCV installed [32-bit](https://qengineering.eu/install-opencv-4.5-on-raspberry-pi-4.html) or [64-bit](https://qengineering.eu/install-opencv-4.5-on-raspberry-64-os.html) OS\u003cbr/\u003e\n- Code::Blocks installed. (```$ sudo apt-get install codeblocks```)\n\n------------\n\n## Tips.\nUse only a ***wired*** Ethernet connection for your RTSP stream. Wi-Fi can be unstable.\u003cbr\u003e\nBecause the RTSP protocol is sensitive to even just one missing frame, the stream can easily crash.\u003cbr\u003e\u003cbr\u003e\nIf you are using the stream in a deep learning app, adjust your resolution and frame rate to the requirements of the deep learning model.\u003cbr\u003e\nIt is not a good idea to send a 1280x960 stream at 30 FPS if your model has a 416x416 input and takes 200 mSec to process a single frame.\u003cbr\u003e\nIt only costs extra memory and processing power.\u003cbr\u003e\u003cbr\u003e\nIf you want to stream UDP or TCP, are sure the streams work with the command line prompt beforehand. If not, they certainly won't work in OpenCV. Often errors are caused by the coding in the pipeline, the addresses or missing modules. If you need to install additional GStreamer modules, you'll need to rebuild your OpenCV also! For more information, see our [website](https://qengineering.eu/install-gstreamer-1.18-on-raspberry-pi-4.html).\n\n------------\n\n## Running the app.\nTo run the application load the project file RTSPcam.cbp in Code::Blocks.\u003cbr\u003e\u003cbr\u003e \nIf you are using a **Jetson Nano**, you have to change the location where OpenCV has stored its header files to `/usr/include/opencv4`\u003cbr\u003e\u003cbr\u003e\nAt line 16 in main.cpp the stream is opened.\n#### RTSP\n```\ncam.Open(\"rtsp://192.168.178.129:8554/test/\");\n```\n#### UDP\n```\ncam.Open(\"udpsrc port=5200 ! application/x-rtp, media=video, clock-rate=90000, payload=96 ! rtpjpegdepay ! jpegdec ! videoconvert ! appsink\", cv::CAP_GSTREAMER);\n```\nSender: RaspiCam with Raspberry Pi **Buster** OS\u003cbr\u003e\n`gst-launch-1.0 -v v4l2src device=/dev/video0 num-buffers=-1 ! video/x-raw, width=640, height=480, framerate=30/1 ! videoconvert ! jpegenc ! rtpjpegpay ! udpsink host=192.168.178.84 port=5200`\u003cbr\u003e\u003cbr\u003e\nSender: RaspiCam with Raspberry Pi **Bullseye** OS\u003cbr\u003e\n`gst-launch-1.0 -v libcamerasrc ! video/x-raw, width=640, height=480, framerate=30/1 ! videoconvert ! jpegenc ! rtpjpegpay ! udpsink host=192.168.178.84 port=5200`\u003cbr\u003e\u003cbr\u003e\nNote, `host=192.168.178.84` being the IP address of the receiver.\n\n#### TCP\n```\ncam.Open(\"tcpclientsrc host=192.168.178.129 port=5000 ! jpegdec ! videoconvert ! appsink\", cv::CAP_GSTREAMER);\n```\nSender: RaspiCam with Raspberry Pi **Buster** OS\u003cbr\u003e\n`gst-launch-1.0 -v v4l2src device=/dev/video0 num-buffers=-1 ! video/x-raw,width=640,height=480, framerate=30/1 ! videoconvert ! jpegenc ! tcpserversink  host=192.168.178.32 port=5000`\u003cbr\u003e\u003cbr\u003e\nSender: RaspiCam with Raspberry Pi **Bullseye** OS\u003cbr\u003e\n`gst-launch-1.0 -v libcamerasrc ! video/x-raw,width=640,height=480, framerate=30/1 ! videoconvert ! jpegenc ! tcpserversink  host=192.168.178.32 port=5000`\u003cbr\u003e\u003cbr\u003e\nNote, `host=192.168.178.32` being the IP address of the sender.\n\n#### RaspiCam (Bullseye)\n```\ncam.Open(\"libcamerasrc ! video/x-raw, width=640, height=480, framerate=30/1 ! videoconvert ! videoscale ! video/x-raw, width=640, height=480 ! appsink\", cv::CAP_GSTREAMER);\n```\n#### RaspiCam (Buster)\n```\ncam.Open(\"v4l2src device=/dev/video0 ! video/x-raw, width=640, height=480, framerate=30/1 ! videoconvert ! videoscale ! video/x-raw, width=640, height=480 ! appsink\", cv::CAP_GSTREAMER);\n```\n#### RaspiCam (only Buster or Bullseye legacy stack)\n```\ncam.Open(0);\n```\n#### Webcam (only Buster OS or Bullseye legacy stack)\n```\ncam.Open(0); //if RaspiCam is not connected\ncam.Open(1); //if RaspiCam is connected\n```\n#### MP4\n```\ncam.Open(\"James.mp4\");\n```\n#### Folder\n```\ncam.Open(\"/home/pi/Pictures/Plants\");\n```\n#### Single file\n```\ncam.Open(\"/home/pi/Pictures/Garden.jpg\");\n```\n### Retrieve the stream\n```cpp\nint main()\n{\n    cv::Mat frame;\n    RTSPcam cam;\n\n    cv::namedWindow(\"Camera\", cv::WINDOW_AUTOSIZE);\n\n    cam.Open(\"rtsp://192.168.178.129:8554/test/\"); //you can dump anything OpenCV eats. (cv::CAP_ANY) BTW,OpenCV first tries FFmpeg\n\n    while(true)\n    {\n        if(!cam.GetLatestFrame(frame)){\n            cout \u003c\u003c \"Capture read error\" \u003c\u003c endl;\n            break;\n        }\n        //place here your time consuming algorithms\n//        cout \u003c\u003c cam.CurrentFileName \u003c\u003c endl;\n        //show frame\n        cv::imshow(\"Camera\",frame);\n        char esc = cv::waitKey(2);\n        if(esc == 27) break;\n    }\n    cv::destroyAllWindows() ;\n    return 0;\n}\n```\n------------\n![output image]( https://qengineering.eu/github/RTSPstream.webp )\u003cbr\u003e\u003cbr\u003e\n[![paypal](https://qengineering.eu/images/TipJarSmall4.png)](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick\u0026hosted_button_id=CPZTM5BB3FCYL) \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fqengineering%2Frtsp-with-opencv","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fqengineering%2Frtsp-with-opencv","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fqengineering%2Frtsp-with-opencv/lists"}