How to deploy RTSP streaming server with VideoWriter and Gstreamer

Hi,

I’m trying to deploy a RTSP server with OpenCV and VideoWriter in Windows.
For doing that I read that I have to rebuild OpenCv from source.

I have follow this link to make OpenCv works in VisualStudio.

To make it work I have to copy manually all dlls in Release folder including all gstreamer dlls.

At this time it seems that I have OpenCv running.
I have done a little example to read some streaming using VideoCapture and VideoWriter as RTSP server.

Here is my source code. It seems that writer is failing to open (see above).

#include <opencv2/core.hpp>
#include <opencv2/imgcodecs.hpp>
#include <opencv2/highgui.hpp>
#include <iostream>
 
using namespace cv;
using namespace std;
 
int main()
{
    std::cout << cv::getBuildInformation();
 
    string command = "appsrc ! videoconvert ! mfxh264enc ! video/x-h264, stream-format=byte-stream, profile=baseline ! send-config=true rtph264pay pt=96 ! udpsink host=127.0.0.1 port=5004";
 
    cv::VideoCapture cap;
    cap.open("http://pendelcam.kip.uni-heidelberg.de/mjpg/video.mjpg");
    double fps = cap.get(CAP_PROP_FPS);
    cv::VideoWriter writer;
    cout << "FPS " << fps;
    cv::Size size = cv::Size(1280, 720);
    writer.open(command, cv::CAP_GSTREAMER, 0, fps, size, true);
    while (cap.isOpened()) {
        cv:Mat img;
        cap.read(img);
        writer.write(img);
        cv::imshow("frame", img);
        if (cv::waitKey(1) > 0)
            break;
    }
 
    cap.release();
    writer.release();
}

GetBuildInformation()

General configuration for OpenCV 4.9.0 =====================================
  Version control:               unknown
 
  Platform:
    Timestamp:                   2024-04-09T09:02:25Z
    Host:                        Windows 10.0.19044 AMD64
    CMake:                       3.29.1
    CMake generator:             Visual Studio 17 2022
    CMake build tool:            C:/Program Files/Microsoft Visual Studio/2022/Professional/MSBuild/Current/Bin/amd64/MSBuild.exe
    MSVC:                        1939
    Configuration:               Debug Release
 
  CPU/HW features:
    Baseline:                    SSE SSE2 SSE3
      requested:                 SSE3
    Dispatched code generation:  SSE4_1 SSE4_2 FP16 AVX AVX2 AVX512_SKX
      requested:                 SSE4_1 SSE4_2 AVX FP16 AVX2 AVX512_SKX
      SSE4_1 (18 files):         + SSSE3 SSE4_1
      SSE4_2 (2 files):          + SSSE3 SSE4_1 POPCNT SSE4_2
      FP16 (1 files):            + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 AVX
      AVX (9 files):             + SSSE3 SSE4_1 POPCNT SSE4_2 AVX
      AVX2 (38 files):           + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2
      AVX512_SKX (8 files):      + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2 AVX_512F AVX512_COMMON AVX512_SKX
 
  C/C++:
    Built as dynamic libs?:      YES
    C++ standard:                11
    C++ Compiler:                C:/Program Files/Microsoft Visual Studio/2022/Professional/VC/Tools/MSVC/14.39.33519/bin/Hostx64/x64/cl.exe  (ver 19.39.33523.0)
    C++ flags (Release):         /DWIN32 /D_WINDOWS /W4 /GR  /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi  /fp:precise     /EHa /wd4127 /wd4251 /wd4324 /wd4275 /wd4512 /wd4589 /wd4819 /MP  /O2 /Ob2 /DNDEBUG
    C++ flags (Debug):           /DWIN32 /D_WINDOWS /W4 /GR  /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi  /fp:precise     /EHa /wd4127 /wd4251 /wd4324 /wd4275 /wd4512 /wd4589 /wd4819 /MP  /Zi /Ob0 /Od /RTC1
    C Compiler:                  C:/Program Files/Microsoft Visual Studio/2022/Professional/VC/Tools/MSVC/14.39.33519/bin/Hostx64/x64/cl.exe
    C flags (Release):           /DWIN32 /D_WINDOWS /W3  /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi  /fp:precise     /MP   /O2 /Ob2 /DNDEBUG
    C flags (Debug):             /DWIN32 /D_WINDOWS /W3  /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi  /fp:precise     /MP /Zi /Ob0 /Od /RTC1
    Linker flags (Release):      /machine:x64  /INCREMENTAL:NO
    Linker flags (Debug):        /machine:x64  /debug /INCREMENTAL
    ccache:                      NO
    Precompiled headers:         YES
    Extra dependencies:
    3rdparty dependencies:
 
  OpenCV modules:
    To be built:                 calib3d core dnn features2d flann gapi highgui imgcodecs imgproc ml objdetect photo python3 stitching ts video videoio
    Disabled:                    world
    Disabled by dependency:      -
    Unavailable:                 java python2
    Applications:                tests perf_tests apps
    Documentation:               NO
    Non-free algorithms:         NO
 
  Windows RT support:            NO
 
  GUI:                           WIN32UI
    Win32 UI:                    YES
    VTK support:                 NO
 
  Media I/O:
    ZLib:                        build (ver 1.3)
    JPEG:                        build-libjpeg-turbo (ver 2.1.3-62)
      SIMD Support Request:      YES
      SIMD Support:              NO
    WEBP:                        build (ver encoder: 0x020f)
    PNG:                         build (ver 1.6.37)
    TIFF:                        build (ver 42 - 4.2.0)
    JPEG 2000:                   build (ver 2.5.0)
    OpenEXR:                     build (ver 2.3.0)
    HDR:                         YES
    SUNRASTER:                   YES
    PXM:                         YES
    PFM:                         YES
 
  Video I/O:
    DC1394:                      NO
    FFMPEG:                      YES (prebuilt binaries)
      avcodec:                   YES (58.134.100)
      avformat:                  YES (58.76.100)
      avutil:                    YES (56.70.100)
      swscale:                   YES (5.9.100)
      avresample:                YES (4.0.0)
    GStreamer:                   YES (1.24.1)
    DirectShow:                  YES
    Media Foundation:            YES
      DXVA:                      YES
 
  Parallel framework:            Concurrency
 
  Trace:                         YES (with Intel ITT)
 
  Other third-party libraries:
    Intel IPP:                   2021.11.0 [2021.11.0]
           at:                   C:/CMake/opencv/opencv/build/3rdparty/ippicv/ippicv_win/icv
    Intel IPP IW:                sources (2021.11.0)
              at:                C:/CMake/opencv/opencv/build/3rdparty/ippicv/ippicv_win/iw
    Lapack:                      NO
    Eigen:                       YES (ver 3.4.0)
    Custom HAL:                  NO
    Protobuf:                    build (3.19.1)
    Flatbuffers:                 builtin/3rdparty (23.5.9)
 
  OpenCL:                        YES (NVD3D11)
    Include path:                C:/CMake/opencv/opencv/3rdparty/include/opencl/1.2
    Link libraries:              Dynamic load
 
  Python 3:
    Interpreter:                 C:/Program Files/Python312/python.exe (ver 3.12.2)
    Libraries:                   C:/Program Files/Python312/libs/python312.lib (ver 3.12.2)
    numpy:                       C:/Users/JuanDYB/AppData/Roaming/Python/Python312/site-packages/numpy/core/include (ver 1.26.4)
    install path:                C:/Program Files/Python312/Lib/site-packages/cv2/python-3.12
 
  Python (for build):            C:/Program Files/Python312/python.exe
 
  Java:
    ant:                         NO
    Java:                        NO
    JNI:                         NO
    Java wrappers:               NO
    Java tests:                  NO
 
  Install to:                    C:/CMake/opencv/opencv/build/install
-----------------------------------------------------------------

Error on VideoWriter

[ WARN:0@0.957] global cap_gstreamer.cpp:2250 cv::CvVideoWriter_GStreamer::close_ OpenCV | GStreamer warning: No source in GStreamer pipeline. Ignore
[ERROR:0@0.960] global cap.cpp:645 cv::VideoWriter::open VIDEOIO(GSTREAMER): raised OpenCV exception:
 
OpenCV(4.9.0) C:\CMake\opencv\opencv\modules\videoio\src\cap_gstreamer.cpp:154: error: (-215:Assertion failed) ptr in function 'cv::`anonymous-namespace'::GSafePtr<struct _GstElement>::get'

Also I’m not sure how to check if it’s working with VLC.
I read that I have to open an sdp file like this. I don’t know if opening rtsp://127.0.0.1:5004/ will work.

c=IN IP4 127.0.0.1
m=video 5004 RTP/AVP 96
a=rtpmap:96 H264/90000

Note: I’m not a C++ developer I would like to make this on C# but at the moment is that I have. I’m running this on a clean Windows HyperV machine with firewall disabled.

Thanks for your help.

I have added to library directories $(OPENCV_DIR)\lib\gstreamer-1.0 directory because I have seen that this is de folder for gstreamer plugins.

With that modification the error is a little bit different but not very descriptive.

[ WARN:0@0.959] global cap_gstreamer.cpp:2250 cv::CvVideoWriter_GStreamer::close_ OpenCV | GStreamer warning: No source in GStreamer pipeline. Ignore
[ERROR:0@1.232] global cap.cpp:645 cv::VideoWriter::open VIDEOIO(GSTREAMER): raised OpenCV exception:
 
OpenCV(4.9.0) C:\CMake\opencv\opencv\modules\videoio\src\cap_gstreamer.cpp:154: error: (-215:Assertion failed) ptr in function 'cv::`anonymous-namespace'::GSafePtr<struct _GstElement>::get'