How can I stream Mat frames to RTSP with Emgu on Windows?

Hi,

I’m using EmguCv and I’m trying to use VideoWriter to create video from Mat frames.
I have tried to create VideoWriter objects using multiple FourCC codecs like XVID, DIVX, MJPG, MPEG … No one works for me.

How can I see codecs available on my Windows 10? I have also tried with codec 0 or codec -1 and no popup is shown to selec codec.
I’m always getting this Exception

“Unable to create VideoWriter. Make sure you have specific Codec installed”

Thanks for your help. With regards,

call cv::getBuildInformation() and check the video I/O section.

MJPG in an .AVI container always works because it’s built-in.

you need to present a MRE. so far, you haven’t presented any code. there’s nothing to debug for anyone.

If you have an issue with Emgu, contact those people. they aren’t just a wrapper around OpenCV. they changed the API in non-trivial ways. it’s up to them to support their stuff.

Hi,
Thanks for your reply. Sorry for not showing an example, I didn’t put because I have been trying it in multiple ways.

I have tried specifing backend and without it.
I have seen also the API guide of EmguCv and it seems to be very similar to OpenCv in C++

var size = new Size(1280, 720);
var codec = VideoWriter.Fourcc('M', 'J', 'P', 'G');
var backends = CvInvoke.WriterBackends;
var backend = backends.FirstOrDefault(back => back.Name == "FFMPEG");
var writer = new VideoWriter(@"C:\video.mp4", codec, fps: 30, size, isColor: true);

Here is the BuildInformation

General configuration for OpenCV 4.8.1 =====================================
  Version control:               4.8.1-255-gbb5bf05043
 
  Extra modules:
    Location (extra):            G:/bb/cv_x64/build/build_x86_64/../opencv_contrib/modules
    Version control (extra):     4.8.0-65-g614c709a
 
  Platform:
    Timestamp:                   2023-10-17T12:19:02Z
    Host:                        Windows 10.0.22000 AMD64
    CMake:                       3.23.0
    CMake generator:             Visual Studio 17 2022
    CMake build tool:            C:/Program Files/Microsoft Visual Studio/2022/Community/MSBuild/Current/Bin/amd64/MSBuild.exe
    MSVC:                        1937
    Configuration:               Debug Release MinSizeRel RelWithDebInfo
 
  CPU/HW features:
    Baseline:                    SSE SSE2 SSE3
      requested:                 SSE3
 
  C/C++:
    Built as dynamic libs?:      NO
    C++ standard:                11
    C++ Compiler:                C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.37.32822/bin/Hostx64/x64/cl.exe  (ver 19.37.32825.0)
    C++ flags (Release):         /DWIN32 /D_WINDOWS /W4 /GR  /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi  /fp:precise     /EHa /wd4127 /wd4251 /wd4324 /wd4275 /wd4512 /wd4589 /wd4819 /MP  /MD /O2 /Ob2 /DNDEBUG 
    C++ flags (Debug):           /DWIN32 /D_WINDOWS /W4 /GR  /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi  /fp:precise     /EHa /wd4127 /wd4251 /wd4324 /wd4275 /wd4512 /wd4589 /wd4819 /MP  /MDd /Zi /Ob0 /Od /RTC1 
    C Compiler:                  C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.37.32822/bin/Hostx64/x64/cl.exe
    C flags (Release):           /DWIN32 /D_WINDOWS /W3  /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi  /fp:precise     /MP   /MD /O2 /Ob2 /DNDEBUG 
    C flags (Debug):             /DWIN32 /D_WINDOWS /W3  /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi  /fp:precise     /MP /MDd /Zi /Ob0 /Od /RTC1 
    Linker flags (Release):      /machine:x64  /INCREMENTAL:NO 
    Linker flags (Debug):        /machine:x64  /debug /INCREMENTAL 
    ccache:                      NO
    Precompiled headers:         YES
    Extra dependencies:          wsock32 comctl32 gdi32 ole32 setupapi ws2_32 G:/bb/cv_x64/build/build_x86_64/install/lib/freetype.lib G:/bb/cv_x64/build/build_x86_64/install/lib/harfbuzz.lib G:/bb/cv_x64/build/build_x86_64/install/lib/libhdf5.lib
    3rdparty dependencies:       libprotobuf ade ittnotify libjpeg-turbo libwebp libpng libtiff libopenjp2 IlmImf zlib quirc
 
  OpenCV modules:
    To be built:                 alphamat aruco bgsegm bioinspired calib3d ccalib core datasets dnn dnn_objdetect dnn_superres dpm face features2d flann freetype fuzzy gapi hdf hfs highgui img_hash imgcodecs imgproc intensity_transform line_descriptor mcc ml objdetect optflow phase_unwrapping photo plot quality rapid reg rgbd saliency shape stereo stitching structured_light superres surface_matching text tracking ts video videoio videostab wechat_qrcode xfeatures2d ximgproc xobjdetect xphoto
    Disabled:                    java python_bindings_generator python_tests world
    Disabled by dependency:      -
    Unavailable:                 cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters cudaimgproc cudalegacy cudaobjdetect cudaoptflow cudastereo cudawarping cudev cvv julia matlab ovis python2 python3 sfm viz
    Applications:                perf_tests
    Documentation:               NO
    Non-free algorithms:         NO
 
  Windows RT support:            NO
 
  GUI:                           WIN32UI
    Win32 UI:                    YES
    VTK support:                 NO
 
  Media I/O: 
    ZLib:                        build (ver 1.2.13)
    JPEG:                        build-libjpeg-turbo (ver 2.1.3-62)
      SIMD Support Request:      YES
      SIMD Support:              NO
    WEBP:                        build (ver encoder: 0x020f)
    PNG:                         build (ver 1.6.37)
    TIFF:                        build (ver 42 - 4.2.0)
    JPEG 2000:                   build (ver 2.5.0)
    OpenEXR:                     build (ver 2.3.0)
    HDR:                         YES
    SUNRASTER:                   YES
    PXM:                         YES
    PFM:                         YES
 
  Video I/O:
    DC1394:                      NO
    FFMPEG:                      YES (prebuilt binaries)
      avcodec:                   YES (58.134.100)
      avformat:                  YES (58.76.100)
      avutil:                    YES (56.70.100)
      swscale:                   YES (5.9.100)
      avresample:                YES (4.0.0)
    GStreamer:                   NO
    DirectShow:                  YES
    Media Foundation:            YES
      DXVA:                      YES
 
  Parallel framework:            Concurrency
 
  Trace:                         YES (with Intel ITT)
 
  Other third-party libraries:
    Eigen:                       YES (ver 3.4.0)
    Custom HAL:                  NO
    Protobuf:                    build (3.19.1)
    Flatbuffers:                 builtin/3rdparty (23.5.9)
 
  OpenCL:                        YES (NVD3D11)
    Include path:                G:/bb/cv_x64/build/opencv/3rdparty/include/opencl/1.2
    Link libraries:              Dynamic load
 
  Python (for build):            C:/python-virt/python37/python.exe
 
  Java:                          
    ant:                         NO
    Java:                        YES (ver 1.8.0.202)
    JNI:                         C:/Program Files/Microsoft/jdk-11.0.16.101-hotspot/include C:/Program Files/Microsoft/jdk-11.0.16.101-hotspot/include/win32 C:/Program Files/Microsoft/jdk-11.0.16.101-hotspot/include
    Java wrappers:               NO
    Java tests:                  NO
 
  Install to:                    G:/bb/cv_x64/build/build_x86_64/install
-----------------------------------------------------------------

Thanks for your help

that looks fine.

what Mats are you trying to write? shape (mind the order of width and height), element type, …

also remember to release() the writer.

your code snippet asks for an .mp4 file. ffmpeg should handle that but it’s not built-in. only .AVI container carrying MJPG is built-in. if there’s an issue with ffmpeg, that will matter. if the issue is with the frames you try to write, it will not matter.

Hi,
The exception is thrown on VideoWriter creation, I’m not able to write any Mat object into de writer.

I’ll try to explain better what I want to do. I’m reading frames from an Intel.RealSense camera and do some processing like motion detection with OpenCv (or EmguCv or OpenCvSharp).

After that I want to stream the processed frames using RTSP, so other clients can play the processed images with the motion detection applied (rectangles inside the image).

Also I need the streaming to be multicast (multiple clients at same time have to be able to play streaming).

For the streaming I think that FFMEPG or GStreamer can do the work (I’m not an expert on that) but I think that RTSP is the commonly used protocol for my target.

I’m open to other options or alternatives if you think that I’m on the wrong way.

I have seen this message in the console when trying to create VideoWriter with MJPG.
Also I have try with OpenCvSharp and the message is the same.

OpenCV: FFMPEG: tag 0x47504a4d/‘MJPG’ is not supported with codec id 7 and format ‘mp4 / MP4 (MPEG-4 Part 14)’
OpenCV: FFMPEG: fallback to use tag 0x7634706d/‘mp4v’

Also I have try to create VideoWriter with OpenCvSharp with codec -1 and this is the codec list in the console.

OpenCV: FFMPEG: format mp4 / MP4 (MPEG-4 Part 14)
fourcc tag 0x7634706d/‘mp4v’ codec_id 000C
fourcc tag 0x31637661/‘avc1’ codec_id 001B
fourcc tag 0x33637661/‘avc3’ codec_id 001B
fourcc tag 0x31766568/‘hev1’ codec_id 00AD
fourcc tag 0x31637668/‘hvc1’ codec_id 00AD
fourcc tag 0x7634706d/‘mp4v’ codec_id 0002
fourcc tag 0x7634706d/‘mp4v’ codec_id 0001
fourcc tag 0x7634706d/‘mp4v’ codec_id 0007
fourcc tag 0x7634706d/‘mp4v’ codec_id 003D
fourcc tag 0x7634706d/‘mp4v’ codec_id 0058
fourcc tag 0x312d6376/‘vc-1’ codec_id 0046
fourcc tag 0x63617264/‘drac’ codec_id 0074
fourcc tag 0x7634706d/‘mp4v’ codec_id 00A3
fourcc tag 0x39307076/‘vp09’ codec_id 00A7
fourcc tag 0x31307661/‘av01’ codec_id 801D
fourcc tag 0x6134706d/‘mp4a’ codec_id 15002
fourcc tag 0x63616c61/‘alac’ codec_id 15010
fourcc tag 0x6134706d/‘mp4a’ codec_id 1502D
fourcc tag 0x6134706d/‘mp4a’ codec_id 15001
fourcc tag 0x6134706d/‘mp4a’ codec_id 15000
fourcc tag 0x332d6361/‘ac-3’ codec_id 15003
fourcc tag 0x332d6365/‘ec-3’ codec_id 15028
fourcc tag 0x6134706d/‘mp4a’ codec_id 15004
fourcc tag 0x61706c6d/‘mlpa’ codec_id 1502C
fourcc tag 0x43614c66/‘fLaC’ codec_id 1500C
fourcc tag 0x7375704f/‘Opus’ codec_id 1503C
fourcc tag 0x6134706d/‘mp4a’ codec_id 15005
fourcc tag 0x6134706d/‘mp4a’ codec_id 15018
fourcc tag 0x6134706d/‘mp4a’ codec_id 15803
fourcc tag 0x7334706d/‘mp4s’ codec_id 17000
fourcc tag 0x67337874/‘tx3g’ codec_id 17005
fourcc tag 0x646d7067/‘gpmd’ codec_id 18807
fourcc tag 0x316d686d/‘mhm1’ codec_id 15817

You said that only .AVI container carrying MJPG is built-in. What is the FourCC code for Avi container with MJPG? I tried with MPEG, AVC1, MP4V, ut seems none of those working.

Thanks so much for your help. With regards,

for VideoWriter, gstreamer only.

Your problem is probably in the output video file suffix, which should match MJPG codec for AVI - now the library tries to find a codec for .mp4 files, and fails. Try .avi for the suffix.

FFMPEG can’t do the work? why? Is there any other option different than using VideoWriter? I was exploring first FFMPEG because is built-in in the Windows compilation, to include GStreamer backend it seems to be neccesary to rebuild OpenCv.

With regards,

opencv’s ffmpeg wrapper does not handle rtsp (or other) streaming (even if ffmpeg itself can do that)

Thanks for your reply.
What do you think is the best way to create an RTSP server from Mat frames? I’m trying to use VideoWriter because it’s the way that I’ve found, but I’m not an expert on that.
To use Gstreamer I think it’s a little bit complicated because I have to rebuild OpenCv distribution for Windows.
Do you know some good guide to follow it?

With regards,