Best way to diagnose VideoCapture not opening the rtmp stream

I am pulling my hair for a few days and I’m out of ideas.

I have two rtmp streams

  • first stream is transcoded directly by myself (I use ffmpeg to transcode) and then I attach to that stream with opencv - VideoCapture can open the stream with no problem
  • second stream is transcoded by 3rd party system (this system captures video through WebRTC and then encodes to h264) - this stream cannot be opened by VideCapture no matter what I do

I can attach with pure ffmpeg to that dodgy stream and I can restream - but this is not ideal as introduces extra network traffic and latency.

I was playing with OPENCV_FFMPEG_CAPTURE_OPTIONS environmental variable (I was trying to remove audio stream, change the video codec, playing with rtmp options) - no joy

So I figured I am trying to solve this problem from wrong end. I should somehow collect underlying ffmpeg logs that should be available when calling VideoCapture. So I tried to set OPENCV_FFMPEG_CAPTURE_OPTIONS="v;debug" but I can see no ffmpeg output when calling VideoCapture.

This is very simple python3 script I was using during tests:

import cv2 as cv
dodgy_cap = cv.VideoCapture()'rtmp://my_local_ip_address/rtmp/dodgy_stream_name')
print(dodgy_cap.isOpened())  # always returns False
healthy_cap = cv.VideoCapture()'rtmp://my_local_ip_address/rtmp/healthy_stream_name')
print(healthy_cap.isOpened())  # always returns True

I collected information about both streams with ffprobe, but even though they look different I cannot see what would be the difference that prevents opencv from opening VideoCapture for dodgy stream…

This is a fragment from (very) verbose log for healthy stream:

RTMP_ClientPacket, received: notify 254 bytes                                                                                                                                                                                               
(object begin)                                                                                                                                                                                                                              
Property: <Name:           no-name., STRING:    onMetaData>                                                                                                                                                                                 
Property: <Name:           no-name., ECMA_ARRAY>                                                                                                                                                                                            
(object begin)                                                                                                                                                                                                                              
Property: <Name:           duration, NUMBER:    0.00>                                                                                                                                                                                       
Property: <Name:              width, NUMBER:    2048.00>                                                                                                                                                                                    
Property: <Name:             height, NUMBER:    1536.00>                                                                                                                                                                                    
Property: <Name:      videodatarate, NUMBER:    0.00>                                                                                                                                                                                       
Property: <Name:          framerate, NUMBER:    6.00>                                                                                                                                                                                       
Property: <Name:       videocodecid, NUMBER:    7.00>                                                                                                                                                                                       
Property: <Name:              title, STRING:    Session streamed by "preview">                                                                                                                                                              
Property: <Name:            comment, STRING:    h264Preview_01_main>                                                                                                                                                                        
Property: <Name:            encoder, STRING:    Lavf58.20.100>                                                                                                                                                                              
Property: <Name:           filesize, NUMBER:    0.00>                                                                                                                                                                                       
(object end)                                                                                                                                                                                                                                
(object end)                                                                                                                                                                                                                                
  duration              0.00
  width                 2048.00
  height                1536.00
  videodatarate         0.00
  framerate             6.00
  videocodecid          7.00
  title                 Session streamed by "preview"
  comment               h264Preview_01_main
  encoder               Lavf58.20.100
  filesize              0.00

(... raw network packets ...)

Input #0, flv, from 'rtmp://my_local_ip_address/rtmp/healthy_stream_name':
    title           : Session streamed by "preview"
    comment         : h264Preview_01_main
    encoder         : Lavf58.20.100
  Duration: 00:00:00.00, start: 159.743000, bitrate: N/A
    Stream #0:0, 41, 1/1000: Video: h264 (High), 1 reference frame, yuv420p(progressive), 2048x1536, 0/1, 6 fps, 6 tbr, 1k tbn

And this is the fragment for dodgy stream:

RTMP_ClientPacket, received: notify 205 bytes                                                                                                                                                                                               
(object begin)                                                                                                                                                                                                                              
Property: <Name:           no-name., OBJECT>                                                                                                                                                                                                
(object begin)                                                                                                                                                                                                                              
Property: <Name:    metadatacreator, STRING: SDK>                                                                                                                                                                               
Property: <Name:            encoder, STRING: Encoder>                                                                                                                                                                           
Property: <Name:       audiocodecid, NUMBER:    10.00>                                                                                                                                                                                      
Property: <Name:    audiosamplesize, NUMBER:    16.00>                                                                                                                                                                                      
Property: <Name:             stereo, BOOLEAN:   FALSE>                                                                                                                                                                                      
Property: <Name:    audiosamplerate, NUMBER:    48000.00>                                                                                                                                                                                   
Property: <Name:       videocodecid, NUMBER:    7.00>                                                                                                                                                                                       
Property: <Name:              width, NUMBER:    640.00>                                                                                                                                                                                     
Property: <Name:             height, NUMBER:    480.00>                                                                                                                                                                                     
(object end)                                                                                                                                                                                                                                
(object end)                                                                                                                                                                                                                                
RTMP_ReadPacket: fd=3                                                                                                                                                                                                                       

(... raw network packets ...)

Input #0, flv, from 'rtmp://my_local_ip_address/rtmp/dodgy_stream_name':
  Duration: N/A, start: 4511.449000, bitrate: N/A
    Stream #0:0, 41, 1/1000: Video: h264 (High), 1 reference frame, yuv420p(progressive, left), 640x480, 0/1, 15.17 fps, 15.08 tbr, 1k tbn, 30 tbc
    Stream #0:1, 124, 1/1000: Audio: aac (LC), 48000 Hz, mono, fltp

Both streams don’t require any authentication (they are not exposed to the outside world)

I have no more ideas how can I debug this problem, please help…

OpenCV using ffmpeg shouldn’t differ from ffmpeg CLI connecting to each server.

create VideoCapture instances with fixed apiPreference (CAP_FFMPEG). the difference in behavior might be explained by OpenCV picking gstreamer as the backend.

when isOpened already says false, see if you can capture that connection attempt, and a successful one, with wireshark.