FFmpeg not working in VideoCapture

[data]

Continuing the discussion from Not able to read Video from Capture Card with GPU (Unsupported Format) while Reading with CPU works fine:

Hello All,
like I shared in the post above, I have encountered problems using FFmpeg in OpenCV:

@data

Opening the camera like this works:

cv::VideoCapture camL(0, cv::CAP_V4L2), camR(1, cv::CAP_V4L2);

While opening the camera like this failed (camL.isOpen() and camR.isOpen() return false):

cv::VideoCapture camL(0, cv::CAP_FFMPEG), camR(1, cv::CAP_FFMPEG);

The problem spans also to cv::cuda::VideoReader I couldn’t solve this until now. But maybe someone has encountered it already and managed to solve it?

Hardware/Software Configuration

System:
Ubuntu 22.04 LTS
OpenCV(4.6.0-dev)
Cuda 11.7

Hardware:
Graphics: Nvidia RTX 3070 TI
Video Capture Card: Magewell Pro Capture Dual HDMI 4K Plus LT

OpenCV

...
--   Video I/O:
--     DC1394:                      NO
--     FFMPEG:                      YES
--       avcodec:                   YES (58.134.100)
--       avformat:                  YES (58.76.100)
--       avutil:                    YES (56.70.100)
--       swscale:                   YES (5.9.100)
--       avresample:                NO
--     GStreamer:                   NO
--     v4l/v4l2:                    YES (linux/videodev2.h)
...

Full Build Log

[cudawarped]

The FFmpeg backend cannot read from video for linux devices (usb camera’s). In the previous post I suggested a possible workaround which works with direct show devices but may not work with video for linux.

As I mensioned before if you can’t patch the FFmpeg backend so that it can read from a v42l device then your only other realistic option is to wrap another parser which can in a class that inherits from cv::cudacodec::RawVideoSource.

I am not sure what your question here is?

[data]

When I use FFmpeg with

ffmpeg -f v4l2 -framerate 25 -video_size 960x540 -i /dev/video0

I can record a video from with the cam. Problems occur, when I want to use FFmpeg Backend within OpenCV. Sorry if I missed to tell that.

[cudawarped]

It has nothing to do with the FFmpeg libs themselves, it is to do with which parts of FFmpeg have been implemented inside of OpenCV.

I would suspect because OpenCV has other backends to read from hardware devices there is no requirement for the FFmpeg backend to do this aswell.

I would change the title of your post because the FFmpeg backend works with VideoCapture it just doesn’t do what you want which makes sense as there are other backends to do it. You have a specific requirement which is to access usb camera’s with cudacodec::VideoReader. It would be more correct to say cudacodec::VideoReader does not work when reading from a usb camera. I am trying to help you with that but it may not be easy to solve.

[data]

Thank you for your effort. I appreciate that very much. I will change the title accordingly.