Not able to read Video from Capture Card with GPU (Unsupported Format) while Reading with CPU works fine

[data]

@cudawarped

OPENCV_FFMPEG_CAPTURE_OPTIONS=input_format;v4l2

Also it is not clear to me how you set these variable in Windows. Is it via UI or PowerShell? Although in both cases the format of this variable seems strange to me.

[crackwitz]

How to change environment variables in C language

and yes the format is strange because the one who implemented that in OpenCV decided that it needs to be strange. there is probably no reason for it.

[cudawarped]

As mentioned in

VideoCapture from camera with CAP_PROP_FORMAT -1 (raw) and DSHOW/FFMPEG/windows

I performed a quick test and it appears that you can retrieve the raw encoded MJPEG data from directshow using CAP_FFMPEG if you slightly modify the source and link OpenCV against libavdevice. You need to add avdevice_register_all(); to void CvCapture_FFMPEG::init() and #include <libavdevice/avdevice.h>. Unfortunatley this means you need to build OpenCV against FFMPEG libs and not use the precompiled dll - opencv_videoio_ffmpegxxx_64.dll. Once the modification is in place you simply ne…

1. You need to specify the device name given in device manager, i.e. “HD Webcam”, not the index as you would when using the Direct Show backend (CAP_DSHOW).

@cudawarped

VideoCapture cap(“video=<NAME_FROM_DEVICE_MANAGER>”);

2. You need to build against FFmpeg not use the pre-built binaries, these won’t include any changes you made to cap_ffmpeg_impl.hpp.

@data

    FFMPEG:                      YES (prebuilt binaries)
      avcodec:                   YES (58.134.100)
      avformat:                  YES (58.76.100)
      avutil:                    YES (56.70.100)
      swscale:                   YES (5.9.100)

[data]

@cudawarped

You need to build against FFmpeg not use the pre-built binaries

Hi,
could you please elaborate on your workflow for installing ffmpeg on Windows? How can I prevent cmake from using the prebuild binaries?

Thank you

[cudawarped]

Sure thing, see this post

Error: 'Your platform doesn't suppport hardware accelerated AV1 decoding.' when i use 'cv2.VideoCapture'

To build against FFmpeg and get AV1 decoding you need to: Build or dowload and extract a build of FFMpeg to <FFMPEG_BUILD>. I would suggest simply downloading from here and depending on your license requirements you will want to download either ffmpeg-n5.0.1-7-g7389a49fd3-win64-gpl-shared-5.0.zip or ffmpeg-n5.0.1-7-g7389a49fd3-win64-lgpl-shared-5.0.zip Create a cmake directory <FFMPEG_BUILD>/cmake. Save the cmake script (ffmpeg-config.cmake) mentioned in the below post to that directory…

[data]

Thanks for the quick reply

@cudawarped

Set CMAKE_PREFIX_PATH or FFMPEG_DIR environmental variables to the directory containing the ffmpeg-config.cmake script.

I extracted the Build and created an directory /cmake containing the mentioned .cmake file.
Afterwards I set the variables via Windows UI, but Cmake (I use cmake-gui on windows) doesn’t catch the FFMpeg Build.

FFMPEG: Downloading opencv_videoio_ffmpeg.dll from https://raw.githubusercontent.com/opencv/opencv_3rdparty/65ec04d4573dcdfa4531f0b9e67f35d8ffff873e/ffmpeg/opencv_videoio_ffmpeg.dll
FFMPEG: Downloading opencv_videoio_ffmpeg_64.dll from https://raw.githubusercontent.com/opencv/opencv_3rdparty/65ec04d4573dcdfa4531f0b9e67f35d8ffff873e/ffmpeg/opencv_videoio_ffmpeg_64.dll
FFMPEG: Downloading ffmpeg_version.cmake from https://raw.githubusercontent.com/opencv/opencv_3rdparty/65ec04d4573dcdfa4531f0b9e67f35d8ffff873e/ffmpeg/ffmpeg_version.cmake

[cudawarped]

I don’t use the GUI.

How did you set the CMake flag OPENCV_FFMPEG_USE_FIND_PACKAGE=ON?

How did you set CMAKE_PREFIX_PATH or FFMPEG_DIR environmental variables?

[data]

@cudawarped

How did you set CMAKE_PREFIX_PATH or FFMPEG_DIR environmental variables?

I used the Environment Editor for the varibles

@cudawarped

How did you set the CMake flag OPENCV_FFMPEG_USE_FIND_PACKAGE=ON?

Same here, althought i wasn’t sure if this would actually work.
How do you use cmake?

[cudawarped]

@data

I used the Environment Editor for the varibles

Inside CMake?

@data

How do you use cmake?

From the command line, that way I can
set FFMPEG_DIR=...
and then run
cmake.exe ... .. OPENCV_FFMPEG_USE_FIND_PACKAGE=ON

Although I would have thought you should be able to achieve the same thing in the GUI if you add OPENCV_FFMPEG_USE_FIND_PACKAGE using Add Entry not Environment. The only problem is you then have to configure again and it might not pick up this additional variable.

[data]

@cudawarped

Inside CMake?

Yes inside cmake-gui you can edit environment variables. I previously set the varibles in the Windows UI for environmental variables, which led them to appear in the Environment Editor of Cmake-gui. With cmd set you can achieve the same.

@cudawarped

The only problem is you then have to configure again and it might not pick up this additional variable.

This might be the issue. I tried opening the gui with cmd, but added the OPENCV_FFMPEG_USE_FIND_PACKAGE=ON Flag. No difference.

[cudawarped]

Works for me in the GUI, if I add OPENCV_FFMPEG_USE_FIND_PACKAGE as a boolean using Add Entry, even if I press configure twice.

I configured first without the setting OPENCV_FFMPEG_USE_FIND_PACKAGE and got the default downloaded version of FFmpeg

FFMPEG:                      YES (prebuilt binaries)
  avcodec:                   YES (58.134.100)
  avformat:                  YES (58.76.100)
  avutil:                    YES (56.70.100)
  swscale:                   YES (5.9.100)
  avresample:                YES (4.0.0)

then I set OPENCV_FFMPEG_USE_FIND_PACKAGE using Add Entry, pressed configure again and CMake picked up my FFmpeg install

FFMPEG:                      YES (find_package)
  avcodec:                   YES (59.18.100)
  avformat:                  YES (59.16.100)
  avutil:                    YES (57.17.100)
  swscale:                   YES (6.4.100)

[data]

Sorry, yes it works, but do you also have these errors?:

CMake Warning at modules/videoio/cmake/detect_ffmpeg.cmake:6 (find_package):
Found package configuration file:

C:/Users/NvidiaPC/FFMPEG_BUILD/cmake/ffmpeg-config.cmake

but it set FFMPEG_FOUND to FALSE so package “FFMPEG” is considered to be
NOT FOUND.
Call Stack (most recent call first):
modules/videoio/cmake/init.cmake:7 (include)
modules/videoio/cmake/init.cmake:11 (add_backend)
cmake/OpenCVModule.cmake:298 (include)
cmake/OpenCVModule.cmake:361 (_add_modules_1)
cmake/OpenCVModule.cmake:385 (ocv_glob_modules)
CMakeLists.txt:931 (ocv_register_modules)

[cudawarped]

@cudawarped

FFMPEG:                      YES (find_package)
  avcodec:                   YES (59.18.100)
  avformat:                  YES (59.16.100)
  avutil:                    YES (57.17.100)
  swscale:                   YES (6.4.100)

Did you see the above in the output, did you download the FFmpeg build?

[data]

@cudawarped

Did you see the above in the output, did you download the FFmpeg build?

I downloaded and extracted the FFmpeg Build like you described in the other post.
Yes, I have the (find_package) instead of the (prebuild binaries):

FFMPEG:                      YES (find_package)
      avcodec:                   YES (58.134.100)
      avformat:                  YES (58.76.100)
      avutil:                    YES (56.70.100)
      swscale:                   YES (5.9.100)
      avresample:                YES (4.0.0)

[cudawarped]

I don’t know what you are doing differently, I just tried again from scratch setting OPENCV_FFMPEG_USE_FIND_PACKAGE=ON and FFMPEG_DIR to the directory containing the cmake script inside of the cmake gui and it picked up FFmpeg straight away.

Does your FFmpeg directory sturcture look similar to the below?

bin
 - avcodec-59.dll
- ...
cmake
 - ffmpeg-config.cmake
doc
...

[data]

I must have casually extracted the folder there and forgot about the original zip folder, so the dir cmake was not with bin in one dir.
However, now I moved them in an structure like this:

\bin\
\cmake\
  ffmpeg-config.cmake
\docs\
...

This gives me a new output, FFmpeg is found, but disabled:

Found FFMPEG: C:/Users/NvidiaPC/FFMPEG_BUILD/ffmpeg-n5.1-latest-win64-gpl-shared-5.1/include (found version "5.1-9-g4e4cc6e56a-20220821") found components: avutil swresample swscale avcodec avformat avfilter avdevice 
FFMPEG is disabled. Can't find suitable libavcodec library (minimal 54.35.0, found #define LIBAVCODEC_VERSION_MINOR  37;#define LIBAVCODEC_VERSION_MICRO 100.37.100).
FFMPEG is disabled. Can't find suitable libavformat library (minimal 54.20.4, found #define LIBAVFORMAT_VERSION_MINOR  27;#define LIBAVFORMAT_VERSION_MICRO 100.27.100).
FFMPEG is disabled. Can't find suitable libswscale library (minimal 2.1.1, found #define LIBSWSCALE_VERSION_MINOR   7;#define LIBSWSCALE_VERSION_MICRO 100.7.100).
FFMPEG libraries version check failed (minimal libav release 9.20, minimal FFMPEG release 1.1.16).

    FFMPEG:                      NO (find_package)
      avcodec:                   YES (#define LIBAVCODEC_VERSION_MINOR  37 #define LIBAVCODEC_VERSION_MICRO 100.37.100)
      avformat:                  YES (#define LIBAVFORMAT_VERSION_MINOR  27 #define LIBAVFORMAT_VERSION_MICRO 100.27.100)
      avutil:                    YES (57.28.100)
      swscale:                   YES (#define LIBSWSCALE_VERSION_MINOR   7 #define LIBSWSCALE_VERSION_MICRO 100.7.100)
      avresample:                NO

[cudawarped]

Sorry it looks like they moved things around in the headers, try
https://github.com/BtbN/FFmpeg-Builds/releases/download/autobuild-2022-06-30-12-32/ffmpeg-n5.0.1-7-g7389a49fd3-win64-gpl-shared-5.0.zip

[data]

Thanks, now FFmpeg is found without erros:

Found FFMPEG: C:/Users/NvidiaPC/FFMPEG_BUILD/include (found version "5.0.1-7-g7389a49fd3-20220630") found components: avutil swresample swscale avcodec avformat avfilter avdevice

    FFMPEG:                      YES (find_package)
      avcodec:                   YES (59.18.100)
      avformat:                  YES (59.16.100)
      avutil:                    YES (57.17.100)
      swscale:                   YES (6.4.100)
      avresample:                NO

Except avresample, but I don’t think this will be a problem.

[data]

I can build OpenCV now, but I can’t use ist anymore.
On Windows I use VS Code with a CMakeLists.txt Configuration.
I can run normal C++ Code with no problem so far, but as soon as I add an OpenCV function (or even cv::Mat), the build exits with 0, but the whole code isn’t executed anymore, there is also no error message.

[cudawarped]

It may be that you haven’t added the path to the FFmpeg dll’s to your path, but in that case you would normally see an error informing you which dll is missing. Either way if you haven’t added the bin directory to your path you need to.