Not able to read Video from Capture Card with GPU (Unsupported Format) while Reading with CPU works fine

cudawarped · August 19, 2022, 2:18pm

Inside CMake?

From the command line, that way I can
set FFMPEG_DIR=...
and then run
cmake.exe ... .. OPENCV_FFMPEG_USE_FIND_PACKAGE=ON

Although I would have thought you should be able to achieve the same thing in the GUI if you add OPENCV_FFMPEG_USE_FIND_PACKAGE using Add Entry not Environment. The only problem is you then have to configure again and it might not pick up this additional variable.

data · August 19, 2022, 2:33pm

Yes inside cmake-gui you can edit environment variables. I previously set the varibles in the Windows UI for environmental variables, which led them to appear in the Environment Editor of Cmake-gui. With cmd set you can achieve the same.

This might be the issue. I tried opening the gui with cmd, but added the OPENCV_FFMPEG_USE_FIND_PACKAGE=ON Flag. No difference.

cudawarped · August 19, 2022, 2:50pm

Works for me in the GUI, if I add OPENCV_FFMPEG_USE_FIND_PACKAGE as a boolean using Add Entry, even if I press configure twice.

I configured first without the setting OPENCV_FFMPEG_USE_FIND_PACKAGE and got the default downloaded version of FFmpeg

FFMPEG:                      YES (prebuilt binaries)
  avcodec:                   YES (58.134.100)
  avformat:                  YES (58.76.100)
  avutil:                    YES (56.70.100)
  swscale:                   YES (5.9.100)
  avresample:                YES (4.0.0)

then I set OPENCV_FFMPEG_USE_FIND_PACKAGE using Add Entry, pressed configure again and CMake picked up my FFmpeg install

FFMPEG:                      YES (find_package)
  avcodec:                   YES (59.18.100)
  avformat:                  YES (59.16.100)
  avutil:                    YES (57.17.100)
  swscale:                   YES (6.4.100)

data · August 19, 2022, 2:58pm

Sorry, yes it works, but do you also have these errors?:

CMake Warning at modules/videoio/cmake/detect_ffmpeg.cmake:6 (find_package):
Found package configuration file:
C:/Users/NvidiaPC/FFMPEG_BUILD/cmake/ffmpeg-config.cmake
but it set FFMPEG_FOUND to FALSE so package “FFMPEG” is considered to be
NOT FOUND.
Call Stack (most recent call first):
modules/videoio/cmake/init.cmake:7 (include)
modules/videoio/cmake/init.cmake:11 (add_backend)
cmake/OpenCVModule.cmake:298 (include)
cmake/OpenCVModule.cmake:361 (_add_modules_1)
cmake/OpenCVModule.cmake:385 (ocv_glob_modules)
CMakeLists.txt:931 (ocv_register_modules)

cudawarped · August 19, 2022, 3:01pm

cudawarped:

FFMPEG:                      YES (find_package)
  avcodec:                   YES (59.18.100)
  avformat:                  YES (59.16.100)
  avutil:                    YES (57.17.100)
  swscale:                   YES (6.4.100)

Did you see the above in the output, did you download the FFmpeg build?

data · August 22, 2022, 9:01am

I downloaded and extracted the FFmpeg Build like you described in the other post.
Yes, I have the (find_package) instead of the (prebuild binaries):

FFMPEG:                      YES (find_package)
      avcodec:                   YES (58.134.100)
      avformat:                  YES (58.76.100)
      avutil:                    YES (56.70.100)
      swscale:                   YES (5.9.100)
      avresample:                YES (4.0.0)

cudawarped · August 22, 2022, 9:16am

I don’t know what you are doing differently, I just tried again from scratch setting OPENCV_FFMPEG_USE_FIND_PACKAGE=ON and FFMPEG_DIR to the directory containing the cmake script inside of the cmake gui and it picked up FFmpeg straight away.

Does your FFmpeg directory sturcture look similar to the below?

bin
 - avcodec-59.dll
- ...
cmake
 - ffmpeg-config.cmake
doc
...

data · August 22, 2022, 10:39am

I must have casually extracted the folder there and forgot about the original zip folder, so the dir cmake was not with bin in one dir.
However, now I moved them in an structure like this:

\bin\
\cmake\
  ffmpeg-config.cmake
\docs\
...

This gives me a new output, FFmpeg is found, but disabled:

Found FFMPEG: C:/Users/NvidiaPC/FFMPEG_BUILD/ffmpeg-n5.1-latest-win64-gpl-shared-5.1/include (found version "5.1-9-g4e4cc6e56a-20220821") found components: avutil swresample swscale avcodec avformat avfilter avdevice 
FFMPEG is disabled. Can't find suitable libavcodec library (minimal 54.35.0, found #define LIBAVCODEC_VERSION_MINOR  37;#define LIBAVCODEC_VERSION_MICRO 100.37.100).
FFMPEG is disabled. Can't find suitable libavformat library (minimal 54.20.4, found #define LIBAVFORMAT_VERSION_MINOR  27;#define LIBAVFORMAT_VERSION_MICRO 100.27.100).
FFMPEG is disabled. Can't find suitable libswscale library (minimal 2.1.1, found #define LIBSWSCALE_VERSION_MINOR   7;#define LIBSWSCALE_VERSION_MICRO 100.7.100).
FFMPEG libraries version check failed (minimal libav release 9.20, minimal FFMPEG release 1.1.16).

    FFMPEG:                      NO (find_package)
      avcodec:                   YES (#define LIBAVCODEC_VERSION_MINOR  37 #define LIBAVCODEC_VERSION_MICRO 100.37.100)
      avformat:                  YES (#define LIBAVFORMAT_VERSION_MINOR  27 #define LIBAVFORMAT_VERSION_MICRO 100.27.100)
      avutil:                    YES (57.28.100)
      swscale:                   YES (#define LIBSWSCALE_VERSION_MINOR   7 #define LIBSWSCALE_VERSION_MICRO 100.7.100)
      avresample:                NO

cudawarped · August 22, 2022, 11:10am

Sorry it looks like they moved things around in the headers, try
https://github.com/BtbN/FFmpeg-Builds/releases/download/autobuild-2022-06-30-12-32/ffmpeg-n5.0.1-7-g7389a49fd3-win64-gpl-shared-5.0.zip

data · August 22, 2022, 11:34am

Thanks, now FFmpeg is found without erros:

Found FFMPEG: C:/Users/NvidiaPC/FFMPEG_BUILD/include (found version "5.0.1-7-g7389a49fd3-20220630") found components: avutil swresample swscale avcodec avformat avfilter avdevice

    FFMPEG:                      YES (find_package)
      avcodec:                   YES (59.18.100)
      avformat:                  YES (59.16.100)
      avutil:                    YES (57.17.100)
      swscale:                   YES (6.4.100)
      avresample:                NO

Except avresample, but I don’t think this will be a problem.

data · August 22, 2022, 2:17pm

I can build OpenCV now, but I can’t use ist anymore.
On Windows I use VS Code with a CMakeLists.txt Configuration.
I can run normal C++ Code with no problem so far, but as soon as I add an OpenCV function (or even cv::Mat), the build exits with 0, but the whole code isn’t executed anymore, there is also no error message.

cudawarped · August 22, 2022, 2:48pm

It may be that you haven’t added the path to the FFmpeg dll’s to your path, but in that case you would normally see an error informing you which dll is missing. Either way if you haven’t added the bin directory to your path you need to.

data · August 23, 2022, 10:22am

Thanks a lot, that fixed the problem with the cv::VideoCapture(). I can now open the Camera like this:

cv::VideoCapture cam("video=Microsoft® LifeCam HD-3000", cv::CAP_FFMPEG);
// cam.set(cv::CAP_PROP_FORMAT, -1); // If activated camera Window is about 1 px high and as wide as my screen

I tested also with cv::cudacodec::VideoReader() and there I get:

OpenCV(4.6.0) C:\...\private.cuda.hpp:112: error: (-213:The function/feature is not implemented) The called functionality is disabled for current build or platform in function 'throw_no_cuda'

I also looked up the post from @alex to this problem:

I did that, but it still gives me the error message.
Should I open up a new thread for this problem or could this be related?

cudawarped · August 23, 2022, 10:41am

Did you build OpenCV afterwards?

You also need to set -DWITH_NVCUVID=ON and check for the below in your CMake output

NVIDIA CUDA: YES (ver 11.7, CUFFT CUBLAS NVCUVID FAST_MATH)

data · August 23, 2022, 12:17pm

Thank you
With both of that done, I have now NCUVID shown as above.

Now I face two more errors, seems like OpenCV is actively trying to stop me from using ths functionallity :

OpenCV(4.6.0) C:\...\OpenCV\opencv_contrib-4.6.0\modules\cudacodec\src\cuvid_video_source.cpp:66: error: (-217:Gpu API call) CUDA_ERROR_FILE_NOT_FOUND [Code = 301] in function 'cv::cudacodec::detail::CuvidVideoSource::CuvidVideoSource'

[dshow @ 000001e43cc99b00] Malformed dshow input string.

My code:

const std::string fname("Microsoft® LifeCam HD-3000");
cv::Ptr<cv::cudacodec::VideoReader> d_reader = cv::cudacodec::createVideoReader(fname);

fname is set to the exact name of my camera from the Windows Device Manager. I tried it without the “®”, but this seems not to be the problem.

cudawarped · August 23, 2022, 1:49pm

Try exactly the same string you used for VideoCapture

const std::string fname("video=Microsoft® LifeCam HD-3000");

data · August 26, 2022, 1:24pm

When doing that, I get another error from cuvid_video_source.cpp:66:

(-217:Gpu API call) CUDA_ERROR_FILE_NOT_FOUND [Code = 301] in function 'cv::cudacodec::detail::CuvidVideoSource::CuvidVideoSource'

cudawarped · August 26, 2022, 2:44pm

That error implies that VideoCapture inside of VideoReader failed to open the stream, however I can’t think of a reason why because the below worked for you

Is there any additional error messages?
Have you tried stepping through the code to see what is happening?
Do you get any additional info if you set the following environmental variables?
OPENCV_LOG_LEVEL=DEBUG
OPENCV_VIDEOIO_DEBUG=1

data · August 26, 2022, 2:51pm

I tried the Debug Log Level.
There is one, looks like the problem to me, what do you think?:

[ WARN:0@0.622] global C:\...\OpenCV\opencv_contrib-4.6.0\modules\cudacodec\src\ffmpeg_video_source.cpp (95) FourccToCodec Unknown codec FOURCC: 0x32595559 (YUY2)

cudawarped · August 26, 2022, 3:19pm

That’s strange, if you reach that warning VideoCapture has opened the web cam and you should not have fallen through to try CuvidVideoSource and see the previous error.

If I were you I would step through the code and see what is throwing an exception inside

github.com

opencv/opencv_contrib/blob/d85887cd8bf2fb567c96632dea1caedd50316995/modules/cudacodec/src/video_reader.cpp#L351


      
          
          
Ptr<VideoReader> cv::cudacodec::createVideoReader(const String& filename, const std::vector<int>& sourceParams, const VideoReaderInitParams params)
          {
              CV_Assert(!filename.empty());
          
          
    Ptr<VideoSource> videoSource;
          
          
    try
              {
                  // prefer ffmpeg to cuvidGetSourceVideoFormat() which doesn't always return the corrct raw pixel format
                  Ptr<RawVideoSource> source(new FFmpegVideoSource(filename, sourceParams));
                  videoSource.reset(new RawVideoSourceWrapper(source, params.rawMode));
              }
              catch (...)
              {
                  if (sourceParams.size()) throw;
                  videoSource.reset(new CuvidVideoSource(filename));
              }
          
          
    return makePtr<VideoReaderImpl>(videoSource, params.minNumDecodeSurfaces, params.allowFrameDrop, params.udpSource);
          }