GPU Sample for VideoReader not working

C++
,
1

Hi everyone,
I’m (still) having trouble executing the OpenCV sample video_reader.cpp. Fixing the (-213: The function/feature is not implemented) results in a new problem.

Error Message:

terminate called after throwing an instance of ‘cv::Exception’
what(): OpenCV(4.5.4-dev) /home/user/openCV2/opencv_contrib-master/modules/cudacodec/src/cuvid_video_source.cpp:65: error: (-210:Unsupported format or combination of formats) Unsupported video source in function ‘CuvidVideoSource’

I’ve searched the Forum again, and it looks like I’m not the only one having trouble with the VideoReader (i.e. here and here).

Nonetheless, I’m just trying to run the sample code that is provided with OpenCV. Nothing fancy really.

Configuration:
I am running Ubuntu 20.04 LTS (Focal Fossa).

FFMPEG 
ffmpeg version n4.4 Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-17ubuntu1~20.04)
  configuration: --enable-cuda --enable-cuda-nvcc --enable-cuvid --enable-nvenc --enable-shared --disable-static --enable-nonfree --enable-libnpp --extra-cflags=-I/usr/local/cuda/include --extra-ldflags=-L/usr/local/cuda/lib64
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
OpenCV

OpenCV Config:

General configuration for OpenCV 4.5.4-dev =====================================
  Version control:               unknown

  Extra modules:
    Location (extra):            /home/user/openCV2/opencv_contrib-master/modules
    Version control (extra):     unknown

  Platform:
    Timestamp:                   2022-01-28T14:07:00Z
    Host:                        Linux 5.13.0-27-generic x86_64
    CMake:                       3.16.3
    CMake generator:             Unix Makefiles
    CMake build tool:            /usr/bin/make
    Configuration:               Release

  CPU/HW features:
    Baseline:                    SSE SSE2 SSE3
      requested:                 SSE3
    Dispatched code generation:  SSE4_1 SSE4_2 FP16 AVX AVX2 AVX512_SKX
      requested:                 SSE4_1 SSE4_2 AVX FP16 AVX2 AVX512_SKX
      SSE4_1 (17 files):         + SSSE3 SSE4_1
      SSE4_2 (2 files):          + SSSE3 SSE4_1 POPCNT SSE4_2
      FP16 (1 files):            + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 AVX
      AVX (5 files):             + SSSE3 SSE4_1 POPCNT SSE4_2 AVX
      AVX2 (32 files):           + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2
      AVX512_SKX (8 files):      + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2 AVX_512F AVX512_COMMON AVX512_SKX

  C/C++:
    Built as dynamic libs?:      YES
    C++ standard:                11
    C++ Compiler:                /usr/bin/g++-9  (ver 9.3.0)
    C++ flags (Release):         -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections  -msse -msse2 -msse3 -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG  -DNDEBUG
    C++ flags (Debug):           -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections  -msse -msse2 -msse3 -fvisibility=hidden -fvisibility-inlines-hidden -g  -O0 -DDEBUG -D_DEBUG
    C Compiler:                  /usr/bin/gcc-9
    C flags (Release):           -fsigned-char -W -Wall -Werror=return-type -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections  -msse -msse2 -msse3 -fvisibility=hidden -O3 -DNDEBUG  -DNDEBUG
    C flags (Debug):             -fsigned-char -W -Wall -Werror=return-type -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections  -msse -msse2 -msse3 -fvisibility=hidden -g  -O0 -DDEBUG -D_DEBUG
    Linker flags (Release):      -Wl,--exclude-libs,libippicv.a -Wl,--exclude-libs,libippiw.a   -Wl,--gc-sections -Wl,--as-needed  
    Linker flags (Debug):        -Wl,--exclude-libs,libippicv.a -Wl,--exclude-libs,libippiw.a   -Wl,--gc-sections -Wl,--as-needed  
    ccache:                      NO
    Precompiled headers:         NO
    Extra dependencies:          m pthread /usr/lib/x86_64-linux-gnu/libGL.so /usr/lib/x86_64-linux-gnu/libGLU.so cudart_static dl rt nppc nppial nppicc nppidei nppif nppig nppim nppist nppisu nppitc npps cublas cufft -L/usr/local/cuda-11.5/lib64 -L/usr/lib/x86_64-linux-gnu
    3rdparty dependencies:

  OpenCV modules:
    To be built:                 aruco barcode bgsegm bioinspired calib3d ccalib core cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters cudaimgproc cudalegacy cudaobjdetect cudaoptflow cudastereo cudawarping cudev cvv datasets dnn dnn_objdetect dnn_superres dpm face features2d flann freetype fuzzy gapi hfs highgui img_hash imgcodecs imgproc intensity_transform line_descriptor mcc ml objdetect optflow phase_unwrapping photo plot python3 quality rapid reg rgbd saliency shape stereo stitching structured_light superres surface_matching text tracking ts video videoio videostab viz wechat_qrcode xfeatures2d ximgproc xobjdetect xphoto
    Disabled:                    world
    Disabled by dependency:      -
    Unavailable:                 alphamat hdf java julia matlab ovis python2 sfm
    Applications:                tests perf_tests apps
    Documentation:               NO
    Non-free algorithms:         NO

  GUI:                           QT5
    QT:                          YES (ver 5.12.8 )
      QT OpenGL support:         YES (Qt5::OpenGL 5.12.8)
    GTK+:                        YES (ver 3.24.20)
      GThread :                  YES (ver 2.64.6)
      GtkGlExt:                  YES (ver 1.2.0)
    OpenGL support:              YES (/usr/lib/x86_64-linux-gnu/libGL.so /usr/lib/x86_64-linux-gnu/libGLU.so)
    VTK support:                 YES (ver 9.0.1)

  Media I/O: 
    ZLib:                        /usr/lib/x86_64-linux-gnu/libz.so (ver 1.2.11)
    JPEG:                        /usr/lib/x86_64-linux-gnu/libjpeg.so (ver 80)
    WEBP:                        build (ver encoder: 0x020f)
    PNG:                         /usr/lib/x86_64-linux-gnu/libpng.so (ver 1.6.37)
    TIFF:                        /usr/lib/x86_64-linux-gnu/libtiff.so (ver 42 / 4.1.0)
    JPEG 2000:                   build (ver 2.4.0)
    OpenEXR:                     /usr/lib/x86_64-linux-gnu/libImath.so /usr/lib/x86_64-linux-gnu/libIlmImf.so /usr/lib/x86_64-linux-gnu/libIex.so /usr/lib/x86_64-linux-gnu/libHalf.so /usr/lib/x86_64-linux-gnu/libIlmThread.so (ver 2_3)
    HDR:                         YES
    SUNRASTER:                   YES
    PXM:                         YES
    PFM:                         YES

  Video I/O:
    DC1394:                      YES (2.2.5)
    FFMPEG:                      YES
      avcodec:                   YES (58.134.100)
      avformat:                  YES (58.76.100)
      avutil:                    YES (56.70.100)
      swscale:                   YES (5.9.100)
      avresample:                YES (4.0.0)
    GStreamer:                   YES (1.16.2)
    v4l/v4l2:                    YES (linux/videodev2.h)

  Parallel framework:            pthreads

  Trace:                         YES (with Intel ITT)

  Other third-party libraries:
    Intel IPP:                   2020.0.0 Gold [2020.0.0]
           at:                   /home/user/openCV2/build/3rdparty/ippicv/ippicv_lnx/icv
    Intel IPP IW:                sources (2020.0.0)
              at:                /home/user/openCV2/build/3rdparty/ippicv/ippicv_lnx/iw
    VA:                          NO
    Lapack:                      NO
    Eigen:                       NO
    Custom HAL:                  NO
    Protobuf:                    build (3.5.1)

  NVIDIA CUDA:                   YES (ver 11.5, CUFFT CUBLAS NVCUVID)
    NVIDIA GPU arch:             35 37 50 52 60 61 70 75 80 86
    NVIDIA PTX archs:

  cuDNN:                         NO

  OpenCL:                        YES (no extra features)
    Include path:                /home/user/openCV2/opencv-master/3rdparty/include/opencl/1.2
    Link libraries:              Dynamic load

  Python 3:
    Interpreter:                 /usr/bin/python3 (ver 3.8.10)
    Libraries:                   /usr/lib/x86_64-linux-gnu/libpython3.8.so (ver 3.8.10)
    numpy:                       /home/source/.local/lib/python3.8/site-packages/numpy/core/include (ver 1.21.3)
    install path:                lib/python3.8/dist-packages/cv2/python-3.8

  Python (for build):            /usr/bin/python2.7

  Java:                          
    ant:                         NO
    JNI:                         NO
    Java wrappers:               NO
    Java tests:                  NO

  Install to:                    /usr/local
-----------------------------------------------------------------

I could really use some help with this. What am I missing?
Best,
Alex

1 Like
2

I’m not sure what your issue is, can you share your video source?

The sample is broken but it shouldn’t result in the error you described. Your error implies that FFMpeg failed to open the video source and it fell back on CuvidVideoSource.

I would try running opencv_test_cudacodec. You will need to download the extra repository and set OPENCV_TEST_DATA_PATH to the testdata path inside it.

The author of the first post didn’t link to NVCUVID and the author of the second post is trying to read from a webcam so its unlikely to be the same issue that your having?

3

Hi cudawarped,
thank you for your reply. The error occurs with any video source! I’ve tried a video capture card, I’ve also tried Webcams. Nothing seems to work! However, if I uncomment all the GPU stuff, I can see the video feed as intended.

The sample provided with openCV is broken? How’s that a thing? Shouldn’t it be removed from the samples if it doesn’t work? Is there a way to request for it to get fixed?
Thanks again

4

Can you confirm that the following

cv::VideoCapture cap(src, cv::VideoCaptureAPIs::CAP_FFMPEG);

works, but

cv::Ptr<cv::cudacodec::VideoReader> reader = cv::cudacodec::createVideoReader(src);

results in the exception you observed?

It seems so.

I guess nobody noticed until I checked it the other day. It was working then a PR was introduced which broke the functionailty and I guess samples are not part of the test coverage. Futhermore OpenCV is an open source not commercial product and the sample is part of the contrib not main repo.

I would suggest it should be fixed. It is currently under investigation, see this issue.

5

OK gotcha. Thanks =)

I just tried the following:

    //cv::VideoCapture reader(fname);
    cv::VideoCapture reader(fname, cv::VideoCaptureAPIs::CAP_FFMPEG);

There is no error message, but the resulting window does not show the feed as before (left). Instead, I see a black square (right).

out
out888×414 188 KB

Anything else I can try?

6

It looks like the FFmpeg backend is not working, are you reading from a web cam? Try a video source.

Do you get any debug output when you run

 cv::VideoCapture reader(fname, cv::VideoCaptureAPIs::CAP_FFMPEG);

what is the value of reader.isOpened()?

7

I’ve tried using a video file. That works!
However, the video capture card and webcam do not work, as described above.

The terminal does not display any messages, so there are no debug outputs I suppose.

reader.isOpened() is 0 for capture card and webcam and 1 for the video file, which is consistent with the images from above.

8

Sorry I missed that I thought by

you meant all video source (i.e. files, rtsp sources etc.) including webcams and usb.

You can’t use VideoReader unless you can use the CAP_FFMPEG backend to read from your source as VideoReader parses the video source using VideoCapture with the FFmpeg backend. Therefore without modification VideoReader will not work on hardware video sources (capture cards, usb or inbuilt webcams).

I tried to help @data with this but we didn’t get very far. In windows you can build the FFmpeg backend to use hardware devices so I had hoped you could do the same with v42l. Anyway you can try the suggestions in

which is essentially build OpenCV against libavdevice and include avdevice_register_all() in void CvCapture_FFMPEG::init() and set OPENCV_FFMPEG_CAPTURE_OPTIONS=input_format;v42l. I have no idea if this will work and don’t have time to test as I’m working on windows at the moment.