Error while running EDSR model: CUDNN_STATUS_NOT_INITIALIZED in function 'UniqueHandle'

Python
,
1

Hey,
I am trying to use EDSR pre-trained model with cv2 dnn_superres. Installed all NVIDIA, CUDA and cuDNN related drivers. The version I am using for this is the following:

AWS EC2: p3.2xlarge
Ubuntu: 22.04.2 LTS
GPU: nvidia tesla v100
NVIDIA-SMI: 520.61.05
Driver Version: 520.61.05
CUDA Version: 11.8
cuDNN: 8.8.1
OpenCV: 4.5.5

The example code I am using is

def pre_trained_model_enhancement(img):
    model_path = "EDSR_x4.pb"
    sr = cv2.dnn_superres.DnnSuperResImpl_create()
    sr.readModel(model_path)
    sr.setPreferableBackend(cv2.dnn.DNN_BACKEND_CUDA)
    sr.setPreferableTarget(cv2.dnn.DNN_TARGET_CUDA)
    sr.setModel("edsr", 4)

    image_height, image_width, channels = img.shape

    if image_width > 600 or image_height > 600:
        if image_width > image_height:
            fac = 600 / image_width
        else:
            fac = 600 / image_height

        img = cv2.resize(img, (int(image_width * fac), int(image_height * fac)))

    return sr.upsample(img)

When I pass the cv2 imread image to the function, it works all fine until the last line where it tries to upsample the image.
It throws this particular error:

cv2.error: OpenCV(4.5.5) /home/ubuntu/opencv-4.5.5/modules/dnn/src/cuda4dnn/csl/cudnn/cudnn.hpp:74: error: (-217:Gpu API call) CUDNN_STATUS_NOT_INITIALIZED in function 'UniqueHandle'

Is this a bug or some kind of issue with the drivers or the OpenCV version?
I have not seen any topic related to this particular error making it difficult to understand. If anyone has experienced this or has any idea please suggest further steps to resolve this.

2

hmm, the superres code hasn’t changed much, but there were tons of changes in the dnn module, – please try with latest 4.7.0 ?

4

Still has the same error with 4.7.0 as well.

cv2.error: OpenCV(4.7.0) /home/ubuntu/opencv/modules/dnn/src/cuda4dnn/csl/cudnn/cudnn.hpp:74: error: (-217:Gpu API call) CUDNN_STATUS_NOT_INITIALIZED in function 'UniqueHandle'
1 Like
5

can you post result of print(cv2.getBuildInformation())?

6 
General configuration for OpenCV 4.7.0 =====================================
  Version control:               unknown

  Extra modules:
    Location (extra):            /home/ubuntu/opencv_contrib/modules
    Version control (extra):     unknown

  Platform:
    Timestamp:                   2023-03-21T12:09:19Z
    Host:                        Linux 5.15.0-1028-aws x86_64
    CMake:                       3.22.1
    CMake generator:             Unix Makefiles
    CMake build tool:            /usr/bin/gmake
    Configuration:               RELEASE

  CPU/HW features:
    Baseline:                    SSE SSE2 SSE3
      requested:                 SSE3
    Dispatched code generation:  SSE4_1 SSE4_2 FP16 AVX AVX2 AVX512_SKX
      requested:                 SSE4_1 SSE4_2 AVX FP16 AVX2 AVX512_SKX
      SSE4_1 (18 files):         + SSSE3 SSE4_1
      SSE4_2 (2 files):          + SSSE3 SSE4_1 POPCNT SSE4_2
      FP16 (1 files):            + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 AVX
      AVX (5 files):             + SSSE3 SSE4_1 POPCNT SSE4_2 AVX
      AVX2 (34 files):           + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2
      AVX512_SKX (8 files):      + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2 AVX_512F AVX512_COMMON AVX512_SKX

  C/C++:
    Built as dynamic libs?:      YES
    C++ standard:                11
    C++ Compiler:                /usr/bin/c++  (ver 11.3.0)
    C++ flags (Release):         -fsigned-char -ffast-math -W -Wall -Wreturn-type -Wnon-virtual-dtor -Waddress -Wsequence-point -Wformat -Wformat-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections  -msse -msse2 -msse3 -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG  -DNDEBUG
    C++ flags (Debug):           -fsigned-char -ffast-math -W -Wall -Wreturn-type -Wnon-virtual-dtor -Waddress -Wsequence-point -Wformat -Wformat-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections  -msse -msse2 -msse3 -fvisibility=hidden -fvisibility-inlines-hidden -g  -O0 -DDEBUG -D_DEBUG
    C Compiler:                  /usr/bin/cc
    C flags (Release):           -fsigned-char -ffast-math -W -Wall -Wreturn-type -Waddress -Wsequence-point -Wformat -Wformat-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections  -msse -msse2 -msse3 -fvisibility=hidden -O3 -DNDEBUG  -DNDEBUG
    C flags (Debug):             -fsigned-char -ffast-math -W -Wall -Wreturn-type -Waddress -Wsequence-point -Wformat -Wformat-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections  -msse -msse2 -msse3 -fvisibility=hidden -g  -O0 -DDEBUG -D_DEBUG
    Linker flags (Release):      -Wl,--exclude-libs,libippicv.a -Wl,--exclude-libs,libippiw.a   -Wl,--gc-sections -Wl,--as-needed -Wl,--no-undefined  
    Linker flags (Debug):        -Wl,--exclude-libs,libippicv.a -Wl,--exclude-libs,libippiw.a   -Wl,--gc-sections -Wl,--as-needed -Wl,--no-undefined  
    ccache:                      NO
    Precompiled headers:         NO
    Extra dependencies:          m pthread cudart_static dl rt nppc nppial nppicc nppidei nppif nppig nppim nppist nppisu nppitc npps cublas cudnn cufft -L/usr/local/cuda-11.8/lib64 -L/usr/lib/x86_64-linux-gnu
    3rdparty dependencies:

  OpenCV modules:
    To be built:                 aruco barcode bgsegm bioinspired calib3d ccalib core cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters cudaimgproc cudalegacy cudaobjdetect cudaoptflow cudastereo cudawarping cudev datasets dnn dnn_objdetect dnn_superres dpm face features2d flann freetype fuzzy gapi hfs highgui img_hash imgcodecs imgproc intensity_transform line_descriptor mcc ml objdetect optflow phase_unwrapping photo plot python3 quality rapid reg rgbd saliency shape stereo stitching structured_light superres surface_matching text tracking ts video videoio videostab wechat_qrcode xfeatures2d ximgproc xobjdetect xphoto
    Disabled:                    world
    Disabled by dependency:      -
    Unavailable:                 alphamat cvv hdf java julia matlab ovis python2 sfm viz
    Applications:                tests perf_tests examples apps
    Documentation:               NO
    Non-free algorithms:         YES

  GUI:                           GTK3
    GTK+:                        YES (ver 3.24.33)
      GThread :                  YES (ver 2.72.4)
      GtkGlExt:                  NO
    VTK support:                 NO

  Media I/O: 
    ZLib:                        /usr/lib/x86_64-linux-gnu/libz.so (ver 1.2.11)
    JPEG:                        /usr/lib/x86_64-linux-gnu/libjpeg.so (ver 80)
    WEBP:                        build (ver encoder: 0x020f)
    PNG:                         /usr/lib/x86_64-linux-gnu/libpng.so (ver 1.6.37)
    TIFF:                        /usr/lib/x86_64-linux-gnu/libtiff.so (ver 42 / 4.3.0)
    JPEG 2000:                   build (ver 2.4.0)
    OpenEXR:                     build (ver 2.3.0)
    HDR:                         YES
    SUNRASTER:                   YES
    PXM:                         YES
    PFM:                         YES

  Video I/O:
    DC1394:                      NO
    FFMPEG:                      YES
      avcodec:                   YES (58.134.100)
      avformat:                  YES (58.76.100)
      avutil:                    YES (56.70.100)
      swscale:                   YES (5.9.100)
      avresample:                NO
    GStreamer:                   NO
    v4l/v4l2:                    YES (linux/videodev2.h)

  Parallel framework:            pthreads

  Trace:                         YES (with Intel ITT)

  Other third-party libraries:
    Intel IPP:                   2020.0.0 Gold [2020.0.0]
           at:                   /home/ubuntu/opencv/build/3rdparty/ippicv/ippicv_lnx/icv
    Intel IPP IW:                sources (2020.0.0)
              at:                /home/ubuntu/opencv/build/3rdparty/ippicv/ippicv_lnx/iw
    VA:                          NO
    Lapack:                      NO
    Eigen:                       NO
    Custom HAL:                  NO
    Protobuf:                    build (3.19.1)

  NVIDIA CUDA:                   YES (ver 11.8, CUFFT CUBLAS FAST_MATH)
    NVIDIA GPU arch:             70
    NVIDIA PTX archs:

  cuDNN:                         YES (ver 8.8.1)

  OpenCL:                        YES (no extra features)
    Include path:                /home/ubuntu/opencv/3rdparty/include/opencl/1.2
    Link libraries:              Dynamic load

  Python 3:
    Interpreter:                 /home/ubuntu/.virtualenvs/cv/bin/python3 (ver 3.10.6)
    Libraries:                   /usr/lib/x86_64-linux-gnu/libpython3.10.so (ver 3.10.6)
    numpy:                       /home/ubuntu/.virtualenvs/cv/lib/python3.10/site-packages/numpy/core/include (ver 1.24.2)
    install path:                lib/python3.10/site-packages/cv2/python-3.10

  Python (for build):            /home/ubuntu/.virtualenvs/cv/bin/python3

  Java:                          
    ant:                         NO
    JNI:                         NO
    Java wrappers:               NO
    Java tests:                  NO

  Install to:                    /usr/local
-----------------------------------------------------------------

Anything unusual in this?

7

Can you run opencv_test-dnn or opencv_test_dnn_superres?

8

Can you please elaborate on that on how to run these tests?

9

You have to build opencv with test using cmake :
-DBUILD_TESTS:BOOL=ON
(windows may be there is a space somewhere with lunux)

10

Okay with this it returned

cmake -D CMAKE_BUILD_TYPE=RELEASE \
        -D CMAKE_INSTALL_PREFIX=/usr/local \
        -D INSTALL_PYTHON_EXAMPLES=ON \
        -D INSTALL_C_EXAMPLES=OFF \
        -D OPENCV_ENABLE_NONFREE=ON \
        -D WITH_CUDA=ON \
        -D WITH_CUDNN=ON \
        -D OPENCV_DNN_CUDA=ON \
        -D ENABLE_FAST_MATH=1 \
        -D CUDA_FAST_MATH=1 \
        -D CUDA_ARCH_BIN=7.0 \
        -D WITH_CUBLAS=1 \
        -D OPENCV_EXTRA_MODULES_PATH=~/opencv_contrib/modules \
        -D HAVE_opencv_python3=ON \
        -D PYTHON_EXECUTABLE=~/.virtualenvs/cv/bin/python \
        -D BUILD_EXAMPLES=ON \
        -D BUILD_PERF_TESTS:BOOL=ON ..
-- ocv_init_download: OpenCV source tree is not fetched as git repository. 3rdparty resources will be downloaded from github.com by default.
-- Detected processor: x86_64
-- Found PythonInterp: /home/ubuntu/.virtualenvs/cv/bin/python (found suitable version "3.10.6", minimum required is "2.7") 
-- Looking for ccache - not found
-- Found ZLIB: /usr/lib/x86_64-linux-gnu/libz.so (found suitable version "1.2.11", minimum required is "1.2.3") 
Cleaning INTERNAL cached variable: WEBP_LIBRARY
Cleaning INTERNAL cached variable: WEBP_INCLUDE_DIR
-- Could NOT find OpenJPEG (minimal suitable version: 2.0, recommended version >= 2.3.1). OpenJPEG will be built from sources
-- OpenJPEG: VERSION = 2.4.0, BUILD = opencv-4.7.0-openjp2-2.4.0
-- OpenJPEG libraries will be built from sources: libopenjp2 (version "2.4.0")
-- Found ZLIB: /usr/lib/x86_64-linux-gnu/libz.so (found version "1.2.11") 
-- libva: missing va.h header (VA_INCLUDE_DIR)
-- found Intel IPP (ICV version): 2020.0.0 [2020.0.0 Gold]
-- at: /home/ubuntu/opencv/build/3rdparty/ippicv/ippicv_lnx/icv
-- found Intel IPP Integration Wrappers sources: 2020.0.0
-- at: /home/ubuntu/opencv/build/3rdparty/ippicv/ippicv_lnx/iw
-- CUDA detected: 11.8
-- CUDA: Using CUDA_ARCH_BIN=7.0
-- CUDA NVCC target flags: -gencode;arch=compute_70,code=sm_70;-D_FORCE_INLINES
-- Could not find OpenBLAS include. Turning OpenBLAS_FOUND off
-- Could not find OpenBLAS lib. Turning OpenBLAS_FOUND off
-- Could NOT find Atlas (missing: Atlas_CLAPACK_INCLUDE_DIR) 
-- Could NOT find JNI (missing: JAVA_INCLUDE_PATH JAVA_INCLUDE_PATH2 JAVA_AWT_INCLUDE_PATH) 
-- Could NOT find Pylint (missing: PYLINT_EXECUTABLE) 
-- Could NOT find Flake8 (missing: FLAKE8_EXECUTABLE) 
-- VTK is not found. Please set -DVTK_DIR in CMake to VTK build directory, or to VTK install subdirectory with VTKConfig.cmake file
-- Checking for module 'gtk+-2.0'
--   No package 'gtk+-2.0' found
-- Checking for module 'libavresample'
--   No package 'libavresample' found
-- Checking for module 'gstreamer-base-1.0'
--   No package 'gstreamer-base-1.0' found
-- Checking for module 'gstreamer-app-1.0'
--   No package 'gstreamer-app-1.0' found
-- Checking for module 'gstreamer-riff-1.0'
--   No package 'gstreamer-riff-1.0' found
-- Checking for module 'gstreamer-pbutils-1.0'
--   No package 'gstreamer-pbutils-1.0' found
-- Checking for module 'gstreamer-video-1.0'
--   No package 'gstreamer-video-1.0' found
-- Checking for module 'gstreamer-audio-1.0'
--   No package 'gstreamer-audio-1.0' found
-- Checking for module 'libdc1394-2'
--   No package 'libdc1394-2' found
-- Module opencv_alphamat disabled because the following dependencies are not found: Eigen
-- freetype2:   YES (ver 24.1.18)
-- harfbuzz:    YES (ver 2.7.4)
-- Could NOT find HDF5 (missing: HDF5_LIBRARIES HDF5_INCLUDE_DIRS) (found version "")
-- Julia not found. Not compiling Julia Bindings. 
-- Module opencv_ovis disabled because OGRE3D was not found
-- No preference for use of exported gflags CMake configuration set, and no hints for include/library directories provided. Defaulting to preferring an installed/exported gflags CMake configuration if available.
-- Failed to find installed gflags CMake configuration, searching for gflags build directories exported with CMake.
-- Failed to find gflags - Failed to find an installed/exported CMake configuration for gflags, will perform search for installed gflags components.
-- Failed to find gflags - Could not find gflags include directory, set GFLAGS_INCLUDE_DIR to directory containing gflags/gflags.h
-- Failed to find glog - Could not find glog include directory, set GLOG_INCLUDE_DIR to directory containing glog/logging.h
-- Module opencv_sfm disabled because the following dependencies are not found: Eigen Glog/Gflags
-- Checking for module 'tesseract'
--   No package 'tesseract' found
-- Tesseract:   NO
-- Allocator metrics storage type: 'long long'
-- Excluding from source files list: modules/imgproc/src/imgwarp.lasx.cpp
-- Excluding from source files list: modules/imgproc/src/resize.lasx.cpp
-- Registering hook 'INIT_MODULE_SOURCES_opencv_dnn': /home/ubuntu/opencv/modules/dnn/cmake/hooks/INIT_MODULE_SOURCES_opencv_dnn.cmake
-- Excluding from source files list: <BUILD>/modules/dnn/layers/layers_common.rvv.cpp
-- Excluding from source files list: <BUILD>/modules/dnn/layers/layers_common.lasx.cpp
-- Excluding from source files list: <BUILD>/modules/dnn/int8layers/layers_common.lasx.cpp
-- imgcodecs: OpenEXR codec is disabled in runtime. Details: https://github.com/opencv/opencv/issues/21326
-- highgui: using builtin backend: GTK3
-- rgbd: Eigen support is disabled. Eigen is Required for Posegraph optimization
-- Building with NVIDIA Optical Flow API 2.0
-- Found 'misc' Python modules from /home/ubuntu/opencv/modules/python/package/extra_modules
-- Found 'mat_wrapper;utils' Python modules from /home/ubuntu/opencv/modules/core/misc/python/package
-- Found 'gapi' Python modules from /home/ubuntu/opencv/modules/gapi/misc/python/package
-- Found 'misc' Python modules from /home/ubuntu/opencv/modules/python/package/extra_modules
-- Found 'mat_wrapper;utils' Python modules from /home/ubuntu/opencv/modules/core/misc/python/package
-- Found 'gapi' Python modules from /home/ubuntu/opencv/modules/gapi/misc/python/package
-- SYCL/OpenCL samples are skipped: SYCL SDK is required
--    - check configuration of SYCL_DIR/SYCL_ROOT/CMAKE_MODULE_PATH
--    - ensure that right compiler is selected from SYCL SDK (e.g, clang++): CMAKE_CXX_COMPILER=/usr/bin/c++
-- 
-- General configuration for OpenCV 4.7.0 =====================================
--   Version control:               unknown
-- 
--   Extra modules:
--     Location (extra):            /home/ubuntu/opencv_contrib/modules
--     Version control (extra):     unknown
-- 
--   Platform:
--     Timestamp:                   2023-03-21T12:09:19Z
--     Host:                        Linux 5.15.0-1028-aws x86_64
--     CMake:                       3.22.1
--     CMake generator:             Unix Makefiles
--     CMake build tool:            /usr/bin/gmake
--     Configuration:               RELEASE
-- 
--   CPU/HW features:
--     Baseline:                    SSE SSE2 SSE3
--       requested:                 SSE3
--     Dispatched code generation:  SSE4_1 SSE4_2 FP16 AVX AVX2 AVX512_SKX
--       requested:                 SSE4_1 SSE4_2 AVX FP16 AVX2 AVX512_SKX
--       SSE4_1 (18 files):         + SSSE3 SSE4_1
--       SSE4_2 (2 files):          + SSSE3 SSE4_1 POPCNT SSE4_2
--       FP16 (1 files):            + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 AVX
--       AVX (5 files):             + SSSE3 SSE4_1 POPCNT SSE4_2 AVX
--       AVX2 (34 files):           + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2
--       AVX512_SKX (8 files):      + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2 AVX_512F AVX512_COMMON AVX512_SKX
-- 
--   C/C++:
--     Built as dynamic libs?:      YES
--     C++ standard:                11
--     C++ Compiler:                /usr/bin/c++  (ver 11.3.0)
--     C++ flags (Release):         -fsigned-char -ffast-math -W -Wall -Wreturn-type -Wnon-virtual-dtor -Waddress -Wsequence-point -Wformat -Wformat-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections  -msse -msse2 -msse3 -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG  -DNDEBUG
--     C++ flags (Debug):           -fsigned-char -ffast-math -W -Wall -Wreturn-type -Wnon-virtual-dtor -Waddress -Wsequence-point -Wformat -Wformat-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections  -msse -msse2 -msse3 -fvisibility=hidden -fvisibility-inlines-hidden -g  -O0 -DDEBUG -D_DEBUG
--     C Compiler:                  /usr/bin/cc
--     C flags (Release):           -fsigned-char -ffast-math -W -Wall -Wreturn-type -Waddress -Wsequence-point -Wformat -Wformat-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections  -msse -msse2 -msse3 -fvisibility=hidden -O3 -DNDEBUG  -DNDEBUG
--     C flags (Debug):             -fsigned-char -ffast-math -W -Wall -Wreturn-type -Waddress -Wsequence-point -Wformat -Wformat-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections  -msse -msse2 -msse3 -fvisibility=hidden -g  -O0 -DDEBUG -D_DEBUG
--     Linker flags (Release):      -Wl,--exclude-libs,libippicv.a -Wl,--exclude-libs,libippiw.a   -Wl,--gc-sections -Wl,--as-needed -Wl,--no-undefined  
--     Linker flags (Debug):        -Wl,--exclude-libs,libippicv.a -Wl,--exclude-libs,libippiw.a   -Wl,--gc-sections -Wl,--as-needed -Wl,--no-undefined  
--     ccache:                      NO
--     Precompiled headers:         NO
--     Extra dependencies:          m pthread cudart_static dl rt nppc nppial nppicc nppidei nppif nppig nppim nppist nppisu nppitc npps cublas cudnn cufft -L/usr/local/cuda-11.8/lib64 -L/usr/lib/x86_64-linux-gnu
--     3rdparty dependencies:
-- 
--   OpenCV modules:
--     To be built:                 aruco barcode bgsegm bioinspired calib3d ccalib core cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters cudaimgproc cudalegacy cudaobjdetect cudaoptflow cudastereo cudawarping cudev datasets dnn dnn_objdetect dnn_superres dpm face features2d flann freetype fuzzy gapi hfs highgui img_hash imgcodecs imgproc intensity_transform line_descriptor mcc ml objdetect optflow phase_unwrapping photo plot python3 quality rapid reg rgbd saliency shape stereo stitching structured_light superres surface_matching text tracking ts video videoio videostab wechat_qrcode xfeatures2d ximgproc xobjdetect xphoto
--     Disabled:                    world
--     Disabled by dependency:      -
--     Unavailable:                 alphamat cvv hdf java julia matlab ovis python2 sfm viz
--     Applications:                tests perf_tests examples apps
--     Documentation:               NO
--     Non-free algorithms:         YES
-- 
--   GUI:                           GTK3
--     GTK+:                        YES (ver 3.24.33)
--       GThread :                  YES (ver 2.72.4)
--       GtkGlExt:                  NO
--     VTK support:                 NO
-- 
--   Media I/O: 
--     ZLib:                        /usr/lib/x86_64-linux-gnu/libz.so (ver 1.2.11)
--     JPEG:                        /usr/lib/x86_64-linux-gnu/libjpeg.so (ver 80)
--     WEBP:                        build (ver encoder: 0x020f)
--     PNG:                         /usr/lib/x86_64-linux-gnu/libpng.so (ver 1.6.37)
--     TIFF:                        /usr/lib/x86_64-linux-gnu/libtiff.so (ver 42 / 4.3.0)
--     JPEG 2000:                   build (ver 2.4.0)
--     OpenEXR:                     build (ver 2.3.0)
--     HDR:                         YES
--     SUNRASTER:                   YES
--     PXM:                         YES
--     PFM:                         YES
-- 
--   Video I/O:
--     DC1394:                      NO
--     FFMPEG:                      YES
--       avcodec:                   YES (58.134.100)
--       avformat:                  YES (58.76.100)
--       avutil:                    YES (56.70.100)
--       swscale:                   YES (5.9.100)
--       avresample:                NO
--     GStreamer:                   NO
--     v4l/v4l2:                    YES (linux/videodev2.h)
-- 
--   Parallel framework:            pthreads
-- 
--   Trace:                         YES (with Intel ITT)
-- 
--   Other third-party libraries:
--     Intel IPP:                   2020.0.0 Gold [2020.0.0]
--            at:                   /home/ubuntu/opencv/build/3rdparty/ippicv/ippicv_lnx/icv
--     Intel IPP IW:                sources (2020.0.0)
--               at:                /home/ubuntu/opencv/build/3rdparty/ippicv/ippicv_lnx/iw
--     VA:                          NO
--     Lapack:                      NO
--     Eigen:                       NO
--     Custom HAL:                  NO
--     Protobuf:                    build (3.19.1)
-- 
--   NVIDIA CUDA:                   YES (ver 11.8, CUFFT CUBLAS FAST_MATH)
--     NVIDIA GPU arch:             70
--     NVIDIA PTX archs:
-- 
--   cuDNN:                         YES (ver 8.8.1)
-- 
--   OpenCL:                        YES (no extra features)
--     Include path:                /home/ubuntu/opencv/3rdparty/include/opencl/1.2
--     Link libraries:              Dynamic load
-- 
--   Python 3:
--     Interpreter:                 /home/ubuntu/.virtualenvs/cv/bin/python3 (ver 3.10.6)
--     Libraries:                   /usr/lib/x86_64-linux-gnu/libpython3.10.so (ver 3.10.6)
--     numpy:                       /home/ubuntu/.virtualenvs/cv/lib/python3.10/site-packages/numpy/core/include (ver 1.24.2)
--     install path:                lib/python3.10/site-packages/cv2/python-3.10
-- 
--   Python (for build):            /home/ubuntu/.virtualenvs/cv/bin/python3
-- 
--   Java:                          
--     ant:                         NO
--     JNI:                         NO
--     Java wrappers:               NO
--     Java tests:                  NO
-- 
--   Install to:                    /usr/local
-- -----------------------------------------------------------------
-- 
-- Configuring done
-- Generating done
-- Build files have been written to: /home/ubuntu/opencv/build
11

What is result of program opencv_test-dnn or opencv_test_dnn_superres?
After building opencv you must find those file in ni folder

12

Not sure if it would make any difference, but are you sure you downloaded cuDNN 8.8.1 for CUDA 11 and not CUDA 12? Also did you install a deb or just extract the archive to /usr/local/cuda?

14

for opencv_test_dnn it says

4984 FAILED TESTS
  YOU HAVE 17 DISABLED TESTS

and for opencv_test_dnn_superres

OpenCL Platforms: 
    NVIDIA CUDA
        dGPU: Tesla V100-SXM2-16GB (OpenCL 3.0 CUDA)
Current OpenCL device: 
    Type = dGPU
    Name = Tesla V100-SXM2-16GB
    Version = OpenCL 3.0 CUDA
    Driver version = 520.61.05
    Address bits = 64
    Compute units = 80
    Max work group size = 1024
    Local memory size = 48 KB
    Max memory allocation size = 3 GB 968 MB 176 KB
    Double support = Yes
    Half support = No
    Host unified memory = No
    Device extensions:
        cl_khr_global_int32_base_atomics
        cl_khr_global_int32_extended_atomics
        cl_khr_local_int32_base_atomics
        cl_khr_local_int32_extended_atomics
        cl_khr_fp64
        cl_khr_3d_image_writes
        cl_khr_byte_addressable_store
        cl_khr_icd
        cl_nv_compiler_options
        cl_nv_device_attribute_query
        cl_nv_pragma_unroll
        cl_nv_copy_opts
        cl_nv_create_buffer
        cl_khr_int64_base_atomics
        cl_khr_int64_extended_atomics
        cl_nv_kernel_attribute
        cl_khr_device_uuid
        cl_khr_pci_bus_info
        cl_khr_external_semaphore
        cl_khr_external_memory
        cl_khr_external_semaphore_opaque_fd
        cl_khr_external_memory_opaque_fd
    Has AMD Blas = No
    Has AMD Fft = No
    Preferred vector width char = 1
    Preferred vector width short = 1
    Preferred vector width int = 1
    Preferred vector width long = 1
    Preferred vector width float = 1
    Preferred vector width double = 1
    Preferred vector width half = 0
TEST: Skip tests with tags: 'mem_6gb', 'verylong'
[==========] Running 4 tests from 2 test cases.
[----------] Global test environment set-up.
[----------] 3 tests from CV_DnnSuperResSingleOutputTest
[ RUN      ] CV_DnnSuperResSingleOutputTest.accuracy_espcn_2
unknown file: Failure
C++ exception with description "OpenCV(4.7.0) /home/ubuntu/opencv/modules/ts/src/ts.cpp:1064: error: (-2:Unspecified error) OpenCV tests: Can't find required data file: dnn_superres/butterfly.png in function 'findData'
" thrown in the test body.
[  FAILED  ] CV_DnnSuperResSingleOutputTest.accuracy_espcn_2 (0 ms)
[ RUN      ] CV_DnnSuperResSingleOutputTest.accuracy_fsrcnn_2
unknown file: Failure
C++ exception with description "OpenCV(4.7.0) /home/ubuntu/opencv/modules/ts/src/ts.cpp:1064: error: (-2:Unspecified error) OpenCV tests: Can't find required data file: dnn_superres/butterfly.png in function 'findData'
" thrown in the test body.
[  FAILED  ] CV_DnnSuperResSingleOutputTest.accuracy_fsrcnn_2 (0 ms)
[ RUN      ] CV_DnnSuperResSingleOutputTest.accuracy_fsrcnn_3
unknown file: Failure
C++ exception with description "OpenCV(4.7.0) /home/ubuntu/opencv/modules/ts/src/ts.cpp:1064: error: (-2:Unspecified error) OpenCV tests: Can't find required data file: dnn_superres/butterfly.png in function 'findData'
" thrown in the test body.
[  FAILED  ] CV_DnnSuperResSingleOutputTest.accuracy_fsrcnn_3 (0 ms)
[----------] 3 tests from CV_DnnSuperResSingleOutputTest (1 ms total)

[----------] 1 test from CV_DnnSuperResMultiOutputTest
[ RUN      ] CV_DnnSuperResMultiOutputTest.accuracy
unknown file: Failure
C++ exception with description "OpenCV(4.7.0) /home/ubuntu/opencv/modules/ts/src/ts.cpp:1064: error: (-2:Unspecified error) OpenCV tests: Can't find required data file: dnn_superres/butterfly.png in function 'findData'
" thrown in the test body.
[  FAILED  ] CV_DnnSuperResMultiOutputTest.accuracy (0 ms)
[----------] 1 test from CV_DnnSuperResMultiOutputTest (0 ms total)

[----------] Global test environment tear-down
[==========] 4 tests from 2 test cases ran. (1 ms total)
[  PASSED  ] 0 tests.
[  FAILED  ] 4 tests, listed below:
[  FAILED  ] CV_DnnSuperResSingleOutputTest.accuracy_espcn_2
[  FAILED  ] CV_DnnSuperResSingleOutputTest.accuracy_fsrcnn_2
[  FAILED  ] CV_DnnSuperResSingleOutputTest.accuracy_fsrcnn_3
[  FAILED  ] CV_DnnSuperResMultiOutputTest.accuracy

 4 FAILED TESTS
15

yes downloaded with cuDNN =8.8.1.3-1+cuda11.8.
And installed via deb.

16

For opencv_test_dnn_superres it should be :

TEST: Skip tests with tags: 'mem_6gb', 'verylong'
[==========] Running 4 tests from 2 test cases.
[----------] Global test environment set-up.
[----------] 3 tests from CV_DnnSuperResSingleOutputTest
[ RUN      ] CV_DnnSuperResSingleOutputTest.accuracy_espcn_2
[       OK ] CV_DnnSuperResSingleOutputTest.accuracy_espcn_2 (91 ms)
[ RUN      ] CV_DnnSuperResSingleOutputTest.accuracy_fsrcnn_2
[       OK ] CV_DnnSuperResSingleOutputTest.accuracy_fsrcnn_2 (46 ms)
[ RUN      ] CV_DnnSuperResSingleOutputTest.accuracy_fsrcnn_3
[       OK ] CV_DnnSuperResSingleOutputTest.accuracy_fsrcnn_3 (49 ms)
[----------] 3 tests from CV_DnnSuperResSingleOutputTest (186 ms total)

[----------] 1 test from CV_DnnSuperResMultiOutputTest
[ RUN      ] CV_DnnSuperResMultiOutputTest.accuracy
[       OK ] CV_DnnSuperResMultiOutputTest.accuracy (1531 ms)
[----------] 1 test from CV_DnnSuperResMultiOutputTest (1531 ms total)

[----------] Global test environment tear-down
[==========] 4 tests from 2 test cases ran. (1719 ms total)
[  PASSED  ] 4 tests.
17

may be you should update cuda as suggested by @cudawarped
try other cuda test

18

You need opencv_extra
and set env variable OPENCV_TEST_DATA_PATH to opencv_extra/testdata path (

19

I don’t think opencv_test_dnn_superres uses the CUDA backend. You can force it to do so by adding

dnn_sr->setPreferableBackend(dnn::DNN_BACKEND_CUDA);
dnn_sr->setPreferableTarget(dnn::DNN_TARGET_CUDA);

here. For me this passed the tests without any problems.

I then tried the model you are using downlaode from

and placed in the PATH_TO_EXTRA/testdata/cv/dnn_suprres directory with the following small changes

and again had not issues. I would therefore suggest you either perform these tests (which you might not want to because you will have to recompile) or run opencv_test_dnn as described below to see if any models which use the cuDNN work on your setup.

To test opencv_test_dnn you first you need to download a model. I suggest GoogLeNet as its relatively small.

When you have the extra repo and you have added the path

export OPENCV_TEST_DATA_PATH=<PATH_TO_EXTRA/testdata>

you will need to go to the PATH_TO_EXTRA/testdata/dnn folder and download the googlenet model using

python download_models.py GoogleNet

Then you can execute the test as

./opencv_test_dnn --gtest_filter=DNNTestNetwork.GoogLeNet/0

and hopefully the output should be something similar to

CTEST_FULL_OUTPUT
OpenCV version: 4.7.0-dev
OpenCV VCS version: 4.7.0-98-g0052d46b8e
Build type: Release
Compiler: /usr/bin/c++  (ver 9.4.0)
Parallel framework: pthreads (nthreads=20)
CPU features: SSE SSE2 SSE3 *SSE4.1 *SSE4.2 *FP16 *AVX *AVX2 *AVX512-SKX?
Intel(R) IPP version: ippIP AVX2 (l9) 2020.0.0 Gold (-) Oct 19 2019
Intel(R) IPP features code: 0x8000
OpenCL is disabled
TEST: Skip tests with tags: 'mem_6gb', 'verylong', 'dnn_skip_opencv_backend', 'dnn_skip_cpu', 'dnn_skip_ocl', 'dnn_skip_ocl_fp16', 'dnn_skip_cuda', 'dnn_skip_cuda_fp32', 'dnn_skip_cuda_fp16', 'dnn_skip_onnx_conformance', 'dnn_skip_parser'
Note: Google Test filter = DNNTestNetwork.GoogLeNet/0
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from DNNTestNetwork
[ RUN      ] DNNTestNetwork.GoogLeNet/0, where GetParam() = CUDA/CUDA
[       OK ] DNNTestNetwork.GoogLeNet/0 (8183 ms)
[----------] 1 test from DNNTestNetwork (8183 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test case ran. (8183 ms total)
[  PASSED  ] 1 test.
20

Hey
So I tried with updated versions but it got chaotic with the different versions provided and somehow the CUDA got updated to 12.1 and with that, the unsupported version error started to come. I am trying all this on an EC2 machine so it is difficult to manage the issues.
Can you tell me the version of all the required drivers you used for this so that I can start fresh with all this?
Thanks for helping with this very exhausting problem.

21

I have it working on CUDA 12.0 with cuDNN 8.8.1 using the latest commit from the master branch of both the main and contrib repositories, in both windows 11 and ubuntu 20.04 (wsl).

I’m not sure if it makes a difference, but I didn’t use the deb I simply copied the cuDNN lib and include folders to /usr/local/cuda/lib64 and /usr/local/cuda/include respectivley.