# Trouble using NVDIA hardware decoder when streaming from Camera **URL:** \ **Category:** C++\ **Tags:** cuda, videoio\ **Created:** [March 12, 2022, 8:47pm UTC](https://forum.opencv.org/t/trouble-using-nvdia-hardware-decoder-when-streaming-from-camera/7908 "2022-03-12T20:47:57Z")\ **Posts on this page:** 15\ **Page:** 1
**Author:** ![christophersmith](https://avatars.discourse-cdn.com/v4/letter/c/c89c15/32.png) [@christophersmith](https://forum.opencv.org/u/christophersmith)\ **Post date:** [March 12, 2022, 8:47pm UTC](https://forum.opencv.org/t/trouble-using-nvdia-hardware-decoder-when-streaming-from-camera/7908/1 "2022-03-12T20:47:57Z")
We are trying to offload encoding work from the CPU in our application. We built OpenCV 4.5.1 to enable CUDA, we can get cuda functions to run on there, but video decode does not use it. We found a reference that said you need to set the environment variable to: OPENCV\_FFMPEG\_CAPTURE\_OPTIONS=“hwaccel;cuvid|video\_codec;h264\_cuvid|vsync;0” This did work when reading from a file, but not when reading from an RTSP camera. Another source had OPENCV\_FFMPEG\_CAPTURE\_OPTIONS=“video\_codec;h264\_cuvid|rtsp\_transport;tcp” as the way to read from RTSP but this did not work Any one know what the setup is to read from RTSP camera into NVIDIA decoder into Opencv? ---
**Author:** ![crackwitz](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/crackwitz/32/14_2.png) [@crackwitz](https://forum.opencv.org/u/crackwitz)\ **Post date:** [March 13, 2022, 1:00am UTC](https://forum.opencv.org/t/trouble-using-nvdia-hardware-decoder-when-streaming-from-camera/7908/2 "2022-03-13T01:00:45Z")
can you get that (rtsp + hw decode) working on a standalone ffmpeg/ffplay? ---
**Author:** ![christophersmith](https://avatars.discourse-cdn.com/v4/letter/c/c89c15/32.png) [@christophersmith](https://forum.opencv.org/u/christophersmith)\ **Post date:** [March 13, 2022, 1:16am UTC](https://forum.opencv.org/t/trouble-using-nvdia-hardware-decoder-when-streaming-from-camera/7908/3 "2022-03-13T01:16:52Z")
The RTSP input works without the decode (no options) that’s how we are doing it. Shouldn’t this be known technology at this point, though? ---
**Author:** ![christophersmith](https://avatars.discourse-cdn.com/v4/letter/c/c89c15/32.png) [@christophersmith](https://forum.opencv.org/u/christophersmith)\ **Post date:** [March 13, 2022, 2:31am UTC](https://forum.opencv.org/t/trouble-using-nvdia-hardware-decoder-when-streaming-from-camera/7908/4 "2022-03-13T02:31:29Z")
I downloaded a recent ffmpeg , the most recent and did get it to work and use both CUDA and the NVDEC blocks. It wanted to use a different name for the hwaccel block tag and its name, now cuda. It did not like the rtsp stuff but worked without it. So we can use at least a modern version of FFMPEG to read it. I’m working with 4.5.1 so its version of the libraries might be different and it did not work with the same data string in the OPENCV\_FFMPEG\_(etc) environment variable as _did_ work for reading from a file. ```auto ffmpeg -hwaccel_output_format cuda -vcodec h264_cuvid -i rtsp://admin:Margin719@192.168.0.101/Src/MediaInput/stream_1 cap001.mp4 ffmpeg version 2022-03-07-git-e645a1ddb9-full_build-www.gyan.dev Copyright (c) 2000-2022 the FFmpeg developers built with gcc 11.2.0 (Rev7, Built by MSYS2 project) configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libdav1d --enable-libdavs2 --enable-libuavs3d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libshaderc --enable-vulkan --enable-libplacebo --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint libavutil 57. 22.100 / 57. 22.100 libavcodec 59. 22.100 / 59. 22.100 libavformat 59. 17.103 / 59. 17.103 libavdevice 59. 5.100 / 59. 5.100 libavfilter 8. 27.100 / 8. 27.100 libswscale 6. 5.100 / 6. 5.100 libswresample 4. 4.100 / 4. 4.100 libpostproc 56. 4.100 / 56. 4.100 Input #0, rtsp, from 'rtsp://admin:Margin719@192.168.0.101/Src/MediaInput/stream_1': Metadata: title : Media Presentation Duration: N/A, start: 0.100000, bitrate: N/A Stream #0:0: Video: h264 (High), yuvj420p(pc, bt709, progressive), 2560x1440 [SAR 1:1 DAR 16:9], 10 fps, 10 tbr, 90k tbn Stream mapping: Stream #0:0 -> #0:0 (h264 (h264_cuvid) -> h264 (libx264)) Press [q] to stop, [?] for help [libx264 @ 000002908c3c0040] using SAR=1/1 [libx264 @ 000002908c3c0040] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 AVX512 [libx264 @ 000002908c3c0040] profile High, level 5.0, 4:2:0, 8-bit [libx264 @ 000002908c3c0040] 264 - core 164 r3094 bfc87b7 - H.264/MPEG-4 AVC codec - Copyleft 2003-2022 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=24 lookahead_threads=4 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=10 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00 Output #0, mp4, to 'cap001.mp4': Metadata: title : Media Presentation encoder : Lavf59.17.103 Stream #0:0: Video: h264 (avc1 / 0x31637661), nv12(pc, bt709, progressive), 2560x1440 [SAR 1:1 DAR 16:9], q=2-31, 10 fps, 10240 tbn Metadata: encoder : Lavc59.22.100 libx264 Side data: cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A frame= 889 fps= 10 q=25.0 size= 30976kB time=00:01:21.80 bitrate=3102.1kbits/s speed=0.943x ``` I’m also a bit confused by the rtsp\_transport … I didn’t make it up, there are documentation for it and other parameters passed in thru the command line. ---
**Author:** ![crackwitz](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/crackwitz/32/14_2.png) [@crackwitz](https://forum.opencv.org/u/crackwitz)\ **Post date:** [March 13, 2022, 11:10am UTC](https://forum.opencv.org/t/trouble-using-nvdia-hardware-decoder-when-streaming-from-camera/7908/5 "2022-03-13T11:10:44Z")
4.5.1 might be too old. try again with the latest version. some related links: - [OE-XX: Hardware-accelerated Video I/O · Issue #19676 · opencv/opencv · GitHub](https://github.com/opencv/opencv/issues/19676) - [videoio(doc): add 'videoio\_hwaccel' group by alalek · Pull Request #19688 · opencv/opencv · GitHub](https://github.com/opencv/opencv/pull/19688) ---
**Author:** ![cudawarped](https://avatars.discourse-cdn.com/v4/letter/c/9dc877/32.png) [@cudawarped](https://forum.opencv.org/u/cudawarped)\ **Post date:** [March 14, 2022, 10:09am UTC](https://forum.opencv.org/t/trouble-using-nvdia-hardware-decoder-when-streaming-from-camera/7908/6 "2022-03-14T10:09:50Z")
> [@christophersmith](#): > > I’m also a bit confused by the rtsp\_transport … I didn’t make it up, there are documentation for it and other parameters passed in thru the command line. On 4.5.1 the default is to stream via tcp so `rtsp_transport;tcp` shouldn’t have any effect at all. > [@christophersmith](#): > > It did not like the rtsp stuff but worked without it. So we can use at least a modern version of FFMPEG to read it. Did it work from rtsp at all? > [@christophersmith](#): > > We built OpenCV 4.5.1 to enable CUDA, we can get cuda functions to run on there, but video decode does not use it. If you built with CUDA then you could try using `cudacodec::VideoCapture();` > [@christophersmith](#): > > I downloaded a recent ffmpeg , the most recent and did get it to work and use both CUDA and the NVDEC blocks. Unless anything has changed the most recent version of FFMpeg won’t work with OpenCV so even if it works stand alone it won’t help you. I will check again but I don’t remember having any issues using older versions of FFMpeg with `video_codec;h264_cuvid` for streaming from RTSP as long as they were compiled with NVDEC support. Are you sure the version of FFMpeg you linked OpenCV against was compiled with NVDEC support? ---
**Author:** ![christophersmith](https://avatars.discourse-cdn.com/v4/letter/c/c89c15/32.png) [@christophersmith](https://forum.opencv.org/u/christophersmith)\ **Post date:** [March 14, 2022, 5:57pm UTC](https://forum.opencv.org/t/trouble-using-nvdia-hardware-decoder-when-streaming-from-camera/7908/7 "2022-03-14T17:57:09Z")
It runs NVDEC properly using Opencv/FFMPEG from a file, only RTSP inputs do not work. Therefore I think the FFMPEG was built with support for that. ---
**Author:** ![christophersmith](https://avatars.discourse-cdn.com/v4/letter/c/c89c15/32.png) [@christophersmith](https://forum.opencv.org/u/christophersmith)\ **Post date:** [March 14, 2022, 6:11pm UTC](https://forum.opencv.org/t/trouble-using-nvdia-hardware-decoder-when-streaming-from-camera/7908/8 "2022-03-14T18:11:29Z")
Yes, on the run above with the parameters I was able to run with the most recent ffmpeg from rtsp, just by using the command line parameters in the post. It didn’t even recognize rtsp\_transport at all which I found surprising as I have seen it listed in other sources as a parameter to allow selection of udp. Our CUDA device is 7.5 which other sources have said is not supported with cudacodec. I think we got an error when we tried that. ---
**Author:** ![cudawarped](https://avatars.discourse-cdn.com/v4/letter/c/9dc877/32.png) [@cudawarped](https://forum.opencv.org/u/cudawarped)\ **Post date:** [March 14, 2022, 7:13pm UTC](https://forum.opencv.org/t/trouble-using-nvdia-hardware-decoder-when-streaming-from-camera/7908/9 "2022-03-14T19:13:38Z")
> [@christophersmith](#): > > Yes, on the run above with the parameters I was able to run with the most recent ffmpeg from rtsp, just by using the command line parameters in the post. It didn’t even recognize rtsp\_transport at all which I found surprising as I have seen it listed in other sources as a parameter to allow selection of udp. That is odd considering [rtsp\_transport](https://www.ffmpeg.org/ffmpeg-protocols.html#rtsp) it is one of the listed options in the official documentation. > [@christophersmith](#): > > Our CUDA device is 7.5 which other sources have said is not supported with cudacodec. I think we got an error when we tried that. Can you post a link to the sources as it should be supported on all valid compute capabilities as long as you download and build against the [Video Codec SDK](https://developer.nvidia.com/nvidia-video-codec-sdk)? ---
**Author:** ![christophersmith](https://avatars.discourse-cdn.com/v4/letter/c/c89c15/32.png) [@christophersmith](https://forum.opencv.org/u/christophersmith)\ **Post date:** [March 14, 2022, 11:31pm UTC](https://forum.opencv.org/t/trouble-using-nvdia-hardware-decoder-when-streaming-from-camera/7908/10 "2022-03-14T23:31:08Z")
which sources - we built against 4.5.1 official release source. Again, since it worked for reading from file. Or do you mean the command line FFMPEG? That was downloaded from the most recent build. I’m now trying 4.5.5 as per the other gentleman’s posts. ---
**Author:** ![christophersmith](https://avatars.discourse-cdn.com/v4/letter/c/c89c15/32.png) [@christophersmith](https://forum.opencv.org/u/christophersmith)\ **Post date:** [March 15, 2022, 3:52am UTC](https://forum.opencv.org/t/trouble-using-nvdia-hardware-decoder-when-streaming-from-camera/7908/11 "2022-03-15T03:52:14Z")
I just tried it with the latest OpenCV. Now I get that h264\_cuvid decoder is not recognized, which worked fine with the old version, and the most up to date FFMPEG. > Blockquote > url: rtsp://admin:Margin719@192.168.0.101/Src/MediaInput/stream\_1 > url: rtsp://admin:Margin719@192.168.0.102/Src/MediaInput/stream\_1 > url: rtsp://admin:Margin719@192.168.0.103/Src/MediaInput/stream\_1 > url: rtsp://admin:Margin719@192.168.0.104/Src/MediaInput/stream\_1 > [h264 @ 000001ddb063f300] cabac\_init\_idc 5 overflow > [h264 @ 000001ddb063f300] decode\_slice\_header error > [h264 @ 000001ddb063f300] no frame! > [h264 @ 000001ddb063f300] deblocking filter parameters 0 8 out of range > [h264 @ 000001ddb063f300] decode\_slice\_header error > [h264 @ 000001ddb063f300] no frame! > [h264 @ 000001ddb063f300] deblocking\_filter\_idc 9 out of range > [h264 @ 000001ddb063f300] decode\_slice\_header error > [h264 @ 000001ddb063f300] no frame! > [h264 @ 000001ddb063f300] deblocking filter parameters 0 8 out of range > [h264 @ 000001ddb063f300] decode\_slice\_header error > [h264 @ 000001ddb063f300] no frame! > [h264 @ 000001ddb063f300] deblocking\_filter\_idc 9 out of range > [h264 @ 000001ddb063f300] decode\_slice\_header error > [h264 @ 000001ddb063f300] no frame! > [h264 @ 000001ddb063f300] QP 4294967279 out of range > [h264 @ 000001ddb063f300] decode\_slice\_header error > [h264 @ 000001ddb063f300] no frame! > [h264 @ 000001ddb063f300] Missing reference picture, default is 0 > [h264 @ 000001ddb063f300] decode\_slice\_header error > [ERROR:0@5.915] global /build/opencv/modules/videoio/src/cap\_ffmpeg\_impl.hpp (1117) open Could not find decoder ‘h264\_cuvid’ > [ERROR:0@5.916] global /build/opencv/modules/videoio/src/cap\_ffmpeg\_impl.hpp (1140) open VIDEOIO/FFMPEG: Failed to initialize VideoCapture > Could not open reference rtsp://admin:Margin719@192.168.0.101/Src/MediaInput/stream\_1 > FS: w: 0 h: 0 > FPS0: 0 ex: > FPS: 0 > Wait for Go: > \< \< \< Game over! \> \> \> > [h264 @ 000001de6634aa40] deblocking\_filter\_idc 9 out of range > [h264 @ 000001de6634aa40] decode\_slice\_header error > [h264 @ 000001de6634aa40] no frame! > [h264 @ 000001de6634aa40] QP 4294967279 out of range > [h264 @ 000001de6634aa40] decode\_slice\_header error > [h264 @ 000001de6634aa40] no frame! > [h264 @ 000001de6634aa40] Missing reference picture, default is 0 > [h264 @ 000001de6634aa40] decode\_slice\_header error > [ERROR:0@9.991] global /build/opencv/modules/videoio/src/cap\_ffmpeg\_impl.hpp (1117) open Could not find decoder ‘h264\_cuvid’ > [ERROR:0@9.992] global /build/opencv/modules/videoio/src/cap\_ffmpeg\_impl.hpp (1140) open VIDEOIO/FFMPEG: Failed to initialize VideoCapture > Could not open reference rtsp://admin:Margin719@192.168.0.102/Src/MediaInput/stream\_1 > FS: w: 0 h: 0 > FPS0: 0 ex: > FPS: 0 > Wait for Go: > \< \< \< Game over! \> \> \> > [ERROR:0@13.011] global /build/opencv/modules/videoio/src/cap\_ffmpeg\_impl.hpp (1117) open Could not find decoder ‘h264\_cuvid’ > [ERROR:0@13.011] global /build/opencv/modules/videoio/src/cap\_ffmpeg\_impl.hpp (1140) open VIDEOIO/FFMPEG: Failed to initialize VideoCapture > Could not open reference rtsp://admin:Margin719@192.168.0.103/Src/MediaInput/stream\_1 > FS: w: 0 h: 0 > FPS0: 0 ex: > FPS: 0 > Wait for Go: > \< \< \< Game over! \> \> \> > [ERROR:0@16.033] global /build/opencv/modules/videoio/src/cap\_ffmpeg\_impl.hpp (1117) open Could not find decoder ‘h264\_cuvid’ > [ERROR:0@16.033] global /build/opencv/modules/videoio/src/cap\_ffmpeg\_impl.hpp (1140) open VIDEOIO/FFMPEG: Failed to initialize VideoCapture > Could not open reference rtsp://admin:Margin719@192.168.0.104/Src/MediaInput/stream\_1 > FS: w: 0 h: 0 > FPS0: 0 ex: > FPS: 0 > Wait for Go: > \< \< \< Game over! \> \> \> I was using the old value that worked for a file just as a quick test: OPENCV\_FFMPEG\_CAPTURE\_OPTIONS=“hwaccel;cuvid|video\_codec;h264\_cuvid|vsync;0” But it is complaining that it doesn’t know the video codec. Now, it knows this codec in the latest FFMPEG, and in the old 4.5.1 code. What is the proper value of the options to the NVIDIA NVDECODE from RSTP? I did get some activity in the Video Decode block of the GPU before the streams failed. ---
**Author:** ![christophersmith](https://avatars.discourse-cdn.com/v4/letter/c/c89c15/32.png) [@christophersmith](https://forum.opencv.org/u/christophersmith)\ **Post date:** [March 15, 2022, 3:53am UTC](https://forum.opencv.org/t/trouble-using-nvdia-hardware-decoder-when-streaming-from-camera/7908/12 "2022-03-15T03:53:13Z")
Oh also the reference to the file doesn’t appear to refer to any file I can find in the installed distribution, or the built source. ---
**Author:** ![cudawarped](https://avatars.discourse-cdn.com/v4/letter/c/9dc877/32.png) [@cudawarped](https://forum.opencv.org/u/cudawarped)\ **Post date:** [March 15, 2022, 12:14pm UTC](https://forum.opencv.org/t/trouble-using-nvdia-hardware-decoder-when-streaming-from-camera/7908/13 "2022-03-15T12:14:33Z")
> [@christophersmith](#): > > which sources - we built against 4.5.1 official release source I was refering to the source of information, sorry for the confusion. The point is `cv::cudacodec::VideoReader()` should work on all versions of CUDA any information to the contrary may be for specific circumstances which I would be interested to investigate if you have a link to the information? Anyway this thread is getting confusing, I have just performed some tests on my build of 4.5.5 to confirm everything is working as it should, and it appears to be. I will try to compress the relevent information below. Essentially for Nvidia hardware decoding you have two (possibly three) easy approaches: 1. Build OpenCV against FFMpeg built against the [Nvidia Video Codec SDK](https://developer.nvidia.com/nvidia-video-codec-sdk). 2. Use `cv::cudacodec::VideoReader()`. 3. Use GStreamer - I haven’t personally tested this with OpenCV and RTSP but I have used the Nvidia plugins when not streaming from RTSP outside of OpenCV without any issues. In my opinion the pros and cons for each are. **Build OpenCV against FFMpeg built against Nvidia Video Codec SDK.** Pros: - You can use the `VideoCapture()` api as is, frames are returned as Mat’s etc., so essentially the hardware decoding is hidden from you and should work without issues with existing code which uses `VideoCapture(). Cons: - FFMpeg needs to be built against the Nvidia Video Codec SDK - if you get similar errors to the following from `VideoCapture::open()` `[ERROR:0@1.112] global /build/opencv/modules/videoio/src/cap_ffmpeg_impl.hpp (1117) open Could not find decoder 'h264_cuvid'` then this is not the case. - If you use the old api pre OpenCV 4.5.2 you need to set environmental variables and make sure you know which codec you require in advance, e.g. setting `OPENCV_FFMPEG_CAPTURE_OPTIONS=video_codec;h264_cuvid` and trying to decode a h265 file will result in the following error `[h264_cuvid @ 00000269ae74a000] Codec type or id mismatches` - Decoding can be as slow as using the CPU (although this may have changed with the latest api), you simply get the benefit of reducing CPU utilization. Procedure: This vaires depending on the version of OpenCV you are using. For all versions of OpenCV adding the environmenal variables: - OPENCV\_FFMPEG\_CAPTURE\_OPTIONS=video\_codec;h264\_cuvid - OPENCV\_FFMPEG\_CAPTURE\_OPTIONS=video\_codec;hevc\_cuvid or even - OPENCV\_FFMPEG\_CAPTURE\_OPTIONS=video\_codec;h264\_qsv if you have an Intel chip. should work as long as the version of FFMpeg you are building against supports these codecs. From [4.5.2 onwards](https://github.com/opencv/opencv/wiki/Video-IO-hardware-acceleration) this was made simpler but it is still fairly new and I am not sure if everything works. You no longer need to add an environmental variable and opening an RTSP source as ```auto VideoCapture cap("rtsp://....", cv::CAP_FFMPEG, { CAP_PROP_HW_ACCELERATION, cv::VIDEO_ACCELERATION_ANY }); ``` should result in hardware decoding and debug output similar to below ```auto [INFO:0@1.315] global repos/opencv/modules/videoio/src/cap_ffmpeg_hw.hpp (276) hw_check_device FFMPEG: Using d3d11va video acceleration on device: NVIDIA GeForce GTX 1060 [INFO:0@1.316] global repos/opencv/modules/videoio/src/cap_ffmpeg_hw.hpp (566) hw_create_device FFMPEG: Created video acceleration context (av_hwdevice_ctx_create) for d3d11va on device 'default' ``` In addition this should work out of the box on windows as it is baked into `opencv_videoio_ffmpegxxx_64.dll` where as `video_codec;h264_cuvid`isn’t and would requir hacking the cmake files to link directly to FFMpeg libs built with hardware support. **cv::cudacodec::VideoReader().** Pros: - The output is a` cv::cuda::GPUMat`. This could also be a con if you require a `cv::Mat`. - You can get the raw encoded output at the same time as the decoded frame if required. This is useful if you want to record without the overhead of encoding by writing the encoded data to a file. - You don’t need to build or find a version of FFMpeg which is built against the Nvidia Video Codec SDK. Cons: - OpenCV needs to be built against both the CUDA and Nvidia Video Codec SDK, if you are not using CUDA for anything else this is probably overkill. - You don’t need to mess around with environmental variables when using versions of OpenCV pre 4.5.2. Procedure: - Build OpenCV again CUDA and the Nvidia VideoCodec SDK. **To summarize:** - If you have the latest version of OpenCV, access to FFMpeg built against the Nvidia Video Codec SDK and don’t use CUDA for processing the resulting frames I would use approach 1). - If you can’t use a version of FFMpeg built against the Nvidia Video Codec SDK or use CUDA to process the resulting frames use approach 2). Personally I use 2) because most of the work I do is on the GPU so I would add the following caveat. I am not sure if there is a limit on the number of concurrent hardware decoding sessions or how stable approach 1) is. You can see the results of some comparisons I ran a while back [here](https://github.com/cudawarped/opencv-experiments/blob/master/nbs/opencv450-video-read-CUDA_10_2_VideoCodecSDK_11_0_10.ipynb) to get an idea of the decoding perfomance of each approach. ---
**Author:** ![christophersmith](https://avatars.discourse-cdn.com/v4/letter/c/c89c15/32.png) [@christophersmith](https://forum.opencv.org/u/christophersmith)\ **Post date:** [March 15, 2022, 5:15pm UTC](https://forum.opencv.org/t/trouble-using-nvdia-hardware-decoder-when-streaming-from-camera/7908/14 "2022-03-15T17:15:04Z")
Thank you for the information, I’m trying out some of them now, hopefully it gives me a path forward. The place where I got the information about the codacodec::VideoReader is here: > > > \##### System information (version) > \- OpenCV =\> 3.4.1 > \- Operating System / Plat…form =\> Ubuntu 16.04 amd64 > \- Compiler =\> g++ 5.4 > > \##### Detailed description > I try to use cudacodec VideoReader to read a mkv : > \`cudacodec::createVideoReader("file.mkv")\` > > OpenCV throws this error : > \`\`\` > OpenCV(3.4.1) Error: The function/feature is not implemented (The called functionality is disabled for current build or platform) in throw\_no\_cuda, file /home/damien/code/lib/opencv-3.4.1/opencv-3.4.1/modules/core/include/opencv2/core/private.cuda.hpp, line 111 > terminate called after throwing an instance of 'cv::Exception' > what(): OpenCV(3.4.1) /home/damien/code/lib/opencv-3.4.1/opencv-3.4.1/modules/core/include/opencv2/core/private.cuda.hpp:111: error: (-213) The called functionality is disabled for current build or platform in function throw\_no\_cuda > \`\`\` > > I compile OpenCV with external cuda-enabled ffmpeg : > \`\`\` > $ ffmpeg > ffmpeg version n3.4.2-6-g1b9b469 Copyright (c) 2000-2018 the FFmpeg developers > built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.9) 20160609 > configuration: --enable-cuda --enable-nvenc --enable-cuvid --enable-nonfree --extra-cflags='-I/usr/local/cuda/include -I/home/damien/code/lib/Video\_Codec\_SDK\_8.1.24/Samples/NvCodec/NvEncoder' --extra-ldflags=-L/usr/local/cuda/lib64 --enable-shared --disable-static > libavutil 55. 78.100 / 55. 78.100 > libavcodec 57.107.100 / 57.107.100 > libavformat 57. 83.100 / 57. 83.100 > libavdevice 57. 10.100 / 57. 10.100 > libavfilter 6.107.100 / 6.107.100 > libswscale 4. 8.100 / 4. 8.100 > libswresample 2. 9.100 / 2. 9.100 > Hyper fast Audio and Video encoder > \`\`\` > > I compile OpenCV with this cmake : > \`cmake --enable-shared -D WITH\_NVCUVID\` > Giving me this OpenCV configuration : > \`\`\` > General configuration for OpenCV 3.4.1 ===================================== > Version control: unknown > > Extra modules: > Location (extra): /home/damien/code/lib/opencv-3.4.1/opencv\_contrib-3.4.1/modules > Version control (extra): unknown > > Platform: > Timestamp: 2018-04-03T15:40:15Z > Host: Linux 4.13.0-37-generic x86\_64 > ... > Video I/O: > DC1394: YES (ver 2.2.4) > FFMPEG: YES > avcodec: YES (ver 57.107.100) > avformat: YES (ver 57.83.100) > avutil: YES (ver 55.78.100) > swscale: YES (ver 4.8.100) > avresample: NO > ... > NVIDIA CUDA: YES (ver 9.1, CUFFT CUBLAS) > NVIDIA GPU arch: 30 35 37 50 52 60 61 70 > NVIDIA PTX archs: > > OpenCL: YES (no extra features) > Include path: /home/damien/code/lib/opencv-3.4.1/opencv-3.4.1/3rdparty/include/opencl/1.2 > Link libraries: Dynamic load > ... > \`\`\` > > After taking a look at the source code, I see that : > > Using cudacodec VideoReader needs the HAVE\_NVCUVID envvar to be set : > https://github.com/opencv/opencv/blob/3.4.1/modules/cudacodec/src/video\_reader.cpp#L49-L54 > > This envvar is set only if CUDA\_nvcuvid\_LIBRARY is set : > https://github.com/opencv/opencv/blob/3.4.1/cmake/OpenCVDetectCUDA.cmake#L36-L38 > > And this variable is set only when we are on WIN32 : > https://github.com/opencv/opencv/blob/3.4.1/cmake/FindCUDA.cmake#L792-L794 > > So I think that there is no way to enable NVCUVID. Do I miss something ? If not, I think that it should be fixed. > > Furthermore, I don't want to use the CuvidVideoSource, but I want to use the shared FFMPEG lib that already can use cuvid, with the FFmpegVideoSource. And I think that there is no need for OpenCV to link to cuvid to use this ffmpeg backend. So the FFMPEG backend should be available event if HAVE\_NVCUVID is not defined. Is it ? > > I can do these changes, but I think that having a confirmation before working on could be interresting. > > \##### Steps to reproduce > \- Compile OpenCV 3.4.1 with CUDA 9.1, NVCUVID > \- try to open a mkv : > > \`\`\`.cpp > cudacodec::createVideoReader("file.mkv") > \`\`\` > > More informations about this problem : http://answers.opencv.org/question/188525/enable-cudacodecvideoreader/ among other places this exchange is relevant: > [![@ZiperRom1](https://us1.discourse-cdn.com/flex020/uploads/opencv/original/2X/6/60e6fd476e7e7a2581f61792a29e0ec01b89ee8b.png)](https://github.com/ZiperRom1) > > ### **[ZiperRom1](https://github.com/ZiperRom1)** commented [on Jun 28, 2018](https://github.com/opencv/opencv/issues/11220#issuecomment-400956082) • > > edited > > So the `cudacodec::createVideoReader` to read video input directly in a `GpuMat` is deprecated since CUDA 7 right ? > > [![@dapicard](https://us1.discourse-cdn.com/flex020/uploads/opencv/original/2X/6/6a6592a4c5011116625d99cc90aca3235226528a.png)](https://github.com/dapicard) > > ContributorAuthor > > ### **[dapicard](https://github.com/dapicard)** commented [on Jun 28, 2018](https://github.com/opencv/opencv/issues/11220#issuecomment-400957355) > > Yes, this is actually what I understand. And I’ve never been able to use cudacodec video reader/writer with CUDA 7+ ---
**Author:** ![cudawarped](https://avatars.discourse-cdn.com/v4/letter/c/9dc877/32.png) [@cudawarped](https://forum.opencv.org/u/cudawarped)\ **Post date:** [March 15, 2022, 5:57pm UTC](https://forum.opencv.org/t/trouble-using-nvdia-hardware-decoder-when-streaming-from-camera/7908/15 "2022-03-15T17:57:47Z")
OK that makes sense what they are saying was probably true in 2018 but it is not since at least 2020, if you stick to the OpenCV 4.5 and above you should be fine. As a side note, they’re talking about CUDA SDK version 7 which is really old not compute capability 7 devices so you should be fine there aswell. If you decide to build cudacodec and are having any issues then I can provide you with build instructions if you want?