Video writing is too slow in OpenCV

Ratheesh · December 18, 2023, 8:11am

Hi Technical Team,

Environment used for my task: Windows 11
CUDA toolkits: 12.1
GPU Name : NVIDIA T1200 Laptop GPU
OpenCV 4.8.0

I have build OpenCV with CUDA support using CMake GUI for C++.

My requirement is “write 4K video @60 FPS”

Current state is “I am able to write @10 FPS only”

Method1:
To achieve this, I tried to enable Hardware acceleration as below, but it is crashing

		cv::VideoWriter VideoWriter;

		VideoWriter.open("fileName.mp4"
                        , cv::VideoWriter::fourcc('m','p','4','v')
                        , 60
                        , cv::Size(3840, 2160)
                        , {cv::VIDEOWRITER_PROP_HW_ACCELERATION, cv::VIDEO_ACCELERATION_ANY}
                        )
						
		VideoWriter.write(imageMat);

If I disable “{cv::VIDEOWRITER_PROP_HW_ACCELERATION, cv::VIDEO_ACCELERATION_ANY}”, it is working but video writing is too slow.

Method2:
Here I have used CUDA write, here also it is crashing and the error is below

“OpenCV: terminate handler is called! The last OpenCV error is:
OpenCV(4.8.0) Error: The function/feature is not implemented (The called functionality is disabled for current build or platform)
in throw_no_cuda, file C:\opencv\opencv-4.8.0\modules\core\include\opencv2/core/private.cuda.hpp, line 112”

cv::Ptr<cv::cudacodec::VideoWriter> d_writer;
cv::cuda::GpuMat d_frame;

d_writer = cv::cudacodec::createVideoWriter("fileName.mp4" 
                                            ,cv::Size(3840, 2160)
					,cv::cudacodec::Codec::H264
					,30
					,cv::cudacodec::ColorFormat::RGB
					);	

d_frame.upload(imageMat);
d_writer->write(d_frame);

My questions are:

Is any other software needs to be installed to enable hardware acceleration?
What is wrong in method2?

Regards,
Ratheesh

cudawarped · December 18, 2023, 10:25am

You need to install Nvidia Video Codec SDK. On windows,

Optional - To decode video on the GPU with the Nvidia Video Codec SDK:

Register and download the latest version of the Complete Video Codec SDK xx.x Package.

Extract and copy the contents of Interface and Lib to the include and lib directories inside your CUDA installation.

from

If you want your 4k files in container formats, mp4 etc. then you will have to wait until OpenCV 4.9.0 is released as opencv_videoio_ffmpeg480_64.dll hasn’t been updated yet to include

github.com/opencv/opencv_contrib

`cudacodec::VideoWriter` add container output

opencv:4.x ← cudawarped:videoio_ffmpeg_add_stream_encapsulation

opened 01:37PM - 05 Oct 23 UTC

cudawarped

+134 -67

Where supported, allows the raw encoded video output from Nvidia's hardware deco…der to be encapsulated in a suitable (mp4/mkv/etc.) container. Dependant on https://github.com/opencv/opencv/pull/24363. Fixes https://github.com/opencv/opencv_contrib/issues/3556 ### Pull Request Readiness Checklist See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request - [x] I agree to contribute to the project under Apache 2 License. - [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV - [x] The PR is proposed to the proper branch - [x] There is a reference to the original bug report and related work - [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable Patch to opencv_extra has the same branch name. - [x] The feature is well documented and sample code can be built with the project CMake

Ratheesh · December 26, 2023, 12:39pm

Hi,

Thanks for your response.

For the method :2, I have followed your suggestion ( Extract and copy the contents of Interface and Lib to the include and lib directories inside ) (Refer the screenshot

). I am still facing the below error

“OpenCV: terminate handler is called! The last OpenCV error is:
OpenCV(4.8.0) Error: The function/feature is not implemented (The called functionality is disabled for current build or platform)
in throw_no_cuda, file C:\opencv\opencv-4.8.0\modules\core\include\opencv2/core/private.cuda.hpp, line 112”
Also i have set environment variable like below

My questions:
am i doing anything wrong here? or any other parameters needs to be initialized for cv::cudacodec::createVideoWriter?

cudawarped · December 26, 2023, 2:44pm

I notice above I did not say and rebuild OpenCV so I’m guessing you didn’t? If not rebuild it and if so check your CMake configuration output for errors and confirm that the Nvidia section contains and entry for NVCUVID and NVCUVENC. e.g.

--   NVIDIA CUDA:                   YES (ver 12.3, CUFFT CUBLAS NVCUVID NVCUVENC)
--     NVIDIA GPU arch:             50 52 60 61 70 75 80 86 89 90
--     NVIDIA PTX archs:            90

All of this is in the guide I linked to.

You don’t need to set any environmental variables, OPENCV_FFMPEG_WRITER_OPTIONS is for cv::VideoWriter not cv::cuda::VideoWriter.

Ratheesh · December 26, 2023, 4:09pm

Hi,
Thankyou for your response.

Yes, I did not rebuild OpenCV.
I dont know the way to check “CMake configuration output for errors”. (or Kindly let me know, How do i check “CMake configuration output for errors”. )

But I have executed below code

void GuiHandler::deviceInfo()
{
std::cout << cv::getBuildInformation()<< std::endl;
}

The Output is:

nvidiaArchOp

I understood that setting of Environment variable “OPENCV_FFMPEG_WRITER_OPTIONS” is useful for cv::VideoWriter and it is not useful for cv::cuda::VideoWriter.

Ratheesh · December 27, 2023, 5:46pm

Hi, I have rebuild the OpenCV but now i am getting below error, Could you please help me to solve this error?

cudawarped · December 28, 2023, 5:57am

Please post the text not images and include all of the output from getBuildInformation.

Those variables were introduced after OpenCV 4.8.0 so it is likely you are using main 4.8.0 and one of the latest commits from the contrib branch. Use commits from the tip of the 4.x branch in both repos or the 4.8.0 tag on both branches.

Ratheesh · December 28, 2023, 4:51pm

Hi,

Thankyou for your response.

After following your instruction (used proper opencv_contrib-4.8.0 ), The above mentioned error is not coming and no crashing issue.

As per my need, It is creating the video file in .mp4 file but this file is not playable. Refer the below image for this.

I am giving the code again for your reference

cv::Ptrcv::cudacodec::VideoWriter d_writer;
cv::cuda::GpuMat d_frame;

d_writer = cv::cudacodec::createVideoWriter(“fileName.mp4”
,cv::Size(3840, 2160)
,cv::cudacodec::Codec::H264
,30
,cv::cudacodec::ColorFormat::RGB
);

d_frame.upload(imageMat);
d_writer->write(d_frame);

My question:
is something missed in the configuration?
is .mp4 container supported for H264 codec?

Note:
Below is the full output for cv::getBuildInformation(); function execution

General configuration for OpenCV 4.8.0 =====================================
Version control: unknown

Extra modules:
Location (extra): C:/opencv/cudaSDK/opencv_contrib-4.8.0/modules
Version control (extra): unknown

Platform:
Timestamp: 2023-12-28T09:16:09Z
Host: Windows 10.0.22000 AMD64
CMake: 3.27.7
CMake generator: Visual Studio 16 2019
CMake build tool: C:/Program Files (x86)/Microsoft Visual Studio/2019/Enterprise/MSBuild/Current/Bin/MSBuild.exe
MSVC: 1929
Configuration: Release

CPU/HW features:
Baseline: SSE SSE2 SSE3
requested: SSE3
Dispatched code generation: SSE4_1 SSE4_2 FP16 AVX AVX2 AVX512_SKX
requested: SSE4_1 SSE4_2 AVX FP16 AVX2 AVX512_SKX
SSE4_1 (18 files): + SSSE3 SSE4_1
SSE4_2 (2 files): + SSSE3 SSE4_1 POPCNT SSE4_2
FP16 (1 files): + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 AVX
AVX (8 files): + SSSE3 SSE4_1 POPCNT SSE4_2 AVX
AVX2 (37 files): + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2
AVX512_SKX (8 files): + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2 AVX_512F AVX512_COMMON AVX512_SKX

C/C++:
Built as dynamic libs?: YES
C++ standard: 11
C++ Compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Enterprise/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe (ver 19.29.30151.0)
C++ flags (Release): /DWIN32 /D_WINDOWS /W4 /GR /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi /fp:fast /EHa /wd4127 /wd4251 /wd4324 /wd4275 /wd4512 /wd4589 /wd4819 /MP /MD /O2 /Ob2 /DNDEBUG
C++ flags (Debug): /DWIN32 /D_WINDOWS /W4 /GR /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi /fp:fast /EHa /wd4127 /wd4251 /wd4324 /wd4275 /wd4512 /wd4589 /wd4819 /MP /MDd /Zi /Ob0 /Od /RTC1
C Compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Enterprise/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe
C flags (Release): /DWIN32 /D_WINDOWS /W3 /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi /fp:fast /MP /MD /O2 /Ob2 /DNDEBUG
C flags (Debug): /DWIN32 /D_WINDOWS /W3 /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi /fp:fast /MP /MDd /Zi /Ob0 /Od /RTC1
Linker flags (Release): /machine:x64 /INCREMENTAL:NO
Linker flags (Debug): /machine:x64 /debug /INCREMENTAL
ccache: NO
Precompiled headers: NO
Extra dependencies: cudart_static.lib nppc.lib nppial.lib nppicc.lib nppidei.lib nppif.lib nppig.lib nppim.lib nppist.lib nppisu.lib nppitc.lib npps.lib cublas.lib cudnn.lib cufft.lib -LIBPATH:C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.1/lib/x64
3rdparty dependencies:

OpenCV modules:
To be built: aruco bgsegm bioinspired calib3d ccalib core cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters cudaimgproc cudalegacy cudaobjdetect cudaoptflow cudastereo cudawarping cudev datasets dnn dnn_objdetect dnn_superres dpm face features2d flann fuzzy gapi hfs highgui img_hash imgcodecs imgproc intensity_transform line_descriptor mcc ml objdetect optflow phase_unwrapping photo plot quality rapid reg rgbd saliency shape stereo stitching structured_light superres surface_matching text tracking ts video videoio videostab wechat_qrcode world xfeatures2d ximgproc xobjdetect xphoto
Disabled: -
Disabled by dependency: -
Unavailable: alphamat cvv freetype hdf java julia matlab ovis python2 python3 sfm viz
Applications: tests perf_tests apps
Documentation: NO
Non-free algorithms: NO

Windows RT support: NO

GUI:
Win32 UI: YES
VTK support: NO

Media I/O:
ZLib: build (ver 1.2.13)
JPEG: build-libjpeg-turbo (ver 2.1.3-62)
SIMD Support Request: YES
SIMD Support: NO
WEBP: build (ver encoder: 0x020f)
PNG: build (ver 1.6.37)
TIFF: build (ver 42 - 4.2.0)
JPEG 2000: build (ver 2.5.0)
OpenEXR: build (ver 2.3.0)
HDR: YES
SUNRASTER: YES
PXM: YES
PFM: YES

Video I/O:
DC1394: NO
FFMPEG: YES (prebuilt binaries)
avcodec: YES (58.134.100)
avformat: YES (58.76.100)
avutil: YES (56.70.100)
swscale: YES (5.9.100)
avresample: YES (4.0.0)
GStreamer: YES (1.21.1)
DirectShow: YES
Media Foundation: YES
DXVA: YES

Parallel framework: Concurrency

Trace: YES (with Intel ITT)

Other third-party libraries:
Intel IPP: 2021.8 [2021.8.0]
at: C:/opencv/cudaSDK/build/3rdparty/ippicv/ippicv_win/icv
Intel IPP IW: sources (2021.8.0)
at: C:/opencv/cudaSDK/build/3rdparty/ippicv/ippicv_win/iw
Lapack: NO
Eigen: NO
Custom HAL: NO
Protobuf: build (3.19.1)
Flatbuffers: builtin/3rdparty (23.5.9)

NVIDIA CUDA: YES (ver 12.1, CUFFT CUBLAS NVCUVID NVCUVENC FAST_MATH)
NVIDIA GPU arch: 75
NVIDIA PTX archs:

cuDNN: YES (ver 8.9.6)

OpenCL: YES (NVD3D11)
Include path: C:/opencv/cudaSDK/opencv-4.8.0/3rdparty/include/opencl/1.2
Link libraries: Dynamic load

Python (for build): NO

Java:
ant: NO
Java: NO
JNI: NO
Java wrappers: NO
Java tests: NO

Install to: C:/opencv/cudaSDK/build/install

cudawarped · December 28, 2023, 5:56pm

You file is a raw h264 encoded file with a .mp4 extension, rename to fileName.h264, play with vlc and see my original comment.

Ratheesh · December 30, 2023, 5:33pm

Hi,
I have build the openCV 4.9.0 and used the respective dll and lib for videoWriter.
Still the problem was not solved (That means, It is creating the video file in .mp4 file but this file is not directly playable format.). As per your instruction it is playing in VLC player by changing its extension to .h264.

But as per your instruction h264 codec has support to .mp4 container also.

My question: Is something needs to be modified in my above code(earlier chat) ?

Below is the full output for cv::getBuildInformation(); function execution

General configuration for OpenCV 4.9.0 =====================================
Version control: unknown

Extra modules:
Location (extra): C:/opencv/490/opencv_contrib-4.9.0/modules
Version control (extra): unknown

Platform:
Timestamp: 2023-12-29T16:58:47Z
Host: Windows 10.0.22000 AMD64
CMake: 3.27.7
CMake generator: Visual Studio 16 2019
CMake build tool: C:/Program Files (x86)/Microsoft Visual Studio/2019/Enterprise/MSBuild/Current/Bin/MSBuild.exe
MSVC: 1929
Configuration: Release

CPU/HW features:
Baseline: SSE SSE2 SSE3
requested: SSE3
Dispatched code generation: SSE4_1 SSE4_2 FP16 AVX AVX2 AVX512_SKX
requested: SSE4_1 SSE4_2 AVX FP16 AVX2 AVX512_SKX
SSE4_1 (18 files): + SSSE3 SSE4_1
SSE4_2 (2 files): + SSSE3 SSE4_1 POPCNT SSE4_2
FP16 (1 files): + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 AVX
AVX (9 files): + SSSE3 SSE4_1 POPCNT SSE4_2 AVX
AVX2 (38 files): + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2
AVX512_SKX (8 files): + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2 AVX_512F AVX512_COMMON AVX512_SKX

C/C++:
Built as dynamic libs?: YES
C++ standard: 11
C++ Compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Enterprise/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe (ver 19.29.30151.0)
C++ flags (Release): /DWIN32 /D_WINDOWS /W4 /GR /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi /fp:fast /EHa /wd4127 /wd4251 /wd4324 /wd4275 /wd4512 /wd4589 /wd4819 /MP /O2 /Ob2 /DNDEBUG
C++ flags (Debug): /DWIN32 /D_WINDOWS /W4 /GR /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi /fp:fast /EHa /wd4127 /wd4251 /wd4324 /wd4275 /wd4512 /wd4589 /wd4819 /MP /Zi /Ob0 /Od /RTC1
C Compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Enterprise/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe
C flags (Release): /DWIN32 /D_WINDOWS /W3 /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi /fp:fast /MP /O2 /Ob2 /DNDEBUG
C flags (Debug): /DWIN32 /D_WINDOWS /W3 /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi /fp:fast /MP /Zi /Ob0 /Od /RTC1
Linker flags (Release): /machine:x64 /INCREMENTAL:NO
Linker flags (Debug): /machine:x64 /debug /INCREMENTAL
ccache: NO
Precompiled headers: NO
Extra dependencies: cudart_static.lib nppc.lib nppial.lib nppicc.lib nppidei.lib nppif.lib nppig.lib nppim.lib nppist.lib nppisu.lib nppitc.lib npps.lib cublas.lib cudnn.lib cufft.lib -LIBPATH:C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.1/lib/x64
3rdparty dependencies:

OpenCV modules:
To be built: aruco bgsegm bioinspired calib3d ccalib core cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters cudaimgproc cudalegacy cudaobjdetect cudaoptflow cudastereo cudawarping cudev datasets dnn dnn_objdetect dnn_superres dpm face features2d flann fuzzy gapi hfs highgui img_hash imgcodecs imgproc intensity_transform line_descriptor mcc ml objdetect optflow phase_unwrapping photo plot quality rapid reg rgbd saliency shape stereo stitching structured_light superres surface_matching text tracking ts video videoio videostab wechat_qrcode world xfeatures2d ximgproc xobjdetect xphoto
Disabled: -
Disabled by dependency: -
Unavailable: alphamat cannops cvv freetype hdf java julia matlab ovis python2 python3 sfm viz
Applications: tests perf_tests apps
Documentation: NO
Non-free algorithms: NO

Windows RT support: NO

GUI:
Win32 UI: YES
VTK support: NO

Media I/O:
ZLib: build (ver 1.3)
JPEG: build-libjpeg-turbo (ver 2.1.3-62)
SIMD Support Request: YES
SIMD Support: NO
WEBP: build (ver encoder: 0x020f)
PNG: build (ver 1.6.37)
TIFF: build (ver 42 - 4.2.0)
JPEG 2000: build (ver 2.5.0)
OpenEXR: build (ver 2.3.0)
HDR: YES
SUNRASTER: YES
PXM: YES
PFM: YES

Video I/O:
DC1394: NO
FFMPEG: YES (prebuilt binaries)
avcodec: YES (58.134.100)
avformat: YES (58.76.100)
avutil: YES (56.70.100)
swscale: YES (5.9.100)
avresample: YES (4.0.0)
GStreamer: YES (1.21.1)
DirectShow: YES
Media Foundation: YES
DXVA: YES

Parallel framework: Concurrency

Trace: YES (with Intel ITT)

Other third-party libraries:
Intel IPP: 2021.11.0 [2021.11.0]
at: C:/opencv/490/build/3rdparty/ippicv/ippicv_win/icv
Intel IPP IW: sources (2021.11.0)
at: C:/opencv/490/build/3rdparty/ippicv/ippicv_win/iw
Lapack: NO
Eigen: NO
Custom HAL: NO
Protobuf: build (3.19.1)
Flatbuffers: builtin/3rdparty (23.5.9)

NVIDIA CUDA: YES (ver 12.1, CUFFT CUBLAS NVCUVID NVCUVENC FAST_MATH)
NVIDIA GPU arch: 75
NVIDIA PTX archs: 90

cuDNN: YES (ver 8.9.6)

OpenCL: YES (NVD3D11)
Include path: C:/opencv/490/opencv-4.9.0/3rdparty/include/opencl/1.2
Link libraries: Dynamic load

Python (for build): NO

Java:
ant: NO
Java: NO
JNI: NO
Java wrappers: NO
Java tests: NO

Install to: C:/opencv/490/build/install

Thanks
Ratheesh

cudawarped · December 31, 2023, 6:19am

Sorry that feature needs to be enabled now the FFMpeg shared library for windows has been updated. If you apply the following patch it should work.

github.com/opencv/opencv_contrib

`cudacodec`: Enable features available with updated ffmpeg dll

opencv:4.x ← cudawarped:cudacodec_enable_win32_for_updated_dll

opened 07:00PM - 30 Dec 23 UTC

cudawarped

+0 -70

Enable disabled windows features (https://github.com/opencv/opencv_contrib/pull/…3569, https://github.com/opencv/opencv_contrib/pull/3488, https://github.com/opencv/opencv_contrib/pull/3542) which are now available in the ffmpeg shared libary. ### Pull Request Readiness Checklist See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request - [x] I agree to contribute to the project under Apache 2 License. - [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV - [x] The PR is proposed to the proper branch - [x] There is a reference to the original bug report and related work - [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable Patch to opencv_extra has the same branch name. - [x] The feature is well documented and sample code can be built with the project CMake

Ratheesh · January 5, 2024, 2:41pm

Hi,

Thankyou , Yes it is working after applying that patch.

cudawarped · January 16, 2024, 9:57am

Its now merged into 4.x so no need for the patch.

Topic		Replies	Views
VideoWriter.write() very slow Python performance , videoio , nvidia	4	3969	November 3, 2022
cv2.cudacodec.createVideoWriter does not work? Python cuda , videoio	8	2729	May 10, 2022
cv::cudacodec::VideoWriter not working with only NVCUVID C++ cuda , videoio , cudacodec	8	1388	December 31, 2023
Status and usage of cudacodec::VideoWriter Python ffmpeg , cuda , videoio , cudacodec	39	708	May 13, 2024
Can only init 5 cv2.cudacodec.createVideoWriter successfully in parallel Python cuda , videoio	4	448	November 20, 2023

Video writing is too slow in OpenCV

Install to: C:/opencv/cudaSDK/build/install

Install to: C:/opencv/490/build/install

Related topics