How to use FFMPEG of my System with OpenCV?

,
1

Hello community,
I use openCV with C++ to process images and write them with the cv::VideoWriter class.
It works just fine, as backend I use cv::CAP_FFMPEG.
My problem here is, sometimes this is a little slow (depending on which codec is used). It bothers my hardware a lot even if I have a nvidia GPU with hardware encoding, because it is not used.
If I use my FFMPEG of my system in terminal, with the right codec, my GPU is doing the job.
When I got this right, OpenCV is not using my system FFMPEG, which means it does not even have access to this functions, here is the output of cv::getBuildInformation():

General configuration for OpenCV 4.5.1 =====================================
Version control: unknown

Extra modules:
Location (extra): /home/mic/opencv-4.5.1/opencv_contrib/modules
Version control (extra): unknown

Platform:
Timestamp: 2020-12-28T15:48:46Z
Host: Linux 5.4.0-58-generic x86_64
CMake: 3.10.2
CMake generator: Unix Makefiles
CMake build tool: /usr/bin/make
Configuration: RELEASE

CPU/HW features:
Baseline: SSE SSE2 SSE3
requested: SSE3
Dispatched code generation: SSE4_1 SSE4_2 FP16 AVX AVX2 AVX512_SKX
requested: SSE4_1 SSE4_2 AVX FP16 AVX2 AVX512_SKX
SSE4_1 (17 files): + SSSE3 SSE4_1
SSE4_2 (2 files): + SSSE3 SSE4_1 POPCNT SSE4_2
FP16 (1 files): + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 AVX
AVX (5 files): + SSSE3 SSE4_1 POPCNT SSE4_2 AVX
AVX2 (31 files): + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2
AVX512_SKX (7 files): + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2 AVX_512F AVX512_COMMON AVX512_SKX

C/C++:
Built as dynamic libs?: YES
C++ standard: 11
C++ Compiler: /usr/bin/c++ (ver 8.4.0)
C++ flags (Release): -fsigned-char -ffast-math -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections -msse -msse2 -msse3 -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG -DNDEBUG
C++ flags (Debug): -fsigned-char -ffast-math -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections -msse -msse2 -msse3 -fvisibility=hidden -fvisibility-inlines-hidden -g -O0 -DDEBUG -D_DEBUG
C Compiler: /usr/bin/cc
C flags (Release): -fsigned-char -ffast-math -W -Wall -Werror=return-type -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections -msse -msse2 -msse3 -fvisibility=hidden -O3 -DNDEBUG -DNDEBUG
C flags (Debug): -fsigned-char -ffast-math -W -Wall -Werror=return-type -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections -msse -msse2 -msse3 -fvisibility=hidden -g -O0 -DDEBUG -D_DEBUG
Linker flags (Release): -Wl,–exclude-libs,libippicv.a -Wl,–exclude-libs,libippiw.a -Wl,–gc-sections -Wl,–as-needed
Linker flags (Debug): -Wl,–exclude-libs,libippicv.a -Wl,–exclude-libs,libippiw.a -Wl,–gc-sections -Wl,–as-needed
ccache: NO
Precompiled headers: NO
Extra dependencies: dl m pthread rt /usr/lib/x86_64-linux-gnu/libGL.so /usr/lib/x86_64-linux-gnu/libGLU.so
3rdparty dependencies:

OpenCV modules:
To be built: aruco bgsegm bioinspired calib3d ccalib core datasets dnn dnn_objdetect dnn_superres dpm face features2d flann freetype fuzzy gapi hfs highgui img_hash imgcodecs imgproc intensity_transform line_descriptor mcc ml objdetect optflow phase_unwrapping photo plot quality rapid reg rgbd saliency shape stereo stitching structured_light superres surface_matching text tracking ts video videoio videostab xfeatures2d ximgproc xobjdetect xphoto
Disabled: world
Disabled by dependency: -
Unavailable: alphamat cnn_3dobj cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters cudaimgproc cudalegacy cudaobjdetect cudaoptflow cudastereo cudawarping cudev cvv hdf java julia matlab ovis python2 python3 sfm viz
Applications: tests perf_tests apps
Documentation: NO
Non-free algorithms: YES

GUI:
GTK+: YES (ver 2.24.32)
GThread : YES (ver 2.56.4)
GtkGlExt: YES (ver 1.2.0)
OpenGL support: YES (/usr/lib/x86_64-linux-gnu/libGL.so /usr/lib/x86_64-linux-gnu/libGLU.so)
VTK support: NO

Media I/O:
ZLib: /usr/lib/x86_64-linux-gnu/libz.so (ver 1.2.11)
JPEG: /usr/lib/x86_64-linux-gnu/libjpeg.so (ver 80)
WEBP: build (ver encoder: 0x020f)
PNG: /usr/lib/x86_64-linux-gnu/libpng.so (ver 1.6.34)
TIFF: /usr/lib/x86_64-linux-gnu/libtiff.so (ver 42 / 4.0.9)
JPEG 2000: build (ver 2.3.1)
OpenEXR: build (ver 2.3.0)
HDR: YES
SUNRASTER: YES
PXM: YES
PFM: YES

Video I/O:
DC1394: YES (2.2.5)
FFMPEG: YES
avcodec: YES (57.107.100)
avformat: YES (57.83.100)
avutil: YES (55.78.100)
swscale: YES (4.8.100)
avresample: YES (3.7.0)
GStreamer: YES (1.14.5)
v4l/v4l2: YES (linux/videodev2.h)

Parallel framework: TBB (ver 2017.0 interface 9107)

Trace: YES (with Intel ITT)

Other third-party libraries:
Intel IPP: 2020.0.0 Gold [2020.0.0]
at: /home/mic/opencv-4.5.1/opencv/build_smi_pc/3rdparty/ippicv/ippicv_lnx/icv
Intel IPP IW: sources (2020.0.0)
at: /home/mic/opencv-4.5.1/opencv/build_smi_pc/3rdparty/ippicv/ippicv_lnx/iw
Lapack: NO
Eigen: NO
Custom HAL: NO
Protobuf: build (3.5.1)

OpenCL: YES (no extra features)
Include path: /home/mic/opencv-4.5.1/opencv/3rdparty/include/opencl/1.2
Link libraries: Dynamic load

Python (for build): /usr/bin/python2.7

Java:
ant: NO
JNI: NO
Java wrappers: NO
Java tests: NO

Install to: /usr/local

For many of the used libraries there is the path of my system included, but not for FFMPEG, there is just written

YES

So, is it possible to use my System FFMPEG and use the codecs together with my GPU?
Or possible to use the nvidia GPU for encoding in any other way?
I was already looking at the [OpenCV: cv::cudacodec::VideoWriter Class Reference](https://cudacodec videowriter) but there is written

Currently only Windows platform is supported.

And Windows is not an option.
Grateful for your support.
BR Michael
.

2

FYI this hasn’t worked on windows for a long time.

3

I have successfully used ffmpeg.exe by forking a process and feeding images to stdin. It is tricky but can be done.

Performance depends on input image size, codec used, hardware acceleration used, and compression chosen. Try less compression to test performance if you are using lossful compression.

Opencv uses a special interface dll to to hook in ffmpeg library. This is a requirement for opencv, but as I mentioned it is possible to grab images and pipe to stdin of an ffmpeg process to compress images into video. With some codecs you must be careful how to finish or you get no video because the to close the video it has to properly write a header at the beginning. FFMPEG does this but it must have the correct close signaling.

4

I thought it was only windows which used its own shared library built on top of ffmpeg?

If this is not the case then I would imagine you can tweak the cmake script to use the following variables to achieve what you want
-DWITH_FFMPEG=ON -DFFMPEG_ROOT_DIR=<path to compiled libs> -DOPENCV_FFMPEG_SKIP_DOWNLOAD=ON
in the same way you can on windows.

1 Like
5

It is possible that it is windows only using the custom shared library - I have not checked it. I will take a look.

6

Hi cudawarped, this sounds good.
I installed FFMPEG via snapd. Can you pls tell me which “path to compiled libs” I need to specify here.
I tried to reference

/snap/ffmpeg/1286/usr/lib

because here are libavcodec.so and so on located. But I am not sure if this is working.
In the build info there is still

FFMPEG YES

Without any path to my system libs.
Can you pls help me on that one?
BR Michael

7

I think it would be better to ask someone more familiar with linux as I would assume something like pkg-config would pick up the version of ffmpeg you have installed and the manual root of hacking the cmake file is the wrong way to do things.