How to have OpenCV built process use self-built ffmpeg?

Building OpenCV in “default” way (CMake/ Code::Blocks/ mingw-64)
it uses a prebuilt ffmpeg, wrapped in “opencv_videoio_ffmpeg480_64.dll”.

My CMake’s output:

Video I/O:
DC1394: NO
FFMPEG: YES (prebuilt binaries)
avcodec: YES (58.134.100)
avformat: YES (58.76.100)
avutil: YES (56.70.100)
swscale: YES (5.9.100)
avresample: YES (4.0.0)
GStreamer: NO
DirectShow: YES

But this ffmpeg seems to offer only “rudimentary” (video) codecs.
Especially the most-used h264 codec with acceptable quality lacks.

My question:
What exactly has to be done to let OpenCV build use a pre-built
(full-featured) ffmpeg of one’s own?
(Which probably means to have let OpenCV built the ffmpeg wrapper anew?)

See

Note: You may have to modify the CMake script if you use a more recent version of FFMpeg than I described there.

1 Like

Thank you very much for your help!

As I don’t know much about building/ dependencies etc. I couldn’t assess at all which is the right direction, and which completely misleading.

I had a closer look at your proposition; and it seems that I either have to build ffmpeg by myself (being at windows, I will have to install m2sys, and doing in unix world),
or somehow have to make ffmpeg use x264 instead of openh264.
Both are external but OpenCV’s ffmpeg asks/ uses openh264, not x264 (?)

The chain becomes longer and longer; and every step in unknown territory
(which gives an uneasy feeling: if the simplest thing goes wrong, I’m at loss again).

The version linked to in the post uses x264

ffmpeg version n5.0.1-5-g240d82f26e-20220611 Copyright (c) 2000-2022 the FFmpeg developers
built with gcc 11.2.0 (crosstool-NG 1.24.0.533_681aaef)
configuration: --prefix=/ffbuild/prefix --pkg-config-flags=–static --pkg-config=pkg-config --cross-prefix=x86_64-w64-mingw32- --arch=x86_64 --target-os=mingw32 --enable-gpl --enable-version3 --disable-debug --enable-shared --disable-static --disable-w32threads --enable-pthreads --enable-iconv --enable-libxml2 --enable-zlib --enable-libfreetype --enable-libfribidi --enable-gmp --enable-lzma --enable-fontconfig --enable-libvorbis --enable-opencl --disable-libpulse --enable-libvmaf --disable-libxcb --disable-xlib --enable-amf --enable-libaom --enable-libaribb24 --enable-avisynth --enable-libdav1d --enable-libdavs2 --disable-libfdk-aac --enable-ffnvcodec --enable-cuda-llvm --enable-frei0r --enable-libgme --enable-libass --enable-libbluray --enable-libmp3lame --enable-libopus --enable-librist --enable-libtheora --enable-libvpx --enable-libwebp --enable-lv2 --enable-libmfx --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt --enable-librav1e --enable-librubberband --enable-schannel --enable-sdl2 --enable-libsoxr --enable-libsrt --enable-libsvtav1 --enable-libtwolame --enable-libuavs3d --disable-libdrm --disable-vaapi --enable-libvidstab --enable-vulkan --enable-libshaderc --enable-libplacebo --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libzimg --enable-libzvbi --extra-cflags=-DLIBTWOLAME_STATIC --extra-cxxflags= --extra-ldflags=-pthread --extra-ldexeflags= --extra-libs=-lgomp --extra-version=20220611
libavutil 57. 17.100 / 57. 17.100
libavcodec 59. 18.100 / 59. 18.100
libavformat 59. 16.100 / 59. 16.100
libavdevice 59. 4.100 / 59. 4.100
libavfilter 8. 24.100 / 8. 24.100
libswscale 6. 4.100 / 6. 4.100
libswresample 4. 3.100 / 4. 3.100
libpostproc 56. 3.100 / 56. 3.100

My feedback after all - all does as expected now! :smile:

There’s only that setting OPENCV_FFMPEG_WRITER_OPTIONS (e.g. to “crf;18|tune;grain”) before starting my app or before building my app has no effect.
It seems to have been done once and forever when building OpenCV itself?
So there would be no way to use different encoding parameters?

I don’t know as I have not tried changing the writer options myself. That said documentation

Note: extra FFmpeg options should be pased in form key;value|key;value|key;value, for example hwaccel;cuvid|video_codec;h264_cuvid|vsync;0 or vcodec;x264|vprofile;high|vlevel;4.0

implies you should be able to tweak some parameters but maybe not all of them.

Yes, that’s exactly the way I tried it very first time (with default OpenH264 encoder).

Console window @ Windows:

Ran MyApp

(VideoWriter VW (“output.mp4”, CAP_FFMPEG, VideoWriter::fourcc(‘a’,‘v’,‘c’,‘1’), capt.get(CAP_PROP_FPS), refS))

Produces a strange avc1 file (quality very bad/ file size very large).

Next:

set OPENCV_FFMPEG_WRITER_OPTIONS=crf;18

then ran my app.

Produces… the same file!

Now my assumption was OpenH264 was not capable to use the env var.
That’s why I installed newest OpenCV 4.8.0 with built-in x264 (with your kind help).

Result:
The produced avc1 file is coded with default crf23 (= standard) alright.
But in both cases!

My next supposition (until now) was that the env var have to be set when building OpenCV (as the names are similar to the CMake entries).
But your link to the documentation contradicts that (thank god).

It would be nice if somebody would check if/ how these env var do have an effect at all.
Maybe a bug/ lack in documenation?

@crackwitz has already raised an issue to this effect.

If it was me I would check that the env var is being passed correctly to OpenCV either by placing a breakpoint

or increasing the log level

    OPENCV_FFMPEG_DEBUG=1
    OPENCV_LOG_LEVEL=DEBUG

I would also double check that OpenCV is using the supplied version of FFMpeg, not the automaitically downloaded OpenCV shared library and that calling that version outside of OpenCV (ffmpeg.exe) with the arguments you mention has the desired result.

you’re supposed to set them at runtime, not before starting your program. either should work.

I checked:

  • my OpenCV uses ffmpeg built-in x264 (shown by mediainfo, and by output of OPENCV_FFMPEG_DEBUG, see below).
  • the ffmpeg used for building OpenCV is capable of crf-setting (tested by command line)

If set OPENCV_FFMPEG_DEBUG=1:

OPENCV_FFMPEG_WRITER_OPTIONS not set:


[OPENCV:FFMPEG:32] 264 - core 164 - H.264/MPEG-4 AVC codec - Copyleft 2003-2022 - x264, the best H.264/AVC encoder - VideoLAN - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd =1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_go p=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
[OPENCV:FFMPEG:40] Reinit context to 1280x720, pix_fmt: yuv420p

Now

set OPENCV_FFMPEG_WRITER_OPTIONS=crf;18.0


[OPENCV:FFMPEG:32] profile High, level 3.1, 4:2:0, 8-bit
[OPENCV:FFMPEG:32] 264 - core 164 - H.264/MPEG-4 AVC codec - Copyleft 2003-2022 - x264, the best H.264/AVC encoder - VideoLAN - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd =1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 blu ray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_go p=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00

Obviously OPENCV_FFMPEG_WRITER_OPTIONS isn’t noticed (?).

Now

set OPENCV_LOG_LEVEL=DEBUG

Output:

[DEBUG:0@0.009] global videoio_registry.cpp:218 VideoBackendRegistry VIDEOIO: Builtin backends(8): FFMPEG(1000); GSTREAMER(990); INTEL_MFX(980); MSMF(970); DSHOW(960); CV_IMAGES(950); CV_MJPEG(940); UEYE(930)
[DEBUG:0@0.009] global videoio_registry.cpp:242 VideoBackendRegistry VIDEOIO: Available backends(8): FFMPEG(1000); GSTREAMER(990); INTEL_MFX(980); MSMF(970); DSHOW(960); CV_IMAGES(950); CV_MJPEG(940); UEYE(930)
[ INFO:0@0.009] global videoio_registry.cpp:244 VideoBackendRegistry VIDEOIO: Enabled backends(8, sorted by priority): FFMPEG(1000); GSTREAMER(990); INTEL_MFX(980); MSMF(970); DSHOW(960); CV_IMAGES(950); CV_MJPEG(940); UEYE(930)
[OPENCV:FFMPEG:40] Reinit context to 1280x720, pix_fmt: yuv420p
[DEBUG:0@0.032] global cap_ffmpeg_impl.hpp:1182 open FFMPEG: stream[0] is video stream with codecID=27 width=1280 height=720
[DEBUG:0@0.032] global cap_ffmpeg_hw.hpp:934 HWAccelIterator FFMPEG: allowed acceleration types (none): ‘’
[DEBUG:0@0.036] global cap_ffmpeg_impl.hpp:2773 open Selected pixel format: bgr24
[DEBUG:0@0.036] global cap_ffmpeg_impl.hpp:2980 open VIDEOIO/FFMPEG: using writer options from environment: crf;18.0
[DEBUG:0@0.036] global cap_ffmpeg_hw.hpp:934 HWAccelIterator FFMPEG: allowed acceleration types (none): ‘’
[OPENCV:FFMPEG:32] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[OPENCV:FFMPEG:32] profile High, level 3.1, 4:2:0, 8-bit
[OPENCV:FFMPEG:32] 264 - core 164 - H.264/MPEG-4 AVC codec - Copyleft 2003-2022 - x264, the best H.264/AVC encoder - VideoLAN - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
[OPENCV:FFMPEG:40] Reinit context to 1280x720, pix_fmt: yuv420p
[DEBUG:0@0.054] global cap_ffmpeg_impl.hpp:1609 retrieveFrame Input picture format: yuv420p
[DEBUG:0@0.057] global parallel.cpp:107 createDefaultParallelForAPI core(parallel): Initializing parallel backend…
[DEBUG:0@0.057] global registry_parallel.impl.hpp:69 ParallelBackendRegistry core(parallel): Builtin backends(3): ONETBB(1000); TBB(990); OPENMP(980)
[DEBUG:0@0.057] global registry_parallel.impl.hpp:94 ParallelBackendRegistry core(parallel): Available backends(3): ONETBB(1000); TBB(990); OPENMP(980)
[ INFO:0@0.057] global registry_parallel.impl.hpp:96 ParallelBackendRegistry core(parallel): Enabled backends(3, sorted by priority): ONETBB(1000); TBB(990); OPENMP(980)
[DEBUG:0@0.057] global parallel.cpp:65 createParallelForAPI core(parallel): trying backend: ONETBB (priority=1000)
[DEBUG:0@0.057] global plugin_parallel_wrapper.impl.hpp:233 getPluginCandidates Found 2 plugin(s) for ONETBB
[ INFO:0@0.057] global plugin_loader.impl.hpp:67 libraryLoad load D:\Guido\develop\opencv\build\bin\opencv_core_parallel_onetbb480_64.dll => FAILED
[ INFO:0@0.057] global plugin_loader.impl.hpp:67 libraryLoad load opencv_core_parallel_onetbb480_64.dll => FAILED
[DEBUG:0@0.057] global parallel.cpp:65 createParallelForAPI core(parallel): trying backend: TBB (priority=990)
[DEBUG:0@0.058] global plugin_parallel_wrapper.impl.hpp:233 getPluginCandidates Found 2 plugin(s) for TBB
[ INFO:0@0.058] global plugin_loader.impl.hpp:67 libraryLoad load D:\Guido\develop\opencv\build\bin\opencv_core_parallel_tbb480_64.dll => FAILED
[ INFO:0@0.058] global plugin_loader.impl.hpp:67 libraryLoad load opencv_core_parallel_tbb480_64.dll => FAILED
[DEBUG:0@0.058] global parallel.cpp:65 createParallelForAPI core(parallel): trying backend: OPENMP (priority=980)
[DEBUG:0@0.058] global plugin_parallel_wrapper.impl.hpp:233 getPluginCandidates Found 2 plugin(s) for OPENMP
[ INFO:0@0.058] global plugin_loader.impl.hpp:67 libraryLoad load D:\Guido\develop\opencv\build\bin\opencv_core_parallel_openmp480_64.dll => FAILED
[ INFO:0@0.058] global plugin_loader.impl.hpp:67 libraryLoad load opencv_core_parallel_openmp480_64.dll => FAILED
[DEBUG:0@0.058] global parallel.cpp:93 createParallelForAPI core(parallel): fallback on builtin code
[DEBUG:0@0.089] global cap_ffmpeg_impl.hpp:1609 retrieveFrame Input picture format: yuv420p
[DEBUG:0@0.119] global cap_ffmpeg_impl.hpp:1609 retrieveFrame Input picture format: yuv420p

Curiously I tried using HEVC for output codec.
It works - producing (nearly) lossless output;
taking very long and producing file size abt. half FFV1.
But crf=28 is default for HEVC - not lossless.

It produces only that (nearly) lossless - setting “crf;18” has no effect here either.

So the problem of wrong handling of codec settings is severe.

Would be nice if anybody would retrace that
and either confirm or reject it,
to possibly enter an issue.