Build OpenCV with MFX for sandy bridge (linux)

1

Hello opencv community,
I struggle with an problem since days. (Ubuntu 18.04.05)

In general I want to grabb a video input (video grabber card sdk, no opencv involved), process the frames and write the frames with the opencv cv::ViedeoWriter in c++. Everything works just fine so far, but the program is used on an old pc with an i5 2400 sandy bridge. So the regular cv::CAP_FFMPEG or cv::CAP_GSTREAMER backend are to slow here and I run out of memory.
After a while I found the cv::INTEL_MFX. But whatever I do, when I build opencv, there is written

Intel Media SDK: NO

I installed the Intel Media SDK with Intel OpenVino and the corresponding instructions, I tried to install the driver and all dependencies, set all the env. variables, but no way. vainfo works just fine:

libva info: VA-API version 1.8.0
libva info: User environment variable requested driver ‘iHD’
libva info: Trying to open /opt/intel/mediasdk/lib64/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_8
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.8 (libva 2.1.0)
vainfo: Driver version: Intel iHD driver for Intel(R) Gen Graphics - 20.2.0 (97e2ea9)

somehow, even with “Intel Media SDK: NO” in the build, I could manage to use the media sdk for my dev system. But the compiled code does not work on my old deployment system (much older hardware/driver on deployment).
So tried to do the same on my deployment system, but for the build and when I execute cv::getBuildInformation() there is no Intel Media SDK. For my dev system cv::getBuildInformation() shows:

Intel Media SDK: YES (/mnt/nfs/msdk/lin-18.4.1/lib64/libmfx.so)

which is weird, because this path does not exist, but it works.

I assume that the driver I found at github media-driver (see link above) is not working. I mean the mentioned architectures of CPUs do not include the sandy bridge. But sandy bridge already had hw acceleration for video/encoding and vainfo shows:

vainfo: VA-API version: 1.8 (libva 2.1.0)
vainfo: Driver version: Intel i965 driver for Intel(R) Sandybridge Desktop - 2.1.0

My questions, is there an older package I have to install to make my old sandy bridge work with hw encoding/cv::INTEL_MFX? How can I configure my environment, so the Media SDK is detected by opencv for the build?

In addition my build command:

sudo cmake -D CMAKE_BUILD_TYPE=RELEASE
-D CMAKE_INSTALL_PREFIX=/usr/local
-D WITH_TBB=ON
-D WITH_CUBLAS=1
-D WITH_V4L=ON
-D WITH_OPENGL=ON
-D OPENCV_GENERATE_PKGCONFIG=ON
-D OPENCV_ENABLE_NONFREE=ON
-D OPENCV_EXTRA_MODULES_PATH=…/…/opencv_contrib/modules
-D WITH_MFX=ON …

Open for every hint,
BR Michael

2

This is OpenCV from OpenVINO, it was built with MediaSDK support.

Check out this article: MediaSDK encode decode backend · opencv/opencv Wiki · GitHub You should set MFX_HOME environment variable and enable WITH_MFX cmake option.

Perhaps your HW is too old for MediaSDK even though it has HW-accelerated video decoding capabilities.

I’d recommend you to try GStreamer VA-API plugin too, it can be more convenient: opencv/samples/cpp/videocapture_gstreamer_pipeline.cpp at master · opencv/opencv · GitHub

3

Hi mshabunin, thank you for your fast response.
The env. is set in .bashrc as well as other variables:

export MFX_HOME=/opt/intel/mediasdk
export LIBVA_DRIVERS_PATH=/opt/intel/mediasdk/lib
export LIBVA_DRIVER_NAME=i965

This works pretty well for the vainfo command:
libva info: VA-API version 1.8.0
libva info: User environment variable requested driver ‘iHD’
libva info: Trying to open /opt/intel/mediasdk/lib/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_8
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.8 (libva 2.1.0)
vainfo: Driver version: Intel iHD driver for Intel® Gen Graphics - 20.2.0 (97e2ea9)

But it seems to have no meaning to openCV.

I figured out that OpenCVDetectMediaSDK.cmake has been removed after version 4.0.1 on github. So maybe it is not even longer possible to get it in the build. I tried 4.0.1. and I could make it work with a part of this description. But my old cpu is still not running and does not use the path set in the environment variables. When it gets to the part, where the cv::VideoWriter is created:

cv::VideoWriter videoWriter(path, cv::CAP_INTEL_MFX, fourcc, fps, *size);

it prints for my dev system:

libva info: VA-API version 1.11.0
libva info: Trying to open /usr/local/lib/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_8
libva info: va_openDriver() returns 0

and works just fine, while my deployment system with the old intel sandbridge cpu prints this:

libva info: VA-API version 1.11.0
libva info: Trying to open /usr/local/lib/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_11
libva error:  /usr/local/lib/dri/iHD_drv_video.so init failed
libva info: va_openDriver() returns 1
libva info: Trying to open /usr/local/lib/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_1_1
libva info: va_openDriver() returns 0
MFX: Can't initialize session

So the deployment system tries another driver, i965, which is fine because it is older, but it can’t make it work. In both cases there are two things fishy:

  • VA-API version is for openCV 1.11.0 while you can see for vainfo it is 1.8
  • the path for the driver is not the path specified in MFX_HOME etc.

How can I use the libva of my vainfo and use the drivers directly from mediasdk?

I already used the provided link for MediaSDK, it is from 2017, it does not work.

Thank you for the plugin link, I have currently no idea how openCV is working with plugins, I am not even able to use the libs/tools of my system for openCV (ffmpeg, gstreamer, intel media sdk), do you have a good tutorial to see how the plugin works and my compiled tools are linked/used?

4

The detection script has been moved here: opencv/detect_msdk.cmake at master · opencv/opencv · GitHub You can add debug messages to this file to find out where the problem is.

And I remembered that you also need libva-dev package for it to work, perhaps this is your issue.

MFX: Can't initialize session
means that there was a problem with MediaSDK initialization, perhaps it is not compatible with your system (MSDK or iHD driver).

Try to use GStreamer backend:

  • install gstreamer1.0-vaapi package and maybe gstreamer1.0-plugins-good and gstreamer1.0-plugins-ugly
  • create VideoCapture with GStreamer pipeline instead of filename: VideoCapture("filesrc location=<filename.mp4> ! qtdemux ! vaapidecodebin ! videoconvert ! appsink sync=false")
5

Hi mshabunin,
thank you for your response. I tried to take a look for the cmake file you linked, I am not an expert of cmake but I found a problem indeed. In the second line there is written:
set(paths "${MFX_HOME}" ENV "MFX_HOME" ENV "INTELMEDIASDKROOT")
if I print the paths afterwards I get this:
ENVMFX_HOMEENVINTELMEDIASDKROOT
Not sure what is wrong here, if this is about my environment or an actual bug. But if I set the paths hardcoded to my media-sdk path, everything seems to be fine, at least the lib is found.
Thank you, I will take a look for the GStreamer, still not sure how this is linked and working. I do not use VideoCapture to get frames, I use the sdk of my grabber card since this is the only way on linux. So I need it only to write the videos.
BR Michael