Opencv 4.5.2 breaking change with FFMPEG HW accelerated decoding

Python
,
1

I have been trying to find documentation for about an hour and gave up, deciding to recompile 4.5.1 to try to recover my setup…
The so called “improvement” of videoio was a cataclysmic change with poor documentation. ChangeLog · opencv/opencv Wiki · GitHub

I am running opencv and using the ffmpeg backend with cuda acceleration using the python API. I have compiled every version of opencv if contrib this far without any issues.

My question is this: What is the 4.5.2 new equivalent to setting up a video stream as I used to do with this:

os.environ["OPENCV_FFMPEG_CAPTURE_OPTIONS"]="hwaccel;cuvid|video_codec;h264_cuvid|vsync;0"
stream = cv2.VideoCapture(self.rtsp_server_uri, cv2.CAP_FFMPEG)

From all the documentation I found, it seems like I need to change to something like this:

os.environ[ "OPENCV_FFMPEG_CAPTURE_OPTIONS"] = "hw_decoders_any;cuda|vsync;0"
stream = cv2.VideoCapture(self.rtsp_server_uri, cv2.CAP_FFMPEG, (cv2.CAP_PROP_HW_ACCELERATION, cv2.VIDEO_ACCELERATION_ANY))

But nothing seems to work.

Edit: confirmed after re-compiling 4.5.1 release that it is back to working normally. This feature on 4.5.2 is breaking things and/or poorly documented.

1 Like
2

I would suggest filing an issue on github and referencing the pull request for the new feature.

3

Did you try

OPENCV_FFMPEG_CAPTURE_OPTIONS="hw_decoders_any;vaapi,vdpau"

I haven’t had chance to yet buy I would be intereseted to see if it is really broken? Are you on Windows or Linux?

1 Like
4

Thanks, I was thinking about doing this but first wanted to make sure I didn’t miss anything by posting here first. I read all the docs I could find and tried them all.
@cudawarped: I am on linux running python. The syntax you have above is for the C API. Yes I tried this on python and it doesn’t work. These are not the codecs I want to use either as I am trying to use the cuda hevc and h264 codecs. It doesn’t even error out. It seem to establish the connection ok but decodes nothing and doesn’t return any video stream, occasionally posting a scrambled image which I suspect is because it is not using hardware decoding or it is using the wrong codec for the h264 streams. It never returns anything for hevc streams.

5

Setting ‘video_codec’ parameter via environment variable

os.environ["OPENCV_FFMPEG_CAPTURE_OPTIONS"]="video_codec;h264_cuvid"

should work in 4.5.2, assuming VideoCapture initialized without new property ‘CAP_PROP_HW_ACCELERATION’ introduced in OpenCV 4.5.2.
It works in my quick check with input from local file.
Here is corresponding code handling this property.
Can you please check on local file, not RTSP server URL?

Starting OpenCV 4.5.2, for acceleration types listed in enum VideoAccelerationType you can request specific HW acceleration (ex, VIDEO_ACCELERATION_D3D11 on Windows) or any HW acceleration (VIDEO_ACCELERATION_ANY) via property CAP_PROP_HW_ACCELERATION.

VideoCapture cap(filename, cv::CAP_ANY,
      { CAP_PROP_HW_ACCELERATION, cv::VIDEO_ACCELERATION_ANY }
);

For acceleration types not listed in enum VideoAccelerationType, experimental parameter in environment variable, example

os.environ[ "OPENCV_FFMPEG_CAPTURE_OPTIONS"] = "hw_decoders_any;cuda"

should work if open VideoCapture with property { CAP_PROP_HW_ACCELERATION, cv::VIDEO_ACCELERATION_ANY} and assuming FFMPEG was build with corresponding codecs (CUDA in this example).
Note that CUDA codecs are different to CUVID, for some reason CUVID not exposed in FFMPEG as acceleration type (enum AVHWDeviceType).

2 Likes
6

Thanks for the quick response but unfortunately I tried most of this as I explained in my first post.

  1. This is equivalent to my default setting from 4.5.1 and no it doesn’t work with an rtsp url. I don’t have any local file to test this on. I will try to create one but I suspect that it won’t work. It does establish connection to the rtsp url fine. It just doesn’t decode properly sending either a black image or a scrambled image.

  2. This is the C API obviously and I posted above the Python equivalent. Again as I said, I tried that too and it doesn’t work

  3. You posted exactly what I said I tried in my original post and it doesn’t work. The result is the same. Connection is established but decoding fails for h264. For HEVC it doesn’t even return anything.

7

I finally got time to try this on 4.5.2 and

 os.environ["OPENCV_FFMPEG_CAPTURE_OPTIONS"] = "video_codec;h264_cuvid"

still works on windows, but

os.environ[ "OPENCV_FFMPEG_CAPTURE_OPTIONS"] = "hw_decoders_any;cuda"

doesn’t seem to have an effect.