CAP_PROP_POS_FRAMES is abnormal slow

Hi, After I had to reinstall my Anaconda version I have now an issue with jumping within a video file.

I am using vs.set(cv2.CAP_PROP_POS_FRAMES, framei) and it take really long time to perform. And it is relevant from where to where I jump.
Below you can find a test code show the problem.
First I just read in 600 frames ==> 11.78 s
Then I make trials with the jumps: from frame 0 to 600 ==> 6.6s
then from 600 to 1200 ==> 12.8s
Then I make the same but just with the retrieve command which is roughly half the time of reading.

Any idea why the vs.set(cv2.CAP_PROP_POS_FRAMES, framei) command is such slow?

Also just see the warning in the output with the GStreamer. even I don’t asked to do anything with it, I have this warning.

Setup is following:

  • Windows 11
  • Anaconda 2.3.1,
  • with an environment using Python 3.9.15
  • OpenCV 4.6.0, just installed in anaconda

Here the code:

import cv2
import time

filename = "E:/DaVinciResolveProjects/20221208_OffsetSystem/20230115_2157/20230115_2157.mkv"
vs = cv2.VideoCapture(filename)

############################################################
print('Task: read 600 frames')
framei = 0
tic = time.time()
while framei< 600:
    framei = framei + 1
    ret, frame = vs.read()
 
toc = time.time()    
print(str(round(toc-tic,3))+'s - '+str(round(600/(toc-tic),3))+'fps')

############################################################
print(' ')
print('Task: Jump from frame 0 to 600')

# rewind back to 0
framei = 0
vs.set(cv2.CAP_PROP_POS_FRAMES, framei)
tic = time.time()

# jump to 600
framei = 600
vs.set(cv2.CAP_PROP_POS_FRAMES, framei)
 
toc = time.time()
print(str(round(toc-tic,3))+'s - '+str(round(600/(toc-tic),3))+'fps')
    
############################################################
print(' ')
print('Task: Jump from frame 600 to 1200')
tic = time.time()
# now jump to 1200
framei = 1200
vs.set(cv2.CAP_PROP_POS_FRAMES, framei)
 
toc = time.time()
print(str(round(toc-tic,3))+'s - '+str(round(600/(toc-tic),3))+'fps')

############################################################
print(' ')
print('Task: Jump from frame 1200 to 1800')
tic = time.time()
# now jump to 1800
framei = 1800
vs.set(cv2.CAP_PROP_POS_FRAMES, framei)
 
toc = time.time()
print(str(round(toc-tic,3))+'s - '+str(round(600/(toc-tic),3))+'fps')

############################################################
print(' ')
print('Task: Jump back from frame 1200 to 0')
tic = time.time()
# now jump to 0
framei = 0
vs.set(cv2.CAP_PROP_POS_FRAMES, framei)
 
toc = time.time()
print(str(round(toc-tic,3))+'s - '+str(round(1200/(toc-tic),3))+'fps')

############################################################
print(' ')
print('Task: grab 600 frames')
framei = 0
tic = time.time()
while framei< 600:
    framei = framei + 1
    ret = vs.grab()
 
toc = time.time()    
print(str(round(toc-tic,3))+'s - '+str(round(600/(toc-tic),3))+'fps')

############################################################
print(' ')
print('Task: continue grab from frame 600-1200')
tic = time.time()
while framei< 1200:
    framei = framei + 1
    ret = vs.grab()
    frame = vs.retrieve()
 
toc = time.time()    
print(str(round(toc-tic,3))+'s - '+str(round(600/(toc-tic),3))+'fps')

This is the output:

[ WARN:0@7115.629] global C:\b\abs_74oeeuevib\croots\recipe\opencv-suite_1664548340488\work\modules\videoio\src\cap_gstreamer.cpp (2386) cv::handleMessage OpenCV | GStreamer warning: your GStreamer installation is missing a required plugin
[ WARN:0@7115.629] global C:\b\abs_74oeeuevib\croots\recipe\opencv-suite_1664548340488\work\modules\videoio\src\cap_gstreamer.cpp (2402) cv::handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module uridecodebin22 reported: Your GStreamer installation is missing a plug-in.
[ WARN:0@7115.629] global C:\b\abs_74oeeuevib\croots\recipe\opencv-suite_1664548340488\work\modules\videoio\src\cap_gstreamer.cpp (1356) cv::GStreamerCapture::open OpenCV | GStreamer warning: unable to start pipeline
[ WARN:0@7115.629] global C:\b\abs_74oeeuevib\croots\recipe\opencv-suite_1664548340488\work\modules\videoio\src\cap_gstreamer.cpp (862) cv::GStreamerCapture::isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
Task: read 600 frames
11.954s - 50.192fps
 
Task: Jump from frame 0 to 600
6.765s - 88.698fps
 
Task: Jump from frame 600 to 1200
13.195s - 45.473fps
 
Task: Jump from frame 1200 to 1800
19.359s - 30.993fps
 
Task: Jump back from frame 1200 to 0
0.056s - 21424.104fps
 
Task: grab 600 frames
6.892s - 87.063fps
 
Task: continue grab from frame 600-1200
11.251s - 53.331fps

I have made some further investigations.

The code first reads 600 frames to check general read performance. Then it should make jumps of 600 frames. I see that the times for the jumps take longer and longer.

Here I will post now several different combinations of Python and OpenCV versions:

Python-Version 3.10.8 + OpenCV-Version 4.6.0:
Read 600 frames ==> 12.02s
5x600 jumps ==>67.5 s

Python-Version 3.9.15 + OpenCV-Version 4.6.0:
Read 600 frames ==> 11.7s
5x600 jumps ==>65.5 s

Python-Version 3.9.15 + OpenCV-Version 4.5.5:
Read 600 frames ==> 11.5 s
5x600 jumps ==>64.6 s

Python-Version 3.8.15 + OpenCV-Version 4.5.4:
Read 600 frames ==> 11.97s
5x600 jumps ==>6.24 s

Python-Version 3.8.15 + OpenCV-Version 4.0.1:
Read 600 frames ==> 10.391s
5x600 jumps ==>2.078 s

Python-Version 3.6.13 + OpenCV-Version 3.4.2:
Read 600 frames ==> 9.1s
5x600 jumps ==>2.019 s

I have found my sort tearm solution…
it is Python-Version 3.8.15 + OpenCV-Version 4.0.1, AND I also dont get this GSTreamer warning message.

But I am curious how everybody is doing these jumps. Especially if the footage is long??

import cv2
import time
import platform

print('Python-Version '+platform.sys.version)
print('OpenCV-Version '+cv2.__version__)

filename = "20230115_2157.mkv"
vs = cv2.VideoCapture(filename)

############################################################
print('Task: read 600 frames')
framei = 0
tic = time.time()
while framei< 600:
    framei = framei + 1
    ret, frame = vs.read()
    #cv2.imshow("Frame", frame)
    #key = cv2.waitKey(1) & 0xFF
 
toc = time.time()    
print(str(round(toc-tic,3))+'s - '+str(round(600/(toc-tic+0.001),3))+'fps')


############################################################
print(' ')
tic0 = time.time()
framei = 0
reps = 0
while reps < 5:
    tic = time.time()
    vs.set(cv2.CAP_PROP_POS_FRAMES, framei)
    framei = framei + 600
    reps = reps + 1
    toc = time.time()
    print('Jump to frame '+str(framei)+': '+str(round(toc-tic,3))+'s - '+str(round(600/(toc-tic+0.001),3))+'fps')

print('total time 5x600 frames jump: '+str(round(toc-tic0,3))+'s')


Python-Version 3.9.15 (main, Nov 24 2022, 14:39:17) [MSC v.1916 64 bit (AMD64)]
OpenCV-Version 4.5.5
Task: read 600 frames

[ WARN:0@1047.406] global C:\Windows\Temp\abs_0aavye6esb\croots\recipe\opencv-suite_1659973594283\work\modules\videoio\src\cap_gstreamer.cpp (2386) cv::handleMessage OpenCV | GStreamer warning: your GStreamer installation is missing a required plugin
[ WARN:0@1047.406] global C:\Windows\Temp\abs_0aavye6esb\croots\recipe\opencv-suite_1659973594283\work\modules\videoio\src\cap_gstreamer.cpp (2402) cv::handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module uridecodebin3 reported: Your GStreamer installation is missing a plug-in.
[ WARN:0@1047.406] global C:\Windows\Temp\abs_0aavye6esb\croots\recipe\opencv-suite_1659973594283\work\modules\videoio\src\cap_gstreamer.cpp (1356) cv::GStreamerCapture::open OpenCV | GStreamer warning: unable to start pipeline
[ WARN:0@1047.406] global C:\Windows\Temp\abs_0aavye6esb\croots\recipe\opencv-suite_1659973594283\work\modules\videoio\src\cap_gstreamer.cpp (862) cv::GStreamerCapture::isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
11.512s - 52.115fps

Jump to frame 600: 0.025s - 23070.915fps
Jump to frame 1200: 6.437s - 93.196fps
Jump to frame 1800: 12.925s - 46.419fps
Jump to frame 2400: 19.554s - 30.682fps
Jump to frame 3000: 25.651s - 23.39fps
total time 5x600 frames jump: 64.594s

setting CAP_PROP_POS_FRAMES is called seeking in the video file. for most video files (formats), this is either slow or only able to jump to specific frames (you can call them “keyframes”)

please learn more about it.

your video file appears to have extremely long “Groups of Pictures (GOP)” or it’s severely fragmented and stored on a HDD. analyze it. ffprobe with various command line switches lets you analyze individual packets and frames.

I will learn about GOPs…
GOP explanation:

python code to read out GOP:

Meanwhile I have checked GOP structure: see below. 250 frames are between keyframes. Is it extreme? I create the video using OBS Recording software and I can adjust the value to keyframe intervall to 1 sec which will be lead to GOP size 60.

File is stored on SSD.
How do you explain, the big difference in the seeking times then?
I mean there is on same hardware same file a huge difference. Please check the results

Python-Version 3.9.15 + OpenCV-Version 4.5.5:
Read 600 frames ==> 11.5 s
5x600 jumps ==>64.6 s

Python-Version 3.8.15 + OpenCV-Version 4.5.4:
Read 600 frames ==> 11.97s
5x600 jumps ==>6.24 s

Python-Version 3.8.15 + OpenCV-Version 4.0.1:
Read 600 frames ==> 10.391s
5x600 jumps ==>2.078 s

Seeking time for the versions > 4.5.5 is proportional to the frame number!
I would assume, that seeking for a frame at the beginning or at the end should not have big difference. But in that versions, it I can seek only with a 90 fps. So if the frame I want to lookup is the frame 9000, then it needs 10 seconds. Absolutely unacceptable. Something must be wrong.

GOP Structure (all GOPs are same):

GOP: IBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBPBBP 250 CLOSED

potentially.

can you test with v4.7.0? anything earlier than that is only helpful in determining a regression. if there was an issue, and it was fixed, it makes no sense at all to deal with old versions.

can you provide such a video file for which this issue happens?

I can not test with 4.7 since it is not available in Anaconda. May be I can conda it somehow, but I don’t know with which command.

The original file I tested with is 15GB. So I have created a smaller file with 170MB and 36 seconds duration.

I have copied my test code and the video file to a Google Drive:
https://drive.google.com/drive/folders/16kEoFWHFoljmPWkwRLu4dsCPc-EWJjFR?usp=sharing

I have tested with this video file, with unchanged performance:

Python-Version 3.10.8 | packaged by conda-forge | (main, Nov 24 2022, 14:07:00) [MSC v.1916 64 bit (AMD64)]
OpenCV-Version 4.6.0
Task: read 600 frames

[ WARN:0@0.108] global C:\b\abs_74oeeuevib\croots\recipe\opencv-suite_1664548340488\work\modules\videoio\src\cap_gstreamer.cpp (2386) cv::handleMessage OpenCV | GStreamer warning: your GStreamer installation is missing a required plugin
[ WARN:0@0.109] global C:\b\abs_74oeeuevib\croots\recipe\opencv-suite_1664548340488\work\modules\videoio\src\cap_gstreamer.cpp (2402) cv::handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module uridecodebin0 reported: Your GStreamer installation is missing a plug-in.
[ WARN:0@0.109] global C:\b\abs_74oeeuevib\croots\recipe\opencv-suite_1664548340488\work\modules\videoio\src\cap_gstreamer.cpp (1356) cv::GStreamerCapture::open OpenCV | GStreamer warning: unable to start pipeline
[ WARN:0@0.109] global C:\b\abs_74oeeuevib\croots\recipe\opencv-suite_1664548340488\work\modules\videoio\src\cap_gstreamer.cpp (862) cv::GStreamerCapture::isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
11.913s - 50.362fps

Jump to frame 400: 0.026s - 22217.103fps
Jump to frame 800: 4.64s - 129.281fps
Jump to frame 1200: 9.194s - 65.252fps
Jump to frame 1600: 13.565s - 44.227fps
Jump to frame 2000: 18.575s - 32.299fps
total time 5x400 frames jump: 46.001s