How to modify the FPS of a video when using cv::VideoCapture to read frames

C++
1

Using cv::VideoCapture to process videos. Wondering what the technique is for changing the frame rate when I write out a new video. I assume I could do something simple like drop every 2nd frame to cut from 60 FPS to 30 FPS.

But is that the right way to do it?
Does that mean if I want to go from 30 FPS to 20 FPS I drop every 3rd frame?

Seems too simple. Is there any interpolation or re-interpretation of the frames that needs to happen between frames to make sure everything is smooth?

3

So explain to me how it works then, if 2/3 * 30 != 20.

But regardless of the number, I suspect that simply dropping every Nth frame is not the way to do it. What is the correct way to resample or change the FPS for video files?

4

There are different approaches, simpler where frames are dropped or duplicated and more complex where frames can be interpolated. This functionality is built in many specialized video-processing libraries (e.g. FFmpeg - ChangingFrameRate – FFmpeg, https://www.reddit.com/r/ffmpeg/comments/dai5f1/smart_resampling_with_ffmpeg/).

There are no ready-to-use functions for this in the OpenCV, but I believe it is possible to implement it using existing functionality.

1 Like
5

I guess it should be noted that OpenCV is a CV tool and the idea is not to produce things that are nice to watch for humans, but video frames that are easy for computers to do things with - and any interpolation might not help with that

6

To be very clear: I wasn’t in any way expecting OpenCV to do this for me. I just happen to use OpenCV for most of my image manipulation, and I’m decently good at it. Since I do use the cv::VideoCapture class to read video files, I figured I would ask if anyone happened to know the name or the technique used to alter the FPS.

7

depends on what effect you want to achieve.

say you have one minute of video at 30 fps.

do you want

  • two minutes at 15 fps: same number of frames, just played slower
  • one minute at 15 fps: same duration but half the frames are dropped (not shown)
  • two minutes at 30 fps: double the frame count, interpolated frames are inserted

?

8

Let’s say I want my 1 minute @ 30 FPS video to be converted to 1 minute @ 25 FPS.

9

okay, you have the choice of

  • keeping 5 of every 6 frames, and dropping the 6th, which looks jittery/jerky
  • interpolating new frames at precise times, taking “adjacent” frames from the source and interpolating linearly, which causes time-varying exposure, i.e. weird motion blur
  • interpolating, but using using optical flow to calculate the frames (“state of the art”)