How to modify the FPS of a video when using cv::VideoCapture to read frames

Using cv::VideoCapture to process videos. Wondering what the technique is for changing the frame rate when I write out a new video. I assume I could do something simple like drop every 2nd frame to cut from 60 FPS to 30 FPS.

But is that the right way to do it?
Does that mean if I want to go from 30 FPS to 20 FPS I drop every 3rd frame?

Seems too simple. Is there any interpolation or re-interpretation of the frames that needs to happen between frames to make sure everything is smooth?

So explain to me how it works then, if 2/3 * 30 != 20.

But regardless of the number, I suspect that simply dropping every Nth frame is not the way to do it. What is the correct way to resample or change the FPS for video files?

There are different approaches, simpler where frames are dropped or duplicated and more complex where frames can be interpolated. This functionality is built in many specialized video-processing libraries (e.g. FFmpeg - ChangingFrameRate – FFmpeg, https://www.reddit.com/r/ffmpeg/comments/dai5f1/smart_resampling_with_ffmpeg/).

There are no ready-to-use functions for this in the OpenCV, but I believe it is possible to implement it using existing functionality.

1 Like

I guess it should be noted that OpenCV is a CV tool and the idea is not to produce things that are nice to watch for humans, but video frames that are easy for computers to do things with - and any interpolation might not help with that

To be very clear: I wasn’t in any way expecting OpenCV to do this for me. I just happen to use OpenCV for most of my image manipulation, and I’m decently good at it. Since I do use the cv::VideoCapture class to read video files, I figured I would ask if anyone happened to know the name or the technique used to alter the FPS.

depends on what effect you want to achieve.

say you have one minute of video at 30 fps.

do you want

  • two minutes at 15 fps: same number of frames, just played slower
  • one minute at 15 fps: same duration but half the frames are dropped (not shown)
  • two minutes at 30 fps: double the frame count, interpolated frames are inserted

?

Let’s say I want my 1 minute @ 30 FPS video to be converted to 1 minute @ 25 FPS.

okay, you have the choice of

  • keeping 5 of every 6 frames, and dropping the 6th, which looks jittery/jerky
  • interpolating new frames at precise times, taking “adjacent” frames from the source and interpolating linearly, which causes time-varying exposure, i.e. weird motion blur
  • interpolating, but using using optical flow to calculate the frames (“state of the art”)