Help with algorithm for describing camera movement

I am extracting frames in a uniform way from a video recorded with a camera, which goes through rooms or landscapes, for later use.
The problem is that when extracting frames uniformly, if the camera has moved too fast or too slow in some sections the frame distribution can be relatively bad. The objects in the videos are usually still. I have tried to measure that displacement using optical flow between images but I have the feeling that this is more focused on measuring the displacement of objects within a video and not so much the camera displacement itself (although in my head they should be relatively analogous things).
Could someone point me in the right direction?
I know that ideally this distinction should be made at the time the frames are pulled and not after the fact, but for this test that check is sufficient.