this looks like an ideal application for optical flow… enough lowpass/blurring might take care of the noise. optical flow is too sophisticated for this situation. try simple autocorrelation (convolution with itself), because I see just translation.
your application sounds like you might wanna use an optical mouse sensor.
edit: I just tried something using filter2D and minMaxLoc applied to adjacent frames. that kinda works. the issue is numerics. the procedure gives you integer values. integrating those up obviously also cumulates the rounding error. you could either upscale the frames, which gives you a few bits more of precision. or you could keep a “keyframe” to calculate against (instead of the previous one), and only update the keyframe if the shift has gotten large enough for the pictures to become ambiguous. I’d recommend less than half a tooth mark because at that point the shift becomes ambiguous (half tooth forward? half backward?). one downside to “keyframes” is that during fast-moving section, you’ll get frames with motion blur, and those are poor keyframes, so how you determine what’s a keyframe gets a little tricky.
filter2D is also fairly wasteful because it calculates the correlation for ALL POSSIBLE shifts. you’re only ever observing a small subset of those (very little in Y direction). it might pay to reach for numpy/scipy or even roll your own correlation loop (I’d recommend sum of squared differences) and accelerate with numba.
from the adjacent-frames approach with 4x upsampling, I get this: [486.75 3.25] the truth is closer to [490, -3] (manually picked) so there you see the issue.
a plot of the cumulative shift (in X):
edit: I played around with keyframes and assessing motion blur by what relative shift each got. resulting shift ([490.5 -2.5]) gets close to what I would pick in an image editor but eh… depending on the tweakables, it’s still varying by a pixel or so in X, and Y… is moving all over the place.
one side effect of working around motion blur is that it also works around rolling shutter, which is very evident during the quickly moving segments.
X position (positive = going left):
Y position (positive = going up):
have fun: ocv-forum-8983.ipynb · GitHub