Excellent observation. I did have autofocus on. I am using this camera for testing purposes and I am not planning to using it long term. I retook some pics/videos with autofocus disabled now. A quick search revealed that OpenCV can be compiled to use the V4L2 backend. Good to know.
Thanks, I will try this next time.
This is the part I am most unsure about to be honest. I followed this tutorial OpenCV: Feature Matching and the Ratio test is brought up in the section Brute-Force Matching with SIFT Descriptors and Ratio Test. To simplify, I am now following the simpler approach described in the section Brute-Force Matching with ORB Descriptors. That is, I am using BFMatcher.match() to find the best matches and then I am keeping the 100 closest ones. This makes a bit more sense to me.
matcher = cv.BFMatcher(normType=cv.NORM_L2, crossCheck=True)
matches = matcher.match(descriptors0, descriptors1)
matches = sorted(matches, key = lambda x:x.distance)
matches - matches[:100]
EDIT: I found the justification for the ratio test. It is taken from the SIFT paper and there is a shorter description at the official OpenCV: Introduction to SIFT at paragraph 5. Keypoint Matching. From what I understand, if you use a KNN matcher with k=2, you want to exclude those 2 matches if they are too close to each other as it could be a symptom of noise.
The focus of the camera is set to infinity; I recalibrated it and recorded a new video. The results are ok most of the time, but sometimes they are still quite off. Perhaps, this is the expected performance. I just wonder how it will work in a real world test where videos may be much noisier and surroundings much more cluttered and dynamic. Looking at the video, it seems that the keypoints are matched correctly even when the rotation matrix is way off.
Video here results.mp4 - Google Drive (Hopefully accessible to anyone now)