Inconsistent results with `recoverPose`

6

Here’s a video showing the comparison between consecutive frames:
results.mp4 - Google Drive. Note: original video is 60 fps but this one I uploaded is at 1 fps for convenience.

I changes to the Brute force matching algorithm to keep it simple and have less parameters to tune

matcher = cv.BFMatcher()
matches = matcher.knnMatch(descriptors0, descriptors1, k=2)

good_matches = []
for m, n in matches:
   if m.distance < 0.7* n.distance:
      good_matches.append(m)

Then the camera pose is estimated with:

def camera_pose(keypoints0, keypoints1, matches, camera_matrix):

    points0 = [keypoints0[i.queryIdx].pt for i in matches]
    points1 = [keypoints1[i.trainIdx].pt for i in matches]

    points0 = np.asarray(points0)
    points1 = np.asarray(points1)

    E, mask_inliers = cv.findEssentialMat(
        points1=points0,
        points2=points1,
        cameraMatrix=camera_matrix,
        method=cv.RANSAC,
        prob=0.99,  # default 0.999
        threshold=1.0,  # default 1.0
    )

    inliers0 = np.asarray(points0)
    inliers1 = np.asarray(points1)

    _, R, t, _ = cv.recoverPose(
        E=E,
        points1=inliers0,
        points2=inliers1,
        cameraMatrix=camera_matrix,
        mask=mask_inliers,
    )

    return (R, t)

In the video I wrote the value for R. Sometimes it is as expected [[1, 0, 0], [0, 1, 0], [0, 0, 1]] but often it is not.

Code is here in case test-camera-pose (github.com)