How can i convert sift keypoint.pt to Point2f in python?

Python
1

I’m trying to use RANSAC to prune false match after SIFT match.

cv.getPerspectiveTransform()

method requires Point2f data ,and in python i got a tuple.

Here is my code

import cv2
import numpy as np


def ransac(pts1, pts2, img_l, img_r, max_iters=500, epsilon=1):
    best_matches = []
    # Number of samples
    N = 4

    for i in range(max_iters):
        # Get 4 random samples from features
        id1 = np.random.randint(0, len(pts1), N)
        id2 = np.random.randint(0, len(pts2), N)
        src = []
        dst = []
        for i in range(N):
            src.append(pts1[id1[i]].pt)
            dst.append(pts2[id2[i]].pt)

        src = np.mat(src)
        dst = np.mat(dst)

        # Calculate the homography matrix H
        H = cv2.getPerspectiveTransform(src, dst)
        Hp = cv2.perspectiveTransform(pts1[None], H)[0]

        # Find the inliers by computing the SSD(p',Hp) and saving inliers (feature pairs) that are SSD(p',Hp) < epsilon
        inliers = []
        for i in range(len(pts1)):
            ssd = np.sum(np.square(pts2[i] - Hp[i]))
            if ssd < epsilon:
                inliers.append([pts1[i], pts2[i]])

        # Keep the largest set of inliers and the corresponding homography matrix
        if len(inliers) > len(best_matches):
            best_matches = inliers

    return best_matches
2

please highlight, where the current problem is.
(e.g. KeyPoint.pt is already a Point2f)

everything below that looks broken. you probably should convert all keypoints to 2d points before (not only 4), and do calculations on those, not the actual kp.

1 Like
3

proposal (untested !):

def ransac(pts1, pts2, img_l, img_r, max_iters=500, epsilon=1):
    p1 = cv2.convert(pts1)
    p2 = cv2.convert(pts2)

    best_matches = []
    # Number of samples
    N = 4

    for i in range(max_iters):
        # Get 4 random samples from features
        id1 = np.random.randint(0, len(pts1), N)
        id2 = np.random.randint(0, len(pts2), N)
        src = np.array([p1[i] for i in id1], dtype=np.float32)
        dst = np.array([p2[i] for i in id2], dtype=np.float32)
        # Calculate the homography matrix H
        H = cv2.getPerspectiveTransform(src, dst)
        Hp = cv2.perspectiveTransform(p1, H)

        # Find the inliers by computing the SSD(p',Hp) and saving inliers (feature pairs) that are SSD(p',Hp) < epsilon
        inliers = []
        for i in range(len(p1)):
            ssd = np.sum(np.square(p2[i] - Hp[i]))
            if ssd < epsilon:
                inliers.append([p1[i], p2[i]])

        # Keep the largest set of inliers and the corresponding homography matrix
        if len(inliers) > len(best_matches):
            best_matches = inliers

    return best_matches

however, i dont think it will work properly !
imo, there are 2 broken assumptions here:

        for i in range(len(pts1)):

what if len(p1) > len(p2) ?

            ssd = np.sum(np.square(p2[i] - Hp[i]))

you wrongly assume, that p2[i] is the corresponding point to the projected p1[i], however, it’s an arbitrary kp from the p2 set, you’re comparing apples to pears

IF you did some descriptor matching before, you should re-use the matches here

1 Like
4

Thanks for replying ! And pointing out the problems ,I’ll fix those problems. I didn’t think much about this approach . I cloned this codes from this repository , code it wrong . I did sift descriptor matching before this , but the result is bad (I only use 2 images for test), only 1 of 6 pairs is the correct match . But I think I got your point. Your proposal and explanation are very helpful ! Also I looked the doc you provide , The convert function is a static function , like this cv2.KeyPoint.convert().