Is it possible to combine SIFT and ORB descriptors?

Python
1

I need to combine SIFT and ORB descriptors of an image.

As you know, SIFT descriptors are of 128-length and ORB descriptors are of 32-length.

A possible way that I have thought is :

  1. Reshaping SIFT descriptors to 32-length. For instance, reshape a (135, 128) descriptor to a (540, 32) descriptor
  2. Concatenating SIFT and ORB descriptors (since at this moment both have 32-length)

Code:

sift_kp, sift_desc = sift.detectAndCompute(img,None)
new_sift_desc = sift_desc.reshape((int(128/32) * sift_desc.shape[0], 32))
orb_kp, orb_img_descriptor = orb.detectAndCompute(img,None)
all_descriptors = np.concatenate((new_sift_desc , orb_img_descriptor), axis=0)

After combinating the descriptors, the idea is to use all_descriptors in order to perform feature matching against another image.

The problem I find with this approach is that binary descriptors (like ORB) and classical ones (like SIFT) use different types of distance (Hamming vs Euclidean).

Thus, I don’t know if it is possible to convert SIFT descriptors into binary or viceversa.
Or if there is a better way to combine these descriptors.

2

none of it is possible. it makes no sense to combine float & binary descriptors
(you already found out about the different distance types)

can you tell us, why you want to do this ?

3

It is for a project in which I am asked to combine two types of descriptors.

Is it possible to use SIFT for searching keypoints and then ORB for computing the descriptors (from those keypoints)?

4

“combine” can mean a lot of things. what is the entire task description and who asks this of you?

yes, that’s possible. the Feature2D API has detect, compute, and detectAndCompute.

you want to use detect to find those salient points, and compute to compute descriptors for the points you have.

SIFT paper: https://www.cs.ubc.ca/~lowe/papers/iccv99.pdf
SIFT detects its points using a difference of gaussians, calculated using a pyramid.

ORB uses the FAST feature detector. Oriented FAST and rotated BRIEF - Wikipedia

OpenCV also has goodFeaturesToTrack. cba to look that up rn.

there are a bunch more approaches to “getting” interesting points/corners.

5

related: