Generating descriptors from keypoints using SIFT

derekdoth · February 16, 2023, 1:11am

I’m attempting to generate descriptors from a set of keypoints for an image of interest using SIFT. Something like this:

import cv2 as cv
import numpy as np

img = np.random.randn(1024,128).astype(np.uint8)

keypoints=[]
keypoints.append(cv.KeyPoint(x=100,y=100,size=1.0))
keypoints.append(cv.KeyPoint(x=500,y=50,size=1.0))
keypoints = np.asarray(keypoints)

sift = cv.SIFT_create()
keypoints, descriptors = sift.compute(img, keypoints)

And I get the following error:

error: OpenCV(4.7.0-dev) error: (-5:Bad argument) in function ‘compute’
Overload resolution failed:

descriptors data type = 17 is not supported

Expected Ptrcv::UMat for argument ‘descriptors’

Can’t parse ‘descriptors’. Sequence item with index 0 has a wrong type

Can’t parse ‘descriptors’. Sequence item with index 0 has a wrong type

Can anyone help me understand what I’m doing wrong?

laurent.berger · February 16, 2023, 8:06am

Hi,
You can read this tutorial :

sift = cv.SIFT_create()
kp, des = sift.detectAndCompute(gray,None)

Here kp will be a list of keypoints and des is a numpy array of shape (Number of Keypoints)×128.

So we got keypoints, descriptors etc.

Now your code is correct and I have got no error using copy and paste:

Python 3.10.8 (tags/v3.10.8:aaaf517, Oct 11 2022, 16:50:30) [MSC v.1933 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import cv2 as cv
>>> import numpy as np
>>>
>>> img = np.random.randn(1024,128).astype(np.uint8)
>>>
>>> keypoints=[]
>>> keypoints.append(cv.KeyPoint(x=100,y=100,size=1.0))
>>> keypoints.append(cv.KeyPoint(x=500,y=50,size=1.0))
>>> keypoints = np.asarray(keypoints)
>>>
>>> sift = cv.SIFT_create()
>>> keypoints, descriptors = sift.compute(img, keypoints)

You have to install opencv or opencv_contrib not both

crackwitz · February 16, 2023, 8:16am

what you’re trying to do looks reasonable.

my guess is, that’s wrong. the usual data type = 17 is not supported says so. it expects a python list of KeyPoints, not a numpy array of dtype=object containing KeyPoints.

just pass the list. should work.

derekdoth · February 16, 2023, 3:31pm

Thanks! That’s actually what I attempted first, and it gave a different error:

error: OpenCV(4.7.0-dev) error: (-5:Bad argument) in function ‘compute’
Overload resolution failed:

descriptors is not a numpy array, neither a scalar

Expected Ptrcv::UMat for argument ‘descriptors’

Can’t parse ‘descriptors’. Sequence item with index 0 has a wrong type

Can’t parse ‘descriptors’. Sequence item with index 0 has a wrong type

derekdoth · February 16, 2023, 3:34pm

Hmm. Strange. Well, I’m glad to know it SHOULD work!
I compiled opencv from source code because it wasn’t available through pip install. I’m not sure if that has anything to do with it. Or maybe there’s a problem with my particular version (4.7.0-dev). What version of opencv are you using?

berak · February 16, 2023, 4:06pm

hmm, tested with both 4.6.0 & 4.7.0 (from pip, py3.8 on colab), no problem
(with or without np.asarray, no difference)

however, (self-built) 4.7.0-dev shows above error !

descriptors is not a numpy array, neither a scalar(w/o asarray)
descriptors data type = 17 is not supported (with asarray)
(please also note, that it’s about the descriptors, not the keypoints !)

will inquire further, but this looks like a regression / bug to me, now.
@derekdoth , maybe you can help solving the mystery
– you still have the build files around ? the actual generated c++ wrapper code should be in build/modules/python_bindings_generator/pyopencv_generated_funcs.h

(look for pyopencv_cv_Feature2D_compute)

also, since this is from base class Feature2D, it should be reproducable also with ORB, AKAZE, etc !

laurent.berger · February 16, 2023, 4:18pm

>>> print(cv.getBuildInformation())

General configuration for OpenCV 4.7.0-dev =====================================
  Version control:               4.7.0-18-ga9d02f096e

  Extra modules:
    Location (extra):            G:/Lib/opencv_contrib/modules
    Version control (extra):     4.7.0-4-g9331902e

  Platform:
    Timestamp:                   2022-12-19T20:18:40Z
    Host:                        Windows 10.0.19044 AMD64
    CMake:                       3.24.2
    CMake generator:             Visual Studio 17 2022
    CMake build tool:            C:/Program Files/Microsoft Visual Studio/2022/Community/MSBuild/Current/Bin/amd64/MSBuild.exe
    MSVC:                        1933
    Configuration:               Debug Release

laurent.berger · February 16, 2023, 4:27pm

I built open 13 january. if there is a bug then it’s afer

berak · February 16, 2023, 4:29pm

@laurent.berger mine is:

  Version control:               4.7.0-85-g58d8a2702a

(later than yours, right ?)

derekdoth · February 16, 2023, 4:33pm

I do still have the build files!
The header file is there, but I can’t find pyopencv_cv_Feature2D_compute in it.

berak · February 16, 2023, 4:37pm

sorry, my bad,
pyopencv_generated_types_contents.h (not funcs)
but please, look again !

anyway, errors reproduced there with ORB instead of SIFT

laurent.berger · February 16, 2023, 4:44pm

yes
If it’s orb too may be it’s this PR feat: named arguments handling in Python interface · opencv/opencv@b07031b · GitHub

derekdoth · February 16, 2023, 4:44pm

Found it!

static PyObject* pyopencv_cv_Feature2D_compute(PyObject* self, PyObject* py_args, PyObject* kw)
{
    using namespace cv;


    Ptr<cv::Feature2D> * self1 = 0;
    if (!pyopencv_Feature2D_getp(self, self1))
        return failmsgp("Incorrect type of self (must be 'Feature2D' or its derivative)");
    Ptr<cv::Feature2D> _self_ = *(self1);
    pyPrepareArgumentConversionErrorsStorage(4);

    {
    PyObject* pyobj_image = NULL;
    Mat image;
    vector_KeyPoint keypoints;
    PyObject* pyobj_descriptors = NULL;
    Mat descriptors;

    const char* keywords[] = { "image", "descriptors", NULL };
    if( PyArg_ParseTupleAndKeywords(py_args, kw, "O|O:Feature2D.compute", (char**)keywords, &pyobj_image, &pyobj_descriptors) &&
        pyopencv_to_safe(pyobj_image, image, ArgInfo("image", 0)) &&
        pyopencv_to_safe(pyobj_descriptors, descriptors, ArgInfo("descriptors", 1)) )
    {
        ERRWRAP2(_self_->compute(image, keypoints, descriptors));
        return Py_BuildValue("(NN)", pyopencv_from(keypoints), pyopencv_from(descriptors));
    }


        pyPopulateArgumentConversionErrors();
    }

    {
    PyObject* pyobj_image = NULL;
    UMat image;
    vector_KeyPoint keypoints;
    PyObject* pyobj_descriptors = NULL;
    UMat descriptors;

    const char* keywords[] = { "image", "descriptors", NULL };
    if( PyArg_ParseTupleAndKeywords(py_args, kw, "O|O:Feature2D.compute", (char**)keywords, &pyobj_image, &pyobj_descriptors) &&
        pyopencv_to_safe(pyobj_image, image, ArgInfo("image", 0)) &&
        pyopencv_to_safe(pyobj_descriptors, descriptors, ArgInfo("descriptors", 1)) )
    {
        ERRWRAP2(_self_->compute(image, keypoints, descriptors));
        return Py_BuildValue("(NN)", pyopencv_from(keypoints), pyopencv_from(descriptors));
    }


        pyPopulateArgumentConversionErrors();
    }


    {
    PyObject* pyobj_images = NULL;
    vector_Mat images;
    vector_vector_KeyPoint keypoints;
    PyObject* pyobj_descriptors = NULL;
    vector_Mat descriptors;

    const char* keywords[] = { "images", "descriptors", NULL };
    if( PyArg_ParseTupleAndKeywords(py_args, kw, "O|O:Feature2D.compute", (char**)keywords, &pyobj_images, &pyobj_descriptors) &&
        pyopencv_to_safe(pyobj_images, images, ArgInfo("images", 0)) &&
        pyopencv_to_safe(pyobj_descriptors, descriptors, ArgInfo("descriptors", 1)) )
    {
        ERRWRAP2(_self_->compute(images, keypoints, descriptors));
        return Py_BuildValue("(NN)", pyopencv_from(keypoints), pyopencv_from(descriptors));
    }

        pyPopulateArgumentConversionErrors();
    }


    {
    PyObject* pyobj_images = NULL;
    vector_UMat images;
    vector_vector_KeyPoint keypoints;
    PyObject* pyobj_descriptors = NULL;
    vector_UMat descriptors;

    const char* keywords[] = { "images", "descriptors", NULL };
    if( PyArg_ParseTupleAndKeywords(py_args, kw, "O|O:Feature2D.compute", (char**)keywords, &pyobj_images, &pyobj_descriptors) &&
        pyopencv_to_safe(pyobj_images, images, ArgInfo("images", 0)) &&
        pyopencv_to_safe(pyobj_descriptors, descriptors, ArgInfo("descriptors", 1)) )
    {
        ERRWRAP2(_self_->compute(images, keypoints, descriptors));
        return Py_BuildValue("(NN)", pyopencv_from(keypoints), pyopencv_from(descriptors));
    }


        pyPopulateArgumentConversionErrors();
    }
    pyRaiseCVOverloadException("compute");

    return NULL;
}

berak · February 16, 2023, 5:02pm

lovely

here’s the older (~4.6.0) one:


static PyObject* pyopencv_cv_Feature2D_compute(PyObject* self, PyObject* py_args, PyObject* kw)
{
    using namespace cv;


    Ptr<cv::Feature2D> * self1 = 0;
    if (!pyopencv_Feature2D_getp(self, self1))
        return failmsgp("Incorrect type of self (must be 'Feature2D' or its derivative)");
    Ptr<cv::Feature2D> _self_ = *(self1);
    pyPrepareArgumentConversionErrorsStorage(4);

    {
    PyObject* pyobj_image = NULL;
    Mat image;
    PyObject* pyobj_keypoints = NULL;
    vector_KeyPoint keypoints;
    PyObject* pyobj_descriptors = NULL;
    Mat descriptors;

    const char* keywords[] = { "image", "keypoints", "descriptors", NULL };
    if( PyArg_ParseTupleAndKeywords(py_args, kw, "OO|O:Feature2D.compute", (char**)keywords, &pyobj_image, &pyobj_keypoints, &pyobj_descriptors) &&
        pyopencv_to_safe(pyobj_image, image, ArgInfo("image", 0)) &&
        pyopencv_to_safe(pyobj_keypoints, keypoints, ArgInfo("keypoints", 1)) &&
        pyopencv_to_safe(pyobj_descriptors, descriptors, ArgInfo("descriptors", 1)) )
    {
        ERRWRAP2(_self_->compute(image, keypoints, descriptors));
        return Py_BuildValue("(NN)", pyopencv_from(keypoints), pyopencv_from(descriptors));
    }


        pyPopulateArgumentConversionErrors();
    }
    

    {
    PyObject* pyobj_image = NULL;
    UMat image;
    PyObject* pyobj_keypoints = NULL;
    vector_KeyPoint keypoints;
    PyObject* pyobj_descriptors = NULL;
    UMat descriptors;

    const char* keywords[] = { "image", "keypoints", "descriptors", NULL };
    if( PyArg_ParseTupleAndKeywords(py_args, kw, "OO|O:Feature2D.compute", (char**)keywords, &pyobj_image, &pyobj_keypoints, &pyobj_descriptors) &&
        pyopencv_to_safe(pyobj_image, image, ArgInfo("image", 0)) &&
        pyopencv_to_safe(pyobj_keypoints, keypoints, ArgInfo("keypoints", 1)) &&
        pyopencv_to_safe(pyobj_descriptors, descriptors, ArgInfo("descriptors", 1)) )
    {
        ERRWRAP2(_self_->compute(image, keypoints, descriptors));
        return Py_BuildValue("(NN)", pyopencv_from(keypoints), pyopencv_from(descriptors));
    }


        pyPopulateArgumentConversionErrors();
    }
    

    {
    PyObject* pyobj_images = NULL;
    vector_Mat images;
    PyObject* pyobj_keypoints = NULL;
    vector_vector_KeyPoint keypoints;
    PyObject* pyobj_descriptors = NULL;
    vector_Mat descriptors;

    const char* keywords[] = { "images", "keypoints", "descriptors", NULL };
    if( PyArg_ParseTupleAndKeywords(py_args, kw, "OO|O:Feature2D.compute", (char**)keywords, &pyobj_images, &pyobj_keypoints, &pyobj_descriptors) &&
        pyopencv_to_safe(pyobj_images, images, ArgInfo("images", 0)) &&
        pyopencv_to_safe(pyobj_keypoints, keypoints, ArgInfo("keypoints", 1)) &&
        pyopencv_to_safe(pyobj_descriptors, descriptors, ArgInfo("descriptors", 1)) )
    {
        ERRWRAP2(_self_->compute(images, keypoints, descriptors));
        return Py_BuildValue("(NN)", pyopencv_from(keypoints), pyopencv_from(descriptors));
    }


        pyPopulateArgumentConversionErrors();
    }
    

    {
    PyObject* pyobj_images = NULL;
    vector_UMat images;
    PyObject* pyobj_keypoints = NULL;
    vector_vector_KeyPoint keypoints;
    PyObject* pyobj_descriptors = NULL;
    vector_UMat descriptors;

    const char* keywords[] = { "images", "keypoints", "descriptors", NULL };
    if( PyArg_ParseTupleAndKeywords(py_args, kw, "OO|O:Feature2D.compute", (char**)keywords, &pyobj_images, &pyobj_keypoints, &pyobj_descriptors) &&
        pyopencv_to_safe(pyobj_images, images, ArgInfo("images", 0)) &&
        pyopencv_to_safe(pyobj_keypoints, keypoints, ArgInfo("keypoints", 1)) &&
        pyopencv_to_safe(pyobj_descriptors, descriptors, ArgInfo("descriptors", 1)) )
    {
        ERRWRAP2(_self_->compute(images, keypoints, descriptors));
        return Py_BuildValue("(NN)", pyopencv_from(keypoints), pyopencv_from(descriptors));
    }


        pyPopulateArgumentConversionErrors();
    }
    pyRaiseCVOverloadException("compute");

    return NULL;
}

if you take a sharp look, it differs wrt. input parsing:
old (mine):

if( PyArg_ParseTupleAndKeywords(py_args, kw, "OO|O:Feature2D.compute", (char**)keywords, &pyobj_image, &pyobj_keypoints, &pyobj_descriptors)

new (yours):

if( PyArg_ParseTupleAndKeywords(py_args, kw, "O|O:Feature2D.compute", (char**)keywords, &pyobj_image, &pyobj_descriptors)

also missing there:

 PyObject* pyobj_keypoints = NULL;

and:

 pyopencv_to_safe(pyobj_keypoints, keypoints, ArgInfo("keypoints", 1))

so, the new version has lost an (input) O in the format string, as well as &pyobj_keypoints in the following argslist !

this is clearly broken
please report to upper echelons

derekdoth · February 16, 2023, 5:18pm

I’ll do it! Thanks for your help!

berak · February 16, 2023, 5:42pm

@laurent.berger that might be it ! (??)

e.g. here:

if i read this correctly,
no code for an arg is generated, if arg_type_info.strict_conversion == False

(is it ??)

laurent.berger · February 16, 2023, 6:40pm

I can say that I remove change in :
modified: modules/core/include/opencv2/core/bindings_utils.hpp
modified: modules/python/src2/gen2.py
modified: modules/python/src2/hdr_parser.py
and now it works

Topic		Replies	Views
Calculate SIFT keypoints for all given points Python features2d , keypoint , sift	4	1472	March 24, 2023
What keypoint data have been used in SIFT descriptor? Python features2d	0	271	November 14, 2022
How to add or modify SIFT 's funtion features2d , xfeatures2d , contrib	13	730	January 27, 2021
SiftFeatureDetector no longer works in version 3.4 C++ features2d	3	1022	December 10, 2021
How can i convert sift keypoint.pt to Point2f in python? Python features2d	3	1530	September 26, 2022

Generating descriptors from keypoints using SIFT

Related topics