Hi all,

I am currently using SURF with the FLANN descriptor, and I would like to test SURF in its GPU version.
But I don’t find any informations about FLANN with SURF GPU. Since FLANN use the descriptors function “detectAndCompute” and GPU SURF have not this function, how can I use FLANN with the GPU version ?

there is no FLANN matcher for gpu descriptors.
if you want to process tem from CUDA, you probably have to use the cuda::BFMatcher

Okay, but a matcher only takes keypoints right ? I saw the function doawnloadKeypoints(descriptor, gpu_keypoints), we can’t do something with this ?
In the flann based matcher page, I also saw some GPU implementations OpenCV: cv::FlannBasedMatcher Class Reference so I was wondering if it was possible to make it through

ah right, i missed that

So I thought the only thing I should change is the descriptor, make GpuMats (even if I dont know how it work atm) and then “download” the keypoints passed to flann. Does it make sense ?

this would give you GPUMats for keypoints & descriptors already
(counterpart to detectAndCompute() on cpu)

So I convert my img and mask first into GpuMats, then I apply operator like detectAndCompute ?

kp, des = surf.detectAndCompute(frame, mask)
gpu_frame = cv.cuda_GpuMat(frame)
gpu_mask = cv.cuda_GpuMat(mask)
kp, des = surf.operator(gpu_frame, gpu_mask)

cannot try (also not from python…), but no, more like:

cv::cuda::SURF_CUDA det = cv::cuda::SURF_CUDA::SURF_CUDA();

cv::GPUMat kp,desc;

Oh okay, but yes I am in python… But if you say it’s the counterpart of detectandcompute I assume it works the same :crossed_fingers: Thank you for your help

Hum, yes I don’t know how to implement this in python in fact ^^’
The only example I found in python with SURF CUDA is this post
[EDIT] And I just found this post for who wants

the ()() operator is probably wrapped as a function in python

try to create an instance and call help(surf) on it, then it’ll get pretty obvious, what to do :wink:

as a sidenote, the cuda python are only documented if cuda is available on the machine, that builds it, pity.

edit: your 1st link seems to be it !

1 Like
class cuda_SURF_CUDA(builtins.object)
 |  Methods defined here:
 |  __new__(*args, **kwargs) from builtins.type
 |      Create and return a new object.  See help(type) for accurate signature.
 |  __repr__(self, /)
 |      Return repr(self).
 |  defaultNorm(...)
 |      defaultNorm() -> retval
 |      .
 |  descriptorSize(...)
 |      descriptorSize() -> retval
 |      .
 |  detect(...)
 |      detect(img, mask[, keypoints]) -> keypoints
 |      .   @brief Finds the keypoints using fast hessian detector used in SURF
 |      .   
 |      .       @param img Source image, currently supports only CV_8UC1 images.
 |      .       @param mask A mask image same size as src and of type CV_8UC1.
 |      .       @param keypoints Detected keypoints.
 |  detectWithDescriptors(...)
 |      detectWithDescriptors(img, mask[, keypoints[, descriptors[, useProvidedKeypoints]]]) -> keypoints, descriptors
 |      .   @brief Finds the keypoints and computes their descriptors using fast hessian detector used in SURF
 |      .   
 |      .       @param img Source image, currently supports only CV_8UC1 images.
 |      .       @param mask A mask image same size as src and of type CV_8UC1.
 |      .       @param keypoints Detected keypoints.
 |      .       @param descriptors Keypoint descriptors.
 |      .       @param useProvidedKeypoints Compute descriptors for the user-provided keypoints and recompute keypoints direction.
 |  downloadKeypoints(...)
 |      downloadKeypoints(keypointsGPU) -> keypoints
 |      .
 |  ----------------------------------------------------------------------
 |  Static methods defined here:
 |  create(...)
 |      create(_hessianThreshold[, _nOctaves[, _nOctaveLayers[, _extended[, _keypointsRatio[, _upright]]]]]) -> retval
 |      .   @param _hessianThreshold Threshold for hessian keypoint detector used in SURF.
 |      .       @param _nOctaves Number of pyramid octaves the keypoint detector will use.
 |      .       @param _nOctaveLayers Number of octave layers within each octave.
 |      .       @param _extended Extended descriptor flag (true - use extended 128-element descriptors; false - use
 |      .       64-element descriptors).
 |      .       @param _keypointsRatio
 |      .       @param _upright Up-right or rotated features flag (true - do not compute orientation of features;
 |      .       false - compute orientation).
 |  ----------------------------------------------------------------------
 |  Data descriptors defined here:
 |  extended
 |      extended
 |  hessianThreshold
 |      hessianThreshold
 |  keypointsRatio
 |      keypointsRatio
 |  nOctaveLayers
 |      nOctaveLayers
 |  nOctaves
 |      nOctaves
 |  upright
 |      upright

Maybe I don’t create my GpuMat correctly. Because I followed the first link and I get this error :

kpGPU, des = surf.detectWithDescriptors(gpu_frame, gpu_mask)
cv2.error: OpenCV(4.4.0) C:\opencv-4.4.0\opencv_contrib-4.4.0\modules\xfeatures2d\src\surf.cuda.cpp:146: error: (-215:Assertion failed) !img.empty() && img.type() == CV_8UC1 in function '`anonymous-namespace'::SURF_CUDA_Invoker::SURF_CUDA_Invoker'

I declare my mask like so :

mask = np.zeros(frame.shape[:2], dtype=np.uint8)

# (these 4 lines create ROIs mask on a image taken as reference)
save = np.zeros(frame.shape[:2], dtype=np.uint8)
points = np.array([list(r.topleft), list(r.topright), list(r.bottomright), list(r.bottomleft)], np.int32)
cv.fillPoly(mask, [points], (255, 255, 255))
cv.bitwise_and(frame, frame, masked, mask)

gpu_mask = cv.cuda_GpuMat(mask)
gpu_frame = cv.cuda_GpuMat(frame)
kpGPU, des = surf.detectWithDescriptors(gpu_frame, gpu_mask)

[EDIT] For those who have the same error with detectWithDescriptors, the image needs to be converted into grayscale

1 Like

Hi there, thanks to you @berak I solved my problem, but I have a last question: does anyone know if this function will be implemented in python ? There is downloadKeypoints, but not downloadDescriptors which is unfortunate… Or at least, do you know if I can write a PR on the github repo ?

downloadKeypoints() needs a special function, because it needs to deserialize a GPUMat int a std::vector<KeyPoint>

the descriptors are just a cv::Mat / numpy array on the cpu, so you’re only a

desc_cpu =

away (i guess)

1 Like

Is that me or this part of OpenCV is not well documented ? I mean, this is quiet hard to learn all of this, especially when no one has used the same things.

Anyway, you are my lord. Thanks, I love you

no, it’s not you…

the python wrappers for cuda are pretty new, and still evolving