Hi all,
I am currently using SURF with the FLANN descriptor, and I would like to test SURF in its GPU version.
But I don’t find any informations about FLANN with SURF GPU. Since FLANN use the descriptors function “detectAndCompute” and GPU SURF have not this function, how can I use FLANN with the GPU version ?
there is no FLANN matcher for gpu descriptors.
if you want to process tem from CUDA, you probably have to use the cuda::BFMatcher
Okay, but a matcher only takes keypoints right ? I saw the function doawnloadKeypoints(descriptor, gpu_keypoints), we can’t do something with this ?
In the flann based matcher page, I also saw some GPU implementations OpenCV: cv::FlannBasedMatcher Class Reference so I was wondering if it was possible to make it through
ah right, i missed that
So I thought the only thing I should change is the descriptor, make GpuMats (even if I dont know how it work atm) and then “download” the keypoints passed to flann. Does it make sense ?
this would give you GPUMats for keypoints & descriptors already
(counterpart to detectAndCompute() on cpu)
So I convert my img and mask first into GpuMats, then I apply operator like detectAndCompute ?
kp, des = surf.detectAndCompute(frame, mask)
==
gpu_frame = cv.cuda_GpuMat(frame)
gpu_mask = cv.cuda_GpuMat(mask)
kp, des = surf.operator(gpu_frame, gpu_mask)
cannot try (also not from python…), but no, more like:
cv::cuda::SURF_CUDA det = cv::cuda::SURF_CUDA::SURF_CUDA();
cv::GPUMat kp,desc;
det(img,kp,desc);Oh okay, but yes I am in python… But if you say it’s the counterpart of detectandcompute I assume it works the same 
Thank you for your help
Hum, yes I don’t know how to implement this in python in fact ^^’
The only example I found in python with SURF CUDA is this post
[EDIT] And I just found this post for who wants
the ()() operator is probably wrapped as a function in python
try to create an instance and call help(surf) on it, then it’ll get pretty obvious, what to do 
as a sidenote, the cuda python are only documented if cuda is available on the machine, that builds it, pity.
edit: your 1st link seems to be it !
class cuda_SURF_CUDA(builtins.object)
| Methods defined here:
|
| __new__(*args, **kwargs) from builtins.type
| Create and return a new object. See help(type) for accurate signature.
|
| __repr__(self, /)
| Return repr(self).
|
| defaultNorm(...)
| defaultNorm() -> retval
| .
|
| descriptorSize(...)
| descriptorSize() -> retval
| .
|
| detect(...)
| detect(img, mask[, keypoints]) -> keypoints
| . @brief Finds the keypoints using fast hessian detector used in SURF
| .
| . @param img Source image, currently supports only CV_8UC1 images.
| . @param mask A mask image same size as src and of type CV_8UC1.
| . @param keypoints Detected keypoints.
|
| detectWithDescriptors(...)
| detectWithDescriptors(img, mask[, keypoints[, descriptors[, useProvidedKeypoints]]]) -> keypoints, descriptors
| . @brief Finds the keypoints and computes their descriptors using fast hessian detector used in SURF
| .
| . @param img Source image, currently supports only CV_8UC1 images.
| . @param mask A mask image same size as src and of type CV_8UC1.
| . @param keypoints Detected keypoints.
| . @param descriptors Keypoint descriptors.
| . @param useProvidedKeypoints Compute descriptors for the user-provided keypoints and recompute keypoints direction.
|
| downloadKeypoints(...)
| downloadKeypoints(keypointsGPU) -> keypoints
| .
|
| ----------------------------------------------------------------------
| Static methods defined here:
|
| create(...)
| create(_hessianThreshold[, _nOctaves[, _nOctaveLayers[, _extended[, _keypointsRatio[, _upright]]]]]) -> retval
| . @param _hessianThreshold Threshold for hessian keypoint detector used in SURF.
| . @param _nOctaves Number of pyramid octaves the keypoint detector will use.
| . @param _nOctaveLayers Number of octave layers within each octave.
| . @param _extended Extended descriptor flag (true - use extended 128-element descriptors; false - use
| . 64-element descriptors).
| . @param _keypointsRatio
| . @param _upright Up-right or rotated features flag (true - do not compute orientation of features;
| . false - compute orientation).
|
| ----------------------------------------------------------------------
| Data descriptors defined here:
|
| extended
| extended
|
| hessianThreshold
| hessianThreshold
|
| keypointsRatio
| keypointsRatio
|
| nOctaveLayers
| nOctaveLayers
|
| nOctaves
| nOctaves
|
| upright
| upright
Maybe I don’t create my GpuMat correctly. Because I followed the first link and I get this error :
kpGPU, des = surf.detectWithDescriptors(gpu_frame, gpu_mask)
cv2.error: OpenCV(4.4.0) C:\opencv-4.4.0\opencv_contrib-4.4.0\modules\xfeatures2d\src\surf.cuda.cpp:146: error: (-215:Assertion failed) !img.empty() && img.type() == CV_8UC1 in function '`anonymous-namespace'::SURF_CUDA_Invoker::SURF_CUDA_Invoker'
I declare my mask like so :
mask = np.zeros(frame.shape[:2], dtype=np.uint8)
# (these 4 lines create ROIs mask on a image taken as reference)
save = np.zeros(frame.shape[:2], dtype=np.uint8)
points = np.array([list(r.topleft), list(r.topright), list(r.bottomright), list(r.bottomleft)], np.int32)
cv.fillPoly(mask, [points], (255, 255, 255))
cv.bitwise_and(frame, frame, masked, mask)
gpu_mask = cv.cuda_GpuMat(mask)
gpu_frame = cv.cuda_GpuMat(frame)
kpGPU, des = surf.detectWithDescriptors(gpu_frame, gpu_mask)
[EDIT] For those who have the same error with detectWithDescriptors, the image needs to be converted into grayscale
Hi there, thanks to you @berak I solved my problem, but I have a last question: does anyone know if this function will be implemented in python ? There is downloadKeypoints, but not downloadDescriptors which is unfortunate… Or at least, do you know if I can write a PR on the github repo ?
downloadKeypoints() needs a special function, because it needs to deserialize a GPUMat int a std::vector<KeyPoint>
the descriptors are just a cv::Mat / numpy array on the cpu, so you’re only a
desc_cpu = desc_cuda.download()
away (i guess)
Is that me or this part of OpenCV is not well documented ? I mean, this is quiet hard to learn all of this, especially when no one has used the same things.
Anyway, you are my lord. Thanks, I love you
no, it’s not you…
the python wrappers for cuda are pretty new, and still evolving