# SURF GPU with FLANN? **URL:** \ **Category:** Python\ **Tags:** flann, surf, gpu, cuda\ **Created:** [May 3, 2021, 1:18pm UTC](https://forum.opencv.org/t/surf-gpu-with-flann/3109 "2021-05-03T13:18:30Z")\ **Posts on this page:** 16\ **Page:** 1
**Author:** ![PommePomme](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/pommepomme/32/2005_2.png) [@PommePomme](https://forum.opencv.org/u/PommePomme)\ **Post date:** [May 3, 2021, 1:18pm UTC](https://forum.opencv.org/t/surf-gpu-with-flann/3109/1 "2021-05-03T13:18:30Z")
Hi all, I am currently using SURF with the FLANN descriptor, and I would like to test SURF in its GPU version. But I don’t find any informations about FLANN with SURF GPU. Since FLANN use the descriptors function “detectAndCompute” and GPU SURF have not this function, how can I use FLANN with the GPU version ? ---
**Author:** ![berak](https://avatars.discourse-cdn.com/v4/letter/b/85f322/32.png) [@berak](https://forum.opencv.org/u/berak)\ **Post date:** [May 3, 2021, 1:31pm UTC](https://forum.opencv.org/t/surf-gpu-with-flann/3109/2 "2021-05-03T13:31:40Z")
there is no FLANN matcher for gpu descriptors. if you want to process tem from CUDA, you probably have to use the [cuda::BFMatcher](https://docs.opencv.org/master/dd/dc5/classcv_1_1cuda_1_1DescriptorMatcher.html#ac821657d8859f70c0592b91d7d48673c) ---
**Author:** ![PommePomme](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/pommepomme/32/2005_2.png) [@PommePomme](https://forum.opencv.org/u/PommePomme)\ **Post date:** [May 3, 2021, 1:42pm UTC](https://forum.opencv.org/t/surf-gpu-with-flann/3109/3 "2021-05-03T13:42:53Z")
Okay, but a matcher only takes keypoints right ? I saw the function doawnloadKeypoints(descriptor, gpu\_keypoints), we can’t do something with this ? In the flann based matcher page, I also saw some GPU implementations [OpenCV: cv::FlannBasedMatcher Class Reference](https://docs.opencv.org/3.4/dc/de2/classcv_1_1FlannBasedMatcher.html#a94398a33b3c40989245f22f59ad78e7a) so I was wondering if it was possible to make it through ---
**Author:** ![berak](https://avatars.discourse-cdn.com/v4/letter/b/85f322/32.png) [@berak](https://forum.opencv.org/u/berak)\ **Post date:** [May 3, 2021, 1:49pm UTC](https://forum.opencv.org/t/surf-gpu-with-flann/3109/4 "2021-05-03T13:49:16Z")
ah right, i missed that ---
**Author:** ![PommePomme](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/pommepomme/32/2005_2.png) [@PommePomme](https://forum.opencv.org/u/PommePomme)\ **Post date:** [May 3, 2021, 1:54pm UTC](https://forum.opencv.org/t/surf-gpu-with-flann/3109/5 "2021-05-03T13:54:55Z")
So I thought the only thing I should change is the descriptor, make GpuMats (even if I dont know how it work atm) and then “download” the keypoints passed to flann. Does it make sense ? ---
**Author:** ![berak](https://avatars.discourse-cdn.com/v4/letter/b/85f322/32.png) [@berak](https://forum.opencv.org/u/berak)\ **Post date:** [May 3, 2021, 1:59pm UTC](https://forum.opencv.org/t/surf-gpu-with-flann/3109/6 "2021-05-03T13:59:08Z")
[this would give you GPUMats](https://docs.opencv.org/3.4/db/d06/classcv_1_1cuda_1_1SURF__CUDA.html#aa0ed78380fe2142a729e424aa195263c) for keypoints & descriptors already (counterpart to detectAndCompute() on cpu) ---
**Author:** ![PommePomme](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/pommepomme/32/2005_2.png) [@PommePomme](https://forum.opencv.org/u/PommePomme)\ **Post date:** [May 3, 2021, 2:09pm UTC](https://forum.opencv.org/t/surf-gpu-with-flann/3109/7 "2021-05-03T14:09:16Z")
So I convert my img and mask first into GpuMats, then I apply operator like detectAndCompute ? ```auto kp, des = surf.detectAndCompute(frame, mask) == gpu_frame = cv.cuda_GpuMat(frame) gpu_mask = cv.cuda_GpuMat(mask) kp, des = surf.operator(gpu_frame, gpu_mask) ``` ---
**Author:** ![berak](https://avatars.discourse-cdn.com/v4/letter/b/85f322/32.png) [@berak](https://forum.opencv.org/u/berak)\ **Post date:** [May 3, 2021, 2:15pm UTC](https://forum.opencv.org/t/surf-gpu-with-flann/3109/8 "2021-05-03T14:15:04Z")
cannot try (also not from python…), but no, more like: ``` cv::cuda::SURF_CUDA det = cv::cuda::SURF_CUDA::SURF_CUDA(); cv::GPUMat kp,desc; det(img,kp,desc); ``` ---
**Author:** ![PommePomme](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/pommepomme/32/2005_2.png) [@PommePomme](https://forum.opencv.org/u/PommePomme)\ **Post date:** [May 3, 2021, 2:18pm UTC](https://forum.opencv.org/t/surf-gpu-with-flann/3109/9 "2021-05-03T14:18:41Z")
Oh okay, but yes I am in python… But if you say it’s the counterpart of detectandcompute I assume it works the same 🤞 Thank you for your help ---
**Author:** ![PommePomme](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/pommepomme/32/2005_2.png) [@PommePomme](https://forum.opencv.org/u/PommePomme)\ **Post date:** [May 3, 2021, 2:23pm UTC](https://forum.opencv.org/t/surf-gpu-with-flann/3109/10 "2021-05-03T14:23:46Z")
Hum, yes I don’t know how to implement this in python in fact ^^’ The only example I found in python with SURF CUDA is [this post](https://stackoverflow.com/questions/62623035/opencv-cuda-surf-performance-vs-cpu-version) [EDIT] And I just found [this post](https://stackoverflow.com/questions/61895456/retrieving-keypoints-from-cv2-cuda-orb-detectandcomputeasync) for who wants ---
**Author:** ![berak](https://avatars.discourse-cdn.com/v4/letter/b/85f322/32.png) [@berak](https://forum.opencv.org/u/berak)\ **Post date:** [May 3, 2021, 2:34pm UTC](https://forum.opencv.org/t/surf-gpu-with-flann/3109/11 "2021-05-03T14:34:06Z")
the `()()` operator is probably [wrapped as a function in python](https://github.com/opencv/opencv_contrib/blob/fc0cec623add0775ce855bbe1841027dbad96f5d/modules/xfeatures2d/include/opencv2/xfeatures2d/cuda.hpp#L171) try to [create an instance](https://github.com/opencv/opencv_contrib/blob/master/modules/xfeatures2d/include/opencv2/xfeatures2d/cuda.hpp#L117) and call `help(surf)` on it, then it’ll get pretty obvious, what to do 😉 as a sidenote, the cuda python are only documented if cuda is available on the machine, that builds it, pity. edit: your 1st link seems to be it ! ---
**Author:** ![PommePomme](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/pommepomme/32/2005_2.png) [@PommePomme](https://forum.opencv.org/u/PommePomme)\ **Post date:** [May 3, 2021, 2:43pm UTC](https://forum.opencv.org/t/surf-gpu-with-flann/3109/12 "2021-05-03T14:43:46Z")
```auto class cuda_SURF_CUDA(builtins.object) | Methods defined here: | | __new__ (*args, **kwargs) from builtins.type | Create and return a new object. See help(type) for accurate signature. | | __repr__ (self, /) | Return repr(self). | | defaultNorm(...) | defaultNorm() -> retval | . | | descriptorSize(...) | descriptorSize() -> retval | . | | detect(...) | detect(img, mask[, keypoints]) -> keypoints | . @brief Finds the keypoints using fast hessian detector used in SURF | . | . @param img Source image, currently supports only CV_8UC1 images. | . @param mask A mask image same size as src and of type CV_8UC1. | . @param keypoints Detected keypoints. | | detectWithDescriptors(...) | detectWithDescriptors(img, mask[, keypoints[, descriptors[, useProvidedKeypoints]]]) -> keypoints, descriptors | . @brief Finds the keypoints and computes their descriptors using fast hessian detector used in SURF | . | . @param img Source image, currently supports only CV_8UC1 images. | . @param mask A mask image same size as src and of type CV_8UC1. | . @param keypoints Detected keypoints. | . @param descriptors Keypoint descriptors. | . @param useProvidedKeypoints Compute descriptors for the user-provided keypoints and recompute keypoints direction. | | downloadKeypoints(...) | downloadKeypoints(keypointsGPU) -> keypoints | . | | ---------------------------------------------------------------------- | Static methods defined here: | | create(...) | create(_hessianThreshold[, _nOctaves[, _nOctaveLayers[, _extended[, _keypointsRatio[, _upright]]]]]) -> retval | . @param _hessianThreshold Threshold for hessian keypoint detector used in SURF. | . @param _nOctaves Number of pyramid octaves the keypoint detector will use. | . @param _nOctaveLayers Number of octave layers within each octave. | . @param _extended Extended descriptor flag (true - use extended 128-element descriptors; false - use | . 64-element descriptors). | . @param _keypointsRatio | . @param _upright Up-right or rotated features flag (true - do not compute orientation of features; | . false - compute orientation). | | ---------------------------------------------------------------------- | Data descriptors defined here: | | extended | extended | | hessianThreshold | hessianThreshold | | keypointsRatio | keypointsRatio | | nOctaveLayers | nOctaveLayers | | nOctaves | nOctaves | | upright | upright ``` Maybe I don’t create my GpuMat correctly. Because I followed the first link and I get this error : ```auto kpGPU, des = surf.detectWithDescriptors(gpu_frame, gpu_mask) cv2.error: OpenCV(4.4.0) C:\opencv-4.4.0\opencv_contrib-4.4.0\modules\xfeatures2d\src\surf.cuda.cpp:146: error: (-215:Assertion failed) !img.empty() && img.type() == CV_8UC1 in function '`anonymous-namespace'::SURF_CUDA_Invoker::SURF_CUDA_Invoker' ``` I declare my mask like so : ```auto mask = np.zeros(frame.shape[:2], dtype=np.uint8) # (these 4 lines create ROIs mask on a image taken as reference) save = np.zeros(frame.shape[:2], dtype=np.uint8) points = np.array([list(r.topleft), list(r.topright), list(r.bottomright), list(r.bottomleft)], np.int32) cv.fillPoly(mask, [points], (255, 255, 255)) cv.bitwise_and(frame, frame, masked, mask) gpu_mask = cv.cuda_GpuMat(mask) gpu_frame = cv.cuda_GpuMat(frame) kpGPU, des = surf.detectWithDescriptors(gpu_frame, gpu_mask) ``` [EDIT] For those who have the same error with detectWithDescriptors, the image needs to be converted into grayscale ---
**Author:** ![PommePomme](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/pommepomme/32/2005_2.png) [@PommePomme](https://forum.opencv.org/u/PommePomme)\ **Post date:** [May 4, 2021, 9:29am UTC](https://forum.opencv.org/t/surf-gpu-with-flann/3109/13 "2021-05-04T09:29:03Z")
Hi there, thanks to you @berak I solved my problem, but I have a last question: does anyone know if [this function](https://docs.opencv.org/master/db/d06/classcv_1_1cuda_1_1SURF__CUDA.html#aa810356f1c5eb81474735ec57a5bdea4) will be implemented in python ? There is downloadKeypoints, but not downloadDescriptors which is unfortunate… Or at least, do you know if I can write a PR on the github repo ? ---
**Author:** ![berak](https://avatars.discourse-cdn.com/v4/letter/b/85f322/32.png) [@berak](https://forum.opencv.org/u/berak)\ **Post date:** [May 4, 2021, 9:41am UTC](https://forum.opencv.org/t/surf-gpu-with-flann/3109/14 "2021-05-04T09:41:39Z")
downloadKeypoints() needs a special function, because it needs to deserialize a GPUMat int a `std::vector` the descriptors are just a cv::Mat / numpy array on the cpu, so you’re only a ``` desc_cpu = desc_cuda.download() ``` away (i guess) ---
**Author:** ![PommePomme](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/pommepomme/32/2005_2.png) [@PommePomme](https://forum.opencv.org/u/PommePomme)\ **Post date:** [May 4, 2021, 10:46am UTC](https://forum.opencv.org/t/surf-gpu-with-flann/3109/15 "2021-05-04T10:46:41Z")
Is that me or this part of OpenCV is not well documented ? I mean, this is quiet hard to learn all of this, especially when no one has used the same things. Anyway, you are my lord. Thanks, I love you ---
**Author:** ![berak](https://avatars.discourse-cdn.com/v4/letter/b/85f322/32.png) [@berak](https://forum.opencv.org/u/berak)\ **Post date:** [May 4, 2021, 11:34am UTC](https://forum.opencv.org/t/surf-gpu-with-flann/3109/16 "2021-05-04T11:34:00Z")
no, it’s not you… the python wrappers for cuda are pretty new, and still evolving