# Python Opencv/cuda/Opengl/imshow #18553 #18588 **URL:** \ **Category:** Python\ **Tags:** opengl, highgui, cuda\ **Created:** [January 11, 2023, 4:46pm UTC](https://forum.opencv.org/t/python-opencv-cuda-opengl-imshow-18553-18588/11581 "2023-01-11T16:46:58Z")\ **Posts on this page:** 17\ **Page:** 1
**Author:** ![manbehindthemadness](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/manbehindthemadness/32/6873_2.png) [@manbehindthemadness](https://forum.opencv.org/u/manbehindthemadness)\ **Post date:** [January 11, 2023, 4:46pm UTC](https://forum.opencv.org/t/python-opencv-cuda-opengl-imshow-18553-18588/11581/1 "2023-01-11T16:46:58Z")
I am attempting to output high resolution imagery in the form of a cv2.cuda\_GpuMat() using python.cv2.imshow using a namedWindow flagged with cv2.WINDOW\_OPENGL. This does not seem to be functional as the console reports that the feature has not been implemented and to explicitly download using cuda\_GpuMat.download(); however, in researching this problem I came across: > > > This pull request fixes https://github.com/opencv/opencv/issues/18553 by making …the Python bindings generator create a declaration for 'imshow' that works with GpuMat inputs under Python. > > \### Pull Request Readiness Checklist > > \- \[X\] I agree to contribute to the project under Apache 2 License. > \- \[X\] To the best of my knowledge, the proposed patch is not based on a code under GPL or other license that is incompatible with OpenCV > \- \[X\] The PR is proposed to proper branch > \- \[X\] There is reference to original bug report and related work > > > \##### System information (version) > > \- OpenCV =\> 4.4.0 > \- Operating System / Pl…atform =\> Linux x86\_64 (Ubuntu 18.04) > \- Python version: 3.6 > > \##### Detailed description > > According to the \[documentation\](https://docs.opencv.org/4.4.0/d7/dfc/group\_\_highgui.html#ga453d42fe4cb60e5723281a89973ee563), if a window was created with OpenGL support, ogl::Buffer , ogl::Texture2D and cuda::GpuMat are supported as input for imshow. This is not working as expected with the Python bindings when trying to visualize a GpuMat structure: > > \`\`\`python > import cv2 > import numpy as np > a = cv2.cuda\_GpuMat(np.array(\[\[1, 1\], \[1, 1\]\], dtype=np.uint8)) > cv2.namedWindow("my window", flags=cv2.WINDOW\_OPENGL|cv2.WINDOW\_NORMAL) > cv2.imshow("my window", a) > \`\`\` > > The Python snippet above throws an error: > > \`\`\` > Traceback (most recent call last): > File "imshow\_test.py", line 6, in \ > cv2.imshow("my window", a) > TypeError: Expected Ptr\ for argument 'mat' > \`\`\` > > After digging a bit into the problem, I got to the conclusion that the functions with GpuMat structures involved don't get properly wrapped in Python unless they are under the C++ 'cuda' namespace (which is not the case for 'imshow'). > > I created a small wrapper for 'imshow' under the 'cuda' namespace (cuda.cuda\_imshow) and this one works as intended in Python when passing a cuda\_GpuMat structure as an input: > > \`\`\`patch > diff -urN opencv/modules/highgui/include/opencv2/highgui.hpp ../../opencv/opencv-4.4.0/modules/highgui/include/opencv2/highgui.hpp > \--- opencv/modules/highgui/include/opencv2/highgui.hpp 2020-09-30 12:02:00.609385852 +0000 > +++ ../../opencv/opencv-4.4.0/modules/highgui/include/opencv2/highgui.hpp 2020-09-23 11:15:30.800408600 +0000 > @@ -172,6 +172,10 @@ > \*/ > > ///////////////////////// graphical user interface ////////////////////////// > +namespace cv { namespace cuda { > +CV\_EXPORTS\_W void cuda\_imshow(const String& winname, InputArray mat); > +}} > + > namespace cv > { > > @@ -389,6 +393,7 @@ > \*/ > CV\_EXPORTS\_W void imshow(const String& winname, InputArray mat); > > + > /\*\* @brief Resizes window to the specified size > > @note > diff -urN opencv/modules/highgui/src/window.cpp ../../opencv/opencv-4.4.0/modules/highgui/src/window.cpp > \--- opencv/modules/highgui/src/window.cpp 2020-09-30 12:02:00.677385931 +0000 > +++ ../../opencv/opencv-4.4.0/modules/highgui/src/window.cpp 2020-09-23 11:15:54.868340916 +0000 > @@ -425,6 +425,11 @@ > #endif > } > > +void cv::cuda::cuda\_imshow( const String& winname, InputArray \_img ) > +{ > + cv::imshow(winname, \_img); > +} > + > void cv::imshow(const String& winname, const ogl::Texture2D& \_tex) > { > CV\_TRACE\_FUNCTION(); > \`\`\` > > I'm not too familiar with the gen2.py script that generates the Python bindings, but modifying that script to contemplate this special case might be the best solution to this issue. > > \##### Steps to reproduce > > See the code snippet in the previous section. > > \##### Issue submission checklist > > - \[X\] I report the issue, it's not a question > - \[X\] I checked the problem with documentation, FAQ, open issues, > - \[X\] I updated to latest OpenCV version and the issue is still there > - \[X\] There is reproducer code and related data files: videos, images, onnx, etc Both indicating that cv2.imshow is able to process image mats directly from CUDA/GPU memory. Am I crazy? If not what is it I’m missing here? ---
**Author:** ![crackwitz](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/crackwitz/32/14_2.png) [@crackwitz](https://forum.opencv.org/u/crackwitz)\ **Post date:** [January 11, 2023, 9:32pm UTC](https://forum.opencv.org/t/python-opencv-cuda-opengl-imshow-18553-18588/11581/2 "2023-01-11T21:32:06Z")
does this work from C++ (build a small C++ demo file and run it)? if yes, it’s a python bindings issue. if that doesn’t work either, maybe the feature itself doesn’t quite work. ---
**Author:** ![manbehindthemadness](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/manbehindthemadness/32/6873_2.png) [@manbehindthemadness](https://forum.opencv.org/u/manbehindthemadness)\ **Post date:** [January 11, 2023, 10:09pm UTC](https://forum.opencv.org/t/python-opencv-cuda-opengl-imshow-18553-18588/11581/3 "2023-01-11T22:09:51Z")
I’m unable to do so, I’m trying to fix an issue on a GPU heavy production media system that’s running a 4K led-wall (and lagging downloading the frames to buffer on CPU. I don’t have the local resources to perform the test. ---
**Author:** ![manbehindthemadness](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/manbehindthemadness/32/6873_2.png) [@manbehindthemadness](https://forum.opencv.org/u/manbehindthemadness)\ **Post date:** [January 11, 2023, 10:12pm UTC](https://forum.opencv.org/t/python-opencv-cuda-opengl-imshow-18553-18588/11581/4 "2023-01-11T22:12:06Z")
Perhaps I can find someone to run a quick test for me, before the client decides to murder me haha. ---
**Author:** ![manbehindthemadness](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/manbehindthemadness/32/6873_2.png) [@manbehindthemadness](https://forum.opencv.org/u/manbehindthemadness)\ **Post date:** [January 11, 2023, 10:22pm UTC](https://forum.opencv.org/t/python-opencv-cuda-opengl-imshow-18553-18588/11581/5 "2023-01-11T22:22:34Z")
Actually looking at: [opencv/highgui.hpp at 4.7.0 · opencv/opencv · GitHub](https://github.com/opencv/opencv/blob/4.7.0/modules/highgui/include/opencv2/highgui.hpp) I don’t see the changes mentioned in the enhancement present in this file ---
**Author:** ![manbehindthemadness](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/manbehindthemadness/32/6873_2.png) [@manbehindthemadness](https://forum.opencv.org/u/manbehindthemadness)\ **Post date:** [January 11, 2023, 10:31pm UTC](https://forum.opencv.org/t/python-opencv-cuda-opengl-imshow-18553-18588/11581/6 "2023-01-11T22:31:36Z")
Alternatively this is interesting: [opencv/highgui.hpp at 725e440d278aca07d35a5e8963ef990572b07316 · opencv/opencv · GitHub](https://github.com/opencv/opencv/blob/725e440d278aca07d35a5e8963ef990572b07316/modules/highgui/include/opencv2/highgui.hpp#L394) ---
**Author:** ![cudawarped](https://avatars.discourse-cdn.com/v4/letter/c/9dc877/32.png) [@cudawarped](https://forum.opencv.org/u/cudawarped)\ **Post date:** [January 12, 2023, 6:53am UTC](https://forum.opencv.org/t/python-opencv-cuda-opengl-imshow-18553-18588/11581/7 "2023-01-12T06:53:12Z")
As @crackwitz alluded to, this is a C++ not python issue. I raised an issue on this a while back see [imshow no longer works for cv::cuda::GpuMat · Issue #22328 · opencv/opencv · GitHub](https://github.com/opencv/opencv/issues/22328). In short it looks like when OpenCV moved highgui to a plugin architechture the support for `GpuMat` was not implemented. This can be overcome in C++ by using ```auto cv::imshow("window_name", cv::ogl::Texture2D(gpuMat)); ``` however from a [quick inspection](https://github.com/opencv/opencv/blob/3d5e3a910faaec30b15eb694d5af7ec5ef435746/modules/core/include/opencv2/core/opengl.hpp#L283) the `cv::ogl::Texture2D` api is not exposed to python. Additionaly if `Texture2D` is wrapped (CV\_EXPORTS\_W) it can’t be passed to any OpenCV functions as they can’t deal with that type. ---
**Author:** ![manbehindthemadness](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/manbehindthemadness/32/6873_2.png) [@manbehindthemadness](https://forum.opencv.org/u/manbehindthemadness)\ **Post date:** [January 12, 2023, 12:50pm UTC](https://forum.opencv.org/t/python-opencv-cuda-opengl-imshow-18553-18588/11581/8 "2023-01-12T12:50:42Z")
Aha! That makes perfect sense! I guess that puts me onto the search for an alternative method, unfortunately I haven’t been able to locate a different solution. Any ideas? ---
**Author:** ![cudawarped](https://avatars.discourse-cdn.com/v4/letter/c/9dc877/32.png) [@cudawarped](https://forum.opencv.org/u/cudawarped)\ **Post date:** [January 12, 2023, 1:48pm UTC](https://forum.opencv.org/t/python-opencv-cuda-opengl-imshow-18553-18588/11581/9 "2023-01-12T13:48:53Z")
> [@manbehindthemadness](#): > > Any ideas? It depends, if your processing a single video stream in a single thread you could try using CUDA streams and displaying the output with a 1 frame delay as I demonstrated in the guide below [https://jamesbowley.co.uk/accelerating-opencv-with-cuda-streams-in-python/](https://jamesbowley.co.uk/accelerating-opencv-with-cuda-streams-in-python/) ---
**Author:** ![manbehindthemadness](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/manbehindthemadness/32/6873_2.png) [@manbehindthemadness](https://forum.opencv.org/u/manbehindthemadness)\ **Post date:** [January 12, 2023, 3:07pm UTC](https://forum.opencv.org/t/python-opencv-cuda-opengl-imshow-18553-18588/11581/10 "2023-01-12T15:07:46Z")
I was just reading about that, yeah it’s a single output stream going to an led wall ![image](https://us1.discourse-cdn.com/flex020/uploads/opencv/original/2X/f/fe5353055d35b442159c5ae17a1a2786f0a47643.jpeg) ---
**Author:** ![manbehindthemadness](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/manbehindthemadness/32/6873_2.png) [@manbehindthemadness](https://forum.opencv.org/u/manbehindthemadness)\ **Post date:** [January 12, 2023, 3:11pm UTC](https://forum.opencv.org/t/python-opencv-cuda-opengl-imshow-18553-18588/11581/11 "2023-01-12T15:11:29Z")
I’m using this in tandem with a Luxonis Depthai RGBD capture device (OAK-D pro) ---
**Author:** ![cudawarped](https://avatars.discourse-cdn.com/v4/letter/c/9dc877/32.png) [@cudawarped](https://forum.opencv.org/u/cudawarped)\ **Post date:** [January 12, 2023, 3:33pm UTC](https://forum.opencv.org/t/python-opencv-cuda-opengl-imshow-18553-18588/11581/12 "2023-01-12T15:33:12Z")
Are you sure its the download which is taking the time? I would time each operation to see where the slow down is. If its the decode then you could try using `cv2.cudacodec.VideoReader()`. ---
**Author:** ![manbehindthemadness](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/manbehindthemadness/32/6873_2.png) [@manbehindthemadness](https://forum.opencv.org/u/manbehindthemadness)\ **Post date:** [January 12, 2023, 5:33pm UTC](https://forum.opencv.org/t/python-opencv-cuda-opengl-imshow-18553-18588/11581/13 "2023-01-12T17:33:25Z")
I have isolated the delay into two locations, upload is hurting me but that’s torch, and cv is actually way faster, on the download side I’m losing out because I am downloading to CPU and then it’s resizing the image to the display, I have the option of uploading back to GPU but that’s just as bad, I want to convert the memory map from torch to cv and then pipe directly to the output without getting tangled up on the cpu ---
**Author:** ![manbehindthemadness](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/manbehindthemadness/32/6873_2.png) [@manbehindthemadness](https://forum.opencv.org/u/manbehindthemadness)\ **Post date:** [January 12, 2023, 5:35pm UTC](https://forum.opencv.org/t/python-opencv-cuda-opengl-imshow-18553-18588/11581/14 "2023-01-12T17:35:09Z")
The really wild thing is if I use 2k it’s instant, like zero delay 30+fps but 4K is a brutal half a second delay and 7 fps ---
**Author:** ![manbehindthemadness](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/manbehindthemadness/32/6873_2.png) [@manbehindthemadness](https://forum.opencv.org/u/manbehindthemadness)\ **Post date:** [January 12, 2023, 11:11pm UTC](https://forum.opencv.org/t/python-opencv-cuda-opengl-imshow-18553-18588/11581/15 "2023-01-12T23:11:18Z")
I also think it’s important to note that the host system in this case is an Nvidia AGX Orin so the behavior might be a little different. ---
**Author:** ![manbehindthemadness](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/manbehindthemadness/32/6873_2.png) [@manbehindthemadness](https://forum.opencv.org/u/manbehindthemadness)\ **Post date:** [January 13, 2023, 5:45pm UTC](https://forum.opencv.org/t/python-opencv-cuda-opengl-imshow-18553-18588/11581/16 "2023-01-13T17:45:04Z")
I would like to thank you, your article was able to provide me with a series of solutions that overcame my bottleneck and satisfied my client. 🤝 ---
**Author:** ![cudawarped](https://avatars.discourse-cdn.com/v4/letter/c/9dc877/32.png) [@cudawarped](https://forum.opencv.org/u/cudawarped)\ **Post date:** [January 14, 2023, 8:03am UTC](https://forum.opencv.org/t/python-opencv-cuda-opengl-imshow-18553-18588/11581/17 "2023-01-14T08:03:39Z")
Nice glad I could help!