OpenGL texture to GpuMat (CUDA)?

Python
,
1

I am trying to pass an OpenGL texture to CUDA , right now I am doing it via glReadPixels to save it as a Python byte object, which needs to be converted to an image with PIL, then to an array using Numpy, before finally using it with OpenCV…

but I saw in the OpenCV docs that there is this:

cv::ogl::Buffer::mapDevice
Maps OpenGL buffer to CUDA device memory.

how do I access this function from Python?

2

unfortunately, it’s not wrapped into the python api

(and even if, you’d have to build from src with both opengl and cuda support)

3

(and even if, you’d have to build from src with both opengl and cuda support)
I am using Cudawarped’s build of OpenCV, so it is with CUDA enabled

is there a workaround for my problem? at the very least I want to avoid transferring to system memory, maybe there is some way to make a copy of the OpenGL data on the GPU and put it into CUDA that way?

4

If you can do this in python (maybe with OpenGL - pycuda 2024.1.2 documentation) then you can pass the raw pointer to the CUDA memory you have mapped from OpenGL using createGpuMatFromCudaMemory().

1 Like
5

I am going to need a bit of hand-holding here with regards to OpenGL … I hope this isn’t straying too far from the purpose of this forum, but all I know is that part of the OpenGL code involves this:
glBindTexture(GL_TEXTURE_2D, TextureID)
the first argument is the type of texture , and the second argument is the name of the texture… Does the name have anything to do with the pointer?

6

no, it’s the id (an integer) of the currently activated opengl gpu texture slot

https://registry.khronos.org/OpenGL-Refpages/gl4/html/glBindTexture.xhtml

7

OpenGL is strange and different from most other APIs for anything.

instead of calling methods on some texture, instead you bind the texture (it is noted in OpenGL’s state that you can’t see directly), and then you call methods/functions on whatever texture is bound, without specifying it in the method/function.

That is my superficial understanding anyway.

8

How do I get the address of the GPU memory out of OpenGL so that I can pass it onto createGpuMatFromCudaMemory() ?

do I do this

glBindTexture(GL_TEXTURE_BUFFER, TextureID)
glGetBufferPointerv(GL_TEXTURE_BUFFER,GL_BUFFER_MAP_POINTER)

or something like that?

9

I think there’s interop from OpenGL to OpenCL, and from OpenCL to CUDA… so that may involve a step via OpenCL. there might be interop from OpenGL to CUDA though I am in no position to have a clue about that.

10

I think I have a plan, to use GL_PIXEL_PACK_BUFFER , like so:

import numpy
import ctypes
from OpenGL.GL import *

pbo = glGenBuffers(1) # create a pixel buffer object
openglframe =np.zeros((imageheight,imagewidth,4),np.uint8) # array with the resolution of the image I want to use, in 4 channels for RGBA, used for the byte size of glBufferData
glBindBuffer(GL_PIXEL_PACK_BUFFER, pbo)
glBufferData(GL_PIXEL_PACK_BUFFER,openglframe ,  GL_DYNAMIC_READ)  # configures the pixel buffer

firstloop = True

tempcudamat = cv2.cuda.GpuMat(height,width,cv2.CV_8UC3) # temporary texture used for mapping the address from OpenGL as a CUDA GpuMat
outputcudamat = cv2.cuda.GpuMat(height,width,cv2.CV_8UC3) # the OpenGL texture should end up as this CUDA GpuMat

while True:
                // There is a function which writes to the texture with TextureID, which runs in this line in my program
                glActiveTexture(GL_TEXTURE0) # activate the OpenGL texture slot
                glBindTexture(GL_TEXTURE_2D, TextureID) # selects the texture
                if firstloop is True:
                    glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, imagewidth,imageheight, 0, GL_RGBA, GL_UNSIGNED_BYTE, None) # configures the texture , runs only once
                    firstloop = False
                glBindBuffer(GL_PIXEL_PACK_BUFFER, pbo) #selects the pixel buffer
                glGetTexImage(GL_TEXTURE_2D, 0, GL_RGBA, GL_UNSIGNED_BYTE,openglframe) # gets the texture into the buffer


                // After this is the Python part
                mapdata = glMapBuffer(GL_PIXEL_PACK_BUFFER,GL_READ_ONLY) # returns a ctype pointer of the pixel buffer
                pointer = ctypes.from_address(mapdata) # convert ctypes pointer to address 

                tempcudamat=cv2.cuda.createGpuMatFromCudaMemory(spoutreceiveheight,spoutreceivewidth,pointer) # give address to createGpuMatFromCudaMemory , and assign the memory to the temporary CUDA texture 
                tempcudamat.copyTo(outputcudamat) # copy the OpenGL texture from temporary CUDA GpuMat 

                // And finally, unmap the pixel buffer in OpenGl
                glUnmapBuffer(GL_PIXEL_PACK_BUFFER)
1 Like
11 
glGetTexImage(GL_TEXTURE_2D, 0, GL_RGBA, GL_UNSIGNED_BYTE)

I am getting hung up on this part, no matter what I try I am getting an ‘invalid operation’ here