Given a Direct3D Texture2D I would like to process it using OpenCV
cv::GpuMat. I am currently building 4.7 with CUDA 11.7 and CUDNN 8.9.
I am having some difficulties getting around CUDA’s interop with Direct3D 11 hence I would like to take a look at
cv::GpuMat, which, from what I’ve read, is for managing OpenCV-related data on the GPU (e.g. using CUDA).
Is there an established workflow for “converting” a Direct3D Texture2D, which I use for my framebuffer, to an OpenCV
cv::GpuMat and vice versa? It is also important that operations take place on the GPU without copying data back and forth between CPU and GPU. If that is not possible, then my hope is that I can at least use pinned memory, which I have already experimented with for CUDA and Torch C++ (libtorch).
I am allmost 100% sure there is no interop between Direct3D11 and
GpuMat. I would think what you have already tried is the best thing to use
Yes, and I am struggling with it. I need to draw to the texture, convert it to a CUDA compatible resource (as you have noted, the
cudaGraphicsResource is the most promising way), process the contents (in my case with Torch) in that same buffer and then render the result to the screen.
When working with Torch, I found out that
torch::from_blob() (with the option to create the resulting tensor on the CUDA-enabled GPU) basically (or so I think) wraps the contents of the CUDA memory block in that tensor. I can access it with both raw CUDA operations as well as Torch tensor operations. My hope is that I can e.g. wrap the OpenCV
cv::Mat to a
cudaArray in a similar way. Since that didn’t work out (don’t know why, but I can’t seem to be able to alter the data in wrapped array) I decided to look into anything else that OpenCV might offer.
Also OpenCV: opencv2/core/directx.hpp File Reference is something I just found. I am not sure what happens underneath though.
that’s for ocl, not cuda ;(