Qt+OpenCV use cv::ogl::texture2D::copyFrom( cv::cuda::GpuMat) to get texture directly

Touch · July 5, 2022, 9:01am

I am writing a module which gets cv::cuda::GpuMat data on gpu and draw it on QOpenGLWidget. To accelarate the speed, I want to avoid downloading data from device to host.

This is the main code

void MyPainter::drawImage(QMatrix4x4 transform_matrix, const cv::Mat &img)
{
    if(parent->context() == 0){
        parent->makeCurrent();
    }

    shader_program->bind();

    cv::cuda::GpuMat gpumat_img;
    qDebug() << "upload img....";
    gpumat_img.upload(img);      //  for test, because other modules are unfished
    qDebug() << gpumat_img.empty() << gpumat_img.type() << "Finish upload";
    qDebug() << gpumat_img.size().width << gpumat_img.size().height;

    cv::ogl::Texture2D test_texture;
    qDebug() << "copy to texture memory....";
    test_texture.copyFrom(gpumat_img);  // here raise an error. 

    qDebug() << "Finish copy";

    test_texture.bind();
    vbo.bind();

    qDebug() << test_texture.size().width << test_texture.size().height;
    shader_program->setUniformValue("matrix", transform_matrix);
    shader_program->enableAttributeArray(MYSHADER_VERTEX_ATTRIBUTE);
    shader_program->enableAttributeArray(MYSHADER_TEXCOORD_ATTRIBUTE);
    shader_program->setAttributeBuffer(MYSHADER_VERTEX_ATTRIBUTE, GL_FLOAT, 0, 2);
    shader_program->setAttributeBuffer(MYSHADER_TEXCOORD_ATTRIBUTE, GL_FLOAT, 0, 2);

    QOpenGLFunctions *gl_function_ptr = QOpenGLContext::currentContext()->functions();
    gl_function_ptr->glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);

    shader_program->release();
    test_texture.release();
    vbo.release();

}

The error tells “OpenCV(4.6.0) Error: Gpu API call (OS call failed or operation not supported on this OS) in `anonymous-namespace’::CudaResource::registerBuffer, file D:\lib\opencv_my\source\opencv-4.6.0\modules\core\src\opengl.cpp, line 176”

But if I use the cv::ogl::texture2D::copyFrom(cv::Mat) instead of cv::ogl::texture2D::copyFrom(cv::cuda::GpuMat), it can work well.

I read some source code of opencv and find the error is appear at here:

    void CudaResource::registerBuffer(GLuint buffer)
    {
        CV_DbgAssert( buffer != 0 );

        if (buffer_ == buffer)
            return;

        cudaGraphicsResource_t resource;
        cudaSafeCall( cudaGraphicsGLRegisterBuffer(&resource, buffer, cudaGraphicsMapFlagsNone) );                   // here raise the error

        release();

        resource_ = resource;
        buffer_ = buffer;
    }

But i can’t find cudaGraphicsGLRegisterBuffer().

My environment

Qt 7.0.0
OpenCV 4.6.0
CUDA 11.6

Apologize for my poor English.

cudawarped · July 5, 2022, 4:53pm

This is something I’ve wanted to try for ages, so please let me know how you get on.

Regarding the error, have you checked the test cases? I had a quick look and didn’t get any errors from

cv::namedWindow("test", cv::WINDOW_OPENGL);
const Size sz(128, 128);
cv::cuda::GpuMat tmp(sz, CV_8UC1);
cv::ogl::Texture2D tex(sz, cv::ogl::Texture2D::DEPTH_COMPONENT, true);
tex.copyFrom(tmp, true);

Touch · July 6, 2022, 9:02am

Thanks! And I tried the code you given. But it shows the same error

OpenCV: terminate handler is called! The last OpenCV error is:
OpenCV(4.6.0) Error: Gpu API call (OS call failed or operation not supported on this OS) in `anonymous-namespace'::CudaResource::registerBuffer, file D:\lib\opencv_my\source\opencv-4.6.0\modules\core\src\opengl.cpp, line 176

I guess it is because of the cuda environment, but I can’t find more information about the error.

!!! I use the same code in my vs+cuda+opencv project. And it can work without error and show an image.

	cv::Mat mat_img;
	mat_img = cv::imread("C:/Users/Touch/Desktop/0.jpg");
	cv::namedWindow("test", cv::WINDOW_OPENGL);
	cv::cuda::GpuMat tmp;
	tmp.upload(mat_img);
	cv::ogl::Texture2D tex;
	tex.copyFrom(tmp, true);
	cv::imshow("test", tex);
	cv::waitKey(0);

cudawarped · July 6, 2022, 9:18am

Can you run bin\opencv_test_cudaarithm.exe --gtest_filter=OpenGL/Texture2D.Constructor1/0 without errors? e.g.

[ RUN ] OpenGL/Texture2D.Constructor1/0, where GetParam() = (128x128, 8UC1)
[ OK ] OpenGL/Texture2D.Constructor1/0 (2926 ms)
[----------] 1 test from OpenGL/Texture2D (2927 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test case ran. (3090 ms total)
[ PASSED ] 1 test.

Do any of the tests from opencv_test_cudaarithm.exe report that they have PASSED

Touch · July 6, 2022, 9:30am

Yes i can pass the test.

OpenCV version: 4.6.0
OpenCV VCS version: unknown
Build type: Debug Release
WARNING: build value differs from runtime: Release
Compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.32.31326/bin/Hostx64/x64/cl.exe  (ver 19.32.31329.0)
Parallel framework: ms-concurrency (nthreads=16)
CPU features: SSE? SSE2? SSE3?
Intel(R) IPP version: disabled
OpenCL is disabled
TEST: Skip tests with tags: 'mem_6gb', 'verylong'
Note: Google Test filter = OpenGL/Texture2D.Constructor1/0
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from OpenGL/Texture2D
[ RUN      ] OpenGL/Texture2D.Constructor1/0, where GetParam() = (128x128, 8UC1)
[       OK ] OpenGL/Texture2D.Constructor1/0 (3 ms)
[----------] 1 test from OpenGL/Texture2D (3 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test case ran. (172 ms total)
[  PASSED  ] 1 test.

I guess the problem is on my qt project environment. Because my visual studio project with opencv+cuda doesn’t have the error.

cudawarped · July 6, 2022, 9:31am

Let me know if you solve the problem, I will try later in a QT project to see if I get the same issue.

Touch · July 6, 2022, 11:17am

OK but i haven’t found any solution yet.

cudawarped · July 6, 2022, 1:34pm

I don’t get any errors in QT running the test code in a class derived from QLabel.

Touch · July 7, 2022, 2:29am

wow. I create a clickbutton on mainWidget and connect the signal and the test code as slot funtion. I think this doesn’t matter.
Could you please show me you .pro file?
And this is my .prg file.

QT       += core gui

greaterThan(QT_MAJOR_VERSION, 4): QT += widgets

CONFIG += c++17

# You can make your code fail to compile if it uses deprecated APIs.
# In order to do so, uncomment the following line.
#DEFINES += QT_DISABLE_DEPRECATED_BEFORE=0x060000    # disables all the APIs deprecated before Qt 6.0.0



# Default rules for deployment.
qnx: target.path = /tmp/$${TARGET}/bin
else: unix:!android: target.path = /opt/$${TARGET}/bin
!isEmpty(target.path): INSTALLS += target

# ===========opengl==============
QT += opengl
QT += core gui openglwidgets
QT += widgets
# ===============================


# ============opencv=============
INCLUDEPATH += D:\lib\opencv_my\build\install\include
#LIBS += D:\lib\opencv_my\build\install\x64\vc17\lib
LIBS += D:\lib\opencv_my\build\install\x64\vc17\lib\opencv_world460.lib
# ===============================


# ============eigen============
INCLUDEPATH += D:\lib\eigen-3.4.0
# =============================


# ============cuda=============
CUDA_DIR = $$(CUDA_PATH)
#message($${CUDA_DIR})
INCLUDEPATH += $${CUDA_DIR}/include
# =============================


# ============vtk==============
#LIBS += D:\lib\VTK-8.2.0\build\install_prefix\lib

# =============================


DISTFILES += \
    draw_image.frag \
    draw_image.vert

Topic		Replies	Views
GpuMat to openCV C++ opengl	1	202	December 12, 2023
Copy cv::cuda::GpuMat in Cuda Kernel C++ cuda	4	1129	October 5, 2022
Copy directX 12 texture to GpuMat C++ gpumat	10	405	November 22, 2023
How to use cv::cudev::GpuMat_ correctly? C++ cuda	4	700	May 26, 2021
Throw error when use cuda::warpAffine C++ cuda , imgproc	2	231	November 8, 2023

Qt+OpenCV use cv::ogl::texture2D::copyFrom( cv::cuda::GpuMat) to get texture directly

Related Topics