Is OpenNI2 still supported?

1

Hi,

I’ve recently built OpenCV (on Windows 10) and wanted to try the rgbd examples, but ran into OpenNI 2 related errors:

example_rgbd_dynafu_demo.exe
[ERROR:0] global C:\Users\george.profenza\Downloads\gp\opencv\modules\videoio\src\cap.cpp (246) cv::VideoCapture::open VIDEOIO(OPENNI2): raised OpenCV exception:

OpenCV(4.5.1-pre) C:\Users\george.profenza\Downloads\gp\opencv\modules\videoio\src\cap_openni2.cpp:378: error: (-2:Unspecified error) CvCapture_OpenNI2::CvCapture_OpenNI2 : Couldn't start IR stream:  in function 'CvCapture_OpenNI2::toggleStream'


OpenCV: terminate handler is called! The last OpenCV error is:
OpenCV(4.5.1-pre) Error: Unspecified error (CvCapture_OpenNI2::CvCapture_OpenNI2 : Couldn't start IR stream: ) in CvCapture_OpenNI2::toggleStream, file C:\Users\george.profenza\Downloads\gp\opencv\modules\videoio\src\cap_openni2.cpp, line 378

How would I go about fixing/getting around this issue ?

(I remember successfully using OpenCV with OpenNI 1 back in 2012 and that worked like a charm. I can see the WITH_OPENNI in CMake however the configuration Video I/O only lists OpenNI2, but no OpenNI (1). Is there a way to try OpenNI 1 if OpenNI 2 doesn’t work?)

Any hints/tips are appreciated.

Thank you,
George

2

Yes, OpenCV supports OpenNI 2.

Did you try to start the camera using OpenNI 2 (try the SDK examples)? The error clearly says that the problem isn’t with OpenCV or OpenNI, but by starting the IR camera.

Otherwise you can use OpenNI (or the SDK of any other camera) without the OpenCV binding. It’s a bit more complicated, but I prefer this solution as it gives much better control of the camera. Then, convert the grabbed buffer to OpenCV Mat for further processing.

Here is a little pseudocode:

uchar *buffer;
Camera mycam;
if(!mycam.open())printf("Camera not opened!!!");
int W,H;
W = mycam.getWidth(); H = mycam.getHeight();
mycam.start();
mycam.grab(buffer);
Mat img(W,H,CV_8UC3,buffer);
cvtColor(img,img,COLOR_RGB2BGR);
//...process img in OpenCV...
3

Hi @kbarni,

Apologies for the delay. Eventually I found the time to test this more thoroughly.

OpenCV appears to support OpenNI 2, however in my particular case just can’t seem to open the capture device.

The SDK examples run without a hitch.

I haven’t drilled down to see why OpenNI (1 or 2) doesn’t work on Windows 10 (Insider Preview).

I did follow your suggestion, and indeed manually interfacing with OpenNI2 and passing the bytes to a cv::Mat does the trick:

#include <OpenNI.h>

#include "opencv2/highgui/highgui.hpp"
#include "opencv2/videoio/videoio.hpp"

#include <iostream>

using namespace cv;
using namespace std;

int main() {
    cout << "opening device" << endl;

    VideoCapture sensor1;

    sensor1.open(CAP_OPENNI_ASUS);

    if (!sensor1.isOpened()) {
        cout << "CAP_OPENNI_ASUS can not open capture object " << endl;
    }

	sensor1.open(CAP_OPENNI2_ASUS);
    
    if (!sensor1.isOpened()) {
        cout << "CAP_OPENNI2_ASUS can not open capture object " << endl;
    }

    openni::Status rc = openni::STATUS_OK;

    openni::Device device;
    openni::VideoStream depth, color;
    const char* deviceURI = openni::ANY_DEVICE;
    rc = openni::OpenNI::initialize();

    printf("After initialization:\n%s\n", openni::OpenNI::getExtendedError());
    rc = device.open(deviceURI);
    if (rc != openni::STATUS_OK)
    {
        printf("SimpleViewer: Device open failed:\n%s\n", openni::OpenNI::getExtendedError());
        openni::OpenNI::shutdown();
        return 1;
    }

    rc = depth.create(device, openni::SENSOR_DEPTH);
    if (rc == openni::STATUS_OK)
    {
        rc = depth.start();
        if (rc != openni::STATUS_OK)
        {
            printf("SimpleViewer: Couldn't start depth stream:\n%s\n", openni::OpenNI::getExtendedError());
            depth.destroy();
        }
    }
    else
    {
        printf("SimpleViewer: Couldn't find depth stream:\n%s\n", openni::OpenNI::getExtendedError());
    }

    rc = color.create(device, openni::SENSOR_COLOR);
    if (rc == openni::STATUS_OK)
    {
        rc = color.start();
        if (rc != openni::STATUS_OK)
        {
            printf("SimpleViewer: Couldn't start color stream:\n%s\n", openni::OpenNI::getExtendedError());
            color.destroy();
        }
    }
    else
    {
        printf("SimpleViewer: Couldn't find color stream:\n%s\n", openni::OpenNI::getExtendedError());
    }

    if (!depth.isValid() || !color.isValid())
    {
        printf("SimpleViewer: No valid streams. Exiting\n");
        openni::OpenNI::shutdown();
        return 2;
    }

    openni::VideoMode vm = depth.getVideoMode();
    int cols, rows;
    cols = vm.getResolutionX();
    rows = vm.getResolutionY();

	openni::VideoFrameRef frame;
    depth.start();

    for (;;) {
        depth.readFrame(&frame);
        
        openni::DepthPixel* dData = (openni::DepthPixel*)frame.getData();
        Mat depthMat(rows, cols, CV_16UC1, dData);
        Mat depthShow;
        const float scaleFactor = 0.05f;
        depthMat.convertTo(depthShow, CV_8UC1, scaleFactor);
        //Mat depth1;
        
        /*if (!sensor1.grab()) {
            cout << "Sensor1 can not grab images." << endl;
            return -1;
        }
    	
        else if (sensor1.retrieve(depthMat, CAP_OPENNI_DEPTH_MAP)) imshow("depthMat", depthMat);
        */

    	if(!depthMat.empty())
    	{
            imshow("depth", depthShow);
    	}
    	
        if (waitKey(30) == 27)   break;

    }

	depth.stop();

    openni::OpenNI::shutdown();
}

OpenCV fails to open OPENNI and OPENNI2 devices:
manual_openni2_depth_x