Python representation of Mat with pointer to user data

Python
, , , ,
1

Hello guys,
how do I represent Mat (Size size, int type, void *data, size_t step=AUTO_STEP) from docs in python?
Im trying to change C++ code of someone to python, and Im stuck.
Here is the code he wrote:

rs2::frameset frameset = pipe.wait_for_frames();
auto disparity_frame = frameset.get_depth_frame();
Mat mDisparity = Mat(Size(imageW, imageH), CV_16UC1, (void*)disparity_frame.get_data());

Any suggestions would be appreciated!

2

in python, all cv::Mat are instead numpy arrays.

please show how you intend to translate the first two lines of your code into python. then we can figure out how the OpenCV/numpy part (line 3) should work.

3

Thank you for your reply crackwitz!

As far, I could translate his code as this (if you need full code, just notice):

frameset = pipe.wait_for_frames()
disparity_frame = frameset.get_depth_frame()

The guy tries to get raw disparity from realsense camera and saving the data in raw.
These are his last lines:

...
auto disparity_frame = frameset.get_depth_frame();
Mat mDisparity = Mat(Size(imageW, imageH), CV_16UC1,(void*)disparity_frame.get_data());

ofstream outFile("Disparity.raw", ofstream::binary);
outFile.write((char*)mDisparity.data, mDisparity.elemSize() * mDisparity.total());
outFile.close();

I discussed with my senior, he suggested me to create array of zeros and assign row by row:

mDisparity = np.zeros((imageH,imageW),dtype=np.uint16)
data = np.asanyarray(disparity_frame.get_data())
for i in range(imageH):
    for j in range(imageW):
        mDisparity[i][j]=data[i][j]

But he said it will be slow.

Also, what does mDisparity.data gets? And why is he multiplying elemSize with total number of elements in array? Im trying to figure out it also.

Kindly, waiting for your reply!

4

he doesn’t seem to know numpy. do this:

data = np.asanyarray(disparity_frame.get_data()) # you gave this
# then try this:
disparity = data.view(np.uint16).reshape((imageH, imageW))

I see that the librealsense python docs are absolutely useless so you have my sympathy.

6

Hello crackwitz, sorry for late reply!
Thank you for your suggestion!

Although, I couldn’t get the desired image/data from the guy’s script, I could manage to get disparity map using StereoBM_create of opencv.

If you are one of OpenCV developers, I wanted to ask something. Currently, I am studying the book “Learning Opencv with python3” using D435i realsense camera.

There, the author gets disparity map using cv2.CAP_OPENNI_DISPARITY_MAP which is in uint8 data type (chapter 4). I could compute disparity map using Stereo_BM from stereo cameras. But the data type of computation is in int16. When I use cv2.imshow, the image looks like this:

642×513 14.2 KB

However, when I use pyplot.imshow(disparity,'gray'), the image it shows is what I desire to get.

But still, I cannot use the data I get in the exercise in book because it is in int16. I tried to change the dtype to uint8, but it will look different.

Any suggestions would appreciate! If you need any additional information,please tell me.

P.s. I cannot put more than one image, I wanted to show the pyplot img, and data from compute.

7

opencv’s imshow() does NOT normalize the 16bit image, for visualization, you probably want to do that on your own

(the disparity values cover only a small spectrum of the [0…0xffff] range, lookup, what you gave to min/max disparity in the blockmatching)

8

Thank you berak for quick reply!
I will look up about normalization, and i think you mean to use Stereo_SGBM. I haven’t tried, but will.

9

something like this