OnSemi/Aptina ap_ColorPipe RGB-32 output conversion to cv::Mat format

I am currently writing an application that reads picture data from multiple OnSemi cameras. I am successfully pulling images of the cameras and storing them into AVI video files.

I decided that I would like to add live video monitoring on-screen during the recording process and to keep it simple I wanted to accomplish this with OpenCV imshow() function.

The issue where I am stuck is conversion from the output of the OnSemi (Aptina) ap_ColorPipe function to a cv::Mat() variable that I can feed to imshow().

I get a pointer pDisplayFrame to an RGB buffer like this (nRGBBitDepth = 32).

pDisplayFrame = ap_ColorPipe(pVideoFileHandle[camera], pInBuffer[camera], InBufferSize[camera], nullptr, nullptr, &nRGBBitDepth);

I then attempt to convert it like this (nWidth = 1920, nHeight = 1082)

displayframe[camera] = cv::Mat(nHeight, nWidth, CV_32S, pDisplayFrame);

And show it like this:

cv::imshow(window_name[camera], displayframe[camera]);

The problem is in the cv::Mat() constructor. I know the ap_ColorPipe produces 32 bit unsigned RGB data. The picture size is 1920 x 1082.

I am am not sure about the OpenCV conversion data type specifier selection. Right now it is set for CV_32U which is not correct; CV_32U does not exist. I am not sure what to pick as I do not understand how this relates to the actual data format of my buffer and how the Mat() constructor interprets that. I did not find good documentation on this either.

I also have another OpenCV program that reads out of the AVI files that I produce with this program. When I look at the Mat data structures it creates in the variable watch window, everything matches after the conversion (col, row, dims, etc.) except for the MatStep variable. The working version has [5760,3] but the conversion above produces [7680,4].

BTW: Trying different avenues, I found a statement in a Qt example like this that works:

QImage *image = new QImage(rgb, rgbWidth, rgbHeight, QImage::Format_RGB32);

But it offered no clues.

32 bits are 4 bytes, so it’s [X R G B] or, [B G R X] .

try

cv::Mat(nHeight, nWidth, CV_8UC4, pDisplayFrame); // 4 channels a 8bits

look up the exact order in your cam’s manual, you might need a RGB → BGR conversion later (opencv uses BGR ordering for most ops)

1 Like

Thank you for confirming this. I was just reading up some more and realized I need to account for the (unused) alpha channel. So, not 8UC3 but 8UC4. It works. The other thing I forgot (being a rookie with OpenCV) was Waitkey(). No BGR conversion needed.

1 Like