YUYV -> RGB conversion

1

Hello,

I have image frames incoming from a camera in the YUYV color space, and want to convert these frames to RGB. I’m currently using the color conversion code enum cv::COLOR_YUV2RGB_YUYV to do the conversion. However, the red and blue channels in the resulting image are switched for me, meaning a red object in the scene is appearing as blue and vice versa. It seems the data buffer in the converted image is laid out as B1 G1 R1 B2 G2 R2 B3 G3 R3, etc, where Ri, Gi, Bi are the RGB values for the ith pixel.

It seems I need to use cv::COLOR_YUV2BGR_YUYV to get the buffer layout in Ri, Gi, Bi order. That’s what I see others also do to make the YUYV->RGB conversion work:

I wondered if this has something to do with OpenCV storing RGB images in BGR layout, so I tried feeding BayerRG8 images from the camera to my application, and used the cv::COLOR_BayerRG2RGB conversion code. This time, the converted image was indeed in the RGB layout, meaning the pixels were laid out in the data buffer of the converted Mat object in Ri, Gi, Bi order.

Is this a bug in OpenCV w.r.t. the YUYV->RGB conversion, or am I missing something?

Thanks.

2

welcome.

did you imshow or imwrite the result? how do you determine that the channel order is mixed up?

opencv’s native order is BGR. when you ask for RGB and imshow/imwrite that, opencv will still assume BGR order, and you will see mixed up channels.

3

@crackwitz I updated my question to indicate that I indeed consider that BGR quirk of OpenCV.

I’m getting the data array from the Mat object and using matplotlib.pyplot.imshow routine to display the image. That’s how I’m seeing the order as switched.

Like I say in my updated question, what’s intriguing to me is how the OpenCV behavior is inconsistent between BayerRG->RGB and YUYV->RGB conversions.

4

matplotlib assumes RGB order.

please provide data and code to reproduce the issue.

5

the source for this conversion, for reference

6

@crackwitz

The data are here: Dropbox - YUYVIssue - Simplify your life

These are 4 files obtained from the camera:
yuv422_yuyv_packed.* - the raw YUYV image and its conversion to RGB as .bmp file done by the camera software
bayerRG.* - the raw BayerRG image and its conversion to RGB as .bmp file done by the camera software

Below is the Python code I wrote to read and convert the .raw files, to help you debug the issue. You’ll find that the OpenCV conversion from BayerRG->RGB as displayed by cv2 matches the .bmp image from the camera software. But, in the YUYV->RGB conversion done by cv2, blue and red channels are switched when compared to the .bmp image from the camera.

NOTE: I was using matplotlib to display images earlier, but modified my code to use cv2.imshow for consistency and simplification.

Seems to me there’s a bug in OpenCV. What do you think?

import cv2
import numpy as np

with open("yuv422_yuyv_packed.raw", "rb") as f:
    yuyv_file_contents=f.read()
with open("bayerRG.raw", "rb") as f:
    bayerRG_file_contents=f.read()

height = 2056
width = 2464
yuyv_num_channels = 2
bayerRG_num_channels = 1

yuyv_image = np.reshape(np.array(bytearray(yuyv_file_contents)), (height, width, yuyv_num_channels))
bayerRG_image = np.reshape(np.array(bytearray(bayerRG_file_contents)), (height, width, bayerRG_num_channels))

yuyv2rgb_image = cv2.cvtColor(yuyv_image, cv2.COLOR_YUV2RGB_YUYV)
bayerRG2rgb_image = cv2.cvtColor(bayerRG_image, cv2.COLOR_BayerRG2RGB)

cv2.imshow('yuyv image', yuyv2rgb_image)
cv2.waitKey(0)
cv2.destroyAllWindows()

cv2.imshow('bayerRG image', bayerRG2rgb_image)
cv2.waitKey(0)
cv2.destroyAllWindows()
7

@crackwitz

Upon further investigation on this, it seems the real bug is with BayerRG → RGB conversion:

In the code and data I uploaded above, the behavior with the YUYV->RGB conversion is actually expected, because cv2.imshow() assumes BGR order. So the red and blue channels are switched w.r.t. the .bmp image from the camera in the case of YUYV->RGB conversion.

But in the case of BayerRG->RGB conversion I attempt above, OpenCV actually produces a BGR image, which matches the camera-produced RGB image only because cv.imshow() switches the red and blue channels.

Thoughts?