Capture RGB Frame Directly using Open CV APIs

C++
,
1

Hello,
I tried capturing RGB frames directly using the following code snippet, Is this the right method (Using Open CV 4.5.1):??

  1. Open the camera Device: camera = cv::VideoCapture(0, cv::CAP_DSHOW);
  2. Set frame width, height and FPS:
    camera.set(cv::CAP_PROP_FRAME_WIDTH, 1920);
    camera.set(cv::CAP_PROP_FRAME_HEIGHT, 1080);
    camera.set(cv::CAP_PROP_FPS, 30);
  3. Set the capture mode to RGB and read it
    camera.set(cv::CAP_PROP_MODE, 1);
    camera.read(rgb_frame);

Thanks for the help.

Regards,
Madhav

2

what would 1 mean here ? (please use enum values, not magic numbers)

also, some things are unclear in your post:
opencv uses bgr pixel ordering, not rgb. you really wanted rgb order ?
cameras usually don’t send bgr images over the wire, more like yuv or jpg compressed ones, so there is already an internal conversion involved. did you want to shortcut that ?

3

I wanted to use enum CAP_MODE_RGB (1) as per open CV 3.4 (OpenCV: Flags for video I/O) but the same enum is not available in opencv 4.5.1 (OpenCV: cv::VideoCapture Class Reference). So just used the value directly.

1 Like
4

Thanks for response.

AI model we used consumes images in RGB format.

Currently we read the image in BGR format and use cv::cvtColor(src, dst, cv::COLOR_BGR2RGB) to convert back to RGB.

Once AI model processed again we convert it back to BGR format using cv::cvtColor() to BGR format for display using cv::imshow().

So we are thinking to eliminate cv::cvtColor() if we can set camera capture mode to RGB.

5

look for swapRB

RGB/BGR conversion is a common need with DNNs. that’s why all the APIs out there have this parameter.

https://docs.opencv.org/3.4/d6/d0f/group__dnn.html#ga29f34df9376379a603acd8df581ac8d7

6

try CAP_PROP_CONVERT_RGB (seems to be supported for DSHOW)

7

Ok, i will try using CAP_PROP_CONVERT_RGB to get RGB format image for further processing.

Do we need to convert the image again back to BGR format before feeding it to cv::imshow()??

8

Thanks for suggestion. Can you please share some usage example/sample code for same.??

9

IF you are using opencv’s dnn, i’d also suggest to leave the images as is
(in BGR), and use the swapRB param in blobFromImage , as you get the (internal) conversion basically for free (it’s just swapping a few channel pointers)

10

no! that’s not how this works.

CAP_PROP_CONVERT_RGB controls whether conversion to OpenCV’s standard color order (BGR) is happening, or the image data is returned raw as is (which can be YUV, or jpeg-compressed data).

it’s already true by default because OpenCV always converts whatever pixel format to BGR (color)

you can’t use this to force RGB order color.

setting or unsetting that property will not help you. Ignore it.

is anyone even reading what I wrote in this thread?

11

yes you can do it~~~ I build source it in ver 3.4.16

apt install libv4l-dev

-DWITH_LIBV4L=ON -DWITH_GSTREAMER=OFF

cmake .. -G"Unix Makefiles" -DCMAKE_INSTALL_PREFIX=/opt/opencv -DCMAKE_BUILD_TYPE=Release -DCMAKE_C_FLAGS_RELEASE="-O3 -g" -DCMAKE_CXX_FLAGS_RELEASE="-O3 -g" -DBUILD_PERF_TESTS=OFF -DBUILD_TESTS=OFF -DBUILD_JPEG=ON -DBUILD_PNG=ON -DBUILD_ZLIB=ON -DWITH_LIBV4L=ON -DBUILD_opencv_python3=ON -DBUILD_EXAMPLES=ON -DWITH_GSTREAMER=OFF -DWITH_CUDA=OFF
cmake --build . --config Release --target install -- -j$CPU_NUM VERBOSE=1

////////because
~~~in cap_gstreamer.cpp 
case CV_CAP_PROP_CONVERT_RGB:
        break;

~~~in cap_libv4l.cpp
case CV_CAP_PROP_MODE:
        int mode;
        mode = cvRound(value);
        if (capture->mode != mode) {
            switch (mode) {
            case CV_CAP_MODE_BGR:
            case CV_CAP_MODE_RGB:
            case CV_CAP_MODE_GRAY:
            case CV_CAP_MODE_YUYV:
                capture->mode = mode;

case CV_CAP_MODE_RGB:
    requestedPixelFormat = V4L2_PIX_FMT_RGB24;
    break;

~~~~in your cpp
cv::VideoCapture cap(0);
    cap.set(CV_CAP_PROP_FRAME_WIDTH,    800);
    cap.set(CV_CAP_PROP_FRAME_HEIGHT,   600);
    cap.set(CV_CAP_PROP_MODE,           CV_CAP_MODE_RGB);
12 
console command: v4l2-ctl --list-formats 
most Pixel Format: is 'YUYV'

cap.set(CV_CAP_PROP_MODE,           CV_CAP_MODE_YUYV);
then send to opencv color convert 
libv4l using c to convert color is slow
v4l2_ioctl
v4l2_dequeue_and_convert
v4lconvert_convert
v4lconvert_convert_pixfmt
https://github.com/philips/libv4l/blob/master/libv4lconvert/libv4lconvert.c

but opencv using this
inline void cvtYUV422toRGB(uchar * dst_data, size_t dst_step, const uchar * src_data, size_t src_step,
                           int width, int height)
{
    YUV422toRGB8Invoker<bIdx, uIdx, yIdx, dcn> converter(dst_data, dst_step, src_data, src_step, width);
    if (width * height >= MIN_SIZE_FOR_PARALLEL_YUV422_CONVERSION)
        parallel_for_(Range(0, height), converter);
    else
        converter(Range(0, height));
}