Demosaic bayer bggr

Hello,

I have a custom USB camer which sends raw bayer RGB image. Now I have a C++ code snippet which is able to show the picture in grayscale. But when i try to use any of the opencv’s built in demosaicing function I always got false colors(using the image sensor built test pattern function). I tried to demosaicing manually the bggr image, but it just produce a greenish version of the false coloring. I attached the grayscale and false coloring and manual demosaicing version of the sensor test pattern. I want to solve this problem, and also want to learn more about this topic. I would be grateful any kind of help or hint in this problem.
manualdemosaic
grayscale
coloredtestpattern

maybe you got some links with info about your hw / sensor ?

show, ofc. !

what does the test pattern look like (what should the camera see here) ?

It is a 400*400 ov426 sensor which sends 10 bit raw RGB images, bayer bggr. The camera datasheet not public, so I have minimal informations (already write multiple email for the vendor about the datashett, but no response yet), for example I do not know what should the camera see. But based on my experiences with a different cemara module i think this test pattern looks strange, also checking the picture when the pattern turned off the colors look odd.

#include <vector>
#include <iostream>
#include <fstream>
#include <stdio.h>
#include <opencv2/opencv.hpp>
#include <opencv2/highgui/highgui.hpp>

int main()
{
    cv::VideoCapture cap(0);
    cap.set(cv::CAP_PROP_CONVERT_RGB, false);
    cap.set(cv::CAP_PROP_FORMAT, -1);
    if (!cap.isOpened()) {
         std::cout << "Error opening video stream or file" << std::endl;
         return -1;
    }
     std::ofstream pixel;
     pixel.open("bayer.csv");
     cv::Mat image; /* Video frame placeholder */
     cap >> image;
     std::cout<<"RGB: "<<cap.get(cv::CAP_PROP_CONVERT_RGB)<<" format: "<<cap.get(cv::CAP_PROP_FORMAT)<<" frame: "<<cap.get(cv::CAP_PROP_FRAME_WIDTH)<<std::endl;
     int frame_size = image.dataend - image.datastart;
     std::cout<<"frame size: "<<frame_size<<" shape: "<<image.dims<<std::endl;
     std::cout<<"frame o dim: "<<image.dims<<" type: "<<image.type()<<" byte elements: "<<image.elemSize()<<" elements: "<<image.total()<<" rows: "<<image.rows<<" cols: "<<image.cols<<std::endl;
    while (1)
    {
        cap >> image;
        /* 1. Capture image as RGB555 -> [X R R R R R G G G G G B B B B B] */
   
         std::vector<unsigned short int> data; /* Placeholder for all image pixels */
         std::vector<unsigned short int> sdata;
   
         for (int i = 0; i < frame_size; i += 3) {        
             data.push_back(0xFFC0 & ((image.data[i + 2]) << 10 | ((image.data[i + 1]) << 5) | (image.data[i])));           
         }
        cv::Mat img_bayer = cv::Mat(image.rows, image.cols, CV_16U);
        cv::Mat img_bayer2 = cv::Mat(image.rows, image.cols, CV_16U);
        cv::Mat img_bayer8 = cv::Mat(image.rows, image.cols, CV_8U);
        cv::Mat rgb_bayer = cv::Mat(image.rows, image.cols, CV_16U);
        cv::Mat img_rgb = cv::Mat(image.rows, image.cols, CV_16UC3);
       
    
    cv::Mat img_converted;
    cv::imshow("bggr",bggr);

    memmove(img_bayer.data, data.data(), data.size() * 2);
    memmove(img_bayer2.data, data.data(), data.size() * 2);
    memmove(img_bayer8.data, data.data(), data.size());
    memmove(img_rgb.data, data.data(), sdata.size() * 2);
    cv::imshow("img_bayer",img_bayer);
    cv::imshow("img_bayergb",img_bayer8);
          
    cv::Mat bayered(400,400,CV_8UC3,cv::Scalar(255));
    cv::Mat rgb(400,400,CV_8UC3,cv::Scalar(0));
    for(int i=0;i<400;i++)
    {
        //cv::Vec3b* rpixels = bggr.ptr<cv::Vec3b>(i);
        if(i%2==0)
        {
            for(int j=0;j<400;j++)
            {
                if(j%2==0)
                {
                    bayered.at<cv::Vec3b>(i,j)[0] = img_bayer.at<uchar>(i,j);//b
                    bayered.at<cv::Vec3b>(i,j)[1] = 0;//g
                    bayered.at<cv::Vec3b>(i,j)[2] = 0;//r
                }
                else
                {
                    bayered.at<cv::Vec3b>(i,j)[0] = 0;//b
                    bayered.at<cv::Vec3b>(i,j)[1] = img_bayer.at<uchar>(i,j);//g
                    bayered.at<cv::Vec3b>(i,j)[2] = 0;//r
                }
            }
        }
        else
        {
            for(int j=0;j<400;j++)
            {
                if(j%2==0)
                {
                    bayered.at<cv::Vec3b>(i,j)[0] = 0;//b
                    bayered.at<cv::Vec3b>(i,j)[1] = img_bayer.at<uchar>(i,j);//g
                    bayered.at<cv::Vec3b>(i,j)[2] = 0;//r
                }
                else
                {
                    bayered.at<cv::Vec3b>(i,j)[0] = 0;//b
                    bayered.at<cv::Vec3b>(i,j)[1] = 0;//g
                    bayered.at<cv::Vec3b>(i,j)[2] = img_bayer.at<uchar>(i,j);//r
                }
            }
        }

    }
  
    cv::imshow("ori", img_bayer2);
    cv::imshow("bayered", bayered);
    cv::cvtColor(img_bayer2, rgb, cv::COLOR_BayerRG2BGR);
    cv::imshow("Display rgb", rgb);
    cv::cvtColor(img_bayer, rgb, cv::COLOR_BayerRG2RGB);
    cv::imshow("Display rgb2", rgb);

   
    if ((char)cv::waitKey(5) >= 0)
        break;
}

return 0;
}

I turned of the test pattern and make some picture with a normal target and used the same de-bayer functions. cv::COLOR_BayerGB2BGR result is the closest to the reality only the white color looks yellow a bit. I mean i can see green and red and blue if I show this color to the camera, but the whole image has a pale yellow overlay.
Display COLOR_BayerGB2BGR

don’t be surprised by the weird coloring.

debayered pictures usually require white balance, gamma mapping, (affine) remixing of color channels, …

looks a bit blurry and the barrel distortion is obvious but otherwise seems fine.

I do see some moire/interference pattern, the horizontal stripes rippling down the image. perhaps the room’s lighting flickers at a particularly high frequency. if you take a picture outdoors (or out the window) that should disappear.

1 Like

Thank you for your response, this comment contained soo many hints(most important I have to learn more about optics). I already starting get to the bottom of these topics. Thank you once more, this was really helpful.